Water quality determination device and water quality determination method

Abstract

To provide a water quality determination device and a water quality determination method that are capable of detecting abnormalities with high accuracy without erroneous detection.SOLUTION: An image data storage unit 101 stores image data of a water surface photographed by a network camera. An image data processing unit 102 processes the image data and outputs the processed image data. An image data expansion unit 103 generates abnormal dummy data from the processed image data. A learning processing unit 104 performs learning using the abnormal dummy data as teacher data, and stores learned parameters in a parameter storage unit 105. An abnormal estimation unit 106 processes the processed image data using the learned parameters and outputs a segmentation map. A water quality assessment unit 107 assesses whether a water quality is normal or abnormal based on a predetermined threshold value and the segmentation map, and outputs water quality assessment result data indicating an assessment result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a water quality determining device and a water quality determining method. [Background technology]

[0002] The final settling tank, which is part of the wastewater treatment process at a sewage treatment plant, purifies sewage by removing activated sludge that has grown in the reaction tank. By periodically detecting and collecting foreign matter such as flocs (a collection of microorganisms contained in activated sludge) and scum (floating debris) that float on the water surface, the quality of the sewage discharged into the river can be stabilized.

[0003] In recent years, there has been a labor shortage at sewage treatment plants, and there is a need to improve the efficiency of maintenance and management of sewage treatment facilities. As a result, there are discussions about replacing some of the work of employees with artificial intelligence (AI).

[0004] Patent Document 1, an example of the prior art, discloses a water quality analysis device and a water quality analysis method that perform water quality analysis corresponding to various characteristics of water quality without special tuning. The water quality analysis device of Patent Document 1 includes a hue analysis unit that learns a convolutional autoencoder (CAE) for image data of a settling tank to extract a feature amount of the hue of the settling tank, classifies the image data into hue clusters by clustering from the feature amount of the hue, and draws a feature space and cluster distribution of the hue, and a floating matter analysis unit that uses CAE to extract a feature amount of floating matter in the settling tank, classifies the image data into clusters of floating matter by clustering from the feature amount of the floating matter, and draws a feature space and cluster distribution of the floating matter.

[0005] Furthermore, Patent Document 2, an example of prior art, discloses a water quality determination device that uses CAE to generate a normal background image based on an input image and determines the water quality by evaluating the difference between the input image and the generated normal background image.

[0006] Furthermore, Patent Document 3, an example of conventional technology, discloses a water quality assessment device that utilizes time-series image data to achieve high robustness even under harsh shooting environments in order to prevent changes in images due to changes in lighting or external light from affecting the assessment of water quality conditions. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2021-170373 A [Patent Document 2] JP 2019-136664 A [Patent Document 3] JP 2021-140522 A Summary of the Invention [Problem to be solved by the invention]

[0008] However, in Patent Document 1, image quality is degraded by disturbance factors such as dirty lenses, bad weather (fog or rain), turbulence on the water surface, and refraction and reflection of water, resulting in a high probability of false positives. In addition, there is very little abnormal data, making it difficult to maintain a balance in the learning process.

[0009] In addition, in Patent Document 2, when there is a disturbance, the disturbed part is detected as an abnormality, resulting in a false positive. In addition, when the illumination of the settling tank is insufficient or when the camera exposure correction is inappropriate due to a sudden change in illumination, noise is likely to occur in the settling tank image. If such noise is mixed into the CAE model during the learning process as a normal feature, there is a problem that the detection accuracy of fine floating objects such as flocs decreases.

[0010] Furthermore, in Patent Document 3, there is a problem in that disturbances adversely affect the determination of the water quality state, making the detection accuracy unstable.

[0011] In addition, when using an unsupervised deep learning model called Convolutional Autoencoder (CAE), the model learns the details and features of a normal background image, and reproduces a normal background image close to the input image by losing some of the information that is not to be extracted. Various foreign objects are detected based on the difference between the reproduced image and the input image, but there is a problem in that it is not possible to distinguish whether the foreign object affects the water quality or is an external disturbance such as a dirty lens that does not affect the water quality.

[0012] There is a deep learning model called supervised semantic segmentation, which is a deep learning method that labels all pixels in an image, but the task of labeling each pixel as normal or abnormal is an extremely high artificial cost. In addition, it is necessary to prevent overlearning in deep neural networks to improve generalization performance, but there is a problem that it is difficult to collect abnormal images in final settling tanks because the frequency of abnormalities is very low.

[0013] Therefore, an object of the present invention is to provide a water quality determining device that is capable of detecting abnormalities with high accuracy without erroneous detection. [Means for solving the problem]

[0014] The water quality assessment device of the first invention for solving the above problems is characterized in that it comprises an image data storage unit that stores image data acquired from a water quality assessment target, an image data processing unit that performs processing on the image data stored in the image data storage unit and outputs processed image data, an image data extension unit that performs data extension on the processed image data and generates abnormal dummy data, a learning processing unit that performs deep learning using the abnormal dummy data as teacher data and outputs learned parameters, a parameter storage unit that stores the learned parameters, an abnormality estimation unit that uses the learned parameters stored in the parameter storage unit to estimate to which class label each pixel of the processed image data belongs, and a water quality assessment unit that performs water quality assessment based on the class label of each pixel estimated by the abnormality estimation unit.

[0015] In addition, the water quality determination device of the second invention for solving the above-mentioned problems is characterized in that, in the first invention, the image data expansion unit selects an area selected from the abnormal image data in response to user operation as block image data, and by binarizing the selected block image data, generates binarized mask image data in which only abnormal pixels are present, accurately separating abnormal pixels from background pixels remaining in the vicinity, combines the block image data and the mask image data to create a block data set having an arbitrary shape and brightness, and combines the block data set with the normal processed image data to generate the abnormal dummy data.

[0016] In addition, the water quality assessment device of the third invention for solving the above-mentioned problems is characterized in that, in the second invention, the image data expansion unit automatically divides the abnormal image data having abnormal pixels into local images of any shape, performs labeling according to the density of the abnormal pixels, and clusters the local areas of the image having abnormal pixels to determine the cut-out position and shape of the block image data.

[0017] In addition, the water quality assessment device of the fourth invention for solving the above problems is characterized in that, in the second or third invention, it further includes an abnormal pixel data storage unit that stores the abnormal pixel data that has been pre-labeled by the image data expansion unit so that it can be used in a water quality assessment device that assesses the water quality of a water quality assessment target other than the water quality assessment target.

[0018] In addition, the water quality assessment device of the fifth invention for solving the above problems is characterized in that, in the fourth invention, it includes, instead of the image data expansion unit, a dummy data generation unit that uses the abnormal pixel data in the abnormal pixel data memory unit and combines it with normal image data for the different water quality assessment targets to generate abnormal dummy data for learning.

[0019] A water quality determination method according to a sixth invention for solving the above problems is a water quality determination method for a water quality determination device including an image data storage unit, an image data processing unit, an image data expansion unit, a learning processing unit, a parameter storage unit, an abnormality estimation unit, and a water quality determination unit, characterized in that the image data storage unit stores image data acquired from a water quality determination target, the image data processing unit performs processing on the image data stored in the image data storage unit to output processed image data, the image data expansion unit performs data expansion on the processed image data to generate abnormal dummy data, the learning processing unit performs deep learning using the abnormal dummy data as teacher data and outputs learned parameters, the parameter storage unit stores the learned parameters, the abnormality estimation unit uses the learned parameters stored in the parameter storage unit to estimate which class label each pixel of the processed image data belongs to, and the water quality determination unit performs water quality determination based on the class label of each pixel estimated by the abnormality estimation unit.

[0020] In addition, the water quality determination method of the seventh invention for solving the above-mentioned problems is characterized in that, in the sixth invention, the image data expansion unit generates abnormal dummy data by selecting an area selected from the abnormal image data in response to a user operation as block image data, binarizing the selected block image data to generate binarized mask image data in which only abnormal pixels are present, accurately separating abnormal pixels from background pixels remaining in the vicinity, combining the block image data and the mask image data to create a block data set having an arbitrary shape and brightness, and combining the block data set with the normal processed image data to generate the abnormal dummy data.

[0021] In addition, the water quality assessment method of the eighth invention for solving the above-mentioned problems is characterized in that, in the seventh invention, the image data expansion unit generating abnormal dummy data includes automatically dividing the abnormal image data having abnormal pixels into local images of any shape and labeling them according to the density of the abnormal pixels, and determining the cut-out position and shape of the block image data by clustering the local areas of the image having abnormal pixels.

[0022] In addition, a water quality assessment method according to a ninth aspect of the present invention for solving the above-mentioned problems is characterized in that, in the seventh or eighth aspect of the present invention, the water quality assessment device further includes an abnormal pixel data storage unit, and the abnormal pixel data storage unit stores the abnormal pixel data that has been pre-labeled by the image data expansion unit so that it can be used in a water quality assessment device that assesses the water quality of a water quality assessment subject that is different from the water quality assessment subject.

[0023] In addition, the water quality assessment method of the 10th invention for solving the above-mentioned problems is characterized in that, in the 9th invention, the water quality assessment device is provided with a dummy data generation unit instead of the image data expansion unit, and the dummy data generation unit uses abnormal pixel data in the abnormal pixel data memory unit to generate abnormal dummy data for learning by combining it with normal image data for the different water quality assessment targets. Effect of the Invention

[0024] According to the present invention, it is possible to detect an abnormality with high accuracy without erroneous detection. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a block diagram showing the configuration of a water quality determining device 100 according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a conceptual diagram showing the configuration of a DeepLabV3+ model adopted in a learning processing unit 104 according to the first embodiment. [Diagram 3]11 is a conceptual diagram showing a data extension method of the image data extension unit 103 in the water quality determination device 100 according to the second embodiment. FIG. [Figure 4] FIG. 11 is a block diagram showing the configuration of a water quality determination system 400 according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0027] A. First embodiment Fig. 1 is a block diagram showing the configuration of a water quality determination device 100 according to a first embodiment of the present invention. In Fig. 1, the water quality determination device 100 includes an image data storage unit 101, an image data processing unit 102, an image data extension unit 103, a learning processing unit 104, a parameter storage unit 105, an abnormality estimation unit 106, and a water quality determination unit 107.

[0028] The image data storage unit 101 stores image data of the water surface obtained by photographing a settling tank of a sewage treatment plant with a net camera (not shown). The image data processing unit 102 performs trimming, normalization, resizing, and data labeling on the image data stored in the image data storage unit 101 into a format that can be input to a deep learning semantic segmentation model used in the learning processing unit 104, and outputs the processed image data. The image data processing unit 102 can be realized by a processor such as an MPU (Micro-Processing Unit) and a CPU (Central Processing Unit).

[0029] The image data extension unit 103 performs data augmentation on the processed image data processed by the image data processing unit 102 to generate abnormal dummy data. The learning processing unit 104 performs learning using the abnormal dummy data supplied from the image data extension unit 103 as teacher data, generates learned parameters, and outputs them to the parameter storage unit 105. The learning processing unit 104 can be realized by a processor such as a CPU, a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit). The parameter storage unit 105 stores parameters of a trained deep convolutional neural network (DCNN) and parameters that are not involved in the learning of the learning processing unit 104 and have been trained separately.

[0030] The abnormality estimation unit 106 processes the processed image data from the image data processing unit 102 using the learned parameters read from the parameter storage unit 105, and finally outputs a segmentation map. The water quality determination unit 107 calculates the area and quantity of normal regions and the area and quantity of abnormal regions based on the segmentation map output from the abnormality estimation unit 106, determines whether the image is normal or abnormal based on a predetermined threshold, and outputs water quality determination result data indicating the determination result.

[0031] FIG. 2 is a conceptual diagram showing an example of a deep learning semantic segmentation model adopted in the learning processing unit 104 according to the first embodiment. In the present invention, a supervised deep learning semantic segmentation model is used, but is not limited to a specific semantic segmentation model in deep learning. Famous deep learning models include U-Net, PSPNet, DeepLab, etc. Any of these deep learning models are effective for the learning method and anomaly detection method proposed in the present invention, but a deep learning model called DeepLabV3+ is used to evaluate the performance of anomaly detection. Therefore, if different deep learning models, hyperparameters, and learning data are used, there is a possibility that the test scores will differ. Semantic segmentation is a technology that classifies images on a pixel-by-pixel basis and assigns a label to each pixel, and is widely used in fields such as object detection and image processing. DeepLabV3+ is one of the algorithms that boasts the highest accuracy in the field of semantic segmentation.

[0032] In order to prevent overlearning and improve generalization performance, it is necessary to prepare a large number of images with labeled abnormalities as training data. However, since the frequency of abnormalities occurring in a sedimentation tank is extremely low, it is difficult to secure the required number of abnormal images. Therefore, in the present invention, a label is attached to each pixel in the training dataset to indicate which class it belongs to, and the abnormal image data is transformed by rotating, inverting, cropping, zooming, adjusting brightness, adding noise, etc., and a new abnormal dummy dataset is created from the original training dataset, thereby securing the required amount of abnormal image data. Then, the original training dataset and the newly created abnormal dummy dataset are combined and input to the learning processing unit 104 of DeepLabV3+.

[0033] FIG. 2 is a conceptual diagram showing the configuration of the DeepLabV3+ model adopted in the learning processing unit 104 according to the first embodiment. As shown in FIG. 2, the DeepLabV3+ model is composed of two parts, an encoder 200 and a decoder 201. The encoder 200 uses a trained deep convolutional neural network (DCNN) to pass an input image (learning image data) 202 through multiple convolution layers and combines feature representations at multiple spatial scales. In addition, feature representations at different resolutions are fused using a method called ASPP (Atrous Spatial Pyramid Pooling), which is a pyramid structure having dilated convolution layers with different rates. This makes it possible to capture a wider range of features while reducing the amount of calculation.

[0034] The decoder 201 upsamples the input image based on the feature map by the convolution layer according to the size of the segmentation map corresponding to the input image, and outputs the segmentation map 203 as shown in the figure. The learning processing unit 104 adjusts the parameters of the deep learning model by minimizing the difference between the output segmentation map 203 and the correct label map. However, in order to obtain a strong feature extraction ability, in this first embodiment, ResNet50 (depth residual network), which is a trained high-precision image classification network, is used as the network of the DCNN part of the encoder 200. The parameters of the DCNN do not participate in the learning of the learning processing unit 104, and are stored in the parameter storage unit 105 together with parameters that have been separately trained.

[0035] Next, a procedure for detecting the amount of foreign matter by the water quality determination device 100 of the first embodiment described above will be described. In the water quality determination device 100 shown in Fig. 1, an image of the water surface of a settling pond in a sewage treatment plant taken by a net camera is stored in an image data storage unit 101. An image data processing unit 102 performs trimming, normalization, resizing, and data labeling on the image data stored in the image data storage unit 101 into a format that can be input to a deep learning semantic segmentation model, and then supplies the processed image data to an image data extension unit 103 and anomaly estimation unit 106.

[0036] The image data extension unit 103 executes data extension processing on the processed image data, creates new abnormal dummy data, and supplies the data to the learning processing unit 104. The learning processing unit 104 performs learning using the DeepLabV3+ model with the abnormal dummy data as training data, generates learned parameters, and outputs the learned parameters to the parameter storage unit 105.

[0037] On the other hand, the anomaly estimation unit 106 first reads the learned parameters generated by the DeepLabV3+ model from the parameter storage unit 105, and uses the learned parameters to estimate the probability of which class each pixel belongs to for the processed image data input from the image data processing unit 102, and generates a two-dimensional array representing the probability distribution of each class label. Then, the two-dimensional array assigns the class label with the highest probability to each pixel. Finally, the anomaly estimation unit 106 combines RGB pixels corresponding to a preset class label according to the class label of each pixel, and outputs a segmentation map. The class labels can be treated as a binary classification problem in which clean water quality and ceiling reflection are normal labels, and flocs and scum are abnormal labels, or as a multi-value classification problem in which multiple water quality states such as normal background, scum, and flocs are classified.

[0038] The abnormality estimation unit 106 supplies the generated segmentation map to the water quality determination unit 107. Based on the generated segmentation map, the water quality determination unit 107 calculates the area and quantity of normal regions and the area and quantity of abnormal regions according to the class label of each pixel. Next, the water quality determination unit 107 determines an abnormality when the calculated area and quantity exceed a preset threshold, and determines a normality when they do not, and outputs water quality determination result data indicating the water quality determination result.

[0039] According to the above-described first embodiment, a supervised convolutional neural network is used to perform estimation using a deep learning model that has learned the characteristics and textures of normal backgrounds, scum, flocs, etc., so that it is possible to distinguish and detect (distinguish anomalies) with high accuracy between disturbances such as dirty lenses, bad weather (fog or rain), rippling water surface, refraction / reflection of water, changes in lighting, and foreign matter such as flocs (aggregates of microorganisms contained in activated sludge) and scum (floating residue), which were difficult to distinguish abnormalities from using conventional unsupervised deep learning methods.

[0040] B. Second embodiment Next, a second embodiment of the present invention will be described. In the second embodiment, although the image data expansion unit 103 is different from that in the first embodiment, the basic configuration of the water quality determination device 100 is the same as that in FIG. 1, and therefore a description thereof will be omitted.

[0041] In general data augmentation methods, processes such as left-right flipping of image data, rotation, enlargement, reduction, change of color, brightness, saturation, and noise addition are included. However, image data generated by general data augmentation methods may be unnatural and have low similarity to the original image data. Conversely, when augmented image data similar to the original image data is generated, there are problems such as little new information and low diversity of the augmented data. Therefore, in the second embodiment, a new data augmentation method is proposed based on the publicly known data augmentation technology described in the first embodiment.

[0042] Fig. 3 is a conceptual diagram showing the data augmentation method of the image data extension unit 103 in the water quality determination device 100 according to the second embodiment. As shown in Fig. 3, the data augmentation method of the second embodiment is composed of two parts: a block data set generation method for generating an abnormal block image data set, and a dummy data generation method for generating dummy data for training and verifying a deep learning model based on the block data set.

[0043] Next, a block data set generation method will be described. First, for the abnormal image data, an experienced person manually selects a region of a scum part that has accumulated like mud or a flock part where floating objects are relatively dense (using an input means such as a mouse, or an input means that specifies coordinates + shape (any shape, specific shape such as a rectangle / ellipse)) or divides the region using an algorithm (program), and performs cropping to generate block image data 301. Next, by binarizing the block image data 301, a binarized binary image (mask image data 302) is generated in which only abnormal pixels exist, with abnormal pixels accurately separated from background pixels remaining in the vicinity. In the binarized binary image (mask image data 302), pixels with a value of 1 are abnormal pixels, and pixels with a value of 0 are normal pixels. The binary image is multiplied by a label value corresponding to the abnormal pixels to create the ground truth of the abnormal block image.

[0044] Usually, block image data of any shape can be obtained by manually cutting out abnormal pixels from the abnormal image data 300, but it is also possible to use a predetermined algorithm to identify an abnormal pixel-dense area, surround it with a bounding box having an appropriate size and shape, and automatically select an image area to be cropped by cropping to cut out the block image data 301. For example, the predetermined algorithm may be a local feature learning method described below. The abnormal image data 300 having abnormal pixels is divided into uniform square local images, and images in which abnormal pixels are relatively dense are labeled with 1 and the others with 0. The density is determined by a convolutional neural network (CNN). In the CNN model, an encoder consisting of one or more convolutional layers and pooling layers is used to extract each local feature of the abnormal image data 300, and a fully connected layer is used to map these features, and the output layer compresses them into values ​​from 0 to 1 using a sigmoid function. This makes it possible to roughly determine the location where abnormal pixels are dense by the output value of the CNN model. Finally, a clustering algorithm such as K-means or DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is used to cluster local regions of the image having abnormal pixels, and the cutout position and shape of the block image data 301 are determined.

[0045] In addition to the above-mentioned predetermined algorithm, a method of creating (abnormal) dummy data 305 during additional learning will be described. First, the area where an abnormality is detected is stored as it is in the block data set. Next, the abnormal image data is divided into meshes to find locations with high density of abnormalities for each type of abnormality. At this time, the area may be further expanded using clustering. In this case, for example, if the vicinity is also abnormal, the area is linked and expanded. Next, an area of ​​a specific shape may be scanned on the image (starting from the top left of the image, the density of abnormalities contained in the area is confirmed while shifting the area) to find an area with high density of abnormalities, thereby generating block image data 301 of a significant size and shape.

[0046] Next, the dummy data generation method will be described. The dummy data generation method randomly extracts block image data 301 generated by the block image data set generation method described above, replaces some pixels with normal background image data, and generates new abnormal image data for learning. The normal background image data 304 is image data that is trimmed according to the size of the model input image and the range to be detected from a settling pond image taken under conditions such as different brightness, time, and angle of view. In order to generate various dummy data, affine transformation and brightness adjustment are performed on the block image data 301 as a data expansion method. Since floating objects and floating scum on the water surface do not have a fixed shape or size, even if affine transformation is performed, the image does not become unnatural. The affine transformation is a transformation that rotates and scales abnormal pixels and randomly combines translation transformation from the center of the model input image. Next, the block image data 301 that has been subjected to multiple affine transformations and brightness adjustments and the mask image data 302 are combined to create a block data set 303 with the original shape and brightness. However, the overlapping pixels when combined are replaced according to the order of overlapping. Finally, the original block dataset 303 is combined with normal background image data 304 randomly selected from multiple background image data, and added to the dataset together with the corresponding class label to finally generate (abnormal) dummy data 305.

[0047] According to the second embodiment described above, background pixels that are strongly influenced by positional relationships, scale, and angle of view are separated from abnormal pixels that are influenced by texture and brightness, and conversion processing is performed on each of them, thereby maintaining diversity and generating more natural dummy data. In addition, even if labeled abnormal data is insufficient, it becomes possible to use a supervised deep learning model.

[0048] C. Third embodiment Next, a third embodiment of the present invention will be described. Fig. 4 is a block diagram showing the configuration of a water quality determining system 400 according to the third embodiment. In Fig. 4, the water quality determining system 400 includes a plurality of water quality determining devices 500, 600, 800 disposed in sedimentation pond A, sedimentation pond B, and sedimentation pond C, respectively. The configuration of the water quality determining device 500 for sedimentation pond A and the configuration of the water quality determining device 600 for sedimentation pond B are substantially the same as the configuration of the water quality determining device 100 of the first embodiment.

[0049] That is, the water quality determination device 500 includes an image data storage unit 501, an image data processing unit 502, an image data expansion unit 503, a learning processing unit 504, a parameter storage unit 505, an abnormality estimation unit 506, and a water quality determination unit 507. The water quality determination device 600 includes an image data storage unit 601, an image data processing unit 602, an image data expansion unit 603, a learning processing unit 604, a parameter storage unit 605, an abnormality estimation unit 606, and a water quality determination unit 607. The water quality determination device 800 includes an image data storage unit 801, an image data processing unit 802, a dummy data generation unit 803, a learning processing unit 804, a parameter storage unit 805, an abnormality estimation unit 806, and a water quality determination unit 807.

[0050] 4, the functions of the image data extension units 503 and 603 are the same as those of the second embodiment described above, but differ in that the abnormal block image data sets output from the image data extension unit 503 of the settling pond A and the image data extension unit 603 of the settling pond B are stored in the abnormal pixel storage unit 700 and managed in a unified manner. That is, the abnormal pixel storage unit 700 stores abnormal pixel data that has been generated in the settling ponds A and B and has been labeled in advance.

[0051] The configuration of the water quality determination device 800 of the sedimentation pond C is almost the same as that of the first embodiment, but is different in that it includes a dummy data generation unit 803 instead of the image data extension unit. The dummy data generation unit 803 reads out abnormal pixel data labeled in advance, such as for sedimentation ponds A and B, from the abnormal pixel storage unit 700, and combines it with normal image data for the sedimentation pond C to generate dummy data for learning. The learning processing unit 804 learns using the dummy data supplied from the dummy data generation unit 803 as teacher data, generates learned parameters, and outputs them to the parameter storage unit 805. Therefore, the parameter storage unit 805 stores learned parameters learned with dummy data generated based on the abnormal pixel data in the water quality determination devices 500 and 600 for the sedimentation ponds A and B.

[0052] The abnormality estimation unit 806 inputs the image data of the settling pond C processed by the image data processing unit 802 into the deep learning semantic segmentation model loaded from the parameter storage unit 805, and finally outputs a segmentation map. The water quality determination unit 807 calculates the area and quantity of normal regions and the area and quantity of abnormal regions based on the segmentation map output from the abnormality estimation unit 806, determines whether the region is normal or abnormal based on a predetermined threshold, and outputs the determination result as water quality determination result data of the settling pond C.

[0053] When learning an anomaly detection model, there are often few abnormal image data or the cost of attaching class labels is high. However, between different settling basins using the same sewage treatment method, the characteristics of abnormal pixels such as scum and flocs are highly consistent. Therefore, in the third embodiment, for settling basin C (in which a new water quality determination device 800 is installed), abnormal pixel data that has been labeled in advance from settling basins A and B is stored in the abnormal pixel storage unit 700, and the abnormal pixel data is reused from the abnormal pixel storage unit 700 and combined with normal image data from settling basin C to generate dummy data for learning. In addition, abnormal pixels from settling basins A and B are also reused from each other to generate new dummy data, which further improves the diversity and accuracy of detection and the generalizability of the model.

[0054] It is difficult to collect all image data with abnormal water quality for a single settling tank and process it as learning data. There are cases where there is no abnormal data at all for a certain settling tank image data. In addition, expert knowledge and experience are required to understand the types and characteristics of abnormal image data and assign appropriate class labels accordingly. The more settling tanks and images there are, the more time it takes to complete the task.

[0055] According to the above-described third embodiment, by utilizing abnormal pixel data created by water quality assessment devices of other sedimentation tanks with similar characteristics, the cost of data preprocessing can be reduced while at the same time increasing the diversity of the learning data, thereby improving the generalizability of the model.

[0056] In addition, it is possible to select and discard the configurations given in the above embodiment, or to change them to other configurations as appropriate, without departing from the spirit of the present invention. [Explanation of symbols]

[0057] 100, 500, 600, 800 water quality determination device 101, 501, 601, 801 Image data storage unit 102, 502, 602, 802 Image data processing section 103, 503, 603 Image data extension 803 Dummy Data Generator 104, 504, 604, 804 Learning processing unit 105, 505, 605, 805 Parameter memory section 106, 506, 606, 806 Abnormality estimation part 107, 507, 607, 807 Water quality determination department

Claims

1. an image data storage unit that stores image data acquired from a water quality assessment target; an image data processing unit that processes the image data stored in the image data storage unit and outputs processed image data; an image data extension unit that performs data extension on the processed image data to generate abnormal dummy data; A learning processing unit that performs deep learning using the abnormal dummy data as teacher data and outputs learned parameters; a parameter storage unit that stores the learned parameters; an anomaly estimation unit that estimates to which class label each pixel of the processed image data belongs, using the learned parameters stored in the parameter storage unit; a water quality determination unit that determines water quality based on the class label of each pixel estimated by the abnormality estimation unit; A water quality determination device comprising:

2. The image data extension unit includes: The water quality determination device described in claim 1, characterized in that an area selected from the abnormal image data in response to user operation is selected as block image data, the selected block image data is binarized to generate binarized mask image data in which only abnormal pixels are present, accurately separating abnormal pixels from background pixels remaining in the vicinity, the block image data and the mask image data are combined to create a block data set having an arbitrary shape and brightness, and the block data set is combined with the normal processed image data to generate the abnormal dummy data.

3. The image data extension unit includes: The water quality determination device of claim 2, characterized in that the abnormal image data having abnormal pixels is automatically divided into local images of arbitrary shapes, labeling is performed according to the density of the abnormal pixels, and the cut-out position and shape of the block image data are determined by clustering the local areas of the image having abnormal pixels.

4. The water quality determination device of claim 2 or 3, further comprising an abnormal pixel data storage unit that stores the abnormal pixel data previously labeled by the image data expansion unit so that it can be used in a water quality determination device that determines the water quality of a water quality determination subject other than the water quality determination subject.

5. The water quality determination device of claim 4, further comprising a dummy data generation unit which, instead of the image data expansion unit, uses the abnormal pixel data from the abnormal pixel data memory unit to generate abnormal dummy data for learning by combining it with normal image data for the different water quality determination subjects.

6. A water quality determination method for a water quality determination device including an image data storage unit, an image data processing unit, an image data expansion unit, a learning processing unit, a parameter storage unit, an abnormality estimation unit, and a water quality determination unit, comprising: The image data storage unit stores image data acquired from a water quality assessment subject; the image data processing unit performs processing on the image data stored in the image data storage unit and outputs processed image data; the image data extension unit performs data extension on the processed image data to generate abnormal dummy data; The learning processing unit performs deep learning using the abnormal dummy data as teacher data and outputs learned parameters; The parameter storage unit stores the learned parameters. the anomaly estimation unit estimates to which class label each pixel of the processed image data belongs, using the learned parameters stored in the parameter storage unit; A water quality determination method, comprising: the water quality determination unit determining water quality based on the class label of each pixel estimated by the abnormality estimation unit.

7. The image data extension unit generates abnormal dummy data, selecting an area selected from the abnormal image data in response to a user operation as block image data; binarizing the selected block image data to generate binarized mask image data in which only abnormal pixels are present, accurately separating the abnormal pixels from the background pixels remaining nearby; combining the block image data and the mask image data to create a block data set having an arbitrary shape and brightness; The water quality determination method according to claim 6, further comprising combining the block data set with normal processed image data to generate the abnormal dummy data.

8. The image data extension unit generates abnormal dummy data, Automatically segmenting the abnormal image data having abnormal pixels into local images of any shape and labeling them according to the density of the abnormal pixels; The water quality determination method according to claim 7, further comprising determining a cutout position and a shape of the block image data by clustering a local area of ​​the image having abnormal pixels.

9. The water quality determination device further includes an abnormal pixel data storage unit, The water quality determination method described in claim 7 or 8, characterized in that the abnormal pixel data storage unit stores the abnormal pixel data previously labeled by the image data expansion unit so that it can be used by a water quality determination device that determines the water quality of a water quality determination subject other than the water quality determination subject.

10. The water quality determination device includes a dummy data generation unit instead of the image data extension unit, The water quality determination method according to claim 9, characterized in that the dummy data generation unit uses the abnormal pixel data from the abnormal pixel data memory unit to generate abnormal dummy data for learning by combining it with normal image data for the different water quality determination subjects.

Images