Rust detection device, rust detection method, and program

Abstract

To provide a rust detection device, a rust detection method, and a program which not only determines the presence or absence of rust, but also detects knobby rust having a high degree of rusting without time and effort.SOLUTION: The rust detection device comprises: a rust detection unit which, on the basis of an image obtained by capturing an object, detects the presence or absence of rust on the object in pixel units of the image; a rusting degree evaluation unit which evaluates a degree of rusting on the object in pixel units of the image on the basis of the image; a rusting degree quantization unit which calculates a ratio of the number of pixels of the object having degrees of rusting equal to or higher than a prescribed value to the whole of the number of pixels of rust on the object, on the basis of a result from detecting the presence or absence of rust on the object by the rust detection unit and a result from evaluating the degree of rusting on the object by the rusting degree evaluation unit; and a determination unit which determines a rusting state on the object as an abnormal state when the ratio exceeds a preliminarily set threshold.SELECTED DRAWING: Figure 3A

Description

[Technical Field]

[0001] The present invention relates to a rust detection device, a rust detection method, and a program. [Background technology]

[0002] A rule-based method for detecting rust is already known, which defines rules such as judging an image to be "rust" if its pixel brightness or color value is above a certain level.

[0003] For example, Patent Document 1 discloses, as an example of such a rule-based method, a rust detection technology that determines whether or not an object is rust based on texture information on the surface of the object. Summary of the Invention [Problem to be solved by the invention]

[0004] However, while conventional technology can determine whether something is rust or not, when trying to detect nodular rust, which is a more advanced form of rust, it is necessary to define rules based on complex characteristics such as the shape, color, and structure of the nodular rust, which is time-consuming and labor-intensive.Furthermore, if each user has different standards for nodular rust, it is necessary to redefine the rules for each user.

[0005] The present invention has been made in view of the above, and aims to provide a rust detection device, a rust detection method, and a program that can not only determine whether something is rust, but also detect advanced nodular rust without requiring much time and effort. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the present invention comprises a rust detection unit that detects the presence or absence of rust on an object on a pixel-by-pixel basis based on an image of the object; a rust degree evaluation unit that evaluates the degree of rust on the object on a pixel-by-pixel basis based on the image; a rust degree quantification unit that calculates the proportion of pixels on the object that have a degree of rust equal to or greater than a predetermined value out of the total number of rust pixels on the object based on the detection result of the presence or absence of rust on the object by the rust detection unit and the evaluation result of the degree of rust on the object by the rust degree evaluation unit; and a judgment unit that judges the rust state of the object to be an abnormal state if the proportion exceeds a predetermined threshold. [Effects of the Invention]

[0007] According to the present invention, it is possible to not only determine whether or not something is rust, but also to detect nodular rust, which is an advanced stage of rust, without taking much time and effort. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a rust detection system according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a PC (or a server) according to the first embodiment. [Figure 3A] FIG. 3A is a diagram illustrating an example of a functional configuration of the rust detection device according to the first embodiment. [Figure 3B] FIG. 3B is a diagram for explaining an example of rust detection and rust level evaluation in the rust detection device according to the first embodiment. [Figure 4A] FIG. 4A is a diagram for explaining an example of a rust detection process performed by the rust detection unit in the rust detection device according to the first embodiment. [Figure 4B] FIG. 4B is a diagram for explaining an example of rust detection processing by the rust detection unit in the rust detection device according to the first embodiment. [Figure 5]FIG. 5 illustrates an example of a functional configuration of the server according to the second embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a functional configuration of a server according to the first modification. [Figure 7] FIG. 7 is a diagram illustrating an example of a hardware configuration of an HMD according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of a rust detection device, a rust detection method, and a program will be described in detail with reference to the accompanying drawings.

[0010] (First embodiment) Fig. 1 is a diagram showing an example of the configuration of a rust detection system according to a first embodiment. As shown in Fig. 1, the rust detection system 1 according to the present embodiment includes an electronic whiteboard 2, an inkjet printer 3, a smartphone 4, a PC (Personal Computer) 5 (or a server 5), an omnidirectional imaging device 6, a video conference terminal 7, a projector 8, and an MFP (Multi-Function Peripheral) 9, and each device is connected to each other via a communication network 100 so as to be able to communicate with each other.

[0011] 2 is a diagram showing an example of the hardware configuration of a PC (or a server) according to the first embodiment. Here, the hardware configuration of the server 5 will be described.

[0012] As shown in FIG. 2, the server 5 is constructed by a computer, and as shown in FIG. 2, includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, a HD 504, a HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a bus line (data bus) 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0013] Of these, the CPU 501 controls the overall operation of the server 5. The ROM 502 stores programs, such as an IPL, used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data, such as programs. The HDD controller 505 controls the reading and writing of various data from and to the HD 504 under the control of the CPU 501. The display 506 displays various information, such as a cursor, menus, windows, characters, or images. The external device connection I / F 508 is an interface for connecting various external devices. In this case, the external devices are, for example, USB (Universal Serial Bus) memories or printers. The network I / F 509 is an interface for data communication using the communication network 100. The bus line 510 is an address bus, a data bus, or the like, for electrically connecting the components, such as the CPU 501, shown in FIG. 1.

[0014] The keyboard 511 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 514 controls reading and writing of various data from a DVD-RW 513, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R, etc. The media I / F 516 controls reading and writing (storing) of data from a recording medium 515, such as a flash memory.

[0015] Fig. 3A is a diagram illustrating an example of the functional configuration of a rust detection device according to the first embodiment. Fig. 3B is a diagram illustrating an example of rust detection and rust level evaluation in the rust detection device according to the first embodiment. In the server 5 (an example of a rust detection device) according to this embodiment, a CPU 501 executes a program stored in a ROM 502 using a RAM 503 or the like as a working area, thereby realizing a rust detection unit 201, a rust level evaluation unit 202, a rust level quantification unit 203, a determination unit 204, and a threshold setting unit 205.

[0016] The imaging unit 200 is an example of an imaging unit that captures an image of an object containing rust. For example, the imaging unit 200 may be a camera included in an external device such as the smartphone 4.

[0017] The rust detection unit 201 is an example of a rust detection unit that detects the presence or absence of rust on an object, pixel by pixel, based on an image of the object captured by the imaging unit 200. Specifically, the rust detection unit 201 detects (extracts) the object from the image by object detection. The rust detection unit 201 may then detect the presence or absence of rust on the detected object, pixel by pixel, based on the color of the image. In this case, the rust detection unit 201 also detects whether the rust is dark or light, depending on the color of the pixel determined to be rust. In this embodiment, the rust detection unit 201 detects dark pixel colors as dark rust and light pixel colors as light rust. The dark and light colors are determined in advance with reference to a sample rust data set. That is, the rust detection unit 201 may detect the presence or absence of dark rust on the object based on preset information that light rust is present around dark rust. Furthermore, the rust detection unit 201 may detect a main object included in an image as a target object, and detect whether or not the object has rust.

[0018] In this embodiment, the rust detection unit 201 first calculates the color difference C between the color (L1, a1, b1) of each pixel in the image output by the imaging unit 200 and a preset rust color (L2, a2, b2) using the following formula (1). Here, the color is expressed as a CIE-LAB value.

number

number

[0019] The rust degree evaluation unit 202 is an example of a rust degree evaluation unit that evaluates the degree of rust on an object for each pixel of the image based on the image captured by the imaging unit 200. Specifically, the rust degree evaluation unit 202 may evaluate the degree of rust on an object (such as the state of rust being rough) for each pixel of the image based on the texture (graininess) of the image.

[0020] In this embodiment, the rust level evaluation unit 202 divides the image captured by the imaging unit 200 into multiple blocks and calculates the granularity of each block. The calculation of granularity D is performed based on the following equation (3) (see Non-Patent Document 1) after detecting the peaks of each block. The granularity calculation method described in Non-Patent Document 1 calculates the amount of graininess (graininess) perceived by humans. It is assumed that graininess is based on a weighted linear sum of the parameters of the average grain amplitude, average grain size, and maximum luminance value, and each weight is calculated using multiple regression analysis with human subjective evaluation as the correct answer. In this method, stimulus images used in the subjective evaluation are generated by varying the average amplitude, average grain size, and maximum luminance value in multiple stages. The generated images are evaluated by multiple people in a paired comparison (evaluating which image has a stronger graininess), and the results are analyzed using the Thurston method to obtain a graininess value for each image.

number

[0021] Furthermore, in this embodiment, the rust degree evaluation unit 202 sets the granularity D' to a value obtained by replacing values ​​below a threshold value preset by the threshold setting unit 205 with 0. The rust degree evaluation unit 202 then outputs a heat map image (the graininess heat map shown in FIG. 3B) that indicates the product, for each pixel, of the output (rust detection result) of the rust detection unit 201 and the granularity D'.

[0022] The rust degree quantification unit 203 is an example of a rust degree quantification unit that calculates the proportion of the number of pixels in the object where the degree of rust is equal to or greater than a predetermined value to the total number of rust pixels in the object included in the image captured by the imaging unit 200, based on the evaluation result of the rust degree of the object by the rust degree evaluation unit 202 and the detection result of the presence or absence of rust in the object by the rust detection unit 201. That is, the rust degree quantification unit 203 calculates a quantitative evaluation value for determining whether the state of rust is abnormal. Specifically, in this embodiment, the rust degree quantification unit 203 finds the number of rust pixels in the object and the number of pixels in the object where the degree of rust is high (for example, nodular rust), and calculates the proportion of the number of pixels in the object where the degree of rust is high (where the degree of rust is equal to or greater than a predetermined value) to the total number of rust pixels in the object as the quantitative evaluation value of the rust degree. The quantitative evaluation value is expressed as a ratio because, if only the number of pixels were considered, the number of pixels would change depending on the distance at which the object is imaged (for example, the closer the distance, the more pixels the object has; the farther the distance, the fewer pixels the object has), and the quantitative evaluation value would change depending on the distance. By using the above ratio as the quantitative evaluation value, the influence of distance can be reduced. Furthermore, the number of rust pixels on the object is defined as the total number of pixels of rust on the object, which is the number of dark rust pixels and light rust pixels detected by the rust detection unit 201 and pixels with a medium or higher graininess as determined by the rust degree evaluation unit 202. Furthermore, the number of pixels of rust with a high degree of rust (e.g., nodular rust) on the object is defined as the number of dark rust pixels detected by the rust detection unit 201 and pixels with a high graininess as determined by the rust degree evaluation unit 202. In this embodiment, the rust degree quantification unit 203 outputs two values ​​(heat map of quantitative evaluation results shown in FIG. 3B): the number O1 of pixels equal to or greater than k2 (e.g., 100) in the output values ​​(pixels in the graininess heat map) of the rust degree evaluation unit 202, and the number O2 of pixels O1 within pixels equal to or greater than k3 (e.g., 100) in the output values ​​(color heat map) of the rust detection unit 201. In other words, the number O1 of pixels is the number of all the rust pixels on the object, and is the number of dark and light rust pixels and pixels with a medium or greater graininess.The pixel number O2 is the number of pixels of rust with a high degree of rust (for example, nodular rust) on the object, and is the number of pixels limited to dark rust and with a high degree of graininess. The rust degree quantification unit 203 calculates these two values ​​(pixel number O. 1、 From the number of pixels O2), the ratio of the number of nodular rust pixels to the total number of rust pixels on the object is calculated, and this is used as a quantitative evaluation value to determine whether the rust condition on the object is abnormal.

[0023] The threshold setting unit 205 is an example of a threshold setting unit that displays a pixel-by-pixel rust level evaluation result based on the quantitative evaluation result (quantitative evaluation value) of the rust level of the object by the rust level quantification unit 203, and a UI that allows the user or the like to intuitively set the threshold. In this embodiment, the threshold setting unit 205 displays a screen (threshold setting screen) on a display unit such as the display 506, the screen including the image captured by the imaging unit 200, the heat map output by the rust level evaluation unit 202, and a slide bar for setting the threshold. The user can adjust the threshold used by the rust level evaluation unit 202 by adjusting the slide bar. The output (heat map) of the rust level evaluation unit 202 resulting from the threshold adjustment using the slide bar is reflected in the threshold setting unit 205 in real time.

[0024] The determination unit 204 is an example of a determination unit that determines the rust state of the object as an abnormal state when the ratio calculated by the rust degree quantification unit 203 exceeds a preset threshold (for example, a threshold set by the threshold setting unit 205). This not only determines whether or not there is rust, but also makes it possible to detect nodular rust without much effort and time. In this embodiment, the determination unit 204 determines that there is an abnormality when both outputs of the two output values ​​set by the rust degree quantification unit 203 exceed preset thresholds.

[0025] 4A and 4B are diagrams illustrating an example of a rust detection process performed by the rust detection unit in the rust detection device according to the first embodiment. Specifically, Fig. 4A and 4B are diagrams illustrating an example of a detection process performed by the rust detection unit 201 that detects the presence or absence of thick rust based on preset information that thin rust is present around thick rust.

[0026] First, the rust detection unit 201 calculates the difference R from a reference rust color (L2, a2, b2) based on the image (original image) captured by the imaging unit 200. Here, the rust detection unit 201 sets separate reference rust colors (L2, a2, b2) for light and dark rust colors, thereby generating a light rust heat map (color Prior) that visualizes light rust areas, and a dark rust heat map (dark rust detection result) that visualizes dark rust areas.

[0027] Next, the rust detection unit 201 blurs the pixel values ​​by shrinking and enlarging the light rust heat map. Finally, the rust detection unit 201 integrates (combines) the light rust heat map with blurred pixel values ​​and the dark rust heat map on a pixel-by-pixel basis to detect dark rust that is surrounded by light rust.

[0028] In this way, the rust detection device according to the first embodiment can not only determine whether or not something is rust, but can also detect advanced nodular rust without much effort or time.

[0029] (Second embodiment) In the present embodiment, a main object included in an image is detected as a target object, and the presence or absence of rust on the object is detected. In the following description, a description of the same configuration as the above-described embodiment will be omitted.

[0030] Fig. 5 is a diagram illustrating an example of the functional configuration of the server according to the second embodiment. Specifically, Fig. 5 is a diagram illustrating an example of a process for detecting a main object from an image captured by the imaging unit 200 and detecting whether or not the detected object contains rust.

[0031] The server 5 according to this embodiment includes an object detection unit 521 in addition to the rust detection unit 201, the rust level evaluation unit 202, the rust level quantification unit 203, the determination unit 204, and the threshold setting unit 205 (see FIG. 3A). The object detection unit 521 detects a main object included in an image captured by the imaging unit 200 as a target object. The rust detection unit 201 then detects whether or not rust is present on the object detected by the object detection unit 521. This makes it possible to prevent rust from being detected on an object other than the target object for which rust is to be detected, thereby improving the accuracy of rust detection.

[0032] For example, the object detection unit 521 uses the image captured by the imaging unit 200 as input to detect the main object contained in the image using the technology described in Non-Patent Document 3. Alternatively, the object detection unit 521 uses the technology described in Non-Patent Document 4 to generate a mask image (object detection mask shown in Figure 3B) based on the image, with objects containing rust in the foreground and everything else in the background. The technologies described in Non-Patent Documents 3 and 4 are methods for realizing object region segmentation. Non-Patent Document 3 enables high-speed calculations by using an EfficientNet based on MovileNet, and is trained using a dataset in which the main object is segmented. Non-Patent Document 4 is a method for extracting regions that match the content described in a prompt. For example, if "objects with rust" is written, regions of objects containing rust can be segmented. The model uses a Vision Transformer, which has a self-attention mechanism and divides the image into patches for processing, allowing for focused processing on important regions. A prompt encoder enables segmentation not only of text but also of specified pixel positions and rectangular regions. In this embodiment, when high-speed processing is required, the technique described in Non-Patent Document 3 is used, and when there are many objects in the image and extraction from them (i.e., more advanced region segmentation) is also desired, the technique described in Non-Patent Document 4 may be used.

[0033] In this way, the rust detection device according to the second embodiment can prevent rust from being detected on an object other than the object for which rust is to be detected, thereby improving the accuracy of rust detection.

[0034] (Variation 1) This modified example is an example in which the degree of rust on an object is evaluated based on statistical information. In the following description, the same configuration as in the above-described embodiment will not be described.

[0035] FIG. 6 is a diagram showing an example of the functional configuration of a server according to Modification 1. In this modification, a rust level evaluation unit 601 evaluates the rust level of an object on a pixel-by-pixel basis based on statistical information of the image. Here, the statistical information may be, for example, non-uniformity (dissimilarity, entropy, homogeneity, etc.) used in texture analysis. As a result, while the above-described embodiment evaluates the rust level of an object based on perception, evaluation based on statistical information of the image eliminates the need for subjective evaluation by a human.

[0036] In measuring non-uniformity, the rust degree evaluation unit 601 first normalizes the image to a gray-level co-occurrence matrix (GLCM (see Non-Patent Document 5)). Then, the rust degree evaluation unit 601 calculates the numerical value of each index (dissimilarity, entropy, etc.) for each pixel using the formula described in Non-Patent Document 5. Each index is the absolute value of the difference between the central pixel value and the surrounding pixel values, and the probability that the central pixel value has a pixel value in the surroundings. The library described in Non-Patent Document 5 can be used for each calculation. The rust degree evaluation unit 601 defines the measurement result of non-uniformity as E. The rust degree evaluation unit 601 may use non-uniformity E instead of granularity D' as an evaluation value for the roughness of the rust, or may use the sum of these values ​​for each pixel. The GLCM of features described in Non-Patent Document 5 is a method for normalizing patch images, and is capable of calculating feature values ​​that are invariant to variations in brightness values. GLCM is a matrix that lists the total number of specific pixel value pairs in a grayscale-converted patch image, with each element of the matrix representing a pixel value pair. Dissimilarity can be calculated from this matrix. For example, dissimilarity is calculated by calculating the absolute value of the difference between the center and periphery of the matrix. In this embodiment, GLCM is used to measure the texture of rust without relying on the maximum brightness value of the image.

[0037] In this way, the rust detection device according to the first modification makes evaluation based on statistical information of the image, eliminating the need for subjective evaluation by a human.

[0038] (Third embodiment) In this embodiment, a threshold setting screen is displayed on a head mounted display (HMD). In the following description, the same configuration as in the above-described embodiment will not be described.

[0039] 7 is a diagram showing an example of the hardware configuration of an HMD according to the third embodiment. In this embodiment, the threshold setting unit 205 may display a threshold setting screen on the HMD 700.

[0040] As shown in Figure 7, the HMD 700 is a computer that includes a CPU 701, a ROM 702, a RAM 753, an external device connection I / F 705, a display 707, an operation unit 708, a media I / F 709, a bus line 710, a speaker 712, an electronic compass 718, a gyro sensor 719, and an acceleration sensor 720.

[0041] Of these, the CPU 701 controls the overall operation of the HMD 700. The ROM 702 stores programs such as IPL used to drive the CPU 701. The RAM 753 is used as a work area for the CPU 701.

[0042] The external device connection I / F 705 is an interface for connecting various external devices, such as a communication management server and earphones with a microphone.

[0043] The display 707 is a type of display unit such as a liquid crystal display or organic EL (Electro Luminescence) display that displays various images.

[0044] The operation unit 708 is an input means for selecting and executing various instructions such as various operation buttons, power switches, physical buttons, and gaze operation circuits that detect and operate the user's gaze, selecting the processing target, moving the cursor, etc.

[0045] The media I / F 709 controls reading and writing (storing) of data from and to a recording medium 709m such as a flash memory, etc. The recording medium 709m includes DVDs, Blu-ray (registered trademark) discs, etc.

[0046] The speaker 712 is a circuit that converts an electrical signal into physical vibrations to produce sounds such as music and voice.

[0047] The electronic compass 718 calculates the direction of the HMD from the Earth's magnetism and outputs the direction information.

[0048] The gyro sensor 719 is a sensor that detects changes in angle (roll angle, pitch angle, yaw angle) that accompany the movement of the HMD.

[0049] The acceleration sensor 720 is a sensor that detects acceleration in three axial directions.

[0050] The bus line 710 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 701.

[0051] In this way, the rust detecting device according to the third embodiment can provide the same effects as those of the first embodiment.

[0052] The program executed by the server 5 of this embodiment is provided by being pre-installed in the ROM 502 or the like. The program executed by the server 5 of this embodiment may be provided by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disk (DVD).

[0053] Furthermore, the program executed by the server 5 of this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed by the server 5 of this embodiment may be provided or distributed via a network such as the Internet.

[0054] The program executed by the server 5 in this embodiment has a modular structure including the above-mentioned units (rust detection unit 201, rust level evaluation unit 202, 601, rust level quantification unit 203, judgment unit 204, threshold setting unit 205, object detection unit 521), and in actual hardware, an example of a processor such as CPU 501 reads and executes the program from the above-mentioned ROM 502, thereby loading the above-mentioned units onto the main memory, and the rust detection unit 201, rust level evaluation unit 202, 601, rust level quantification unit 203, judgment unit 204, threshold setting unit 205, and object detection unit 521 are generated on the main memory.

[0055] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0056] Note that the information processing device, such as the server 5, which is an example of a rust detection device, is not limited to an image forming device as long as it has a communication function. The information processing device may be, for example, a PJ (Projector), an IWB (Interactive White Board: a white board with an electronic blackboard function that allows mutual communication), an output device such as digital signage, a HUD (Head Up Display) device, industrial machinery, an imaging device, a sound collection device, a medical device, a network home appliance, an automobile (Connected Car), a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game console, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, a desktop PC, or the like.

[0057] For example, aspects of the present invention are as follows. <1> a rust detection unit that detects the presence or absence of rust on an object on a pixel-by-pixel basis based on an image of the object; a rust degree evaluation unit that evaluates the degree of rust in the object on a pixel-by-pixel basis based on the image; a rust degree quantification unit that calculates the ratio of the number of pixels in the object having a degree of rust equal to or greater than a predetermined value to the total number of rust pixels in the object based on the detection result of the presence or absence of rust in the object by the rust detection unit and the evaluation result of the degree of rust in the object by the rust degree evaluation unit; a determination unit that determines the rust state of the object as an abnormal state when the ratio exceeds a predetermined threshold value; A rust detection device comprising: <2> The rust detection unit detects the presence or absence of rust on the object on a pixel-by-pixel basis based on the color of the image. <1> The rust detection device according to claim 1. <3> the rust detection unit detects the presence or absence of thick rust on the object based on preset information that thin rust exists around thick rust. <2> The rust detection device according to claim 1. <4> the rust degree evaluation unit evaluates the degree of rust on the object based on graininess, which is a human perception; <1> from <3> The rust detection device according to any one of the above. <5> a threshold setting unit that displays a pixel-by-pixel rust degree evaluation result based on the calculation result of the ratio by the rust degree quantification unit and an UI that allows intuitive setting of the threshold value, <1> from <4> The rust detection device according to any one of the above. <6> further comprising an object detection unit that detects a main object included in the image as the target object; The rust detection unit detects whether or not rust is present on the object. <1> from <5> The rust detection device according to any one of the above. <7> a rust detection unit that detects the presence or absence of rust on an object on a pixel-by-pixel basis based on an image of the object; a rust degree evaluation unit that evaluates the degree of rust in the object based on statistical information for each pixel of the image, based on the image; a rust degree quantification unit that calculates the ratio of the number of pixels in the object having a degree of rust equal to or greater than a predetermined value to the total number of rust pixels in the object based on the detection result of the presence or absence of rust in the object by the rust detection unit and the evaluation result of the degree of rust in the object by the rust degree evaluation unit; a determination unit that determines the rust state of the object as an abnormal state when the ratio exceeds a predetermined threshold value; A rust detection device comprising: <8> A rust detection method performed by a rust detection device, comprising: detecting the presence or absence of rust on the object on a pixel-by-pixel basis based on an image of the object; evaluating the degree of rust on the object on a pixel-by-pixel basis based on the image; Calculating the ratio of the number of pixels in the object with a degree of rust equal to or greater than a predetermined value to the total number of pixels in the object with rust, based on the result of detecting the presence or absence of rust in the object and the result of evaluating the degree of rust in the object; determining that the rust state of the object is abnormal when the ratio exceeds a predetermined threshold value; A rust detection method comprising: <9> Computer, a rust detection unit that detects the presence or absence of rust on an object on a pixel-by-pixel basis based on an image of the object; a rust degree evaluation unit that evaluates the degree of rust in the object on a pixel-by-pixel basis based on the image; a rust degree quantification unit that calculates the ratio of the number of pixels in the object having a degree of rust equal to or greater than a predetermined value to the total number of rust pixels in the object based on the detection result of the presence or absence of rust in the object by the rust detection unit and the evaluation result of the degree of rust in the object by the rust degree evaluation unit; a determination unit that determines the rust state of the object as an abnormal state when the ratio exceeds a predetermined threshold value; A program to make it function as such. [Explanation of symbols]

[0058] 5 Server 200 Imaging unit 201 Rust detection unit 202,601 Rust Degree Evaluation Section 203 Rust Degree Quantification Department 204 Judgment section 205 Threshold setting unit 501 CPU 502 ROM 503 RAM 506 Display 521 Object detection unit [Prior art documents] [Patent documents]

[0059] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-013260 [Non-patent literature]

[0060] [Non-Patent Document 1] Tokumichi Tsumura and Atsuki Yoshii, "Device-independent texture management based on a graininess evaluation model," Journal of the Imaging Society of Japan, 2019. [Non-patent document 2] Virtanen, Pauli ; Gommers, Ralf ; Burovski, Eugene ; Oliphant, Travis E. ; Cournapeau, David ; Weckesser, Warren ; Alexbrc ; Peterson, Pearu ; Endolith ; Van Der Walt, Stefan ; Wilson, Josh ; Mayorov, Nikolai ; Laxalde, Dennis ; Brett, Matthew ; Millman, Jarrod ; Ark ; Eric-Jones ; Nelson, Andrew ; Kern, Robert ; Moore, Eric ; Leslie, Tim ; Perktold, Joseph ; Carey, CJ ; Feng, Yu ; Haberland, Matt ; Vanderplas, Jake ; Cowlicks ; Larson, Eric ; Polat, Ilhan ; Reddy, Tyler, "scipy / scipy: SciPy 1.1.0" “scipy.signal.find_peaks”<URL:https: / / ui.adsabs.harvard.edu / abs / 2018zndo...1241501V / abstract.>

Table 3

Fashion 4

Claims

1. a rust detection unit that detects the presence or absence of rust on an object on a pixel-by-pixel basis based on an image of the object; a rust degree evaluation unit that evaluates the degree of rust in the object on a pixel-by-pixel basis based on the image; a rust degree quantification unit that calculates the ratio of the number of pixels in the object having a degree of rust equal to or greater than a predetermined value to the total number of rust pixels in the object based on the detection result of the presence or absence of rust in the object by the rust detection unit and the evaluation result of the degree of rust in the object by the rust degree evaluation unit; a determination unit that determines the rust state of the object as an abnormal state when the ratio exceeds a predetermined threshold value; A rust detection device comprising:

2. The rust detection device according to claim 1 , wherein the rust detection unit detects the presence or absence of rust on the object on a pixel-by-pixel basis based on the color of the image.

3. The rust detection device according to claim 2 , wherein the rust detection unit detects the presence or absence of thick rust on the object based on preset information that thin rust exists around thick rust.

4. The rust detection device according to claim 1 , wherein the rust level evaluation unit evaluates the rust level of the object based on graininess, which is a human perception.

5. The rust detection device of claim 1, further comprising a threshold setting unit that displays a pixel-by-pixel rust degree evaluation result based on the calculation result of the ratio by the rust degree quantification unit and a UI that enables intuitive setting of the threshold value based on the operation of the threshold value.

6. further comprising an object detection unit that detects a main object included in the image as the target object; The rust detection device according to claim 1 , wherein the rust detection unit detects whether or not rust is present on the object.

7. a rust detection unit that detects the presence or absence of rust on an object on a pixel-by-pixel basis based on an image of the object; a rust degree evaluation unit that evaluates the degree of rust in the object based on statistical information for each pixel of the image, based on the image; a rust degree quantification unit that calculates the ratio of the number of pixels in the object having a degree of rust equal to or greater than a predetermined value to the total number of rust pixels in the object based on the detection result of the presence or absence of rust in the object by the rust detection unit and the evaluation result of the degree of rust in the object by the rust degree evaluation unit; a determination unit that determines the rust state of the object as an abnormal state when the ratio exceeds a predetermined threshold value; A rust detection device comprising:

8. A rust detection method performed by a rust detection device, comprising: detecting the presence or absence of rust on the object on a pixel-by-pixel basis based on an image of the object; evaluating the degree of rust on the object on a pixel-by-pixel basis based on the image; A step of calculating a detection result of the presence or absence of rust in the object and a ratio of the number of pixels in the object having a degree of rust of a predetermined value or more to the total number of pixels in the object having rust; determining that the rust state of the object is abnormal when the ratio exceeds a predetermined threshold value; A rust detection method comprising:

9. Computer, a rust detection unit that detects the presence or absence of rust on an object on a pixel-by-pixel basis based on an image of the object; a rust degree evaluation unit that evaluates the degree of rust in the object on a pixel-by-pixel basis based on the image; a rust degree quantification unit that calculates the ratio of the number of pixels in the object having a degree of rust equal to or greater than a predetermined value to the total number of rust pixels in the object based on the detection result of the presence or absence of rust in the object by the rust detection unit and the evaluation result of the degree of rust in the object by the rust degree evaluation unit; a determination unit that determines the rust state of the object as an abnormal state when the ratio exceeds a predetermined threshold value; A program to make it function as such.

Images