Smoke detection device, method, and program

Abstract

This invention provides a smoke detection device, a smoke detection method, and a smoke detection program that can detect smoke with greater accuracy. [Solution] The smoke detection device 1000 of the present invention acquires an image, determines the brightness difference between each pixel in the acquired image and adjacent pixels, obtains statistical features based on the brightness differences of each pixel in the image obtained, and determines the presence or absence of smoke based on the obtained statistical features.

Description

Technical Field

[0001] The present invention relates to a smoke detection device, a smoke detection method, and a smoke detection program for detecting smoke.

Background Art

[0002] When detecting an object from an image obtained by imaging the object with a camera, if there is smoke between the object and the camera, the object may be hidden by the smoke or the image may be blurred, resulting in a decrease in the detection accuracy of the object. Therefore, smoke detection is desired. The technology related to this smoke detection is disclosed in, for example, Patent Document 1. In this Patent Document 1, a histogram of the maximum luminance is created based on an image obtained by imaging the surface of a molten metal container, and in the middle stage of slag scraping, based on the presence or absence of peaks that can be formed in each of the luminance portions corresponding to slag, the luminance portions corresponding to dust generation, and the luminance portions corresponding to molten metal, a method for detecting the presence or absence of dust generation is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, if the smoke detection is incorrect, there is a risk of affecting a predetermined process subsequently performed according to the smoke detection result, and an improvement in the smoke detection accuracy is desired .

[0005]

Means for Solving the Problems

[0006] As a result of various studies, the inventors have found that the above objective can be achieved by the present invention as described below. That is, a smoke detection device according to one aspect of the present invention comprises: an image acquisition unit that acquires an image; a feature processing unit that, for each pixel in the image acquired by the image acquisition unit, determines the brightness difference between the pixel and adjacent pixels adjacent to that pixel, and calculates statistical features based on the brightness differences of each pixel in the image obtained; and a smoke determination unit that determines the presence or absence of smoke based on the statistical features obtained by the feature processing unit. Preferably, in the above-described smoke detection device, the smoke determination unit determines that there is smoke if the statistical features obtained by the feature processing unit satisfy a predetermined condition (first condition), and determines that there is no smoke if the statistical features obtained by the feature processing unit do not satisfy the first condition.

[0007] When smoke is captured in an image, it blurs the image, resulting in a relatively small difference in brightness between adjacent pixels. The smoke detection device described above calculates the brightness difference between adjacent pixels, obtains statistical features based on the brightness differences of each pixel in the image, and determines the presence or absence of smoke based on these statistical features, thereby enabling more accurate smoke detection.

[0008] In another embodiment, the smoke detection device described above further includes a classification processing unit that divides the image acquired by the image acquisition unit into a plurality of regions, the feature processing unit obtains statistical features in the image of each of the plurality of regions classified by the classification processing unit, and the smoke determination unit determines whether or not there is smoke in the image of each of the plurality of regions classified by the classification processing unit.

[0009] This type of smoke detection device divides an image into multiple regions and detects smoke in each region, so it can detect smoke even if it is only partially visible in the image.

[0010] In another embodiment, the smoke detection device described above further includes a sorting processing unit that divides the image acquired by the image acquisition unit into a plurality of regions, the feature processing unit obtains statistical features in the image of each of the plurality of regions sorted by the sorting processing unit, the smoke determination unit determines whether there is smoke in the image of each of the plurality of regions sorted by the sorting processing unit, and finally determines whether there is smoke in the image acquired by the image acquisition unit based on the number of regions determined to have smoke by the determination. Preferably, in the smoke detection device described above, the smoke determination unit finally determines that there is smoke if the number of regions determined to have smoke by the determination satisfies a predetermined condition (second condition), and finally determines that there is no smoke if the number of regions determined to have smoke by the determination does not satisfy the second condition.

[0011] Such smoke detection devices divide an image into multiple regions, initially detect smoke in each region, and then ultimately determine the presence or absence of smoke in the entire image based on the number of regions where smoke is detected. Therefore, even if smoke is captured in various parts of the image, the presence or absence of smoke can be evaluated for the entire image.

[0012] In another embodiment, in the smoke detection device described above, the adjacent pixel is one of the following: a pixel directly adjacent to the pixel, a pixel adjacent to the pixel with one pixel space between them, or a pixel adjacent to the pixel with two pixels space between them.

[0013] According to this, a smoke detection device can be provided in which the adjacent pixels are any of the following: pixels directly adjacent to the pixel in question, pixels adjacent to the pixel with one pixel between them, or pixels adjacent to the pixel with two pixels between them.

[0014] In another embodiment, in the smoke detection device described above, the statistical feature is the correlation value of the co-occurrence matrix in the image acquired by the image acquisition unit.

[0015] According to this, a smoke detection device can be provided that uses the correlation value of the co-occurrence matrix in an image as a statistical feature.

[0016] In another embodiment, in these smoke detection devices described above, the image is captured so as to include the surface of the molten metal.

[0017] Such smoke detection devices can be used in factories that manufacture metal products because they detect smoke in images that include the surface of molten metal.

[0018] A smoke detection method according to another aspect of the present invention comprises an image acquisition step of acquiring an image; a feature processing step of determining the brightness difference between each pixel in the image acquired in the image acquisition step and adjacent pixels adjacent to that pixel, and determining statistical features based on the brightness differences of each pixel in the image obtained; and a smoke determination step of determining the presence or absence of smoke based on the statistical features obtained in the feature processing step.

[0019] This smoke detection method, as described above, calculates the brightness difference between adjacent pixels, obtains statistical features based on the brightness differences of each pixel in the image, and determines the presence or absence of smoke based on these obtained statistical features, thereby enabling more accurate smoke detection.

[0020] Another aspect of the present invention is a smoke detection program that causes a computer to function as a smoke detection device as described above.

[0021] According to this, a smoke detection program can be provided, and this smoke detection program will have the same effects as the smoke detection devices described above. [Effects of the Invention]

[0022] The smoke detection device, smoke detection method, and smoke detection program according to the present invention can detect smoke with greater accuracy. [Brief explanation of the drawing]

[0023] [Figure 1] This is a block diagram showing the configuration of the smoke detection device in the embodiment. [Figure 2]This diagram illustrates the object to be detected by the smoke detection device. [Figure 3] This is the first figure to explain the method of calculating the concurrent occurrence matrix. [Figure 4] This is a second figure, following Figure 3, which illustrates the method for calculating the simultaneous occurrence matrix. [Figure 5] As an example, the following is a sample image. [Figure 6] Figure 5 shows an image of the simultaneous occurrence matrix calculated from the sample image. [Figure 7] This figure shows the correlation values ​​of the simultaneous occurrence matrix calculated from the sample images shown in Figure 5. [Figure 8] This figure shows sample images with and without smoke. [Figure 9] This figure shows a histogram of the correlation values ​​of the simultaneous occurrence matrix calculated from the sample images shown in Figure 8. [Figure 10] This is a flowchart showing the operation of the smoke detection device. [Figure 11] This diagram illustrates an example in the smoke-free scenario. [Figure 12] This is a diagram illustrating an example in the case of smoke. [Modes for carrying out the invention]

[0024] Hereinafter, one or more embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the disclosed embodiments. In each figure, components denoted by the same reference numerals are identified as identical components, and their descriptions are omitted where appropriate. In this specification, general reference numerals are used without subscripts, while individual components are indicated by subscripts.

[0025] The smoke detection device according to this embodiment is a device for detecting smoke and comprises an image acquisition unit, a feature processing unit, and a smoke determination unit. The image acquisition unit acquires an image. The feature processing unit calculates the brightness difference between each pixel in the image acquired by the image acquisition unit and adjacent pixels adjacent to that pixel, and calculates statistical features based on the brightness differences of each pixel in the image calculated. The smoke determination unit determines the presence or absence of smoke based on the statistical features calculated by the feature processing unit. The smoke detection device, as well as the smoke detection method and smoke detection program implemented therein, will be described in more detail below.

[0026] Figure 1 is a block diagram showing the configuration of a smoke detection device in an embodiment. Figure 2 is a diagram illustrating the detection target for which smoke is detected by the smoke detection device. Figures 3 and 4 are diagrams illustrating the calculation method of the simultaneous occurrence matrix. Figure 3A shows the initial state before starting the calculation of the simultaneous occurrence matrix, Figure 3B shows the calculation result when the pixel value of the pixel of interest is 0, Figure 4A shows the calculation result when the pixel value of the pixel of interest is 1, and Figure 4B shows the calculation result when the pixel value of the pixel of interest is 2. Figure 5 is a diagram showing a sample image as an example. Figure 6 is a diagram illustrating the simultaneous occurrence matrix calculated from the sample image shown in Figure 5. Figure 7 is a diagram showing the correlation value of the simultaneous occurrence matrix calculated from the sample image shown in Figure 5. The horizontal axis of Figure 7 is the degree of noise superimposed on the image, and its vertical axis is the correlation value (GLCM-correlation) of the simultaneous occurrence matrix. Figure 8 is a diagram showing a sample image without smoke and a sample image with smoke. Figure 8A shows the ninth image, No. 9, from the first to tenth sample images No. 1 to No. 10 shown in Figure 5, and Figure 8B shows image No. 9-with-smoke, which is image No. 9 from Figure 8A with noise superimposed. Figure 9 is a histogram of the correlation values ​​of the co-occurrence matrix calculated from the sample images shown in Figure 8. Figure 9A shows the histogram of the correlation values ​​of the co-occurrence matrix for image No. 9 from Figure 8A, and Figure 9B shows the histogram of the correlation values ​​of the co-occurrence matrix for image No. 9-with-smoke from Figure 8B. The horizontal axis of Figures 9A and 9B represents the correlation value by class, and the vertical axis of each of them represents the frequency [%].

[0027] The smoke detection device 1000 in this embodiment is a device for detecting smoke in a target space, and for example, as shown in Figure 1, comprises an image acquisition unit 1, a control processing unit 2, an input unit 3, an output unit 4, an interface (IF unit) 5, and a storage unit 6.

[0028] The image acquisition unit 1 is connected to the control processing unit 2 and is a device that acquires images according to the control of the control processing unit 2. The image acquisition unit 1 may be, for example, an imaging unit 1 that generates an image of a detection target, such as smoke. This imaging unit 1 is connected to the control processing unit 2 by wire or wireless so that it can transmit image data. The imaging unit 1 only needs to be able to obtain an image of brightness, so it can be configured with, for example, a monochrome or color digital camera. Such an imaging unit 1 is placed in an appropriate location (location, position) depending on the detection target. The detection target may be any location where smoke generation is a concern, but here, as an example, in order to detect smoke generated in the surrounding environment of molten metal produced in a furnace that handles molten metal (e.g., blast furnace, electric furnace, converter, desiliconization furnace, dephosphorization furnace, etc.), the imaging unit 1 is placed in a location (location, position) where it can image including the surface of the molten metal. More specifically, as shown in Figure 2, for example, the imaging unit 1 is positioned above the molten metal container MB so that it can capture images of the molten metal MT contained within the molten metal container MB, allowing for an overhead view of the molten metal MT. Since slag SL may form on the surface of the molten metal MT produced in the blast furnace, the slag SL (and its location) can be determined by referring to the image generated by the imaging unit 1, and the slag SL is scraped out of the molten metal container MB by the slag dragger SD. Smoke SM may form on the surface of the molten metal MT in this manner.

[0029] The image acquisition unit 1 may be configured to include, for example, an interface circuit for inputting and outputting data to and from an external device. The external device is a storage medium such as a USB (Universal Serial Bus) memory and an SD card (registered trademark) that stores the images of the object to be detected. Alternatively, the external device is a drive device that reads data from a recording medium such as a CD-ROM (Compact Disc Read Only Memory), CD-R (Compact Disc Recordable), DVD-ROM (Digital Versatile Disc Read Only Memory), and DVD-R (Digital Versatile Disc Recordable) that records the images of the object to be detected. Alternatively, the image acquisition unit 1 may be, for example, a communication interface circuit that sends and receives communication signals to and from an external device, and the external device is a server device that manages the images of the object to be detected and is connected to the communication interface circuit via a network (WAN (Wide Area Network, including a public communication network)) or LAN (Local Area Network). In this case, if the image acquisition unit 1 is an interface circuit or a communication interface circuit, the image acquisition unit 1 may also be used in conjunction with the IF unit 5 (i.e., the IF unit 5 may be used as the image acquisition unit 1).

[0030] The input unit 3 is connected to the control processing unit 2 and is a device that inputs various commands, such as a command to instruct the start of detection, and various data necessary for operating the smoke detection device 1000, such as the name of the plant (factory) and the name of the location within the plant (factory). For example, it may be a keyboard, mouse, and multiple input switches to which predetermined functions are assigned. The output unit 4 is connected to the control processing unit 2 and is a device that outputs commands and data input from the input unit 3, as well as images and detection results acquired by the image acquisition unit 1, in accordance with the control of the control processing unit 2. For example, it may be a display device such as a CRT display, LCD (liquid crystal display device), or organic EL display, or a printing device such as a printer.

[0031] The input unit 3 and output unit 4 may be configured as touch panels. In this configuration, the input unit 3 is a position input device that detects and inputs the operating position, such as a resistive or capacitive touch panel, and the output unit 4 is a display device. In this touch panel, a position input device is provided on the display surface of the display device, and one or more candidate input contents that can be input to the display device are displayed. When the user touches the display position that displays the input content they want to input, the position input device detects that position, and the display content displayed at the detected position is input to the smoke detection device 1000 as the user's operation input. With such a touch panel, the user can easily understand the input operation intuitively, thus providing a smoke detection device 1000 that is easy for the user to use.

[0032] The IF unit 5 is connected to the control processing unit 2 and, in accordance with the control of the control processing unit 2, is a circuit that inputs and outputs data to and from external devices, for example. Examples include an RS-232C serial communication interface circuit, an interface circuit using the Bluetooth® standard, and an interface circuit using the USB standard. Alternatively, the IF unit 5 may be a communication interface circuit that sends and receives communication signals to and from external devices, such as a data communication card or a communication interface circuit conforming to the IEEE 802.11 standard.

[0033] The memory unit 6 is connected to the control processing unit 2 and is a circuit that stores various predetermined programs and various predetermined data in accordance with the control of the control processing unit 2. The various predetermined programs include, for example, a control processing program, and the control processing program includes, for example, a control program, a feature processing program, a smoke detection program, and a classification processing program. The control program is a program that controls each of the parts 1, 3 to 6 of the smoke detection device 1000 according to the function of each part. The feature processing program is a program that calculates the brightness difference between each pixel in the image acquired by the image acquisition unit 1 and adjacent pixels adjacent to that pixel, and calculates statistical features based on the brightness differences of each pixel in the image calculated. The smoke detection program is a program that determines the presence or absence of smoke based on the statistical features obtained by the feature processing program. The classification processing program is a program that divides the image acquired by the image acquisition unit 1 into multiple regions. The aforementioned various predetermined data include, for example, images acquired by the image acquisition unit 1, the name of the detection target, the date, time, minutes, and seconds of detection, the results of each process during the detection process, and the detection result, which are all data necessary for executing these programs.

[0034] Such a storage unit 6 may include, for example, a non-volatile memory element such as ROM (Read Only Memory) or a rewritable non-volatile memory element such as EEPROM (Electrically Erasable Programmable Read Only Memory). Furthermore, the storage unit 6 includes RAM (Random Access Memory) which serves as the working memory of the control processing unit 2, storing data generated during the execution of the predetermined program. The storage unit 6 may also be configured to include a hard disk drive (HDD) or solid-state drive (SSD) with a relatively large storage capacity.

[0035] The control processing unit 2 is a circuit for detecting smoke by controlling each part 1, 3 to 6 of the smoke detection device 1000 according to the function of each part. The control processing unit 2 is configured, for example, with a CPU (Central Processing Unit) and its peripheral circuits. When the control processing program is executed, the control unit 21, classification processing unit 22, feature quantity processing unit 23, and smoke determination unit 24 are functionally configured in the control processing unit 2.

[0036] The control unit 21 controls each of the parts 1, 3 to 6 of the smoke detection device 1000 according to the function of each part, and is in charge of the overall control of the smoke detection device 1000.

[0037] The feature processing unit 23 calculates the brightness difference between each pixel in the image acquired by the image acquisition unit 1 and its adjacent pixels, and then calculates statistical features based on these brightness differences for each pixel in the image. In this embodiment, the statistical features are, for example, the correlation values ​​(one index representing similarity) of the co-occurrence matrices in the image acquired by the image acquisition unit 1.

[0038] The Gray Level Co-occurrence Matrix represents the probability P that the brightness value of a point located at a predetermined distance (displacement) δ (=(r, θ)) from a point with brightness value i is j. ij This is a matrix whose elements are [elements]. The statistical features of this co-occurrence matrix can be used to characterize the texture, which is the spatial distribution pattern of brightness in an image (quantification of texture features).

[0039] To find the simultaneous occurrence matrix, first, we have the pairs PA of brightness values ​​i and j in the image. ij Number of occurrences PN ij And this is required, and the number of occurrences PN ij PA ij By dividing by the total number of possible combinations (total number of combinations) AN, the probability P can be calculated. ij The simultaneous occurrence matrix is ​​obtained by finding (P ij =PN ij / AN).

[0040] More specifically, assume that the target image (original image) SP for which the co-occurrence matrix is to be obtained is a 5×5 (5 rows and 5 columns) image SP having pixel values (luminance values) as shown on the left side of the paper surface in FIG. 3A. First, the number of occurrences PN ij of the pair PA ij of the luminance value i and the luminance value j in the original image SP ij is used as an element PN 8 to prepare a matrix (pair occurrence number matrix) PNM as shown on the right side of the paper surface in FIG. 3A. The size of this pair occurrence number matrix PNM is determined according to the number of luminance gradations in the original image. For example, when the gradation is 256 (=2 ij ) gradations (0 to 255), the size of the pair occurrence number matrix PNM is 256×256 (256 rows and 256 columns). Each row (the 0th row to the 255th row) is each luminance value i of the target pixel which is the pixel for which the occurrence number PN

[0041] <0000?]] ij of the pair is to be obtained, and each column (the 0th column to the 255th column) is each luminance value j of the adjacent pixel which is a pixel separated by a predetermined distance from the target pixel of the luminance value i and forms a pair with it. Therefore, the pair occurrence number matrix PNM is a square matrix. Note that the number of gradations of the original image is not limited to 256 gradations and may be arbitrary.The predetermined distance is set appropriately in advance, for example, according to the size of the smoke to be detected and the actual length captured in one pixel. If the center position of the pixel (the intersection of two diagonals assuming the pixel is rectangular) is defined as the pixel position, then in this embodiment, for example, the distance is one of the lengths of one pixel, two pixels, or three pixels in the horizontal direction (θ=0°) to the right of the page. That is, adjacent pixels to the pixel of interest are either pixels directly adjacent to the pixel of interest (the pixel in question), pixels adjacent to the pixel of interest with one pixel between them, or pixels adjacent to the pixel of interest with two pixels between them. In the examples shown in Figures 3 and 4, the adjacent pixels are pixels directly adjacent to the pixel of interest (the pixel in question). In this example, the direction θ of the predetermined distance is defined as the horizontal direction to the right of the paper (θ=0°), but it is not limited to this. For example, it may be any of the following directions: diagonally upward to the right of the paper (e.g., θ=45°), upward on the paper (θ=90°), diagonally left to the right of the paper (e.g., θ=135°), left to the left of the paper (θ=180°), diagonally downward to the left of the paper (e.g., θ=225°), downward at the bottom of the paper (θ=270°), or diagonally downward to the right of the paper (e.g., θ=315°).

[0042] Once the predetermined distance (the adjacent pixels) is defined in this way, in the 0th row, first, the element PN in the 0th row and 0th column of the PNG pair generation matrix is ​​defined. 00 PN is the number of occurrences of a pixel value of 0 in adjacent pixels directly to the right (to the right, θ=0°) of the pixel of interest with a brightness value of 0. 00 Therefore, in the example shown in Figure 3A, there are 0, and PN 00 = 0. Next, the element PN of the pair occurrence matrix PNG. 01 PN is the number of occurrences of a pixel value of 1 in the adjacent pixels directly to the right of the pixel of interest with a brightness value of 0. 01 Therefore, in the example shown in Figure 3A, there is one, as shown in Figure 3B, and PN 00 = 1. Similarly, for element PN in row 0, column 255. 0255 Up to each element PN ij This is required.

[0043] In the first row, we first have the element PN of the set occurrence matrix PNG. 10PN is the number of occurrences of a pixel value of 0 in the adjacent pixels directly to the right of the pixel of interest with a brightness value of 1. 10 Therefore, in the example shown in Figure 3A, there are 0, and PN 10 = 0. Next, the element PN of the pair occurrence matrix PNG. 11 PN is the number of occurrences of a pixel value of 1 in adjacent pixels directly to the right of the pixel of interest with a brightness value of 1. 11 Therefore, in the example shown in Figure 3A, there are 0, and PN 11 = 0. Next, the element PN of the pair occurrence matrix PNG. 12 This is the number of occurrences of pixel value 2 in the adjacent pixels directly to the right of the pixel of interest with a brightness value of 1, PN. 12 Therefore, in the example shown in Figure 3A, there are two, as shown in Figure 4A, and PN 12 = 2. Next, the element PN of the set occurrence matrix PNG. 13 This is the number of occurrences of a pixel value of 3 in the adjacent pixel directly to the right of the pixel of interest with a brightness value of 1, which is PN. 13 Therefore, in the example shown in Figure 3A, there is one, as shown in Figure 4A, and PN 13 = 1. Similarly, for element PN in row 1, column 255. 1255 Up to each element PN ij This is required.

[0044] In the second row, first, the element PN of the set occurrence matrix PNG. 20 PN is the number of occurrences of a pixel value of 0 in the adjacent pixels directly to the right of the pixel of interest with a brightness value of 2. 20 Therefore, in the example shown in Figure 3A, there are 0, and PN 20 = 0. Next, the element PN of the pair occurrence matrix PNG. 21 PN is the number of occurrences of a pixel value of 1 in the adjacent pixel directly to the right of the pixel of interest with a brightness value of 2. 21 Therefore, in the example shown in Figure 3A, there are 0, and PN 21 = 0. Next, the element PN of the pair occurrence matrix PNG. 22 This is the number of occurrences of a pixel value of 2 in adjacent pixels directly to the right of the pixel of interest with a brightness value of 2, PN. 22 Therefore, in the example shown in Figure 3A, there is one, as shown in Figure 4B, and PN 22 = 1. Next, the element PN of the set occurrence matrix PNG.23 This is the number of occurrences of a pixel value of 3 in the adjacent pixel directly to the right of the pixel of interest with a brightness value of 2, which is PN. 23 Therefore, in the example shown in Figure 3A, there are two, as shown in Figure 4B, and PN 32 = 1. Similarly, for element PN in row 2, column 255. 2255 Up to each element PN ij This is required.

[0045] Similarly, for element PN with 255 rows and 255 columns, see below. 255255 Up to each element PN ij The following is obtained: the number of occurrences per pair matrix PNG is obtained.

[0046] Then, each element PN of the set occurrence matrix PNG ij By dividing each by the total number of combinations AN (256 × 256 = 65536 in the example shown in Figure 3A), each element P of the concurrent occurrence matrix can be obtained. ij The simultaneous occurrence matrix is ​​then determined.

[0047] As an example, Figure 6 illustrates the imaged simultaneous occurrence matrices for the first to tenth sample images No. 1 to No. 10 shown in Figure 5. The first sample image No. 1 is a brightness gradient image where the left edge of the page has the darkest brightness (darkest brightness), and the brightness gradually increases from left to right, with the right edge of the page having the brightest brightness. The second to tenth sample images No. 2 to No. 10 are images in which noise is superimposed on the first sample image No. 1 at random pixel positions, with the number of noise particles gradually increasing in numerical order from the second sample image No. 2 to the tenth sample image No. 10. Since the first sample image No. 1 is a brightness gradient image, the brightness values ​​of the noise are adjusted between the left and right halves of the image, so the left and right boundary lines appear in the second to tenth sample images No. 2 to No. 10. For each of the first to tenth sample images No. 1 to No. 10, a co-occurrence matrix is ​​obtained, and in Figure 6, the elements PN of the co-occurrence matrix are shown. ij The element PN ijEach simultaneous occurrence matrix is ​​imaged by assigning brightness values ​​in proportion to the value of . ij The larger the value, the brighter the brightness value assigned. From Figure 6, for example, as in the first sample image No. 1, the smoother the image, the brighter it becomes in a linear fashion from the top left to the bottom right (element PN). ij It was found that the value of is increasing, and conversely, the grainier the image, such as in the 10th sample image No. 10, the more sparsely bright it is (element PN). ij (It can be seen that the values ​​are scattered.)

[0048] The correlation value CV of the simultaneous occurrence matrix obtained in this way is calculated by Equation 1, and the closer the value is to 1, the smoother the image, and the closer the value is to 0, the rougher the image. Each parameter μ in Equation 1 i. , σ i. , μ .j , σ .j , p i. , p .j This is expressed by equations 2-1 to 2-6. Here, N g This is the number of grayscale levels, and in the example shown in Figure 3, it starts from a brightness value of 0, so N g = 255 (i = 0 to 255, j = 0 to 255).

[0049]

number

[0050]

number

[0051] FIG. 7 shows a graph in which each correlation value CV of each co-occurrence matrix for the first to tenth sample images No. 1 to No. 10 shown in FIG. 5 is plotted with ●. For the first and second sample images No. 1 and No. 2, which are relatively smooth images, each correlation value CV of the co-occurrence matrix is close to 1, and for the ninth and tenth sample images No. 9 and No. 10, which are relatively rough images, each correlation value CV of the co-occurrence matrix is close to 0. As shown in FIG. 7, the characteristics of each correlation value CV of the co-occurrence matrix are demonstrated in the first to tenth sample images No. 1 to No. 10 shown in FIG. 5.

[0052] When smoke is reflected in an image, the luminance difference between adjacent pixels becomes relatively small due to blurring by the smoke. Therefore, it can be seen that the correlation value CV of the co-occurrence matrix in the image changes depending on the degree to which smoke is reflected in the image. If it is relatively large, it can be determined that smoke is reflected in the image.

[0053] Returning to FIG. 1, the smoke determination unit 24 determines the presence or absence of smoke based on the statistical feature amount obtained by the feature amount processing unit 23. More specifically, the smoke determination unit 24 determines that there is smoke when the statistical feature amount obtained by the feature amount processing unit 23 satisfies a predetermined condition (first condition), and determines that there is no smoke when the statistical feature amount obtained by the feature amount processing unit 23 does not satisfy the first condition. In the present embodiment, since the correlation value CV of the co-occurrence matrix is used as the statistical feature amount, more specifically, a threshold value (first smoke determination threshold value) Th1 (0 < Th1 ≦ 1) is set in advance by a plurality of samples. The smoke determination unit 24 compares the statistical feature amount obtained by the feature amount processing unit 23 with the first smoke determination threshold value Th1. As a result of the comparison, when the statistical feature amount obtained by the feature amount processing unit 23 is greater than or equal to the first smoke determination threshold value Th1 (or exceeds it), it is determined that there is smoke as a case where the first condition is satisfied. As a result of the comparison, when the statistical feature amount obtained by the feature amount processing unit 23 is less than the first smoke determination threshold value Th1 (or does not exceed it (the following case)), it is determined that there is no smoke as a case where the first condition is not satisfied.

[0054] When detecting smoke for the entire image, the first smoke determination threshold Th1 is a relatively high value where 0 < Th1 ≤ 1. For example, it is appropriately set to 0.7, 0.8, 0.9, etc.

[0055] Although the presence or absence of smoke may be determined for the entire image in this way, in this embodiment, a partitioning processing unit 22 is provided, and the presence or absence of smoke is determined for each image of each region obtained by partitioning the image.

[0056] This partitioning processing unit 22 partitions the image acquired by the image acquisition unit 1 into a plurality of regions. The regions have a rectangular (including square) shape in order to partition the image without gaps. The plurality of regions may be of the same size, different sizes, or partially different sizes (a mixture of the same size and different sizes). When locally determining the presence or absence of smoke, regions of a relatively small size are used, and when globally determining the presence or absence of smoke, regions of a relatively large size are used. The sizes of the plurality of regions are appropriately set according to the usage of the smoke detection device 1000, etc.

[0057] When partitioning this image into a plurality of regions, the feature quantity processing unit 23 obtains the statistical feature quantity in the image of each region for each of the plurality of regions partitioned by the partitioning processing unit 22, and the smoke determination unit 24 determines the presence or absence of smoke in the image of each region for each of the plurality of regions partitioned by the partitioning processing unit 22.

[0058] In one example, the image acquired by the image acquisition unit 1 is partitioned into 16 first to sixteenth regions of 4×4, and for each image of the first to sixteenth regions, each co-occurrence matrix is obtained, each correlation value CV of each co-occurrence matrix is obtained, and by comparing each correlation value CV of each co-occurrence matrix with the first smoke determination threshold Th1, the presence or absence of smoke in each case is determined.

[0059] Figure 9 shows the correlation values ​​CV for each co-occurring matrix obtained for each image in each of the 1st to 16th regions. As shown in Figure 8A, the 9th sample image No. 9 shown in Figure 5 is divided into 16 regions in a 4x4 grid, and Figure 9A shows the histograms of the correlation values ​​CV for each co-occurring matrix obtained for each image in each of the 1st to 16th regions. As shown in Figure 9A, since the 9th sample image No. 9 is a grainy image, approximately 95% of the correlation values ​​CV for each co-occurring matrix obtained for each image in each of the 1st to 16th regions belong to the 0-0.1 class, and approximately 5% belong to the 0.1-0.2 class. Note that multiple copies of the 9th sample image No. 9 were prepared, and the correlation values ​​CV for each were calculated. On the other hand, Figure 9B is a histogram of the correlation values ​​CV of each co-occurring matrix obtained for each image in each of the 16 regions, after dividing image No. 9-with-soke, which is obtained by blurring a portion of the 9th sample image No. 9 shown in Figure 5 as pseudo-smoke, into 16 regions in a 4x4 grid. As shown in Figure 9B, in the parts without the pseudo-smoke, since the original 9th ​​sample image No. 9 is a grainy image, approximately 55% of the correlation values ​​CV of each co-occurring matrix obtained for each image in each region belong to the 0-0.1 class and approximately 15% belong to the 0.1-0.2 class. In the parts with the pseudo-smoke, since the image is smooth, approximately 30% of the correlation values ​​CV of each co-occurring matrix obtained for each image in each region belong to the 0.9-1 class. In Figure 9, the correlation values ​​CV of the co-occurrence matrix are classified into classes of 0.1 each, with 0 ≤ CV ≤ 1. As shown in Figure 9, it is demonstrated that smoke detection is possible based on each correlation value CV of the co-occurrence matrix, as shown in the 9th sample image No. 9 and image No. 9-with-soke in Figure 8.

[0060] When detecting smoke in an image of a region, the first smoke detection threshold Th1 is set to a value closer to 1 than when determining the presence or absence of smoke in the entire image as described above, and can be set appropriately to, for example, 0.85, 0.9, or 0.95. If the imaging unit 1 is fixedly positioned relative to the detection target, each part of the detection target that is captured in each region of the image can be defined, so the first smoke detection threshold Th1 may be finely adjusted for each region (Th1 ± △, △; increase / decrease range) according to the characteristics of each part of the detection target that is captured in each region.

[0061] When the control unit 21 determines whether or not smoke is present using the smoke determination unit 24, it outputs the determination result to the output unit 4, which then outputs the determination result to the outside. The control unit 21 may output the determination result to the output unit 4 only if the determination result indicates the presence of smoke.

[0062] The control processing unit 2, input unit 3, output unit 4, IF unit 5, and storage unit 6 in the smoke detection device 1000 can be configured by, for example, a desktop or notebook computer.

[0063] Next, the operation of this embodiment will be described. Figure 10 is a flowchart showing the operation of the smoke detection device.

[0064] When the smoke detection device 1000 with this configuration is powered on, it performs the initialization of each necessary part and starts operating. The control processing unit 2 is functionally configured with a control unit 21, a classification processing unit 22, a feature quantity processing unit 23, and a smoke determination unit 24 through the execution of its control processing program.

[0065] When smoke detection is initiated, in Figure 10, the smoke detection device 1000, controlled by the control unit 21 of the control processing unit 2, acquires an image with the image acquisition unit 1 and stores it in the storage unit 6 (S1).

[0066] Next, the smoke detection device 1000 divides the image acquired by the image acquisition unit 1 into multiple regions using the division processing unit 22 of the control processing unit 2 and stores them in the storage unit 6 (S2).

[0067] Next, the smoke detection device 1000 uses the feature processing unit 23 of the control processing unit 2 to determine the brightness value of each pixel in the image of each of the multiple regions divided by the division processing unit 22, and stores it in the storage unit 6 (S3). In this embodiment, for each of the multiple regions, in order to determine the simultaneous occurrence matrix in the image of that region, the PNM pair occurrence matrix in the image of that region is determined and stored in the storage unit 6.

[0068] Next, the smoke detection device 1000 uses the feature processing unit 23 of the control processing unit 2 to obtain a simultaneous occurrence matrix in the image of each of the multiple regions divided by the division processing unit 22, obtain statistical features of the simultaneous occurrence matrix (correlation value CV in this embodiment), and store them in the storage unit 6 (S4).

[0069] Next, the smoke detection device 1000, using the smoke determination unit 24 of the control processing unit 2, determines for each of the multiple regions divided by the division processing unit 22 whether or not smoke is present in the image of that region, based on the statistical features (correlation value CV in this embodiment) of the co-occurrence matrix in the image of that region, and stores the result of the determination in the storage unit 6 (S5).

[0070] Then, the smoke detection device 1000 outputs the result of the determination to the output unit 4 via the control unit 21 of the control processing unit 2, and the output unit 4 outputs the determination result to the outside (S6), thus ending this process. The control unit 21 may also output the result of the determination to an external device via the IF unit 5 if necessary.

[0071] Next, an embodiment will be described. Figure 11 is a diagram illustrating an embodiment without smoke. Figure 11A shows the first image PC1 including the surface of the molten metal, and Figure 11B is a histogram of each correlation value CV of each co-occurring matrix obtained from each region obtained by dividing the first image PC1 shown in Figure 11A into 4x4 sections. Figure 11C shows the second image PC2 including the surface of the molten metal, and Figure 11D is a histogram of each correlation value CV of each co-occurring matrix obtained from each region obtained by dividing the second image PC2 shown in Figure 11C into 4x4 sections. Figure 11E shows the third image PC3 including the surface of the molten metal, and Figure 11F is a histogram of each correlation value CV of each co-occurring matrix obtained from each region obtained by dividing the third image PC3 shown in Figure 11E into 4x4 sections. Figure 12 is a diagram illustrating an embodiment with smoke. Figure 12A shows the fourth image PC4, which includes the surface of the molten metal, and Figure 12B is a histogram of the correlation values ​​CV of each co-occurring matrix obtained from each region obtained by dividing the fourth image PC4 shown in Figure 12A into a 4x4 grid. Figure 12C shows the fifth image PC5, which includes the surface of the molten metal, and Figure 12D is a histogram of the correlation values ​​CV of each co-occurring matrix obtained from each region obtained by dividing the fifth image PC5 shown in Figure 12C into a 4x4 grid. Figure 12E shows the sixth image PC6, which includes the surface of the molten metal, and Figure 12F is a histogram of the correlation values ​​CV of each co-occurring matrix obtained from each region obtained by dividing the sixth image PC6 shown in Figure 12E into a 4x4 grid. In Figures 11B, 11D, 11F, 12B, 12D, and 12F, the horizontal axis represents the correlation value by class, and the vertical axis represents the frequency. For class 0, 0 ≤ CV < 0.45; for class 1, 0.45 ≤ CV < 0.55; for class 2, 0.55 ≤ CV < 0.65; for class 3, 0.65 ≤ CV < 0.75; for class 4, 0.75 ≤ CV < 0.85; for class 5, 0.85 ≤ CV < 0.95; and for class 6, 0.95 ≤ CV < 1.

[0072] The first to third images PC1 to PC3 shown in Figures 11A, 11C, and 11E are smoke-free images. As shown in Figures 11B, 11D, and 11F, the correlation values ​​CV of the co-occurrence matrices appear in the histograms of these images for classes 0 to 4, while they hardly appear in classes 5 and 6.

[0073] In contrast, the fourth to sixth images PC4-PC6 shown in Figures 12A, 12C, and 12E are images with smoke, and as shown in Figures 12B, 12D, and 12F, the correlation value CV of the co-occurrence matrix also appears in the histograms of class 5 and class 6.

[0074] Therefore, by setting the second smoke detection threshold Th2 to approximately 0.85, smoke can be detected based on an image of the molten metal surface.

[0075] As described above, the smoke detection device 1000 in the embodiment, as well as the smoke detection method and smoke detection program implemented therein, can detect smoke with greater accuracy because they determine the brightness difference between adjacent pixels, obtain statistical features based on the brightness difference of each pixel in the image obtained, and determine the presence or absence of smoke based on the obtained statistical features. A histogram like the one in Patent Document 1 is information of the entire image and does not take spatial information into consideration, making it difficult to detect smoke occurring locally in the image. However, the smoke detection device 1000, smoke detection method, and smoke detection program compare adjacent pixels by obtaining the brightness difference between adjacent pixels, taking spatial information into consideration, so they can detect smoke occurring locally.

[0076] The smoke detection device 1000, smoke detection method, and smoke detection program described above divide an image into multiple regions and detect smoke in each region, so smoke can be detected even if only a portion of the image contains smoke.

[0077] The smoke detection device 1000, smoke detection method, and smoke detection program described above can be used in factories that manufacture metal products because they detect smoke in images that include the surface of molten metal. For example, in a desulfurization process that removes sulfur from pig iron, if smoke is captured in the image when determining the bare molten metal area of ​​the desulfurization slag or the molten iron temperature by image analysis, the measurement accuracy deteriorates. However, the deterioration of measurement accuracy can be suppressed by excluding the area determined to have smoke by the smoke detection device 1000, smoke detection method, and smoke detection program from the calculation of the bare molten metal area or the molten iron temperature.

[0078] According to the embodiment, a smoke detection device 1000, a smoke detection method, and a smoke detection program can be provided in which the adjacent pixels are any of the following: a pixel directly adjacent to the pixel, a pixel adjacent to the pixel with one pixel between it and the pixel, and a pixel adjacent to the pixel with two pixels between it and the pixel.

[0079] According to the embodiment, a smoke detection device 1000, a smoke detection method, and a smoke detection program can be provided that use the correlation value of the co-occurrence matrix in the image as a statistical feature.

[0080] In the above-described embodiment, smoke was detected for each of the multiple regions, but smoke may be detected for the entire image based on the number of regions in which smoke was determined to be present based on the detection results for each region. In this case, the division processing unit 22 divides the image acquired by the image acquisition unit 1 into multiple regions, similar to the above-described embodiment, and the feature processing unit 23 calculates statistical features in the image of each of the multiple regions divided by the division processing unit 22, similar to the above-described embodiment. The smoke determination unit 24, in this case, first determines whether smoke is present in the image of each of the multiple regions divided by the division processing unit 22, and finally determines whether smoke is present in the image acquired by the image acquisition unit 1 based on the number of regions in which smoke was determined to be present. More specifically, the smoke detection unit 24 ultimately determines that there is smoke in the image acquired by the image acquisition unit 1 if the number of regions determined to have smoke by the above determination satisfies a predetermined condition (second condition), and ultimately determines that there is no smoke in the image acquired by the image acquisition unit 1 if the number of regions determined to have smoke by the above determination does not satisfy the second condition. In this embodiment, the correlation value CV of the co-occurrence matrix is ​​used as a statistical feature. More specifically, a threshold (second smoke determination threshold) Th2 (for example, 50% or 60% of the number of divided regions) is set in advance using multiple samples. The smoke determination unit 24 compares the number of regions determined to have smoke with the second smoke determination threshold Th2. If, as a result of the comparison, the number of regions determined to have smoke is equal to or greater than (or exceeds) the second smoke determination threshold Th2, the unit determines that the second condition is met and that there is smoke. If, as a result of the comparison, the number of regions determined to have smoke is less than (or less than or equal to) the second smoke determination threshold Th2, the unit determines that the second condition is not met and that there is no smoke.

[0081] Such a smoke detection device 1000, smoke detection method, and smoke detection program divide an image into multiple regions, detect smoke in each region on an initial basis, and ultimately determine the presence or absence of smoke in the entire image based on the number of regions in which smoke is determined to be present. Therefore, even if smoke is captured in various parts of the image, the presence or absence of smoke can be evaluated for the entire image.

[0082] To illustrate the present invention, the embodiments have been adequately and fully described above with reference to the drawings. However, those skilled in the art should recognize that it is easy to modify and / or improve upon the embodiments described above. Therefore, unless such modifications or improvements implemented by those skilled in the art fall outside the scope of the claims, such modifications or improvements shall be considered to be included within the scope of the claims. [Explanation of Symbols]

[0083] 1000 Smoke detection device 1 Image acquisition unit 2 Control Processing Unit 3. Input section 4 Output section 5. Interface section (IF section) 6 Memory section 21 Control Unit 22 Sectioning Processing Unit 23 Feature Processing Unit 24 Smoke detection section

Claims

1. Image acquisition unit that acquires images, A feature processing unit calculates the brightness difference between each pixel in the image acquired by the image acquisition unit and adjacent pixels, and calculates statistical features based on the brightness differences of each pixel in the image obtained, The system includes a smoke determination unit that determines the presence or absence of smoke based on statistical features obtained by the feature processing unit. Smoke detection device.

2. The system further includes a segmentation processing unit that divides the image acquired by the image acquisition unit into multiple regions. The feature processing unit obtains statistical features in the image of each of the multiple regions divided by the division processing unit, The smoke detection unit determines, for each of the multiple regions divided by the division processing unit, whether or not there is smoke in the image of that region. The smoke detection device according to claim 1.

3. The system further includes a segmentation processing unit that divides the image acquired by the image acquisition unit into multiple regions. The feature processing unit obtains statistical features in the image of each of the multiple regions divided by the division processing unit, The smoke determination unit determines whether smoke is present in the image of each of the multiple regions divided by the division processing unit, and finally determines whether smoke is present in the image acquired by the image acquisition unit based on the number of regions determined to have smoke. The smoke detection device according to claim 1.

4. The adjacent pixel is one of the following: a pixel directly adjacent to the pixel, a pixel adjacent to the pixel with one pixel between it and the pixel, or a pixel adjacent to the pixel with two pixels between it and the pixel. The smoke detection device according to claim 1.

5. The aforementioned statistical feature is the correlation value of the co-occurrence matrix in the image acquired by the image acquisition unit. The smoke detection device according to claim 1.

6. The aforementioned image is an image taken so as to include the surface of the molten metal. The smoke detection device according to claim 1.

7. The image acquisition process involves obtaining an image, A feature processing step which involves determining the brightness difference between each pixel in the image acquired in the image acquisition step and adjacent pixels adjacent to that pixel, and then determining statistical features based on the brightness differences of each pixel in the image obtained, The system includes a smoke determination step that determines the presence or absence of smoke based on the statistical features obtained in the feature processing step. Smoke detection method.

8. A smoke detection program for causing a computer to function as a smoke detection device according to any one of claims 1 to 6.

Images