Characteristic amount extraction device, and characteristic amount extraction method

Abstract

To provide a technology that extracts an amount of characteristic about a spark time sequence, using a plurality of spark images.SOLUTION: A characteristic amount extraction device comprises: an image acquisition unit; an area division unit that divides a plurality of spark images acquired by the image acquisition unit for each predetermined area, and thereby prepares a plurality of divided images; a number-of-pixels detection unit that detects the number of pixels of the spark; a three-dimensional plot unit that prepares a three-dimensional plot map serving as a three-axis map representing the number of pixels, a detection area and imaging time from the number of pixels detected in the number-of-pixels detection unit, the detection area, and the imaging time of the plurality of spark images; a two-dimensional graph conversion unit that converts the three-dimensional plot map to a two-dimensional map with the detection area and imaging time as an axis; a two-dimensional streamline identification unit that identifies a two-dimensional streamline of the spark from the two-dimensional graph; and a characteristic amount extraction unit that extracts the amount of characteristic from the two-dimensional streamline.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a feature quantity extraction device and a feature quantity extraction method.

Background Art

[0002] Conventionally, spark observation is known as a means for identifying the material of steel and tools and the deterioration of steel and tools. Patent Document 1 discloses a technique of photographing sparks generated when a steel material is rubbed with a camera that exposes for a certain period of time and extracting feature quantities from the obtained image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When extracting the feature quantity of a spark from a single spark image, information regarding the time series of the spark cannot be calculated. Further, when a plurality of sparks are generated during exposure, overlapping of the sparks may occur in the image, and thus the recognition accuracy of the sparks may decrease.

Means for Solving the Problems

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to the first aspect of the present disclosure, there is provided a feature quantity extraction device that extracts feature quantities of sparks generated from a tool of a machining device during machining or a steel material to be machined by the machining device. This feature quantity extraction device includes an image acquisition unit that acquires a plurality of spark images obtained by imaging the sparks in time series by an imaging device, a region division unit that generates a plurality of divided images by dividing the plurality of spark images acquired by the image acquisition unit for each predetermined region in any one of a division direction of an X direction and a Y direction orthogonal to the X direction, a pixel number detection unit that detects the number of pixels of the sparks in each of the plurality of divided images in the plurality of spark images generated by the region division unit, a three-dimensional plot unit that creates a three-dimensional plot map, which is a three-axis map representing the pixel number, the detection region, and the imaging time, from the pixel number detected by the pixel number detection unit, the detection region that is the region in the division direction of the divided image in which the pixel number is detected, and the imaging time of the plurality of spark images, a two-dimensional graph conversion unit that converts the three-dimensional plot map into a two-dimensional graph with the detection region and the imaging time as axes, a two-dimensional streamline identification unit that identifies a two-dimensional streamline, which is a streamline on the two-dimensional graph of the sparks, from the two-dimensional graph, and a feature quantity extraction unit that extracts the feature quantity of the sparks from the two-dimensional streamline. According to this aspect, information regarding the time series of the sparks can be calculated from the time series of spark images acquired by the imaging device. Further, since the imaging device images the sparks in time series, it is possible to suppress the occurrence of overlap of a plurality of sparks in the image. (2) In the above aspect, the two-dimensional streamline identification unit may identify the two-dimensional streamline by any one of extracting, as representative points, points that satisfy a predetermined extraction condition at each imaging time for each detection region of the two-dimensional graph, connecting the representative points for each detection region, and representing the two-dimensional streamline by a line segment approximated by a predetermined function for the representative points for each detection region. According to this aspect, the two-dimensional streamline identification unit can easily identify the two-dimensional streamline in the two-dimensional graph. (3) In the above-described embodiment, further provided with a rupture identification unit, the rupture identification unit may identify that a rupture has occurred when the number of the representative points of the two-dimensional streamline increases for each of the detection regions in the direction of the imaging time. According to this embodiment, the rupture identification unit can easily identify a rupture in the image of the spark. (4) In the above-described embodiment, the feature amount extraction unit may extract information on the speed, scattering time, and flying distance of the spark as feature amounts using the length and angle of the two-dimensional streamline identified by the two-dimensional streamline identification unit. According to this embodiment, the feature amount extraction unit can calculate information on the speed, scattering time, and flying distance of the spark. (5) In the above-described embodiment, the feature amount extraction unit may extract a change in the number of the sparks in a time series as a feature amount from the number of the two-dimensional streamlines identified by the two-dimensional streamline identification unit. According to this embodiment, the feature amount extraction unit can calculate information on a change in the number of the sparks. (6) In the above-described embodiment, a binarization processing unit that performs binarization on the plurality of images including the time series information acquired by the imaging unit with a predetermined threshold value is provided, and the region division unit may divide the plurality of binarized images in the X direction or the Y direction for each of the predetermined lengths. According to this embodiment, the shape of the spark can be made clearer by binarizing the spark image. (7) According to the second aspect of the present disclosure, there is provided a feature quantity extraction method for extracting feature quantities of sparks generated from a tool of a machining apparatus during machining or a steel material to be machined by the machining apparatus. This feature quantity extraction method includes an image acquisition step of acquiring a plurality of spark images obtained by imaging the sparks in time series by an imaging device, a region division step of generating a plurality of divided images by dividing the plurality of spark images acquired in the image acquisition step for each predetermined region in any one of a dividing direction of an X direction and a Y direction orthogonal to the X direction, a pixel number detection step of detecting the number of pixels of the sparks in each of the plurality of divided images in the plurality of spark images generated in the region division step, a three-dimensional plotting step of creating a three-dimensional plot map which is a three-axis map representing the number of pixels, the detection region, and the imaging time from the number of pixels detected in the pixel number detection step, the detection region which is the region in the dividing direction of the divided image in which the number of pixels is detected, and the imaging time of the plurality of spark images, a two-dimensional graph conversion step of converting the three-dimensional plot map into a two-dimensional graph with the detection region and the imaging time as axes, a two-dimensional streamline identification step of identifying a two-dimensional streamline which is a streamline of the sparks on the two-dimensional graph from the two-dimensional graph, and a feature quantity extraction step of extracting the feature quantity of the sparks from the streamline. According to this aspect, information regarding the time series of the sparks can be calculated from a plurality of images including time series information acquired by the imaging device. Further, since the imaging device images the sparks in time series, it is possible to suppress the occurrence of overlapping of a plurality of sparks in the image. The present disclosure can be realized in various forms, and in addition to the above-described form, it can be realized in forms such as a computer program for extracting the feature quantity of the sparks and a non-transitory recording medium recording the computer program.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Mode for Carrying Out the Invention

[0008] A. First Embodiment: FIG. 1 is an explanatory drawing showing the configuration of the feature quantity extraction system 10 in the present embodiment. The feature quantity extraction system 10 includes an imaging device 12 and a feature quantity extraction device 14.

[0009] As shown in FIG. 1, the feature quantity extraction system 10 of the present embodiment images the spark C generated when the steel material A contacts the tool B. The tool B is a tool of a machining device during machining, and the steel material A is an object to be machined by the machining device that is machined by the tool B. The feature quantity extraction device 14 of the feature quantity extraction system 10 extracts the feature quantity of the spark C based on the image of the spark C imaged by the imaging device 12. Note that the feature quantity extraction device 14 also identifies the rupture D included in the spark C.

[0010] The imaging device 12 generates a plurality of spark images by continuously imaging the spark C in time series. The imaging device 12 is, for example, a camera. In the present embodiment, the imaging device 12 continuously images the spark C at very short time intervals. Here, the very short time interval refers to a time interval in which, when focusing on one streamline among the streamlines that are the lines of the spark C, the process from the generation to the disappearance of the streamline can be captured by a plurality of images. In the present embodiment, the time interval is a shooting speed of 10,000 frames / second.

[0011] The feature amount extraction device 14 is, for example, a server. The feature amount extraction device 14 is not limited to a single server and may be configured by a plurality of servers. Further, the feature amount extraction device 14 may be a personal computer or the like.

[0012] As shown in FIG. 1, the feature amount extraction device 14 includes a CPU 15 that performs arithmetic processing, a ROM 17 that stores a control program of the CPU 15, etc., and a RAM 19 that is a data work area. The CPU 15 reads and executes a program stored in an auxiliary storage device 18 such as an HDD or an SDD, and cooperates with the above-described hardware to execute various functions according to the control program.

[0013] FIG. 2 is an explanatory diagram showing the configuration of the CPU 15. The CPU 15 includes an image acquisition unit 21, a luminance conversion processing unit 22, a binarization processing unit 23, a region division unit 24, a pixel number detection unit 25, a three-dimensional plot unit 26, a two-dimensional graph conversion unit 27, a two-dimensional streamline identification unit 28, a feature amount extraction unit 29, and a rupture identification unit 30.

[0014] The image acquisition unit 21 acquires a plurality of spark images generated by the imaging device 12 and stores them in the RAM 19. In the present embodiment, the plurality of spark images are each color images.

[0015] The luminance conversion processing unit 22 converts the luminance of each pixel of a plurality of images captured by the image acquisition unit 21, thereby converting the images into grayscale images consisting only of luminance values, and stores the plurality of converted spark images in the RAM 19.

[0016] The binarization processing unit 23 performs binarization on the plurality of grayscale images after conversion by the luminance conversion processing unit 22 using a predetermined threshold value, and stores each of the obtained plurality of binarized images in the RAM 19 as spark images.

[0017] The region division unit 24 generates a plurality of divided images by dividing the plurality of binarized images into predetermined regions in either the X direction which is the width direction or the Y direction which is the height direction. The X direction and the Y direction are orthogonal to each other. Note that the region division unit 24 sets the division direction to the X direction when the traveling direction of the spark is the X direction, and sets the division direction to the Y direction when the traveling direction of the spark is the Y direction.

[0018] The pixel number detection unit 25 detects the number of pixels of the spark in each of the plurality of divided images in the plurality of binarized images generated by the region division unit 24.

[0019] The three-dimensional plot unit 26 creates a three-dimensional plot map which is a three-axis map representing the number of pixels, the detection region which is the region in the division direction of the divided image, and the imaging time of the plurality of spark images, from the number of pixels detected by the pixel number detection unit 25, the detection region, and the imaging time of the plurality of spark images. Details of the three-dimensional plot map will be described later.

[0020] The two-dimensional graph conversion unit 27 converts the three-dimensional plot map created by the three-dimensional plot unit 26 into a two-dimensional graph with the detection region and the imaging time as axes. Details of the conversion from the three-dimensional plot map to the two-dimensional graph will be described later.

[0021] The two-dimensional streamline specifying unit 28 specifies the two-dimensional streamline, which is the streamline on the two-dimensional graph of the spark C, from the two-dimensional graph obtained by the two-dimensional graph conversion unit 27. Details of this method for specifying the two-dimensional streamline will be described later.

[0022] The feature quantity extraction unit 29 extracts the feature quantity of the spark from the two-dimensional streamline specified by the two-dimensional streamline specifying unit 28. Details of the method for extracting the feature quantity of the spark will be described later.

[0023] When the rupture specifying unit 30 counts the representative points constituting the two-dimensional streamline specified by the two-dimensional streamline specifying unit 28 for each detection region in the direction of the imaging time and the number of representative points increases, it specifies that a rupture has occurred. Details of the method for specifying the rupture will be described later.

[0024] Figure 3 is a flowchart showing the feature quantity extraction process by the feature quantity extraction device 14. In step S1 of Figure 3, the feature quantity extraction device 14 acquires a plurality of spark images generated by the imaging device 12 and stores them in the RAM 19. Here, the feature quantity extraction device 14 may receive the spark images via wired or wireless means. In this embodiment, the plurality of spark images are each color images with a resolution of 1280×720 pixels.

[0025] In step S2 of Figure 3, the luminance conversion processing unit 22 converts the luminance of each pixel of the plurality of spark images acquired by the image acquisition unit 21, thereby converting the images into grayscale images consisting only of luminance values, and stores the plurality of converted images in the RAM 19. As a method for converting from a color image to a grayscale image, for example, there is a method of converting from the RGB values of each pixel in the spark image to the luminance value Y in the YIQ color space system using a predetermined conversion formula. Note that the method for converting from a color image to a grayscale image is not limited to the above method.

[0026] In step S3 of FIG. 3, the binarization processing unit 23 performs binarization processing on the plurality of grayscale images after conversion by the luminance conversion processing unit 22 using a predetermined threshold value, and stores the obtained plurality of images in the RAM 19. Thereby, the shape of the spark can be made clearer. In the present embodiment, for the grayscale image, when the luminance value Y is greater than a predetermined value, it is set as a white pixel, and when it is equal to or less than the predetermined value, it is set as a black pixel. For this reason, the spark is recognized as white. On the other hand, when the luminance value Y is greater than a predetermined value, it may be set as a black pixel, and when it is equal to or less than the predetermined value, it may be set as a white pixel. In the present disclosure, in order to simplify the drawing, white and black are inverted, and the spark is illustrated as black.

[0027] In step S4 of FIG. 3, the region division unit 24 performs a region division process of dividing the plurality of spark images after the binarization process by the binarization processing unit 23 into predetermined regions in any one of the division directions, which is the X direction in the width direction and the Y direction in the height direction. Thereby, a plurality of divided images are created. In the present embodiment, the division direction is the X direction, and a region of 20 pixels is set as one division region.

[0028] In step S5 of FIG. 3, the pixel number detection unit 25 performs a pixel number detection process of detecting the number of pixels of the spark in each of the plurality of divided images in the plurality of images obtained by the region division unit 24.

[0029] FIG. 4 is a diagram for explaining the region division process and the pixel number detection process. The diagram shown on the left side of FIG. 4 represents a state in which the region division unit 24 divides the image Im in the X direction to generate a plurality of divided images X1 to X8. Note that in FIG. 4, eight divided images are shown for simplicity of the drawing. However, in the present embodiment, since an image with a resolution of 1280×720 pixels is divided by 20 pixels in the X direction, 64 divided images are generated. The diagram shown on the left side of FIG. 4 shows the spark images Im1 to ImtN continuously acquired in time series from time t1 to time tN. Note that the spark images Im1 to ImtN are images obtained by converting a color image acquired by the imaging device 12 into a grayscale image by the luminance conversion unit 22 and then performing binarization processing by the binarization unit 23. The thick black line in the image represents a streamline which is a spark line. In the present embodiment, since the spark is imaged at a very short time interval, the trajectory from the generation to the disappearance of the spark is acquired by a plurality of images. The spark is generated at time t2, grows as time elapses, and disappears at time tN. The region division unit 24 divides each of the spark images Im1 to ImtN into eight divided regions in the X direction, and generates eight divided images X1 to X8 corresponding to the eight divided regions. In the present embodiment, the origin side of the X axis is the root side of the spark, and the +X direction side is the tip side of the spark. Note that in the diagram shown on the left side of FIG. 4, for convenience of explanation, the image is divided into eight divided images. However, the region division unit 24 can divide the image into N divided images, where N is an arbitrary integer. For example, using an image with a resolution of 720×1280 pixels, divided images may be created every 20 pixels in the X direction. Note that the resolution of the image and the size of the divided images are not limited to the above.

[0030] The figure shown on the right side of FIG. 4 represents the state in which the pixel number detection unit 25 has detected the number of pixels of the sparks in the divided image X2. Specifically, for each of the spark images Im1 to ImtN shown in FIG. 4, the number of pixels of the sparks included in the divided image X2 is detected. At time t2, a part of the spark is located in the divided image X2. At this time, when the spark located in the divided image X2 is enlarged, pixels constituting the spark are observed as shown in the image P2 shown on the right side of FIG. 4. The pixel number detection unit 25 detects the pixel number from the image P2 and outputs it. In the divided image X2, sparks are observed from time t2 to t5. In particular, at time t4, since all the observed sparks are located in the divided image X2, as shown in the image P4, the number of pixels detected in the divided image X2 is the maximum. After time t6, since the sparks are observed at a position larger than the divided image X2, the value output by the pixel number detection unit 25 in the divided image X2 becomes 0. In FIG. 4, for the convenience of explanation, the pixel number detection unit 25 only detects the number of pixels in the divided image X2, but actually detects the number of pixels in all the divided images from X1 to X8. Also, in the following description, the position in the X direction may be referred to as the "X position".

[0031] In step S6 of FIG. 3, the three-dimensional plotting unit 26 performs a three-dimensional plotting process of creating a three-dimensional plot map, which is a three-axis map representing the pixel number, the detection region, and the imaging time, from the pixel number detected by the pixel number detection unit 25, the detection region which is the region in the division direction of the divided image where the pixel number is detected, and the imaging times of the plurality of spark images.

[0032] In step S7 of FIG. 3, the two-dimensional graph conversion unit 27 performs a two-dimensional graph conversion process of converting the three-dimensional plot map obtained by the three-dimensional plotting unit 26 into a two-dimensional graph with the detection region and the imaging time as axes.

[0033] FIG. 5 is a diagram for explaining various processes of the three-dimensional plot process and the two-dimensional graph conversion unit process. The three-dimensional plot map PL1 shown in the upper diagram of FIG. 5 is generated by the three-dimensional plot unit 26 based on the detection values in the divided image X2 shown in FIG. 4. The three-dimensional plot map PL1 is represented by the detection axis X, the time axis T, and the pixel axis Pn, which are three orthogonal axes. The detection axis X is an axis indicating the positions of the respective divided regions (detection regions) of the divided images X1 to X8. The time axis T is an axis indicating the imaging time. The pixel axis Pn is an axis indicating the number of pixels of the spark detected by the pixel number detection unit 25. In the present embodiment, since the image is divided in the X direction, the detection region indicates the X position. However, when the image is divided in the Y direction, the detection region indicates the Y position.

[0034] The diagram shown in the center of FIG. 5 shows a state in which the pixel number detection unit 25 detects the number of pixels of the spark included in the divided images X1 to X8 for each of the images Im1 to ImtN shown in FIG. 4, and the three-dimensional plot unit 26 creates the three-dimensional plot map PL2.

[0035] The diagram shown at the bottom of FIG. 5 is a diagram for explaining the two-dimensional graph conversion process. The two-dimensional graph conversion unit 27 converts the three-dimensional plot map PL2 shown in the center of FIG. 5 into a two-dimensional graph G1 with the detection axis X and the time axis T as axes. In the two-dimensional graph G1, the plotting method is changed according to the value of the number of pixels detected at each plane coordinate position [X, T] in the three-dimensional plot map PL2. When the number of pixels detected at each plane coordinate position [X, T] is less than a predetermined value, the plot is made white, and when it is more than the predetermined value, the plot is hatched. Also, the larger the hatching of the slant lines, the larger the number of detected pixels. Note that the two-dimensional graph G1 may be in color, and the magnitude of the number of pixels may be represented by color.

[0036] In step S8 of FIG. 3, the two-dimensional streamline specifying unit 28 performs a two-dimensional streamline specifying process for specifying the two-dimensional streamline of the spark from the two-dimensional graph G1 acquired by the two-dimensional graph conversion unit 27. For each detection region of the two-dimensional graph, the two-dimensional streamline specifying unit 28 extracts, as representative points, points that satisfy the extraction conditions determined in advance at each of the imaging times, and specifies the two-dimensional streamline by either connecting the representative points for each detection region or representing the representative points for each detection region by line segments approximated by a predetermined function.

[0037] The predetermined extraction conditions are conditions that enable extraction of points (coordinates on the two-dimensional graph) representing the spark at each imaging time for each detection region of the two-dimensional graph. In the present embodiment, the predetermined extraction conditions are, for each detection region of the two-dimensional graph, the condition that the point indicating the maximum value of the number of pixels at each imaging time, and, when there is no maximum value, the point indicating the maximum value of the number of pixels. Note that the predetermined extraction conditions are not limited to the above.

[0038] FIG. 6 is a diagram for explaining the two-dimensional streamline identification process and the feature amount extraction process. In the diagram shown above FIG. 6, the two-dimensional streamline identification unit 28 shows a state in which representative points Bn for each detection region are extracted from the two-dimensional graph G1 shown in FIG. 5. The two-dimensional streamline identification unit 28 extracts, as representative points Bn, points that satisfy the extraction conditions in each of the detection regions X1 to X8 of the two-dimensional graph G1. For example, at position X1 of the two-dimensional graph G1 shown in FIG. 5, pixels are detected at time t2 and time t3. Here, when comparing the number of pixels at time t2 and the number of pixels at time t3, since the value of the number of pixels at time t2 is larger, it can be seen that the number of pixels is the largest at time t2. Also, at position X1, there is no maximum value of the number of pixels. Therefore, as shown in the upper diagram of FIG. 6, the two-dimensional streamline identification unit 28 extracts the planar coordinate position [X1, t2] as the representative point B1 of position X1. At position X2 of the two-dimensional graph G1, pixels are detected from time t2 to time t5. Here, the number of detected pixels increases from time t2 to time t4 and decreases at time t5. That is, at position X2, the number of pixels detected at time t4 is the maximum. Therefore, as shown in the upper diagram of FIG. 6, the two-dimensional streamline identification unit 28 extracts the planar coordinate position [X2, t4] as the representative point B2 of position X2. When representative points B3 to B7 are extracted using the same method for positions X3 to X7, the two-dimensional graph G2 shown above FIG. 6 is obtained.

[0039] In the diagram shown below FIG. 6, the two-dimensional streamline identification unit 28 shows a state in which the two-dimensional streamline R is identified based on the two-dimensional graph G2. The two-dimensional streamline identification unit 28 identifies the two-dimensional streamline R by either connecting the representative points B1 to B7 in the two-dimensional graph G2 or representing it by a line segment approximated by a predetermined function. In the present embodiment, the two-dimensional streamline identification unit 28 identifies the two-dimensional streamline R by approximating the representative points B1 to B7 with a linear function. The two-dimensional streamline identification unit 28 may also identify the two-dimensional streamline R by approximating the representative points B1 to B7 with a higher-order function of two dimensions or more. Note that the process of extracting the representative points Bn can also be said to be the same process as the process of thinning by contraction of the morphological process.

[0040] In step S9 of FIG. 3, the feature amount extraction unit 29 extracts the feature amount of the spark from the streamline R specified by the two-dimensional streamline specifying unit 28. For example, the feature amount extraction unit 29 extracts information on the speed, scattering time, and flying distance of the spark as feature amounts using the length and angle of the two-dimensional streamline R specified by the two-dimensional streamline specifying unit 28. Further, the feature amount extraction unit 29 extracts the change in the number of sparks in time series as a feature amount from the number of two-dimensional streamlines R specified by the two-dimensional streamline specifying unit 28.

[0041] The process of the feature amount extraction unit 29 extracting the feature amount of the spark will be described with reference to the lower diagram of FIG. 6. The two-dimensional streamline R is a line segment approximated by a linear function indicating the trajectory of the spark generated at time t2, starting from R1, which is the generation position of the spark, and ending at R2, which is the extinction position of the spark. The distance ΔX from the X position X1 at the starting point R1 to the X position X7 at the ending point R2 is the flying distance of the spark in the X direction. The time Δt from the imaging time t2 at the starting point R1 to the imaging time tN - 2 at the ending point R2 is the scattering time of the spark. Here, the length RL of the two-dimensional streamline R can be calculated from the following formula (1) using the angle Δd formed with the X-axis. RL≒ΔX / cosΔd ··· Formula (1) Also, the speed V of the spark can be calculated using the following formula (2). V = RL / Δt ··· Formula (2) Note that the speed V of the spark in this embodiment is the speed of the spark in the X direction. In contrast, when the region dividing unit 24 divides the image in the Y direction, the speed V of the spark is the speed of the spark in the Y direction. Also, in this embodiment, since the main traveling direction of the spark is the X direction, the speed of the spark in the X direction can be regarded as the speed V of the spark.

[0042] FIG. 7 is a diagram showing the change in the number of sparks in a time series in a two-dimensional graph. Five two-dimensional streamlines R11 to R15 are shown in FIG. 7. These are obtained by acquiring five two-dimensional streamlines R shown in FIG. 6. The five two-dimensional streamlines R11 to R15 are acquired in order from the two-dimensional streamline R11 located on the lower side to the two-dimensional streamline R15 located on the upper side. Here, the feature quantity extraction unit 29 calculates the change in the number of sparks in a time series. The change in the number of sparks in a time series is, for example, the change in the number of acquisitions of the two-dimensional streamline R per unit time. In FIG. 7, paying attention to the intervals on the time axis for each two-dimensional streamline R among the five two-dimensional streamlines R11 to R15, the intervals for each two-dimensional streamline R are large in the region F1 from the two-dimensional streamline R11 to R13, while the intervals for each two-dimensional streamline R are small in the region F2 from the two-dimensional streamline R13 to R15. That is, it can be seen that the number of acquisitions of the two-dimensional streamline R per unit time increases as time passes. Note that the above-described change in the number of sparks can be utilized for estimating the state of the grindstone as the tool B shown in FIG. 1. For example, an increase in the number of acquisitions of the two-dimensional streamline R per unit time is presumed to be related to the falling off or wear state of the grindstone. Thereby, the discrimination accuracy of the deterioration of the tool can be improved.

[0043] FIG. 8 is a diagram for explaining the region division process, pixel detection process, three-dimensional plot process, and two-dimensional graph conversion process at the time of spark breakdown. The diagram shown on the left side of FIG. 8 represents the state in which the region division unit 24 divides the image in the X direction. The diagram shown on the left side of FIG. 8 shows the spark images Im1 to Im13 continuously acquired from time t1 to time t13. Note that the images Im1 to Im13 are images obtained by converting the color image acquired by the imaging device 12 into a grayscale image by the luminance conversion unit 22 and then performing binarization processing by the binarization unit 23, similar to the diagram shown on the left side of FIG. 4. The spark is generated at time t2 (not shown), grows over time, and then disappears at time t13. At time t7, one spark 1a is observed, but at time t8, a breakdown D1 occurs and it branches into two sparks 1a and 1b. Among the two branched sparks 1a and 1b, spark 1a follows a trajectory approximately parallel to the X direction, similar to up to time t7. Among the two branched sparks 1a and 1b, spark 1b follows a trajectory diagonally downward with respect to the X direction. The region division unit 24 divides each of the spark images Im1 to Im13 in the X direction and creates eight divided images X1 to X8. In the present embodiment, the origin side of the X-axis is the root side of the spark, and the +X direction side is the tip side of the spark.

[0044] The three-dimensional plot map PL3 shown in the upper right side of FIG. 8 is generated by the three-dimensional plot unit 26 using the spark images Im1 to Im13 shown on the left side of FIG. 8. In the three-dimensional plot map PL3, plots are continuously detected at position X6 from time t7 to time t13. This is because, as shown in the spark images Im7 to 13, spark 1a is detected from time t7 to t10, and spark 1b is detected from time t9 to t13.

[0045] The two-dimensional graph G4 shown on the lower right side of FIG. 8 is obtained by performing two-dimensional conversion processing on the three-dimensional plot map PL3. In the two-dimensional graph G4, the plotting method is changed according to the value of the number of detected pixels at each plane coordinate position [X, T] in the three-dimensional plot map PL3. When the number of detected pixels at each plane coordinate position [X, T] is less than a predetermined value, the plot is made white, and when it is more than the predetermined value, the plot is hatched. Also, the greater the density of the hatching, the greater the number of detected pixels. When the plot is black, it indicates that the number of detected pixels is the largest. Note that the two-dimensional graph G4 may be in color, and the magnitude of the number of pixels may be represented by color.

[0046] FIG. 9 is a diagram for explaining two-dimensional streamline identification processing at the time of spark breakdown. The diagram shown above FIG. 9 shows a state in which the two-dimensional streamline identification unit 28 has extracted representative points Bn for each detection region from the two-dimensional graph G4 shown in FIG. 8. The two-dimensional streamline identification unit 28 extracts representative points Bn that satisfy the above-described extraction conditions in each of the detection regions X1 to X8 of the two-dimensional graph G4. For example, at position X1 of the two-dimensional graph G4 shown in FIG. 8, pixels derived from spark 1a are detected at times t2 and t3. Here, when comparing the number of pixels at time t2 and the number of pixels at time t3, since the number of pixels at time t2 is larger, it can be seen that the number of pixels at time t2 is the maximum value. Also, there is no maximum value of the number of pixels at position X1. Therefore, as shown in the two-dimensional graph G5 of FIG. 9, the two-dimensional streamline identification unit 28 extracts the plane coordinate position [X1, t2] as the representative point B1a at position X1 of spark 1a. At position X2 of the two-dimensional graph G4, pixels derived from spark 1a are detected from time t2 to t5. Here, the number of detected pixels increases from time t2 to time t4 and decreases at time t5. That is, at position X2, the number of detected pixels becomes maximum at time t4. Therefore, as shown in the upper diagram of FIG. 6, the two-dimensional streamline identification unit 28 extracts the plane coordinate position [X2, t4] as the representative point B2a at position X2.

[0047] At position X6 of the two-dimensional graph G4 shown in FIG. 8, pixels derived from spark 1a and spark 1b are detected from time t7 to time t13. When comparing the number of pixels from time t7 to time t13, it reaches a maximum at time t10. As shown in the spark image Im9, at time t9, in addition to spark 1a, a part of spark 1b is also detected, but here it is regarded as the representative point B1a of spark 1a. Therefore, as shown in the two-dimensional graph G5 of FIG. 9, the two-dimensional streamline specifying unit 28 extracts the plane coordinate position [X6, t9] as the representative point B6a at the position X6 of spark 1a. When observing the number of pixels after time t10 at position X6 of the two-dimensional graph G4 shown in FIG. 8, the number of pixels decreases from time t10 to time t11, while the number of pixels increases from time t11 to time t12, and the number of pixels decreases again from time t12 to time t13. Thus, as shown in the two-dimensional graph G5 of FIG. 9, the two-dimensional streamline specifying unit 28 extracts the plane coordinate position [X6, t12] as the representative point B6b at the position X6 of spark 1b. In the present embodiment, when a plurality of representative points Bn are extracted in the same detection region, the two-dimensional streamline specifying unit 28 regards the representative point Bn with an earlier imaging time as the representative point Bna derived from the spark generated from the root. On the other hand, the two-dimensional streamline specifying unit 28 regards the representative point Bn with a later imaging time as the representative point Bnb derived from the branched spark. Note that the method for specifying the branched spark is not limited to the above. The two-dimensional streamline specifying unit 28 does not necessarily regard the representative point Bn with a later imaging time as the representative point Bnb derived from the branched spark. In this case, the two-dimensional streamline specifying unit 28 may regard the representative point Bn with a later imaging time as the representative point Bna derived from the spark generated from the root.

[0048] In the figure shown below Fig. 9, a state is shown where the two-dimensional streamline identification unit 28 has identified two-dimensional streamlines Ra and Rb based on the two-dimensional graph G5. The two-dimensional streamline identification unit 28 identifies the two-dimensional streamline Ra by either connecting the representative points B1a to B8a in the two-dimensional graph G5 or representing it by a line segment approximated by a predetermined function. Also, the two-dimensional streamline identification unit 28 identifies the two-dimensional streamline Rb by either connecting the representative points B6b to B7b or representing it by a line segment approximated by a predetermined function. In the present embodiment, the two-dimensional streamline identification unit 28 identifies the two-dimensional streamlines Ra and Rb by approximating the representative points B1a to B8a and the representative points B6b to B7b with a linear function. Note that the two-dimensional streamline identification unit 28 may identify the two-dimensional streamlines Ra and Rb by approximating the representative points B1a to B8a and the representative points B6b to B7b with a higher-order function of two dimensions or more.

[0049] In the two-dimensional graph G6 shown in Fig. 9, the two-dimensional streamline Ra represents the locus where the spark 1a generated at time t2 (not shown) in the spark image Im of Fig. 8 moves in a direction approximately parallel to the X position and disappears at time t13. On the other hand, the two-dimensional streamline Rb represents the locus traced by the spark 1b that branched at time t8 until time t13. In the two-dimensional graph G6 shown in Fig. 9, the two-dimensional streamline Ra starts from R3, which is the generation position of the spark 1a, and ends at R4, which is the disappearance position of the spark 1a. The two-dimensional streamline Rb is generated at the branch point R5, which is the point where the two-dimensional streamline Ra intersects. Note that, as shown in the spark image Im13 of Fig. 8, since the branched spark has not disappeared at time t13, the end point of the two-dimensional streamline Rb is not shown in the two-dimensional graph G6.

[0050] In step S10 of Fig. 3, when the rupture identification unit 30 counts the representative points Bn of the two-dimensional streamline R for each detection region in the direction of the imaging time and the number of the representative points Bn increases, the rupture identification unit 30 identifies that a rupture has occurred.

[0051] FIG. 10 is a diagram for explaining the rupture identification process. The two-dimensional streamlines Ra and Rb shown in the two-dimensional graphs G7 to G9 of FIG. 10 are the same as the two-dimensional streamlines Ra and Rb in the two-dimensional graph G6 of FIG. 9. The rupture identification unit 30 counts the representative points B1a to B8a and the representative points B6b to B7b in the two-dimensional streamlines Ra and Rb for each detection region in the direction of the imaging time. The direction of the imaging time is the direction from the representative point B1a to B8a as shown in FIG. 10. Counting for each detection region means counting the number of representative points Bn in each region from position X1 to position X8.

[0052] The two-dimensional graph G7 shown above FIG. 10 shows a state where the rupture identification unit 30 is counting the number of representative points Bn at the starting point of the two-dimensional streamline Ra. At this time, since the rupture identification unit 30 detects only the representative point B1a located at position X1, the detection value is 1. The two-dimensional graph G8 shown in the center of FIG. 10 shows a state where the rupture identification unit 30 is counting the number of representative points Bn near the center of the two-dimensional streamline Ra. At this time, since the rupture identification unit 30 detects only the representative point B5a located at position X5, the detection value is 1. The two-dimensional graph G9 shown below FIG. 10 shows a state where the rupture identification unit 30 is counting the number of representative points Bn near the branch point of the two-dimensional streamlines Ra and Rb. At this time, since the rupture identification unit 30 detects the representative points B6a and B6b located at position X6, the detection value is 2. From the above, it can be seen that the number of representative points Bn increased from position X5 to position X6. The rupture identification unit 30 identifies that a rupture has occurred when the number of representative points Bn increases. Therefore, in FIG. 10, it is identified that there was a rupture from position X5 to position X6. In this embodiment, the case where the spark bifurcated into two was identified as a rupture, but the present disclosure is not limited to this. The rupture identification unit 30 identifies a rupture when the spark bifurcates into an arbitrary integer N. Since the number of ruptures is related to the deterioration of the steel material A shown in FIG. 1, the rupture identification unit 30 can improve the identification accuracy of the deterioration of the steel material A by identifying the ruptures in the image of the spark.

[0053] In addition to identifying the rupture of the above-described sparks, the rupture identification unit 30 can also identify the intersection of two sparks that occur at the same time. When the representative points Bn of two intersecting two-dimensional streamlines R are counted for each detection region in the direction of the imaging time, the rupture identification unit 30 identifies that an intersection has occurred when the number of representative points Bn decreases and then increases. Specifically, this is the case where the total of the representative points Bna and Bnb of two two-dimensional streamlines Ra and Rb is two at position X5, but decreases to one at position X6 and then increases to two at position X7 (illustration omitted). At this time, the rupture identification unit 30 identifies that the two sparks have intersected. According to this form, the rupture identification unit 30 can distinguish between the rupture and intersection of the sparks.

[0054] According to the above-described embodiment, the feature amount extraction unit 29 can calculate information regarding the time series of the sparks from the time-series spark images acquired by the imaging device 12. In addition, since the imaging device 12 captures the sparks in time series, it is possible to suppress the occurrence of overlapping of a plurality of sparks in the image.

[0055] B. Other Embodiments: (B1) In the above-described first embodiment, the side with the smaller X position is the base side of the spark, and the side with the larger X position is the tip side of the spark. In contrast, the side with the larger X position may be the base side of the spark, and the side with the smaller X position may be the tip side of the spark.

[0056] (B2) In the above-described first embodiment, the feature amount extraction device 14 includes the luminance conversion processing unit 22 and the binarization processing unit 23. In contrast, the feature amount extraction device 14 may not include the luminance conversion processing unit 22 and the binarization processing unit 23.

[0057] (B3) In the above-described first embodiment, the two-dimensional streamline specifying unit 28 extracted, as the representative point Bn, the point where the maximum or maximum value of the number of pixels was located in each of the detection regions X1 to X8 of the two-dimensional graph G1. In contrast, the two-dimensional streamline specifying unit 28 does not necessarily have to extract, as the representative point Bn, the point where the maximum or maximum value of the number of pixels is located in each of the detection regions X1 to X8 of the two-dimensional graph G1. For example, the two-dimensional streamline specifying unit 28 may use, as the representative point Bn, the average value of the imaging times at which the number of pixels was detected in each of the detection regions X1 to X8 of the two-dimensional graph G1. In this case, the average value of the number of pixels in each detection region may be used as the number of pixels Pn at the representative point Bn.

[0058] (B4) In the above-described first embodiment, the rupture specifying unit 30 specified that a rupture had occurred when the number of representative points Bn increased when the representative points Bn of the two-dimensional streamline R were counted for each detection region in the direction of the imaging time. In contrast, the rupture specifying unit 30 does not necessarily have to specify that a rupture has occurred when the number of representative points Bn increases. When the two-dimensional streamline R is obtained by approximating a plurality of representative points Bn by a function, the rupture specifying unit 30 may recognize the two-dimensional streamline R as a set of points and specify that a rupture has occurred when the number of points increases.

[0059] The present disclosure is not limited to the above-described embodiments, and can be realized in various configurations without departing from the gist thereof. For example, the technical features of the embodiments corresponding to the technical features in each form described in the summary of the invention can be appropriately replaced or combined in order to solve some or all of the above-described problems or to achieve some or all of the above-described effects. Further, if the technical feature is not described as essential in this specification, it can be appropriately deleted.

Explanation of Reference Numerals

[0060] 10... Feature extraction system, 12... Imaging device, 14... Feature extraction device, 15... CPU, 17... ROM, 18... Auxiliary storage device, 19... RAM, 21... Image capture processing unit, 22... Luminance conversion processing unit, 23... Binarization processing unit, 24... Region division unit, 25... Pixel count detection unit, 26... 3D plotting unit, 27... 2D graph conversion unit, 28... 2D streamline identification unit, 29... Feature extraction unit, 30... Rupture identification unit

Claims

1. A feature quantity extraction device that extracts feature quantities of sparks generated from a tool of a machining device during machining or a steel material to be machined by the machining device, comprising: an image acquisition unit that acquires a plurality of spark images obtained by imaging the sparks in time series by an imaging device; a region division unit that generates a plurality of divided images by dividing the plurality of spark images acquired by the image acquisition unit for each predetermined region in any one of a division direction of an X direction and a Y direction orthogonal to the X direction; a pixel number detection unit that detects the number of pixels of the sparks in each of the plurality of divided images in the plurality of spark images generated by the region division unit; a three-dimensional plot unit that creates a three-dimensional plot map, which is a three-axis map representing the pixel number, the detection region, and the imaging time, from the pixel number detected by the pixel number detection unit, the detection region that is the region in the division direction of the divided image in which the pixel number is detected, and the imaging time of the plurality of spark images; a two-dimensional graph conversion unit that converts the three-dimensional plot map into a two-dimensional graph with the detection region and the imaging time as axes; a two-dimensional streamline identification unit that identifies a two-dimensional streamline, which is a streamline on the two-dimensional graph of the sparks, from the two-dimensional graph; a feature quantity extraction unit that extracts the feature quantity of the sparks from the two-dimensional streamline.

2. The feature quantity extraction device according to claim 1, wherein the two-dimensional streamline identification unit extracts, for each detection region of the two-dimensional graph, a point that satisfies a predetermined extraction condition at each imaging time as a representative point, and identifies the two-dimensional streamline by either connecting the representative points for each detection region or representing the representative points for each detection region by a line segment approximated by a predetermined function.

3. The feature quantity extraction device according to claim 2, further comprising a rupture identification unit, wherein the rupture identification unit identifies that a rupture has occurred when the number of representative points increases when the representative points of the two-dimensional streamline are counted for each detection region in the direction of the imaging time.

4. The feature quantity extraction device according to claim 1, The feature quantity extraction unit is a feature quantity extraction device that extracts information on the speed, scattering time, and flying distance of the spark as feature quantities using the length and angle of the two-dimensional streamline specified by the two-dimensional streamline specifying unit.

5. The feature quantity extraction device according to claim 1, wherein the feature quantity extraction unit extracts, as a feature quantity, a change in the number of the sparks in a time series from the number of the two-dimensional streamlines specified by the two-dimensional streamline specifying unit.

6. The feature quantity extraction device according to claim 1, comprising a binarization processing unit that performs binarization on the plurality of spark images acquired by the image acquisition unit using a predetermined threshold value, wherein the region division unit divides the plurality of binarized spark images for each of the predetermined regions in the division direction.

7. A feature quantity extraction method for extracting feature quantities of sparks generated from a tool of a machining device during machining or a steel material that is a machining target of the machining device, the method including: an image acquisition step of acquiring a plurality of spark images obtained by imaging the sparks in time series by an imaging device; a region division step of generating a plurality of divided images by dividing the plurality of spark images acquired in the image acquisition step for each of predetermined regions in any one of a division direction of an X direction and a Y direction orthogonal to the X direction; a pixel number detection step of detecting the number of pixels of the spark in each of the plurality of divided images in the plurality of spark images generated in the region division step; a three-dimensional plotting step of creating a three-dimensional plot map that is a three-axis map representing the number of pixels, the detection region, and the imaging time from the number of pixels detected in the pixel number detection step, the detection region that is the region in the division direction of the divided image in which the number of pixels is detected, and the imaging time of the plurality of spark images; a two-dimensional graph conversion step of converting the three-dimensional plot map into a two-dimensional graph with the detection region and the imaging time as axes; a two-dimensional streamline specifying step of specifying a two-dimensional streamline that is a streamline of the spark on the two-dimensional graph from the two-dimensional graph; and a feature quantity extraction step of extracting the feature quantity of the spark from the streamline.

Images