Monitoring method, monitoring device, and program

Abstract

According to a monitoring method of the present invention, a tool 2 is imaged a plurality of times to acquire a plurality of pieces of captured image data (imaging step). Then, on the basis of the acquired plurality of pieces of captured image data, image processing is performed so that a cutting fluid 5 reflected in the captured image data becomes unclear, in order to generate an image for confirming the state of the tool 2 (image generation step).

Description

Monitoring method, monitoring device, and program

[0001] The present invention relates to a monitoring method, a monitoring device, and a program for checking the state of a monitoring target that exists in an environment where liquid splashes.

[0002] For example, tools such as drills used in machine tools have a limited lifespan, and their edges may wear or chip over time. Patent Document 1 discloses a conventional technology for capturing an image of a tool (a monitored object) using a camera while using cutting fluid as part of an optical path. This technology allows for efficient tool monitoring while maintaining the supply of cutting fluid to the tool. However, when cutting fluid is supplied to the tool, the cutting fluid splashes around the tool, flows over the tool surface, or droplets of cutting fluid temporarily adhere to the tool. This inevitably results in degradation of the image quality of the tool captured by the camera due to the inclusion of the cutting fluid. Therefore, the conventional technology of Patent Document 1 fails to capture a clear image of the tool being monitored, potentially making it difficult to clearly confirm the tool's condition.

[0003] Japanese Patent Application Publication No. 11-114781

[0004] An object of the present invention is to enable the state of a monitored object that exists in an environment where liquid is splashed to be clearly confirmed.

[0005] The monitoring method of the present invention is a monitoring method for checking the status of a monitored object that exists in an environment where liquid is scattered, and is characterized by including an imaging step of capturing images of the monitored object multiple times to obtain multiple frames of captured image data, and an image generation step of processing the captured image data based on the multiple frames so that the liquid scattered in the environment that is captured in the captured image data becomes unclear, thereby generating an image for checking the status of the monitored object.

[0006] In addition, the monitoring method according to the present invention is characterized in that the image generation step smoothes the multiple captured image data acquired in the imaging step in the time axis direction to generate an image for checking the status of the monitored object.

[0007] In addition, the monitoring method of the present invention is characterized in that the image generation step extracts portions from the multiple captured image data in which the whole or part of the monitored object is clearly depicted, and synthesizes the extracted portions of the captured image data to generate an image for checking the status of the monitored object.

[0008] Furthermore, in the monitoring method of the present invention, in which a rotating tool is the monitoring object, the imaging step is characterized in that imaging is performed a plurality of times at an imaging period that is an integral multiple of the rotation period of the tool.

[0009] Furthermore, the present invention is characterized in that in the monitoring method in which a rotating tool is the monitoring object, the imaging step performs imaging a plurality of times by shifting the phase of the imaging cycle with respect to the rotation cycle of the tool.

[0010] In addition, the monitoring device of the present invention is a monitoring device for checking the status of a monitoring object that exists in an environment where liquid is splashed, and includes an imaging unit that images the monitoring object and outputs captured image data, and an image processing unit that performs image processing based on the captured image data sent from the imaging unit, wherein the imaging unit images the monitoring object multiple times and outputs multiple frames of captured image data to the image processing unit, and the image processing unit processes the images based on the multiple frames of captured image data input from the imaging unit so that liquid splashing in the environment that is reflected in the captured image data becomes unclear, thereby generating an image for checking the status of the monitoring object.

[0011] In addition, the program of the present invention is a program executed by a computer used in a monitoring device for checking the status of a monitored object that exists in an environment where liquid is splashed, and is characterized by including: an image data acquisition step of inputting multiple frames of captured image data obtained by capturing multiple images of the monitored object; and an image generation step of processing the multiple frames of captured image data so that the liquid splashing in the environment that is captured in the captured image data becomes unclear, based on the multiple frames of captured image data, thereby generating an image for checking the status of the monitored object.

[0012] According to the present invention described above, by processing the image based on multiple captured image data so that the liquid scattered in the environment captured in the captured image data becomes unclear, the image portion of the monitored object becomes clear and its condition can be properly confirmed.

[0013] Fig. 1 is a configuration diagram showing an overview of a monitoring device according to each embodiment of the present invention. Fig. 2 is a flowchart showing the functions of a camera control unit and an image processing unit according to each embodiment of the present invention. Fig. 3 is a diagram for explaining pixel value averaging. Figs. 4A to 4E are diagrams (reference photographs) showing examples of images for checking the state of a tool obtained by the inventors in an experiment according to the first embodiment.

[0014] 1: Machine tool, 2: Tool, 3: Cover, 4: Work chamber, 5: Cutting fluid, 6: Supply nozzle, 10: Camera, 11: Camera control unit, 12: Image processing unit, 13: Computer

[0015] Hereinafter, an embodiment of the present invention in which a tool of a machine tool is the object of monitoring will be described in detail with reference to the drawings. However, as will be described later, the object of monitoring is not limited to the tool of a machine tool.

[0016] 1 is a configuration diagram showing an overview of a monitoring device according to an embodiment of the present invention. A machine tool 1 has a tool 2 disposed inside a work chamber 4 surrounded by a cover 3, and cutting fluid 5 is supplied to the tool 2 from a supply nozzle 6. The cutting fluid 5 is supplied for purposes such as cooling the tool 2 to reduce frictional heat generated in the tool 2 during operation, but the supplied cutting fluid 5 may splash around the tool 2 or temporarily adhere to the tool 2.

[0017] A camera 10 is disposed in the work chamber 4 facing the tool 2, and an image of the tool 2 is captured by this camera 10. When capturing an image of the tool 2 by this camera 10, it is inevitable that the cutting fluid 5 supplied to the tool 2 and the cutting fluid 5 scattered around the tool 2 will be captured in the image.

[0018] To ensure visibility, the camera 10 is configured to prevent the cutting fluid 5 from adhering to the lens surface. For example, the camera 10 of this embodiment is housed in a housing unit that includes a pivoting window and an air nozzle that sprays air onto the pivoting window. With this configuration, the pivoting window prevents the cutting fluid 5 from adhering to the lens surface, and the centrifugal force of the pivoting window and the air sprayed onto the pivoting window blow away any cutting fluid 5 that has adhered to the pivoting window, thereby ensuring the visibility of the camera 10.

[0019] The monitoring device includes as its components a camera 10, a camera control unit 11, and an image processing unit 12. Of these, the camera 10 and the camera control unit 11 constitute an imaging unit for capturing an image of the tool 2 and outputting the captured image data. The camera control unit 11 and the image processing unit 12 are specifically constituted by a computer 13.

[0020] In this embodiment, the camera 10 is fixed at a predetermined position, and the focus of the camera 10 is also fixed on the tool 2 to take photographs. However, if the tool 2 moves, the camera 10 may be moved to follow the movement of the tool 2, and photographs may be taken from the same position relative to the tool 2.

[0021] The camera control unit 11 outputs a photographing command signal to the camera 10. When the camera 10 receives the photographing command signal from the camera control unit 11, the camera 10 captures an image of the tool 2 and outputs the captured image data to the image processing unit 12. The image processing unit 12 performs image processing based on the captured image data sent from the camera 10, and generates an image for checking the state of the tool 2.

[0022] Next, the functions of the camera control unit 11 and the image processing unit 12 in this embodiment will be described with reference to Fig. 2. Fig. 2 is a flowchart showing the functions of the camera control unit and the image processing unit in this embodiment. In this embodiment, while cutting fluid 5 is being supplied to the tool 2 that is not in operation, the camera 10 captures images of the tool 2 multiple times to obtain multiple pieces of captured image data. Then, based on these multiple pieces of captured image data, image processing is performed so that the cutting fluid 5 captured in the captured image data becomes unclear, and an image for checking the state of the tool 2 is generated.

[0023] 2, the camera control unit 11 outputs a photographing command signal to the camera 10 at time intervals (step S1). In response to this photographing command signal, the camera 10 repeatedly captures images of the tool 2 and outputs photographed image data acquired by the image capture. This photographed image data is input to the image processing unit 12 (step S2). These steps S1 and S2 constitute an imaging step (photographed image data acquisition step).

[0024] Here, it is preferable to set the output interval of the photographing command signal (i.e., the interval at which the camera 10 photographs the tool 2) to a fixed cycle, but this is not limited to this in the present embodiment, and the command signal may be output to photograph the tool 2 at any different time interval.

[0025] Next, the image processing unit 12 executes image processing to generate an image for checking the state of the tool 2 (step S3). This step S3 is an image generation step. In the image generation step, based on the multiple pieces of photographed image data sent from the camera 10, image processing is performed so that the cutting fluid 5 reflected in the photographed image data is blurred, thereby generating an image for checking the state of the tool 2.

[0026] In this embodiment, the image generation step is performed in the following procedure. That is, a plurality of frames of captured image data are smoothed (averaged) in the time axis direction to generate a composite image. Specifically, for a plurality of frames (e.g., 60 frames) of captured image data, pixel values ​​of a plurality of pixels (e.g., 60 pixels) corresponding to the same position in the captured image data are averaged to generate an image (an image for checking the state of the tool 2) in which each pixel of the captured image data is set to the average value of the plurality of pixel values.

[0027] FIG. 3 is a diagram illustrating pixel value averaging. For example, in FIG. 3, a pixel at a specific coordinate position in the captured image data is focused on, and 60 frames of the pixel in question from 60 frames of captured image data are arranged side by side. The shade of the color of each pixel corresponds to a pixel value, and averaging the pixel values ​​for these 60 frames shown in FIG. 3 results in a pixel value of 38. By averaging pixel values ​​in this way, the change in the pixel value is smoothed out and made gentler.

[0028] By averaging pixel values ​​across the entire region of the captured image data of the tool 2, moving parts of the captured image data (for example, parts where the cutting fluid 5 is flowing, or parts where the cutting fluid 5 is temporarily attached and then removed) are smoothed (blurred), making the cutting fluid 5 less noticeable. On the other hand, stationary parts of the captured image data (i.e., parts of the tool 2) are not blurred, and a clear image is highlighted. As a result, the part of the tool 2 in the image for checking the condition of the tool 2 generated by averaging is clear, making it possible to clearly check the condition of the tool 2. This image for checking the condition of the tool 2 can be used to evaluate the condition of the tool 2 (for example, the degree of wear or damage) by comparing it with captured image data of a new tool 2, for example.

[0029] In this embodiment, the photographed image data acquisition step and the image generation step described above are executed by a program pre-installed in the computer 13 shown in FIG.

[0030] 4A to 4E are diagrams (reference photographs) showing examples of images for checking the state of the tool 2 obtained by the inventors through an experiment of the first embodiment. Specifically, the image in FIG. 4A is photographed image data of the tool 2 that has not undergone the image generation step. The image in FIG. 4B is an image of the tool 2 generated by averaging five pieces of photographed image data. The image in FIG. 4C is an image of the tool 2 generated by averaging ten pieces of photographed image data. The image in FIG. 4D is an image of the tool 2 generated by averaging thirty pieces of photographed image data. And the image in FIG. 4E is an image of the tool 2 generated by averaging sixty pieces of photographed image data.

[0031] In the image of Figure 4A, the tool 2 is captured along with the adhering cutting fluid 5a and the flowing cutting fluid 5b. However, from Figure 4B to Figure 4E, it can be seen that these cutting fluids 5a and 5b become blurred and unclear, thereby making the outline and surface condition of the tool 2 clearer.

[0032] Second Embodiment Next, a second embodiment of the present invention will be described. As shown in Fig. 1, the monitoring device used in the second embodiment also includes a camera 10, a camera control unit 11, and an image processing unit 12 as components. The camera 10 and the camera control unit 11 have the same functions as those in the first embodiment, and perform the imaging steps (steps S1 and S2) shown in Fig. 2.

[0033] The second embodiment differs from the first embodiment in that the image processing unit 12 has a function of extracting portions in which the whole or part of the tool 2 is clearly depicted from a plurality of captured image data, and synthesizing the images of the extracted portions to generate an image for checking the state of the tool 2. That is, the content of the image generation step (step S3) in Fig. 2 is different.

[0034] When the state in which the cutting fluid 5 is being supplied to the tool 2 is photographed with the camera 10 at time intervals and multiple frames of photographed image data are obtained, the cutting fluid 5 captured in each photographed image data is flowing, and even if it adheres to the tool 2, it flows off together with the cutting fluid 5 that is sprayed on later, so the situation is changing. Therefore, when each of the multiple photographed image data is observed, the parts of the tool 2 that are blurred due to the influence of the cutting fluid 5 often differ depending on the photographed image data.

[0035] For example, in some photographed image data, the outline and surface state of the tip portion of the tool 2 may be unclear due to the cutting fluid 5 being reflected therein, but the outline and surface state of the middle portion of the tool 2 may be clear without the cutting fluid 5 being reflected therein. On the other hand, in other photographed image data, the outline and surface state of the middle portion of the tool 2 may be unclear due to the cutting fluid 5 being reflected therein, but the outline and surface state of the tip portion of the tool 2 may be clear without the cutting fluid 5 being reflected therein.

[0036] In such a case, by dividing the images of each captured image data, extracting the middle part (clear part) of the tool 2 from the former captured image data, and extracting the tip part (clear part) of the tool 2 from the latter captured image data, and combining each clear part, it is possible to generate an image for checking the condition of the entire tool 2 in a clear manner.

[0037] To explain the image generation step executed by the image processing unit 12 more specifically, each of the multiple pieces of photographed image data acquired by the camera 10 is divided into multiple partial images. For example, one piece of photographed image data is divided into 5 blocks x 5 blocks, for a total of 25 pixel blocks (partial images). This process is executed for all photographed image data.

[0038] Next, a Fourier transform is applied to each partial image to decompose it into spatial frequency components. Here, the clearer the partial image, the more high-spatial frequency components it contains. Similar image processing is performed on all partial images (60 partial images x 25 partial images = 1500 partial images) of multiple (e.g., 60) captured image data to decompose them into spatial frequency components.

[0039] Next, the spatial frequency components of multiple partial images corresponding to the same part (position) of the captured image data are compared, and the partial image with the highest value of high spatial frequency components is selected as the clearest partial image in that part (position).

[0040] The same process is performed on the partial images (25 partial images) at all positions of the captured image data, and the clearest partial image is selected for each position. Then, the selected clearest partial images are combined for each position of the captured image data to generate an image for checking the condition of the tool 2. In the image for checking the condition of the tool 2 generated in this way, the tool 2 portion is clear, making it possible to clearly check the condition of the tool 2. This image for checking the condition of the tool 2 can be used to evaluate the condition of the tool 2 (for example, the degree of wear or damage) by comparing it with the captured image data of a new tool 2, for example.

[0041] In this embodiment as well, the above-described photographed image data acquisition step and image generation step are executed by a program pre-installed in the computer 13 shown in FIG.

[0042] [Modifications and Applications] The present invention is not limited to the above-described embodiments, and various modifications and applications are possible without departing from the spirit and scope of the present invention. For example, in the above-described embodiments, the operation of the tool 2 is stopped, and the stationary tool 2 is photographed multiple times by the camera 10 to obtain multiple pieces of photographed image data. However, in most cases, the tool 2 is in operation when the cutting fluid 5 is supplied.

[0043] Therefore, a configuration may be adopted in which a rotating tool 2 such as a drill is imaged multiple times to acquire multiple pieces of captured image data, with the tool 2 being imaged multiple times at an imaging period that is an integer multiple of the rotation period of the tool 2. If the rotating tool 2 is imaged at an imaging period that is an integer multiple of the rotation period, the same surface of the tool 2 can be imaged at the timing when it rotates to a position facing the camera 10, making it possible to acquire multiple pieces of captured image data for the same surface of the tool 2. By executing an image generation step based on these image captures to generate an image for checking the state of the tool 2, the state of a specific surface of the tool 2 can be grasped from the generated image for checking the state.

[0044] Furthermore, a rotating tool 2 such as a drill can be configured to acquire multiple images of the tool 2 at an imaging cycle that is half the rotation cycle of the tool 2. If the rotating tool 2 is imaged at an imaging cycle that is half the rotation cycle, in addition to the imaging surface (front surface) of the tool 2 that is obtained when imaging is performed at an imaging cycle that is an integer multiple of the rotation cycle of the tool 2, the back surface can be imaged at the timing when the back surface rotates to a position facing the camera 10, making it possible to acquire multiple images of the front and back surfaces of the tool 2. By executing an image generation step based on these images and generating images for checking the state of the front and back surfaces of the tool 2, the state of the front and back surfaces of the tool 2 can be grasped from the generated state confirmation images.

[0045] Furthermore, a configuration may be adopted in which, for a rotating tool 2 such as a drill, multiple images are captured by shifting the phase of the imaging cycle relative to the rotation cycle of the tool 2, thereby acquiring multiple pieces of captured image data. For example, if imaging is performed by shifting the phase of the imaging cycle by 30 degrees relative to the rotation cycle of the tool 2, the image of the tool 2 surface that is 30 degrees shifted from the image surface obtained when imaging is performed at an imaging cycle that is an integer multiple of the rotation cycle of the tool 2 can be captured at the timing when the surface rotates to a position facing the camera 10, making it possible to acquire multiple pieces of captured image data for that surface of the tool 2. By executing an image generation step based on these images to generate a state confirmation image of the tool 2, the state of any surface of the tool 2 can be grasped from the generated state confirmation image.

[0046] By slightly shifting the phase of the imaging cycle and similarly acquiring image data, and then executing an image generation step based on these images to generate an image for checking the condition of tool 2, it is also possible to grasp the condition of the entire circumference of tool 2.

[0047] The present invention can monitor not only cutting processes using a rotating tool 2, but also tools 2 of machine tools 1 that perform various processes involving the supply of machining fluid, such as grinding processes. Furthermore, the present invention can be applied to various applications in which objects other than the tool 2 of a machine tool 1 that exist in an environment where liquid is splashed are monitored and their status is checked. For example, the present invention can be applied to monitoring workpieces and chips produced by cutting processes and checking their status. Furthermore, the present invention can be applied to monitoring tableware placed inside a dishwasher and washed with a spray of cleaning fluid, and checking the surface status (e.g., whether dirt has been removed).

[0048] In the above-described embodiment, the camera 10 is housed in a housing unit equipped with a pivoting window and an air nozzle. However, the method for ensuring the field of view of the camera 10 is not limited to this. For example, an air curtain may be generated by flowing air parallel to the lens surface of the camera 10, or the camera 10 may be housed in a housing unit with a pinhole to prevent the cutting fluid 5 from adhering to the lens surface. Alternatively, a wiper may be provided on the lens surface to wipe away any cutting fluid adhering to the lens surface. Alternatively, the camera 10 may be positioned in the work chamber 4 at a distance from the tool 2 that is out of the reach of the cutting fluid 5, and an image of the tool 2 may be captured using a telephoto lens with a long focal length.

Claims

1. A monitoring method for checking the status of a monitoring target that exists in an environment where liquid is scattered, comprising: an imaging step of capturing images of the monitoring target multiple times to obtain multiple pieces of captured image data; and an image generation step of performing image processing on the multiple pieces of captured image data so that liquid scattered in the environment that is captured in the captured image data becomes unclear, thereby generating an image for checking the status of the monitoring target.

2. The monitoring method according to claim 1, characterized in that the image generation step smoothes the multiple captured image data acquired in the imaging step in the time axis direction to generate an image for checking the status of the monitored object.

3. The monitoring method described in claim 1, characterized in that the image generation step extracts portions from the multiple captured image data in which the whole or part of the monitored object is clearly depicted, and synthesizes the extracted portions of the captured image data to generate an image for checking the status of the monitored object.

4. A monitoring method as claimed in any one of claims 1 to 3, in which a rotating tool is the monitored object, characterized in that the imaging step involves taking images multiple times at an imaging period which is an integer multiple of the rotation period of the tool.

5. A monitoring method as claimed in any one of claims 1 to 3, in which a rotating tool is the monitored object, characterized in that the imaging step involves taking images multiple times by shifting the phase of the imaging cycle with respect to the rotation cycle of the tool.

6. A monitoring device for checking the status of a monitoring target that exists in an environment where liquid is splashed, comprising: an imaging unit that images the monitoring target and outputs captured image data; and an image processing unit that performs image processing based on the captured image data sent from the imaging unit, wherein the imaging unit images the monitoring target multiple times and outputs multiple pieces of captured image data to the image processing unit, and the image processing unit processes the image based on the multiple pieces of captured image data input from the imaging unit so that liquid splashing in the environment that is captured in the captured image data becomes unclear, thereby generating an image for checking the status of the monitoring target.

7. A program executed by a computer used in a monitoring device for checking the status of a monitoring target that exists in an environment where liquid is splashed, the program including: an image data acquisition step of inputting multiple captured image data obtained by capturing images of the monitoring target multiple times; and an image generation step of generating an image for checking the status of the monitoring target by processing the captured image data based on the multiple captured image data so that liquid splashed in the environment that is captured in the captured image data becomes unclear.

Images