Chip detection device

Abstract

To detect chips generated when a machine tool machines a work-piece.SOLUTION: An information processing device, which detects chips generated when a machine tool machines a work-piece, comprises: a chip image storing part that stores a chip image showing the generated chips; a background image storing part that stores a background image that is a photographed image of a predetermined region in the machine tool which is not yet processed; a mesh dividing part that mesh-divides the processed photographed image into a plurality of mesh images; and a chip information determining part that indicates, on the mesh images, portions where the chips are generated in the processed photographed image, by inputting the processed photographed image to a chip determination model made to learn shapes of the chips on the basis of the chip image and the background image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for detecting chips generated when a workpiece is machined by a machine tool.

Background Art

[0002] Conventionally, chips generated when a workpiece is machined by a machine tool have become a factor in damaging the workpiece during machining, or have become a factor in causing a decrease in the operating rate due to machining defects, malfunction of the machine, and unintended operation stop. For this reason, as a technique for automatically detecting and removing chips generated during machining, for example, Japanese Patent Application Laid-Open No. 7-108435 discloses a chip removal device that captures images of a table and a workpiece and detects the positions of chips (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, chips generated during machining vary in shape, color, size, aggregation state, etc. due to differences in machining conditions, tools used, workpiece materials, etc., and there are countless variations. Also, regarding the in-machine environment that serves as the background of the image, there are conditions such as structure and position, types of workpieces and jigs, presence or absence of coolant (cleaning liquid), illuminance, etc., and there are various patterns in their combinations.

[0005] Therefore, when trying to detect chips from the entire single image as in Patent Document 1, a large number of chips and the in-machine environment are intricately reflected in the image in a wide variety of combinations. For this reason, it is difficult to detect each chip precisely and with high accuracy by image processing, and the image processing for detecting all chips requires an enormous amount of computation and time. [Means for solving the problem]

[0006] An information processing device in one aspect of the present invention is an information processing device for detecting chips generated when a workpiece is processed by a machine tool, and comprises: a chip image storage unit that stores chip images representing chips; a background image storage unit that stores a background image which is an image taken before processing a predetermined area inside the machine tool; a mesh division unit that divides the image taken after processing into a plurality of mesh images; and a chip information determination unit that, by inputting the image taken after processing to a chip determination model which has learned the shape of chips based on the chip images and the background images, indicates on the mesh images the locations where chips have been generated in the image taken after processing. [Effects of the Invention]

[0007] According to the present invention, chips generated when a workpiece is processed by a machine tool can be easily detected. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing one embodiment of a chip detection device and a machine tool equipped therewith. [Figure 2] This figure shows an area image divided into multiple mesh images in this embodiment. [Figure 3] This diagram shows how multiple partial images are combined to form an area image in this embodiment. [Figure 4] This figure shows the division process by the mesh division unit of this embodiment. [Figure 5] This diagram illustrates the "class" which is the chip information in this embodiment. [Figure 6] This figure shows the determination process using the chip determination model of this embodiment. [Figure 7] This figure shows an example of mesh image features calculated by a chip information determination unit that employs a non-machine learning algorithm. [Figure 8]This is an example of a display screen showing (a) the judgment result before correction and (b) the judgment result after correction in the teaching mode of this embodiment. [Figure 9] This flowchart shows the chip detection method performed by the chip detection device of this embodiment. [Figure 10] This is a flowchart showing the chip detection process in this embodiment. [Figure 11] This is a flowchart showing the teaching process in this embodiment. [Figure 12] This is a block diagram showing the configuration of the learning image synthesis device in this embodiment. [Modes for carrying out the invention]

[0009] Hereinafter, an embodiment of a chip detection device, a machine tool equipped therewith, and a chip detection method will be described with reference to the drawings.

[0010] The chip detection device 1 of this embodiment detects chips generated when a workpiece is processed by a machine tool 10. Specifically, as shown in Figure 1, the chip detection device 1 controls the imaging means 11 provided on the machine tool 10 to acquire an image of the target area for chip detection. Then, as shown in Figure 2, the image is divided into multiple mesh images, and chip information related to the chips is determined for each of the mesh images. The following describes each configuration in detail.

[0011] In this invention, chip information is a concept that includes detectable information regarding chips, such as the presence or absence of chips, the amount of chips, the shape of the chips, the type of chips, or combinations thereof. In this embodiment, as will be described later, a "class" indicating the ease of cleaning the chips is determined as the chip information.

[0012] The machine tool 10 is a machine for performing machining such as cutting and grinding on workpieces made of metal, wood, stone, resin, etc. In this embodiment, the machine tool 10 is numerically controlled according to a drive signal output from a numerical control device as the chip detection device 1. In this embodiment, the chip detection device 1 and the numerical control device of the machine tool are configured by the same device, but it is not limited thereto. A separate computer from the machine tool may be used as the chip detection device 1. Also, as shown in FIG. 1, the machine tool 10 has a photographing means 11 for photographing the interior of the machine, a table 12 for placing the workpiece, a cleaning means 13 for cleaning the chips, and a lighting means 14 for lighting the interior of the machine.

[0013] The photographing means 11 photographs a target area for detecting chips. In this embodiment, the photographing means 11 is constituted by a camera (hereinafter referred to as an ATC camera) that can be attached to and detached from the spindle of the machine tool 10 in the same manner as a tool by an Automatic Tool Changer (ATC). Then, according to a photographing control signal from an image data acquisition unit 43 described later, an area image covering a preset target area is photographed at a preset photographing timing.

[0014] In this embodiment, as the photographing means 11, an ATC camera that can be attached to and detached from the spindle of the machine tool is used in consideration of the merits that installation inside the machine is not required and the time exposed to coolant mist, etc. is relatively short. Since the ATC camera can be attached to the spindle of the machine tool, if the spindle of the machine tool is moved, the camera can also be moved following it. Therefore, the photographing range and the photographing direction of the camera can be easily changed. Although the ATC camera is described as an example because the photographing range and the photographing direction of the camera can be easily changed, it is not limited to this configuration. One or a plurality of fixed cameras may be installed inside the machine. According to this configuration, there is a merit that the photographing time is relatively short and photographing can be performed during machining.

[0015] Also, when the angle of view of the imaging means 11 is smaller than the target area, as shown in FIG. 3, in the case of an ATC camera, a plurality of partial images are captured while moving the spindle, and in the case of a fixed camera, a plurality of partial images are captured by a plurality of fixed cameras. Then, by synthesizing each partial image, an area whole image covering the entire target area may be configured.

[0016] The cleaning means 13 cleans and removes chips generated by the processing of the workpiece. In the present embodiment, the cleaning means 13 is constituted by a programmable nozzle capable of controlling the injection direction of the cleaning liquid. Then, in accordance with a cleaning control signal from a cleaning processing unit 48 described later, a cleaning liquid such as a coolant is injected onto a desired area on the workpiece or the table 12.

[0017] In the present embodiment, a programmable nozzle is used as the cleaning means 13, but it is not limited to this configuration. For example, a plurality of cleaning nozzles with a fixed injection direction may be installed so as to cover the entire target area, and the cleaning liquid may be injected from the cleaning nozzle closest to the area where cleaning is required.

[0018] The lighting means 14 illuminates the interior of the machine so as to facilitate the detection of chips. In the present embodiment, as shown in FIG. 1, the lighting means 14 is fixed inside the machine and is constituted by a programmable light capable of controlling on / off switching, brightness, irradiation direction, etc. However, it is not limited to this configuration, and the light attached to the ATC camera may be used. Also, in order to suppress reflection due to lighting, the lighting means 14 may be rotated or a polarizing filter may be attached to the camera lens of the imaging means 11.

[0019] Next, the chip detection device 1 is composed of a computer such as a numerical control device that controls the machine tool 10. As shown in Figure 1, it mainly includes a display input means 2 that allows the user to input instructions and displays the judgment results, a storage means 3 that stores various data and functions as a working area when the calculation processing means 4 performs calculations, and a calculation processing means 4 that executes various calculations by running the chip detection program 1a installed in the storage means 3 and functions as one of the components described later. The following describes each component in detail.

[0020] It should be noted again that the chip detection device may be a separate device from the numerical control device that controls the machine tool 10. The chip detection device only needs to include at least a mesh division unit that divides at least a portion of an area image taken of the area to be detected into a plurality of mesh images with a predetermined mesh size, and a chip information determination unit that determines the chip information for each of the mesh images using predetermined determination parameters for determining chip information.

[0021] The display input means 2 is a touch panel and combines an input function for receiving instructions from the user and a display function for displaying chip determination results, etc. In this embodiment, a touch panel is used as the display input means 2, but the system is not limited to this configuration, and a display means with only a display function and an input means with only an input function may be provided separately.

[0022] The storage means 3 consists of a hard disk, ROM (Read Only Memory), RAM (Random Access Memory), flash memory, etc., and as shown in Figure 1, it has a program storage unit 31, a determination parameter storage unit 32, a determination result storage unit 33, a cleaning condition storage unit 34, and a cleaning count storage unit 35.

[0023] The program storage unit 31 has a chip detection program 1a installed for controlling the chip detection device 1 of this embodiment. The arithmetic processing means 4 then executes the chip detection program 1a, causing the chip detection device 1, acting as a computer, to function as the various components described later.

[0024] The usage of the chip detection program 1a is not limited to the configuration described above. For example, the chip detection program 1a may be stored on a non-temporary storage medium that can be read by a computer, such as a CD-ROM or USB memory, and then read and executed directly from that storage medium. It may also be used from an external server using cloud computing or an ASP (Application Service Provider) method.

[0025] The determination parameter storage unit 32 stores determination parameters that have been set in advance for determining chip information related to chips. In this embodiment, as will be described later, a chip determination model is adopted as the algorithm for determining chip information, which is obtained by applying learning parameters to a learning and inference model, which is a machine learning algorithm. For this reason, the determination parameters stored are learning parameters obtained by pre-training using training data consisting of a set of mesh images and chip information.

[0026] Furthermore, as will be described later, this embodiment includes a mesh size selection unit 44 for selecting the mesh size of the mesh image. For this reason, the determination parameter storage unit 32 stores learning parameters as determination parameters for each of several different mesh sizes. In this embodiment, the determination parameter storage unit 32 also has hyperparameters set separately that affect characteristics such as the learning efficiency of the chip determination model.

[0027] The determination result storage unit 33 stores the determined chip information for each mesh image as the determination result. In this embodiment, the chip information determination unit 46, described later, determines the probability of each mesh image belonging to a class that indicates the ease of cleaning the chips, as chip information.

[0028] For example, if there are three classes, "Class 0" with no chips, "Class 1" with few chips, and "Class 2" with many chips, the probability P0 of being "Class 0", the probability P1 of being "Class 1", and the probability P2 of being "Class 2" are determined. Therefore, the determination result storage unit 33 stores the above-mentioned probabilities P0, P1, and P2 for each class as determination results, associated with each mesh image.

[0029] Furthermore, the determination result is not limited to the probability for each class; the class with the highest probability may simply be stored as the determination result. Also, increasing the number of classes makes determination more difficult and performance improvement less likely, so it is preferable to limit the number to a few. The simplest example of classes would be two classes: "Class 0" with no chips and "Class 1" with chips.

[0030] The cleaning condition storage unit 34 stores cleaning conditions for determining whether or not cleaning of chips is necessary. In this embodiment, the cleaning conditions are set to be a combination of the probability for each class and the position of the mesh area corresponding to each mesh image. For example, when the three classes described above are set, an efficient and high-performance determination algorithm is realized by evaluating the following cleaning conditions (1) to (4) in descending order.

[0031] (1) Of P0, P1, and P2, the maximum value is P2: Wash (2) P1 + P2 ≥ 99%: Wash (3) The mesh area is on table 12 and P1 + P2 ≥ P0: Wash (4) None of the above conditions (1) to (3) are met: Do not wash

[0032] In this embodiment, cleaning conditions are set by combining chip information and the position of the mesh area, but this is not the only option; any condition set using chip information is acceptable. For example, if the two simplest classes described above are set, cleaning conditions may be set such that cleaning is not performed if the mesh area is in class 0 (no chips), and cleaning is performed if it is in class 1 (chips present), regardless of the position of the mesh area.

[0033] The cleaning count memory unit 35 stores the number of times chips have been continuously cleaned. In this embodiment, the cleaning count memory unit 35 stores the continuous cleaning count, which is the number of times the target area has been continuously cleaned, and the local continuous cleaning count, which is the number of times the same mesh area has been continuously cleaned. These cleaning counts are then used to prevent situations where the automatic cleaning operation never finishes, as will be described later.

[0034] Next, the arithmetic processing unit 4 is composed of a CPU (Central Processing Unit) and the like, and by executing the chip detection program 1a installed in the storage unit 3, it functions as a target area setting unit 41, a drive control unit 42, an image data acquisition unit 43, a mesh size selection unit 44, a mesh division unit 45, a chip information determination unit 46, a cleaning necessity determination unit 47, a cleaning processing unit 48, a teaching processing unit 49, and an additional learning unit 50, as shown in Figure 1. Each component will be described in more detail below.

[0035] The target area setting unit 41 sets the target area in which chips are to be detected. In this embodiment, the target area setting unit 41 sets any target area, such as the entire machine, the table 12, the protector, the area around the workpiece, a part of these areas, or a combination thereof, in response to instructions input by the user using the display input means 2.

[0036] In this embodiment, the target area setting unit 41 sets the target area manually by the user, but this configuration is not limited to this, and it may be set automatically. Specifically, the target area setting unit 41 may refer to the machining program to obtain the shape of the workpiece in each machining process and automatically set the target area to include the entire workpiece, or it may automatically set only the machining path as the target area.

[0037] The drive control unit 42 controls the motors of each axis of the machine tool 10. In this embodiment, the drive control unit 42 generates drive control signals that cause the machine tool 10 to perform cutting operations, etc., based on the machining program, and outputs them to the motors of each axis. Furthermore, when an ATC camera is used as the imaging means 11, the drive control unit 42 controls an automatic tool changer (ATC) (not shown) to change the tool to the ATC camera.

[0038] The image data acquisition unit 43 acquires an image of the entire target area. In this embodiment, the image data acquisition unit 43 outputs a shooting control signal to the shooting means 11 at a preset shooting timing to acquire an image of the entire area. Furthermore, if the shooting means 11 acquires a partial image smaller than the target area, it is instructed to take multiple partial images to cover the target area and then combines them to acquire an image of the entire area.

[0039] The timing of the photo shoot may be manually set by the user, or it may be set automatically according to predetermined settings. These settings may include intervals between processes, tool changes, regular time intervals, or when a predetermined amount of material has been processed. The amount of material processed can be estimated from the processing program, etc.

[0040] The mesh size selection unit 44 selects the mesh size when dividing the image of the entire area into multiple mesh images. In this embodiment, the mesh size selection unit 44 selects one or more mesh sizes as candidates from a set of pre-configured mesh sizes based on predetermined criteria.

[0041] Furthermore, this invention simplifies and improves the accuracy of chip detection by dividing an image of the entire area into a mesh image. For this reason, the mesh size is an important factor that affects the accuracy of chip detection, and it is preferable to consider the following characteristics of chips when selecting the mesh size.

[0042] Specifically, as mentioned above, chips have various variations and patterns, so no two-dimensional chips are completely identical. However, the edges that make up the shape of the chips share the common characteristic of being relatively complex and irregular. Furthermore, this characteristic is not lost even when numerous chips accumulate and overlap, making it impossible to recognize the individual shapes.

[0043] On the other hand, the workpiece and jig that form the background of the chips generally have little irregularity like the chips, and when compared on the same scale as the chips, their edge shapes are simple. Therefore, in this embodiment, it is preferable that the mesh size selection unit 44 selects a mesh size that is sufficient to recognize the shape of the chips based on the sizes of the chips, workpiece and jig, and that simplifies the edge shapes of the workpiece and jig that form the background of the chips in the mesh image. By selecting such a mesh size, it becomes easier to distinguish the edge shapes of the workpiece and jig from the chips, and false detections are reduced.

[0044] The mesh size selection unit 44 is not limited to the above configuration, and the user may manually select from a set of mesh sizes. However, since the chip detection model that determines chip information needs to have a fixed mesh size, if multiple mesh sizes are available, a chip detection model trained on mesh images of each mesh size is required. On the other hand, if the mesh size used is always fixed, there is no need to provide the mesh size selection unit 44.

[0045] The mesh division unit 45 divides the area image into multiple mesh images. In this embodiment, the mesh division unit 45 generates multiple mesh images by dividing the area image acquired by the image data acquisition unit 43 using the mesh size selected by the mesh size selection unit 44. In this embodiment, as shown in Figure 2, a grid-like or lattice-like mesh is used, but it is not limited to this shape, and rhombic, triangular, honeycomb shapes, etc., may also be used. In this embodiment, the entire area image is divided into mesh images, but it is not limited to this, and areas where chip detection is not required may not be divided, and only at least a part of the area image may be divided. Also, the captured image of the area and the grid-like mesh information may be stored separately in association. In other words, the mesh division unit 45 is not limited to generating individual mesh images, but may also superimpose the captured image and grid information and process the area of ​​the captured image divided by the grid as a mesh area. In this case, the mesh area corresponds to the mesh image described above.

[0046] Furthermore, in this embodiment, when the image data acquisition unit 43 synthesizes multiple partial images to acquire an area image, the mesh division unit 45 performs one of the following processes during the division process, as shown in Figure 4. (i) If there is no overlap at the boundary between one sub-image and another sub-image, the boundary of the mesh image is made to match the boundary of each sub-image. (ii) If there is overlap at the boundary between one sub-image and another sub-image, extract a mesh image containing the boundary from only one of the sub-images.

[0047] By having the mesh division unit 45 perform the process described in (i) or (ii) above, multiple different sub-images will not be mixed within the same mesh image. As a result, false detections caused by differences in brightness or positional misalignment between sub-images are reduced.

[0048] The chip information determination unit 46 determines chip information for each mesh image. In this embodiment, the chip information determination unit 46 determines chip information for each mesh image divided by the mesh division unit 45 using determination parameters read from the determination parameter storage unit 32, and stores the determination result in the determination result storage unit 33.

[0049] Furthermore, in this embodiment, the chip information determination unit 46 determines a "class" indicating the ease of cleaning the chips as chip information. This class is determined by comprehensively considering the amount, density, size, length, shape, etc. of the chips. Specifically, as shown in Figure 5, chips that are easy to clean, such as those that are few, scattered, small, short, and not easily caught, are classified as a lower class, while chips that are difficult to clean, such as those that are many, dense, large, long, and easily caught, are classified as a higher class.

[0050] In this embodiment, the chip information determination unit 46 is composed of a chip determination model obtained by applying learning parameters to a learning and inference model, which is a machine learning algorithm. Specifically, it uses a convolutional neural network (CNN), which is a deep learning method that is relatively robust to environmental changes and is particularly specialized for image classification.

[0051] When determining chip information using the chip determination model described above, the chip information determination unit 46 first performs preprocessing on each mesh image to improve determination accuracy, such as noise reduction and image size conversion. Next, as shown in Figure 6, the chip information determination unit 46 inputs the mesh images into a trained chip determination model that has been prepared in advance in the determination parameter storage unit 32 by applying the trained parameters, and outputs the probability of each class as chip information for each mesh image.

[0052] In this embodiment, a convolutional neural network is used as the machine learning algorithm, but it is not limited to this, and other machine learning algorithms may be used. For example, when using a machine learning algorithm that uses non-image input data, such as a support vector machine (SVM), filtering or other processing is performed on the mesh image to calculate the shape features of each mesh image. Then, by inputting these shape features into a trained chip classification model, a class may be determined for each mesh image.

[0053] Furthermore, while this embodiment employs a machine learning algorithm for the chip information determination unit 46, it is not limited to this, and non-machine learning algorithms such as deterministic algorithms may also be used. For example, mesh images tend to become more complex and have more high-frequency image components as the amount of chips increases. Therefore, as shown in Figure 7, the chip information determination unit 46 performs frequency analysis such as a Fast Fourier Transform (FFT) and calculates spectral statistics for each mesh image as features. Then, based on the relationship between these features and a threshold value set in advance as a determination parameter, the class of each mesh image as chip information may be determined.

[0054] However, the Fast Fourier Transform is highly dependent on the environment. If small screw holes or coolant splashes other than chips are captured in the mesh image, the high-frequency components increase, making misidentification more likely. For this reason, it may be possible to prepare an area image in advance when there are no chips, divide the difference image between this and the area image to be judged into a mesh, and then perform a Fast Fourier Transform to eliminate environment-dependent components.

[0055] Furthermore, in this embodiment, the chip information determination unit 46 determines the chip information using a single algorithm, but the configuration is not limited to this, and the determination may be made comprehensively based on the determination results of multiple algorithms.

[0056] If multiple mesh sizes are selected by the mesh size selection unit 44, the chip information determination unit 46 performs a determination for all mesh sizes. The chip information for each mesh size is then stored as a determination result in the determination result storage unit 33.

[0057] The cleaning necessity determination unit 47 determines whether or not chip cleaning is necessary for each mesh image. In this embodiment, the cleaning necessity determination unit 47 compares the cleaning conditions stored in the cleaning condition storage unit 34 with the chip information for each mesh image stored in the determination result storage unit 33. It then determines that cleaning is necessary for mesh images that satisfy the cleaning conditions.

[0058] The cleaning processing unit 48 controls the cleaning means 13 to perform chip cleaning. In this embodiment, the cleaning processing unit 48 calculates the position (actual position within the machine tool 10) of the mesh area corresponding to the mesh image that has been determined to require cleaning, and sprays cleaning fluid from the cleaning means 13 toward that mesh area.

[0059] The location of the mesh area can be determined, for example, by associating an image memory (not shown) that stores the area image with the coordinate system within the machine tool 10. In this embodiment, the cleaning means 13 is configured with a programmable nozzle, so that coolant liquid is sprayed onto all mesh areas that are determined to require cleaning, while changing the spray direction.

[0060] Furthermore, in this embodiment, each time the cleaning unit 48 performs a cleaning operation, it updates the number of consecutive cleanings, which are recorded in the cleaning count storage unit 35, for continuously cleaning the target area, and the number of local consecutive cleanings, which are recorded for continuously cleaning the same mesh area. The cleaning unit 48 then performs cleaning if the number of consecutive cleanings and the number of local consecutive cleanings are below a predetermined threshold. On the other hand, if at least one of the number of consecutive cleanings and the number of local consecutive cleanings reaches a predetermined threshold, it performs a predetermined abnormality processing to avoid a situation where the automatic cleaning operation never finishes.

[0061] The abnormality handling is not particularly limited as long as it is a process to prevent the automatic cleaning operation from never finishing. For example, it may stop the operation of the machine tool 10 and issue an alarm. Alternatively, it may record and notify that a threshold has been reached, and then suspend cleaning of that mesh area for a "cleaning interruption period" set in advance by the user, while continuing to clean other mesh areas and operating the machine tool 10.

[0062] The threshold values ​​for the number of continuous cleaning cycles and the threshold values ​​for the number of localized continuous cleaning cycles can be individually set by the user. Furthermore, while methods for notifying the user of the alarm include, for example, outputting a warning sound from a speaker (not shown), the system is not limited to this method, as long as it can notify the user. Specifically, a message may be displayed on the display input means 2, or a light (not shown) may be flashed or illuminated. Additionally, if there is no user around the machine tool 10, an error message may be sent to the user's mobile device.

[0063] The teaching processing unit 49 confirms and corrects the determination results made by the chip information determination unit 46. In this embodiment, the teaching processing unit 49 first reads the determination result of each mesh image from the determination result storage unit 33 and displays it on the display input means 2 as shown in Figure 8(a). Specifically, different coloring is applied to each different determination result, and each mesh image is displayed in a manner that allows for identification.

[0064] Furthermore, when displaying the judgment results, the chip information judgment unit 46 may calculate the judgment probability for each mesh image, and mesh images for which the judgment probability for any class is not very high may be highlighted as having a high probability of being incorrect (lack of confidence in the judgment result) to assist the user in making corrections. For example, in the example shown in Figure 8(a), the images are classified into the following four groups based on the judgment probability P0 for class 0 (no chips) and the judgment probability P1 for class 1 (with chips).

[0065] P1 is 0-30% (P0 is 70-100%): Class 0 (no chips), "confident" P1 is 30-50% (P0 is 50-70%): "Not confident" P1 is 50-70% (P0 is 30-50%): "Not confident" P1 is 70-100% (P0 is 0-30%): Class 1 (with chips) - "Confident"

[0066] Next, when a user who has confirmed the above judgment result touches and selects an incorrectly judged mesh image, and inputs chip information as the correct judgment result, the teaching processing unit 49 acquires the judgment result and corrects the judgment result stored in the judgment result storage unit 33. Then, as shown in Figure 8(b), the corrected judgment result is displayed on the display input means 2.

[0067] The additional learning unit 50 performs additional learning using new training data. In this embodiment, the additional learning unit 50 obtains learning parameters that reflect the correct judgment results by performing additional learning on the chip judgment model using training data consisting of a set of judgment results (correct chip information) corrected by the teaching processing unit 49 and a mesh image. The learning parameters are then updated in the judgment parameter storage unit 32 as new judgment parameters.

[0068] In this embodiment, the additional learning unit 50 is installed in the local chip detection device 1 and performs additional learning in the background, but the configuration is not limited to this. That is, training data may be uploaded to a learning server on the network for additional learning, and the obtained learning parameters may be downloaded and used as new judgment parameters.

[0069] Furthermore, in this embodiment, two operating modes are provided for the cleaning process by the chip detection device 1: an automatic cleaning mode and a teaching mode, and the device is configured to be able to set to either operating mode. The automatic cleaning mode is a mode in which the cleaning process is automatically executed using the judgment result of the chip information determination unit 46 as is. The teaching mode is a mode in which the cleaning process is executed using the judgment result corrected by the teaching processing unit 49.

[0070] Next, the operation of the chip detection device 1 and the machine tool 10 equipped therewith, as well as the chip detection method, will be described with reference to Figure 9. In the following description, we will focus on the case where the chips are cleaned after detection.

[0071] When detecting chips using the chip detection device 1 and the machine tool 10 equipped therewith according to this embodiment, the determination parameters are stored in the determination parameter storage unit 32 in advance. At this time, one or more sample images of chips are first prepared, and these sample images are divided into predetermined mesh sizes. As a result, a large number of mesh images are acquired as training data for training the chip determination model. Then, the chip determination model is trained with this training data, and the optimized determination parameters are stored in the determination parameter storage unit 32.

[0072] Furthermore, in systems using machine learning, such as the chip detection device 1 of this embodiment, it is necessary to prepare and train a large amount of training data for the learning model in advance before operation. However, the process of preparing this training data is very time-consuming and laborious, which is a barrier to implementation.

[0073] In this respect, the training data used in this embodiment consists of mesh images and their classes. While a large amount of training data is required to obtain a certain level of performance and versatility, the preparation of the training data is easy for the following reasons: (1) Many mesh images, which are the learning units, can be obtained from a single area image. (2) Even with the same area image, different training data can be easily increased by simply performing operations such as shifting or rotating the division boundaries of the mesh image. (3) Object detection algorithms such as Object Detection and Semantic Segmentation require positional information of all chips captured in the area image. In contrast, in this embodiment, by dividing the area image with a mesh size that is sufficient for cleaning, it is not necessary to identify the precise location of the chips within the area image, and it can be reduced to simple class classification. (4) When class classification is used as the chip information, training data is created by simply assigning one numerical value (class) to each mesh image.

[0074] This allows for the input of training data for chip detection in at least specific environments by simply taking a few area images. In other words, optimization can be achieved for individual machines, environments, and processing methods with a relatively small number of teaching operations.

[0075] Next, in order to detect chips, first, the target area setting unit 41 sets the target area in which chips are to be detected (step S1), and then the drive control unit 42 controls the machine tool 10 to process the workpiece (step S2). As a result, as shown in Figure 1, chips are scattered on the table 12 and the workpiece, so the image data acquisition unit 43 acquires an area image of the target area from the shooting means 11 (step S3).

[0076] Once an area image is acquired, the chip detection process according to the present invention is executed (step S4). The chip detection process related to step S4 will be described below with reference to Figure 10.

[0077] In the chip detection process, first, the mesh size selection unit 44 selects a mesh size (step S21). In this embodiment, the mesh size selection unit 44 selects one or more candidates that are likely to be mesh sizes that are large enough to recognize the shape of the chip and that simplify the edge shape of the workpiece or jig that forms the background of the chip. This makes it easier to distinguish the chip from the background and suppresses false detections, thereby improving detection accuracy.

[0078] Next, the mesh division unit 45 divides the area image into multiple mesh images using one of the mesh sizes selected by the mesh size selection unit 44 (step S22). As a result, when using the chip information, it is only necessary to identify the mesh area corresponding to each mesh image, and it is not necessary to accurately identify the location of all chips. Therefore, the amount of computation required for chip detection is significantly reduced.

[0079] Furthermore, in this embodiment, when acquiring an area image by combining multiple partial images, the mesh division unit 45 performs the process described in (i) or (ii) above depending on whether or not there is overlap at the boundary of each partial image. As a result, multiple different partial images are not mixed within the same mesh image. Therefore, false detections caused by differences in brightness or positional misalignment between partial images are reduced.

[0080] Next, the chip information determination unit 46 determines chip information for one mesh image (step S23), and repeats this until determination is completed for all mesh images within the target area (step S24:NO). As a result, it is only necessary to determine chip information for mesh images divided into predetermined mesh sizes, thus simplifying and improving the accuracy of the chip detection process.

[0081] Once the determination is complete for all mesh images within the target area (step S24: YES), the chip information determination unit 46 determines whether the determination has been completed for all mesh sizes within the target area (step S25). If there are any mesh sizes that have not yet been determined (step S25: NO), the process returns to step S21, and the division process (step S22) and determination process (step S23) for those mesh sizes are repeated.

[0082] On the other hand, once the determination for all mesh sizes is complete (step S25: YES), the chip information determination unit 46 stores the determination result for each mesh size in the determination result storage unit 33 (step S26), and terminates this process.

[0083] Next, returning to Figure 9, it is determined whether the operating mode is set to teaching mode (step S5). If it is set to automatic cleaning mode (step S5: NO), the process proceeds to step 7, which will be described later. On the other hand, if it is set to teaching mode (step S5: YES), the teaching processing unit 49 executes the teaching process (step S6). The teaching process related to step S6 will be explained below with reference to Figure 11.

[0084] In the teaching process, first, the teaching processing unit 49 displays the determination result on the display input means 2 (step S31) and accepts the user's selection of a mesh image. At this time, if there are determination results for multiple mesh sizes, the user is allowed to select the optimal mesh size by visual inspection, and by using that mesh size, the accuracy of the chip information determination is improved. Next, it is determined whether or not any mesh image has been selected (step S32), and if no mesh image has been selected (step S32: NO), the selection of a mesh image is accepted unless the teaching mode is terminated (step S35: NO).

[0085] On the other hand, when a user selects any mesh image (step S32: YES), the teaching processing unit 49 prompts the user to input correct chip information for the selected mesh image and obtains this chip information as a determination result (step S33). As a result, the correct determination result, which has been confirmed and corrected by the user's visual inspection, is stored in the determination result storage unit 33 (step S34), and the determination result is displayed on the display input means 2 (step S35).

[0086] Subsequently, when the end of teaching mode is instructed (step S36: YES), the additional learning unit 50 starts additional learning in the background at a predetermined timing, acquires judgment parameters that reflect the corrected judgment results, and updates the judgment parameters in the judgment parameter storage unit 32 with these judgment parameters (step S37). By performing the above teaching process, the accuracy of the judgment results is improved, and the chip judgment performance is optimized by updating the judgment parameters for individual machines, environments, and processing.

[0087] Next, returning to Figure 9, after the teaching process (step S6) is performed, or in the case of automatic cleaning mode (step S5: NO), the cleaning necessity determination unit 47 determines whether or not chip cleaning is necessary (step S7). In this embodiment, the cleaning conditions are set by combining chip information and the position of the mesh area, so the cleaning process is efficiently executed according to the amount and position of the detected chips.

[0088] For example, one could set a cleaning condition that cleans only the mesh area corresponding to the mesh image determined to be Class 2 (high amount of chips) among the three classes mentioned above, or one could set a cleaning condition that does not clean areas outside of table 12. By setting such cleaning conditions, the time required for cleaning and the amount of cleaning solution consumed can be reduced.

[0089] If the determination in step S7 indicates that cleaning is not necessary (step S7: NO), the cleaning unit 48 initializes the number of continuous cleaning cycles and the number of localized continuous cleaning cycles (step S13). Unless otherwise instructed to end the machining process (step S15: NO), the process returns to step S1 and the subsequent steps are repeated. This makes it possible to machine the workpiece with the chips removed.

[0090] On the other hand, if the determination in step S7 indicates that cleaning is necessary (step S7: YES), the cleaning unit 48 determines whether the number of consecutive cleaning cycles has reached a predetermined threshold (step S8). If the determination indicates that the number of consecutive cleaning cycles has reached the threshold (step S8: YES), the cleaning unit 48 executes abnormality processing without performing the cleaning operation (step S14). This prevents cases where chips that were supposed to have been cleaned simply move to another mesh area and are not actually removed, causing the automatic cleaning operation to never finish.

[0091] Furthermore, even if the number of continuous cleaning cycles is less than the threshold as a result of the determination in step S8 (step S8: YES), the cleaning processing unit 48 separately determines whether the number of local continuous cleaning cycles has reached a predetermined threshold (step S9). If the number of local continuous cleaning cycles reaches the threshold as a result of this determination (step S9: YES), the cleaning processing unit 48 executes abnormality processing without performing cleaning operations (step S14). This avoids cases where chips that are difficult to remove by cleaning due to being stuck to or caught on the workpiece are repeatedly cleaned, or cases where screws on the jig or dirt inside the machine are mistakenly detected as chips and the cleaning continues indefinitely.

[0092] On the other hand, if both the number of continuous washes and the number of local continuous washes are below the threshold (step S8: NO, step S9: NO), the washing processing unit 48 calculates the mesh area corresponding to each mesh image that has been determined to require washing (step S10), and washes each mesh area with the washing means 13 (step S11). This ensures that chips are automatically and efficiently washed according to the chip information. Furthermore, since only the mesh areas that require washing are washed, the amount of washing solution sprayed is reduced compared to washing the entire machine, making it economical and environmentally friendly.

[0093] Finally, the cleaning processing unit 48 updates the continuous cleaning count and the local continuous cleaning count stored in the cleaning count storage unit 35 (step S12), then returns to step S3 and repeats the subsequent processing. As a result, as long as the continuous cleaning count and the local continuous cleaning count do not reach the threshold, the chip information is determined and cleaning is performed according to the cleaning conditions.

[0094] In this embodiment, the process ends when the instruction to end processing is given (step S15: YES).

[0095] The chip detection device 1, chip detection program 1a, and chip detection method of this embodiment, as described above, provide the following effects: 1. Chips generated when a workpiece is processed by the machine tool 10 can be detected easily and with high accuracy. 2. Chips can be detected with high accuracy using machine learning. 3. Chips can be easily detected using image features. 4. The chip detection performance can be improved by optimizing the judgment parameters according to the usage environment and individual processing. 5. The influence of the chip background can be reduced, suppressing false detections. 6. False detections can be suppressed when multiple partial images are combined and used as an area image. 7. Chips can be automatically and efficiently cleaned according to the chip information. 8. Cleaning can be done efficiently according to the amount and location of the detected chips. 9. It is possible to prevent the cleaning operation from being repeated indefinitely due to false detection of chips, etc.

[0096] Furthermore, the chip detection device 1 and the machine tool 10 equipped therewith, as well as the chip detection method according to the present invention, are not limited to the embodiments described above and can be modified as appropriate.

[0097] For example, in the embodiment described above, chips are cleaned using chip information, but the configuration is not limited to this, and the chip detection device 1 does not need to have a cleaning function. If a chip cleaning function is not provided, the cleaning condition storage unit 34, the cleaning count storage unit 35, the cleaning necessity determination unit 47, and the cleaning processing unit 48 become unnecessary, and in the flowchart shown in Figure 9, the processes from step S7 to step S14 become unnecessary.

[0098] Furthermore, chip information can be used in various ways beyond simply cleaning chips. For example, if chip information can identify mesh areas where chips tend to accumulate in large quantities or frequently, it can be used as reference information when setting cleaning conditions, such as cleaning those mesh areas early or constantly. Similarly, information on mesh areas where chips tend to accumulate can be used by machine tool vendors when designing machines that are less prone to chip buildup.

[0099] Furthermore, although the chip detection device 1 has a teaching mode in the embodiment described above, this is not an essential configuration. If the teaching mode is not required, there is no need to provide the teaching processing unit 49, and the processes in steps S5 and S6 in the flowchart shown in Figure 9 become unnecessary. Also, the teaching process shown in Figure 11 becomes unnecessary.

[0100] Furthermore, in the embodiment described above, the cleaning operation is controlled using both the number of continuous cleaning cycles and the number of localized continuous cleaning cycles. However, the configuration is not limited to this, and control may be performed using only one of them. In this case, in the flowchart shown in Figure 9, either step S8 or step S9 becomes unnecessary.

[0101] Furthermore, while the above-described embodiment uses learning parameters as determination parameters, the configuration is not limited to this, and thresholds for classifying chip information based on various features of the mesh image may also be used. The above features are not particularly limited as long as they can represent the characteristics of the chip, and various features used in image processing can be used, such as the complexity of edge components in the mesh image, the proportion of linear components in the edge components of the mesh image, the brightness of the mesh image, and luminance statistics.

[0102] The threshold parameter for judgment may be set by optimizing it by calculating the feature score for each mesh image obtained by dividing the sample image and aggregating the scores for each classification of chip information, or by combining multiple features.

[0103] Furthermore, in the embodiment described above, the chip information determines a "class" indicating how easy the chips are to clean, but it is not limited to this, and may also include the presence or absence of chips, the amount of chips, the shape of the chips, the type of chips, or a combination thereof. For example, if the amount and shape of the chips are used as chip information, it is possible to set a cleaning condition that even in mesh areas with a small amount of chips, chips with shapes that are difficult to clean will be cleaned.

[0104] Furthermore, in the above-described embodiment, the chip detection device 1 is configured to allow additional training of new training data locally. However, the configuration is not limited to this, and training data may be transmitted to a learning device connected to the chip detection device 1 via a network for additional training. Alternatively, training data for additional training may be collected from multiple chip detection devices 1 and shared in a learning device to improve versatility.

[0105] Furthermore, training images for deriving the judgment parameters used in the above-described chip detection device 1 may be generated by a training image synthesis device 1200 as shown in Figure 12. The training image synthesis device 1200 includes a chip image storage unit 1201 that stores chip images representing various types of chips, and a background image storage unit 1202 that stores background images captured from various parts of the machine. Furthermore, the training image synthesis device 1200 includes an image generation unit 1203 that generates training images by randomly combining chip images and background images.

[0106] The chip image may be an image of actual chips taken against a plain background. The background image may be an image of the actual interior of the aircraft. The image generation unit 1203 combines the chip images with the aircraft interior photographs, albeit randomly, to make them appear as if they are actually scattered, thereby creating multiple environmental patterns and automatically generating a large amount of training data. This makes it easy to prepare training data.

[0107] Furthermore, the image generation unit 1203 creates scratches and other imperfections on the background image. Prepare as many patterns as possible.

[0108] The training images generated by the training image synthesis device 1200 are provided to the training device 1250 and stored in the training image storage unit 1251. These are then used by the determination parameter derivation unit 1252 to derive the determination parameters. The determination parameters are provided to the chip determination model derivation unit 1253 and used to generate the chip determination model. The learning image synthesis device 1200 may provide the chip determination model to the chip detection device 1 via the network 1260.

[0109] The learning image synthesis device 1200 and the learning device 1250 may be provided within the chip detection device 1, but in that case, the chip detection device 1 will need to be equipped with high-speed resources such as a GPU.

[0110] Furthermore, actual images from inside the machine tool may be sent to the learning image synthesis device 1200 via the network 1260 and stored in the background image storage unit 1202. In this case, it becomes possible to generate more accurate training data (training images).

[0111] The chip image storage unit 1201 preferably stores images of various chips that differ in color, shape, and size. Furthermore, it is preferable that it stores images of chips that have been exposed to coolant and images of chips that have not been exposed to coolant.

[0112] The background images stored in the background image storage unit 1202 are classified into areas with a lot of chips, areas with little chips, and areas with no chips. When combining them with the chip image, the weighting of each area may be changed before combining them.

[0113] The image generation unit 1203 may process the background image by adding effects such as paint peeling, scratches, and stains, and then combine it with the chip image. When combining, the size and rotation of the chip image may also be changed. Preferably, the image generation unit 1203 generates multiple types of training images by changing the overall color, brightness, and lighting of the combined image.

[0114] The image generation unit 1203 prepares and pastes ground truth data indicating the presence or absence of chips for each region of the mesh image during synthesis.

[0115] The training image synthesis device 1200 creates a very large amount of training data by combining background images from various models, various chip images, and varying weights and densities, and then applying various effects (brightness, coolant mist application). The training image synthesis device 1200 may also generate training images classified by model.

[0116] With the above configuration, training data can be generated very efficiently.

[0117] Furthermore, in the above-described embodiment, the chip detection device 1 is implemented as a function of the numerical control device, but the configuration is not limited to this. In other words, the chip detection device 1 may be configured by a computer separate from the numerical control device.

[0118] The above-described embodiments are illustrative and not restrictive in all respects, and can be modified and altered as appropriate by those skilled in the art. The scope of the present invention is defined not by the embodiments described above, but by the claims. Furthermore, the scope of the present invention includes modifications from embodiments within the scope equivalent to the claims.

[0119] This application claims priority based on Japanese Patent Application No. 2019-031127, filed on 25 February 2019, and incorporates all of its disclosures herein.

Claims

[Claim 1] A chip detection device for detecting chips generated when a workpiece is processed by a machine tool, A mesh division unit divides at least a portion of an area image taken of the target area for detecting the chips into multiple identical mesh images with a mesh size that allows for the analysis of the edges constituting the outer shape of individual chips from a state in which many chips are accumulated and overlapping, and simplifies the analysis of the background edge shape. A chip information determination unit performs an analysis on the mesh image, including the analysis of the edges and the analysis of the background, calculates a single feature quantity for the entire mesh image, and determines the chip information for each of the mesh images using a predetermined determination parameter and the feature quantity to determine the chip information relating to the chips. It has, The aforementioned determination parameters are learning parameters obtained by pre-training using training data consisting of a set of mesh images and chip information. The chip information determination unit is a chip detection device that outputs chip information for each of the mesh images of the same size with the mesh size by inputting the mesh image into a chip determination model to which the learned parameters have been applied.

Images