Information processing equipment and machine tools

Abstract

The present invention performs processing of: (i) receiving an image data, dividing a portion of the image data into a plurality of grids including a first grid, a second grid, a third grid, and a fourth grid, and dividing a portion of the image data into a plurality of regions including (a) a first region including the first grid and the third grid and (b) a second region including the second grid and the fourth grid; (ii) detecting an object on the first grid; and (iii) generating, when an object has been detected on the first grid, a signal to control a fluid discharging unit to discharge a fluid such that the fluid flows on the first region including the first grid.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus.

Background Art

[0002] In the above technical field, Patent Document 1 discloses a technique for detecting a location where chips generated during machining adhere and accumulate.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, since the technique described in the above document detects chips with a thermal detector instead of a visual sensor such as a camera, the detection accuracy of chips is not sufficient.

[0005] An object of the present invention is to provide a technique for solving the above problems.

Means for Solving the Problems

[0006] To achieve the above object, the present invention has the configuration described in the claims. [[ID=…]]

Effects of the Invention

[0007] According to the present invention, an object within a machine tool can be accurately detected.

Brief Description of the Drawings

[0008] [Figure 1] It is a block diagram showing the configuration of an information processing apparatus according to the first embodiment. [Figure 2A] It is a block diagram showing the configuration of a machine tool system according to the second embodiment. [Figure 2B] This figure illustrates the contrast index used in the machine tool system according to the second embodiment. [Figure 3] This is a block diagram showing the configuration of the information processing device according to the second embodiment. [Figure 4] This figure illustrates the machine learning of the information processing device according to the second embodiment. [Figure 5] This is a flowchart illustrating the processing flow of the reliability determination unit according to the second embodiment. [Figure 6] This diagram illustrates the chip evacuation process in a machine tool system according to the second embodiment. [Figure 7] This diagram illustrates the chip evacuation process in a machine tool system according to the second embodiment. [Figure 8] This figure shows another example of image data. [Figure 9] This figure shows an example of a score graph screen. [Modes for carrying out the invention]

[0009] The embodiments will be described in detail below with reference to the drawings. However, the components described in the following embodiments are merely illustrative and are not intended to limit the technical scope of the present invention to them alone.

[0010] [First Embodiment] An information processing device 100 as a first embodiment will be described with reference to Figure 1. The information processing device 100 is a device for detecting objects 130 located inside a machine tool 110. Examples of objects 130 include chips, as well as raw material powder and lumps remaining inside the machine tool after additional processing. Objects 130 are objects that are removed by cleaning, and can also be called objects to be removed.

[0011] The information processing device 100 includes a processing unit 101. The processing unit 101 uses image data 120 obtained by imaging the inside of the machine tool 110 to detect an object 130 from inside the machine tool 110.

[0012] The processing unit 101 associates the image data 120 with grid information for dividing it into a plurality of grids including a first grid, a second grid, a third grid, and a fourth grid, and also associates it with area information for dividing the inside of the machine tool into a plurality of areas including a first area (area B) including the first grid and the third grid and a second area (area A) including the second grid and the fourth grid.

[0013] The plurality of grids are used to detect the degree of presence of an object (such as the presence or absence of an object, the amount of an object, the probability of the presence of an object, etc.) within one grid. The plurality of areas are areas divided according to the structure within the machine tool, and are used to discharge fluid for each area to move the object within the area outside the area. Therefore, when the grid information and the area information are overlaid on the image, a plurality of grids are included within one area. For example, in FIG. 1, the first grid and the third grid are included within area B. Area B further includes a plurality of other grids. Also, one aggregate formed by collecting a plurality of grids may be set as one area.

[0014] In the processing unit 101, for example, when an object is detected in the first grid, if the first grid in which the object is detected is included in the first area (area B), a signal is transmitted to the machine tool to instruct it to discharge fluid to move the remaining object on the structure of the machine tool corresponding to the first area (area B).

[0015] According to the above configuration, since an object can be accurately detected from the image obtained by imaging the inside of the machine tool, the object inside the machine tool can be accurately moved. As a result, the time for removing the object within the processing area of the machine tool can be shortened. Furthermore, since the removal operation of the object by a person can be eliminated or the number of removal operations of the object by a person can be reduced, the operating rate of the machine tool can be increased.

[0016] [Second Embodiment] The machine tool system 200 according to the second embodiment will be described with reference to FIGS. 2A to 6. As shown in FIG. 2A, the machine tool system 200 includes an information processing apparatus 201 and a machine tool 202. In the first embodiment, an example in which the information processing apparatus 201 is provided separately from the machine tool 202 in FIG. 1 is shown. However, the information processing apparatus 201 may be mounted inside the machine tool 202. The information processing apparatus 201 may be connected to the machine tool 202 via a network such as a LAN (Local Area Network) or the Internet. The information processing apparatus 201 may be, for example, a server provided on the cloud.

[0017] (Overall system configuration) The machine tool 202 of the present embodiment is a machine for performing machining such as cutting and grinding on a workpiece 221 made of metal, wood, stone, resin, etc. using a tool 223 attached to a tool spindle 222. However, it is not limited to this form, and other machine tools for removal machining or addition machining may be used. Further, the machine tool may be a composite machining machine that combines these functions. The coolant injection unit 224 injects a coolant 225 to cool the tool 223 and the workpiece 221. Further, the coolant injection unit 224 of the present embodiment may inject the coolant 225 to effectively move the object. The object is, for example, the chips 226 generated by cutting the workpiece 221 with the tool 223. Note that instead of a liquid such as coolant, gas such as air may be used as long as it is a fluid. Therefore, instead of the coolant injection unit 224, a fluid injection unit such as an air blow unit that blows air may be used. The fluid injection unit may include a nozzle that can swing in the X and Z directions.

[0018] The machine tool 202 further includes a camera 227, a control unit 228, and a coolant pump 229.

[0019] The camera 227 is an imaging unit that acquires image data by imaging the inside of the machine tool 202. Here, a camera 227 is provided as an example of the imaging unit.

[0020] The information processing device 201 includes a chip processing unit 211, an image reliability determination unit 212, and a notification unit 213.

[0021] The chip processing unit 211 uses image data 210 obtained by imaging the inside of the machine tool 202 (particularly the machining area) with a camera 227 to detect the chips 226 to be removed from inside the machine tool 202. Specifically, the image data 210 is divided into small regions (gridded), and the presence or absence of chips 226, which is the degree of chip presence, is determined for each grid region. In this embodiment, the image is divided into grids, but it is not limited to this. The image data may not be divided, and the image and grid information divided into grids may be associated.

[0022] The chip processing unit 211 associates image data and grid information with region information that divides the image into multiple regions. These multiple regions are areas divided according to the structure within the machine tool 202, such as region A and region B shown in Figure 1. If the region divided by the grid is a small region, then region A and region B can be called medium regions. As shown in Figure 1, region A is the image region including the pallet, and region B is the image region including the bottom of the machining region between the side cover and the pallet. Alternatively, a collection of multiple grids may be divided as a single region. In this case, the diagonal lines dividing the regions superimposed on the image data example shown in Figure 1 will be zigzag lines corresponding to the size of the grid, rather than straight lines.

[0023] The chip processing unit 211 detects the presence of chips. In other words, the chip processing unit 211 detects chips. The presence of chips includes not only the presence or absence of chips 226 within a single grid (small area), but also the size of the chips, the amount of chips, the shape of the chips, the type of chips, and the prevalence rate of chips within a single grid (small area). Information on the size of the chips is one example of the presence of chips, as it presupposes the presence of chips. The chip processing unit 211 may also detect multiple pieces of information, such as the size and type of chips.

[0024] The chip processing unit 211 extracts feature quantities from the image data 210. The feature quantities extracted by the chip processing unit 211 include the information content, frequency components, contrast, and brightness distribution of the captured image. The chip processing unit 211 has the function of improving detection accuracy by machine learning a large number of training images, where the presence or absence of chips is known, using a neural network.

[0025] The control unit 228 controls the coolant pump 229 while simultaneously controlling the camera 227's imaging. For example, the control unit 228 controls the coolant pump 229 to stop the injection of coolant 225, then controls the camera 227 to take an image, and after the image is taken, controls the coolant pump 229 to inject coolant again. Regarding the coolant injection operation, the processing unit 301 generates a signal that commands the release of fluid to move objects remaining on the structure of the machine tool 202 (e.g., a pallet) corresponding to an area (e.g., area A) divided according to the structure of the machine tool 202. When the control unit 228 obtains a fluid release command signal by polling the processing unit 301, it controls the coolant pump 229 to inject coolant 225. The control unit 228 also controls the fluid injection unit, including the nozzle and oscillating mechanism, according to the fluid release command, causing the nozzle to oscillate toward the area to be cleaned. Similarly, in the first embodiment, the processing unit 101 generates a signal that commands the release of fluid to move the object remaining on the structure of the machine tool 110 (for example, the pallet) corresponding to the region (for example, region A) that is divided according to the structure within the machine tool 202.

[0026] Furthermore, the control unit 228 determines whether or not it is time to take a photograph for chip detection, according to the machining program (G-code) or user operation. The control unit 228 may also turn off the coolant pump 229 and take a photograph of the chip at times such as when there is a break in machining during running (when another side of the workpiece is machined), when the tool is changed by the ATC, or when instructed by the user.

[0027] The control unit 228 controls the coolant injection unit 224 based on the location of the chips detected by the chip processing unit 211 (the grid where the chips were detected), the amount of chips within one grid, the amount of chips within one area, etc., to effectively inject coolant. The control unit 228 controls the coolant injection unit 224 to either direct the chips 226 toward the chute 230 and push the chips 226 away with the coolant, or to sweep away the chips 226 with a broom.

[0028] The chip processing unit 211 may make incorrect judgments (inferences) due to the influence of lighting conditions, contrast, mist, etc. If the inference is incorrect, chips may not be removed even if they are present, or conversely, the unit may attempt to remove chips unnecessarily even if there are none.

[0029] Therefore, the image reliability determination unit 212 calculates the image reliability (image confidence) from the image data feature factors that affect the inference result, as an indicator of whether the chip processing unit 211 is performing inference correctly. In other words, the image reliability determination unit 212 determines the image reliability, which indicates whether the image data 210 is unsuitable for detecting chips 226, based on the feature quantities of the image data 210.

[0030] The notification unit 213 notifies the user that the image data 210 is unsuitable for detecting chips 226, based on the determination result of the image reliability determination unit 212. The display control unit 320 performs processing to display image data (for example, the top of Figure 1, Figures 6, 7, and 8) or a screen (for example, Figure 9) on the display unit (touch panel, see Figure 3) of the information processing device 201 or on the display unit 251 of the operation panel 250 installed on the machine tool 202.

[0031] (Image confidence level) The image reliability determination unit 212 uses three indicators as image reliability: brightness, blur, and mist density. Specifically, the image reliability determination unit 212 includes a brightness determination unit 231, a blur determination unit 232, and a mist determination unit 233, and determines whether the estimation accuracy of chip detection has decreased due to the effects of brightness, blur, and mist. Here, the thresholds for brightness determination, blur determination, and mist determination may be changed according to the environment.

[0032] Each image confidence level is calculated based on image characteristic factors such as brightness value, frequency distribution, and contrast, or combinations thereof. The chip processing unit 211 then determines whether it is functioning correctly based on the magnitude of the image confidence level (0 to 1). If the image confidence level of the image data to be judged is low, the inappropriate chip removal behavior can be avoided by not using that image data. If the decrease in image confidence is due to sudden or transient factors, it is expected to improve over time.

[0033] In this embodiment, the above three indicators are used as the image reliability, but the present invention is not limited thereto, and other indicators (for example, sharpness, contrast, number of gradations, noise level, number of blown-out highlights, number of crushed blacks, peak signal-to-noise ratio, etc.) may be used.

[0034] If the image reliability remains low for a certain period, the user will be notified by displaying an alert. This will allow the user to recognize that the chip processing unit 211 is not functioning properly and to attempt to improve the environment. Specifically, the chip processing unit 211 can be made to function normally by performing maintenance on the camera 227, reviewing the shooting conditions, cleaning the lens, changing the lighting, and installing and operating the mist collector.

[0035] (light / dark judgment) The brightness determination unit 231 determines that the image data is unsuitable for chip detection because it is too bright or too dark. Specifically, it monitors the brightness of the image using the luminance value of the image. More specifically, it uses the average value of the luminance value as the brightness index value and determines whether it is greater than or equal to a threshold. If the brightness index value remains less than or greater than the threshold for a certain period of time, it causes the notification unit 213 to notify that the image data is unsuitable for chip detection because it is too bright or too dark. The notification unit 213 provides notifications such as, "The image is too dark, so chip detection is not possible. Please check the operation status of the light," or "The image is too bright, so chip detection is not possible. Please check the camera aperture."

[0036] (Dementia detection) The blur detection unit 232 determines that the image data is unsuitable for chip detection because it is blurred. Specifically, it monitors the degree of blurring of the image using a value normalized within the range of 0 to 1 from the maximum value (maximum drop) of the image data after applying a differential filter (Sobel filter). Alternatively, the blurring may be determined by calculating the OTF (Optical Transfer Function, also known as the response function) of the image data and determining how much contrast can be maintained at what spatial frequency (amplitude characteristics, MTF: Modulation Transfer Function).

[0037] If the indicator of image blur level remains below a predetermined threshold for a certain period of time, the notification unit 213 will notify that the image is unsuitable for detecting the object to be removed because it was out of focus. For example, the notification unit 213 will notify that "The image is blurry, so chip detection is not possible. Please check the camera lens."

[0038] (Mist judgment) The mist determination unit 233 determines that the image captured in that state is unsuitable for chip detection due to the density of the mist generated when the workpiece or tool is cooled by the coolant sprayed into the machine tool. Specifically, the mist detection unit 233 calculates an index of mist density from a combination of the image's brightness value, frequency distribution, and contrast. In particular, in this embodiment, the product of three features, "degree of darkness," "edge amount," and "contrast," is used as the mist index. This is because all of these features decrease as the degree of mist increases. "Degree of darkness," "edge amount," and "contrast" are calculated from the original image data before grid division. If the amount of mist increases, it may be misinterpreted as the absence of chips. Therefore, if the mist index value remains below the threshold for a certain period of time, the notification unit 213 will notify that the concentration of the coolant mist is the reason why it is unsuitable for chip detection. For example, the notification unit 213 will notify, "The mist is too dense to detect chips. Please check the operation status of the mist collector."

[0039] Here, as an example, we define three features as follows: "degree of darkness," "edge amount," and "contrast."

[0040] "Degree of darkness": A value obtained by subtracting the normalized average brightness value per pixel (within the range of 0 to 1) from 1. "Edge amount" = The Sobel max, which is the maximum value (maximum drop) of the image data after applying a differential filter (Sobel filter), normalized to a range of 0 to 1, with the theoretical maximum value being 1140. "Contrast" = A value normalized from 0 to 1, with the amount of work done by contrast correction (luminance histogram equalization) set to a maximum value of 255. The amount of work done in contrast correction (luminance histogram equalization) is the numerical value representing the absence of gaps in the histogram after equalization processing of the luminance histogram (the number of luminance values ​​whose frequency is not zero) (Figure 2B).

[0041] Furthermore, the variance, standard deviation, or mean of the differential filter output can be used as the "edge amount." The variance, standard deviation of the luminance, or Michelson contrast (a value calculated using the maximum and minimum luminance of all pixels) can also be used as the "contrast."

[0042] Alternatively, you could weight the three features—"degree of darkness," "edge size," and "contrast"—before summing them up.

[0043] By performing brightness / contrast detection, blur detection, and mist detection using the methods described above, the influence of image variations can be suppressed, and robust detection results can be obtained.

[0044] Figure 3 is a block diagram illustrating the internal configuration of the information processing device 201. The information processing device 201 in this embodiment includes a processing unit 301, storage 302, and a touch panel 303. However, it is not limited to these. The information processing device only needs to have at least a processing unit 301 for detecting objects. Instead of a display unit 303 such as a touch panel of a tablet terminal, a display unit 251 of a machine tool's control panel 250 may be used. The storage may be provided on an external server, and various data may be provided to the information processing device 201 via a network.

[0045] The processing unit 301 performs various calculations and realizes various functions. The storage unit 302 stores various data and also functions as a working area when the processing unit 301 performs calculations. Specifically, the storage unit 302 stores the chip detection program 321, the judgment parameters 323, the judgment result 324, the cleaning conditions 325 (including cleaning path information according to the number of cleanings and area), the in-machine image 327, the image reliability judgment program 328, and the notification message 329.

[0046] The touch panel 303 combines an input function that accepts instructions from the user and a display function that displays chip detection results, etc.

[0047] The chip detection program 321 uses a chip determination model 322, which is formed by applying determination parameters 323 to a learning and inference model. The determination parameters 323 are parameters for detecting chips, and are obtained by pre-training the learning and inference model using partial images (or image portions corresponding to one grid) and information about chips as training data. The determination parameters 323 are stored for multiple different mesh sizes. In addition, learning parameters that affect the characteristics of the chip determination model 322, such as the learning efficiency, may be set separately in the storage 302.

[0048] As shown in Figure 4, the judgment result 324 is the result of running the chip detection program on a partial image 401 (a portion of the image corresponding to one grid) obtained by dividing the image 327 into a grid. For example, the chip judgment model 322 uses the concept of "class" as the judgment result 324. This "class" is determined by comprehensively considering the amount, density, size, length, shape, etc. of the chips. Specifically, the lower the condition in which the chips are easy to clean, such as having few chips, scattered chips, small chips, short chips, and chips that are difficult to clean, the higher the condition in which the chips are difficult to clean, such as having many chips, crowded chips, large chips, long chips, and chips that are easy to get caught on. For each class, the probability of that class is stored as the judgment result 324. For example, if three classes are set: "Class 0" for no chips, "Class 1" for few chips, and "Class 2" for many chips, the probability P0 of being "Class 0", the probability P1 of being "Class 1", and the probability P2 of being "Class 2" are stored for each partial image.

[0049] Returning to Figure 3, the cleaning conditions 325 include information such as conditions for determining whether chip cleaning is necessary and the number of cleaning cycles. For example, the number of cleaning cycles includes the number of continuous cleaning cycles, which is the number of times the machine interior is cleaned continuously, and the number of local continuous cleaning cycles, which is the number of times each mesh area is cleaned continuously.

[0050] The processing unit 301 is composed of, for example, a CPU (Central Processing Unit). By executing the chip detection program 321, the processing unit 301 can function as a region division unit 311, an imaging control unit 312, a grid division unit 313, a chip processing unit 314, and a coolant control unit 318. Furthermore, by executing the image reliability determination program 328, the processing unit 301 can function as an image reliability determination unit 212 and a notification unit 213.

[0051] The imaging control unit 312 outputs a shooting control signal to the camera 227 at a preset shooting timing, acquires an image 327, and stores it in the storage 302. The shooting timing can be between processes, during tool changes, at regular time intervals, or when a predetermined amount of processing is reached.

[0052] The grid division unit 313 divides the in-machine image data into a grid so that it can be handled as partial images, and associates the image data with grid information. This allows chip detection to be performed for each grid, thus enabling accurate chip detection. The grid division unit 313 divides the image into partial images of a determined size based on the size of the chip, workpiece, and jig. This makes it easier to distinguish between the edge shape of the workpiece and jig and the chip, reducing false detections. The grid is not limited to a checkerboard pattern (a mesh of square holes), but may also be a rhombus, triangle, honeycomb, etc. Furthermore, it is not necessary to divide the entire image into partial images; only the area where chip detection is required may be divided.

[0053] As shown in Figure 1, the region division unit 311 divides the image data into multiple regions, including a first region (region B) and a second region (region A), according to the internal structure of the machine tool. For example, region A is the region of image data that includes images of the pallet, and region B is the region relating to the bottom of the machining region. Regions F and G are regions of image data that include images of the side walls of the machine tool and the opening and closing doors provided on the sides of the machine tool. Chips remaining inside the machine tool include chips remaining on the pallet and chips stuck to the side walls. To efficiently move these, the method, direction, and injection path (cleaning path) of coolant injection should be determined for each region. The method, direction, and injection path (cleaning path) of coolant injection are included in the cleaning conditions 325 and stored in the storage 302.

[0054] By efficiently moving the chips towards the chute, the accumulation of chips in a short time can be reduced. The chips that reach the chute are then discharged and removed to a chip box outside the machine tool via a chip conveyor. For example, region B in Figure 6 is the bottom of the machining area and has a horizontal surface B1 and an inclined surface B2 that slopes inward from the horizontal surface. When chips are detected from the first grid corresponding to the inclined surface B2 within region B, as shown in Figure 6, coolant liquid is first flowed in a straight line over the highest part of the inclined surface B2. The coolant flows along the inclined surface B2 towards the horizontal surface B1, and in the process, it also flows over the inclined surface B2 corresponding to the first grid where the chips were detected. The chips are pushed away by the coolant and reach the horizontal surface B1. After the injection of coolant to the inclined surface B2 is completed, the control unit 228 controls the injection of coolant in a zigzag pattern onto the horizontal surface B1, pushing the chips on the horizontal surface towards the chute. A fluid flow method that drops debris from above to below and then moves the accumulated debris to a chute is efficient when there is not a large amount of debris.

[0055] The chip processing unit 314 performs preprocessing on each partial image to improve the accuracy of the determination, such as noise reduction and image size conversion. Then, for each partial image, the chip processing unit 314 executes the chip detection program 321 to determine the presence or absence of chips using the chip determination model 322, and saves the determination result 324.

[0056] The chip processing unit 314 uses, for example, a convolutional neural network (CNN), a deep learning method that is relatively robust to environmental changes and is particularly specialized for image classification. However, it is not limited to this, and other machine learning algorithms may also be used.

[0057] When using machine learning algorithms such as Support Vector Machines (SVMs), filtering is performed on the partial images to calculate the shape features of each partial image. Then, these shape features are input into a trained chip detection model to determine the class of each partial image. Furthermore, non-machine learning algorithms such as deterministic algorithms may also be used. For example, partial images tend to become more complex and have more high-frequency image components as there is more chipping. Therefore, the chip processing unit 314 may perform frequency analysis such as Fast Fourier Transform (FFT), compare the spectral statistics of each partial image with the detection parameters, and determine the class. Alternatively, an image of the entire area without chipping may be prepared in advance, and the difference image between this and the entire area image to be determined may be meshed and subjected to Fast Fourier Transform to eliminate environment-dependent components.

[0058] The chip processing unit 314 reads the judgment results 324 of each partial image from the storage unit 302 and displays them on the touch panel 303. Specifically, it displays each partial image in a manner that allows the judgment results to be identified (for example, different colors for different judgment results).

[0059] For example, each grid is classified into the following four groups based on the probability P0 of class 0 (no chips) and the probability P1 of class 1 (with chips), and is masked with a different color.

[0060] Group 1: P1 is 0-30% (P0 is 70-100%): Red Group 2: P1 is 30-50% (P0 is 50-70%): Blue Group 3: P1 is 50-70% (P0 is 30-50%): Green Group 4: P1 is 70-100% (P0 is 0-30%): Yellow

[0061] As a result, for example, if the detection probability for chips in the first grid is 0.98, P1 is 98%, and yellow (group 4) will be displayed on that grid. Also, if the detection probability for chips in the second grid is 0 or 12, P1 will be 12%, and red will be displayed on the second grid. Note that although the detection probability for chips in the second grid is 0.12, it may also be assumed that P0 is 88%.

[0062] After reviewing the judgment results from the image, the user can touch and select any incorrectly judged portion of the image and input the correct judgment result (a different color). For example, they can change a portion of the image colored in Group 2 to Group 1, or a portion of the image colored in Group 3 to Group 4. The teaching processing unit 316 corrects the judgment result 324 in the storage 302 in response to the user's input of a change in the judgment result.

[0063] The coolant control unit 318 controls the coolant pump 229 via the control unit 228. Specifically, in observation mode, which captures the internal conditions of the machine and displays them on the touch panel 303, and in chip detection mode, which detects chips with the chip processing unit 211, the coolant pump 229 is turned off. Alternatively, the coolant control unit 318 controls the coolant spray unit 224 via the control unit 228 according to the determination result of the chip processing unit 314, and sprays coolant to the target location. Whether or not to clean each area is determined by whether the amount of deposit in each area exceeds a threshold (set value). The amount of deposit in each area is calculated from the determination result of each grid included in the area. No chips: 0 points Few chips: 3 points Lots of chips: 8 points Example: Image capture and AI judgment result for a given area A (composed of 200 grids): None x165, less x30, more x5 → Deposition score = 30x3 + 8x5 ≈ 130 > Threshold for region A In that case, clean the entire area A. In other words, the processing unit 301 calculates a score related to the probability of chip presence for each of the multiple grids obtained by dividing a portion of the image data. Then, the processing unit 301 sets a region containing a predetermined number of grids and determines whether or not to perform cleaning on a region-by-region basis according to the total score for the entire region. Specifically, if the total score of the regions divided according to the structure of the machine tool 202 exceeds a threshold, the processing unit 301 generates a signal to control the injection of fluid from the fluid injection unit so that fluid flows into that region, and transmits it to the control unit 228 of the machine tool 202. The total score described above (also called "regional accumulation amount" or "accumulation score") is an example of a "score related to the accumulation of objects in a region".

[0064] For example, suppose chips are detected from the first and third grids corresponding to the inclined surface B2 in Figure 6. In this case, the chip processing unit 314 acquires the region information divided by the region division unit 311 and transmits the information that chips were detected in region B to the coolant control unit 318. The coolant control unit 318 sends a signal to the control unit to control the cleaning in region B based on the cleaning conditions, so that the coolant liquid is sprayed linearly on the inclined surface and then in a zigzag pattern on the horizontal surface. The control unit, such as an NC, receives the signal transmitted from the coolant control unit 318 and drives the coolant nozzles based on it to control the direction and amount of coolant discharge. In other words, the coolant control unit 318 sends a signal to the control unit to spray coolant based on the coolant path to be sprayed on the part of the machine tool corresponding to the region, depending on the region. The coolant path is also for washing away chips from the machine tool and is also called the cleaning path. For example, in region F, which includes the side of the machine tool, if chips are detected at the 5th grid, the cleaning path will flow coolant in a straight line horizontally along the side of the machine tool corresponding to the 6th grid, which is a higher position on the side. In this way, cleaning paths are set for each region and stored in storage. For example, in region A, which corresponds to the image region including the pallet, a zigzag cleaning path is set. On the other hand, in regions G and F, such as the side, a straight cleaning path is set.

[0065] Figure 5 is a flowchart showing the processing flow in the image reliability determination unit 212.

[0066] In step S501, upon acquiring image data, the image reliability determination unit 212 performs the brightness index calculation process (S503), the blur index calculation process (S505), and the mist index calculation process (S507) in parallel. Furthermore, in steps S509 to S513, comparisons with each threshold are performed in parallel, and if any index is below the threshold, the corresponding notification message 329 is read from the storage 302 and notified on the touch panel 303 (S515 to S519).

[0067] The above-described embodiment is summarized below. The imaging control unit 312 controls the camera 227 at a predetermined timing to acquire image data. The grid division unit 313 divides the image data into partial images using a grid. The region division unit 311 divides the image data into multiple regions, each composed of a predetermined number of partial images, according to the internal structure of the machine tool. As shown in Figure 7, for example, region A includes the pallet, and region B includes the bottom of the machining region. Coolant cleaning paths are pre-set for each region. For example, a zigzag cleaning path is set for region A. In addition, a zigzag cleaning path is set on the horizontal surface B1 in region B, and a linear cleaning path is set on the inclined surface B2.

[0068] The chip processing unit 314 determines the probability of chip presence for each of the partial images divided by the grid. Furthermore, for each region corresponding to the internal structure of the machine tool, the chip processing unit 314 calculates a total score regarding the accumulation of the object in the entire region based on the probability of chip presence in multiple partial images contained within that region. If the total score regarding accumulation in any region exceeds a threshold, the coolant control unit 318 controls the injection of coolant along the cleaning path of that region. If the total score of region A exceeds the threshold, coolant is injected along the zigzag cleaning path of region A. If the total score of region B exceeds the threshold, coolant is injected along the zigzag cleaning path of the horizontal surface B1, and then along the straight cleaning path of the inclined surface B2.

[0069] Figure 8 shows another example of image data. Figures 1, 6, and 7 show examples of image data taken from above inside a machine tool, but image data taken from diagonally above, as in Figure 8, may also be used. Figure 8 will be used to explain how to set each region. The information processing device sets the fluid flow method for each region, and when the amount of object in a region exceeds a certain amount, it processes the information to move the object according to the set fluid flow method. The amount and rate of accumulation of objects differ depending on the location, such as tables, pallets, side walls, and slopes. Therefore, it is preferable for the information processing device to set a region for each component within the image. However, since it takes time to finely identify components by image processing, it is preferable to divide the image into grids in advance and set multiple grids as a single set of regions in advance. This method of setting regions will be explained in detail below.

[0070] First, as shown in Figure 8, the image is divided into a grid. The image can be a still image, a video, or live video. In Figure 8, the grid is shown as dotted lines, but whether or not to display it is optional. Whether the image data is taken from above or from an oblique angle above, the area corresponding to the internal structure of the machine tool is defined by the selected partial image (hereinafter referred to as "selected partial image"). The area illustrated in Figure 8 includes the first selected partial image 520 and the second selected partial image 530. The first selected partial image 520 is surrounded by the selected partial image 521 adjacent to the top of the first selected partial image 520, the selected partial image 522 adjacent to the right, the selected partial image 523 adjacent to the bottom, and the selected partial image 524 adjacent to the left. Furthermore, the second selected portion image 530 is surrounded by a selection portion image 531 adjacent to the upper side of the second selected portion image 530, a selection portion image 532 adjacent to the right side, and a selection portion image 533 adjacent to the left side of the second selected portion image 530. The region of the embodiment described above includes the first selected portion image 520 and the second selected portion image 530. Also, the region of the example embodiment described above includes the first selected portion image 520 and the four selection portion images adjacent to it, and the second selected portion image 530 and the three selection portion images adjacent to it. In other words, any shape can be extracted from the image data and used as a region. Thus, the selected region is "a region formed by selecting partial images according to the shape inside the machine tool," and is "a region that includes (i) the selected first selected partial image 520, (ii) four selected partial images adjacent to the selected first selected partial image 520, (iii) the selected second selected partial image 530, and (iv) three selected partial images adjacent to the selected second selected partial image 530." If this selected region corresponds to a pallet, for example, it becomes region A in Figure 8 and Figure 1. In this way, by setting a region formed by combining partial images partitioned by a grid, flexible region setting is possible depending on the machine configuration, and the fluid flow method corresponding to the machine structure can be set. The receiving unit of the information processing device 201 accepts the selection of a region including the first selected partial image 520 and the second selected partial image 530.The region data that associates the candidate region with the multiple sub-images contained within that region may be stored in the storage 302, allowing for the simultaneous selection of a region containing multiple sub-images. In this case, the receiving unit of the information processing device 201 accepts the selection of the region to which the selected sub-image belongs, for example, by touch operation on the touch panel 303 (display unit). For example, a region of a size corresponding to the shape inside the machine tool, such as the shape of the pallet 14, can be arbitrarily selected.

[0071] The display control unit 320 of the information processing device 201 may perform a process to display the score graph screen illustrated in Figure 9 on the touch panel 303 (display unit).

[0072] The score graph screen displays the total score of each area for each shooting session in chronological order, as shown in Figure 9. The score graph screen is displayed on the touch panel 303 (display unit). The display control unit 320 instructs the touch panel 303 (display unit) to display the threshold level of the total score on the graph screen that displays the total score for each shooting session in chronological order. The score graph screen also displays a threshold line 402 that indicates the threshold level. A circle mark 404 is displayed for the total score of an area in a session where the threshold is exceeded. The machine tool performs cleaning when the total score of an area exceeds the threshold. In other words, a circle mark 404 indicates that cleaning was performed in that session.

[0073] In this example, the threshold is set to "100," as shown by threshold line 402. For example, the total score of the area from the first to the fourth time does not exceed the threshold, so cleaning is not performed. Consequently, chips increase during this time, and the total score of the area rises. When the total score of the area exceeds the threshold in the fifth time, cleaning is performed. Cleaning reduces the chips, and the total score of the area in the sixth time decreases. In this way, the increase in the total score of the area when cleaning is not performed and the decrease in the total score of the area when cleaning is performed are repeated. The user can easily understand the changes in the chip situation in the area according to the internal structure of the machine tool by looking at the score graph screen.

[0074] Furthermore, according to this embodiment, it is possible to determine whether the image obtained by imaging the inside of the machine tool is suitable for detecting the object to be removed. The reliability of the detection result of the object to be removed can then be judged, and if the reliability is low, means of solving the problem can be considered. Ultimately, this enables more accurate detection of the object to be removed inside the machine tool.

[0075] [others] The present invention may be applied to a system consisting of multiple devices or to a single device. Furthermore, the present invention is also applicable when an information processing program that realizes the functions of the embodiment is supplied to a system or device and executed by a built-in processor. The technical scope of the present invention includes programs installed on a computer to realize the functions of the present invention on a computer, or a medium storing such a program, a server that downloads such a program, and a processor that executes such a program. In particular, at least a non-transitory computer-readable medium storing a program that causes a computer to execute the processing steps included in the above-described embodiment is included in the technical scope of the present invention.

Claims

1. An information processing apparatus for processing image data captured by the imaging unit of a machine tool, which comprises an imaging unit for photographing an object inside the machine tool and a fluid injection unit for injecting a fluid to move the object, (i) Receiving the image data, dividing a portion of the image data into a plurality of grids including a first grid, a second grid, a third grid, and a fourth grid, and dividing a portion of the image data into a plurality of regions that are divided according to the structure of the machine tool, including (a) a first region including the first grid and the third grid and (b) a second region including the second grid and the fourth grid. (ii) The object is detected with respect to the first grid, (iii) Information processing apparatus comprising a processing unit that generates a signal for controlling the injection of fluid from the fluid injection unit so that fluid flows into the first region including the first grid when the object is detected on the first grid.

2. The information processing apparatus according to claim 1, wherein the processing unit generates the signal when the object is detected in the first grid and the score relating to the accumulation of the object in the first region exceeds a threshold.

3. An imaging unit that photographs objects inside a machine tool, A fluid injection unit that injects fluid to move the object, (i) Receiving image data captured by the imaging unit, dividing a portion of the image data into a plurality of grids including a first grid, a second grid, a third grid, and a fourth grid, and dividing a portion of the image data into a plurality of regions divided according to the structure of the machine tool, including (a) a first region including the first grid and the third grid and (b) a second region including the second grid and the fourth grid. (ii) The object is detected with respect to the first grid, (iii) A machine tool comprising a processing unit that generates a signal for controlling the injection of fluid from the fluid injection unit so that fluid flows into the first region including the first grid when the object is detected on the first grid.

4. A processing unit that uses image data obtained by imaging the inside of a machine tool to detect objects to be removed from inside the machine tool, A determination unit analyzes the image data and determines an image confidence level based on the feature quantities of the image data to indicate whether or not the image data is unsuitable for detecting the object to be removed. Based on the determination result of the determination unit, a notification unit notifies that the image data is unsuitable for detecting the object to be removed, Equipped with an information processing device.

5. The determination unit determines that the image data is unsuitable for detecting the object to be removed due to the density of the coolant mist sprayed inside the machine tool, The information processing apparatus according to claim 4, wherein the notification unit notifies that the concentration of the coolant mist is unsuitable for detecting the object to be removed.

6. The information processing apparatus according to claim 4 or 5, wherein the determination unit performs a determination using the brightness, edge, and contrast of the image data.

7. The information processing apparatus according to claim 1, characterized in that the method, direction, and path of fluid injection from the fluid injection unit are determined for each of the plurality of regions.

8. The machine tool according to claim 3, characterized in that the method, direction, and path of fluid injection from the fluid injection unit are determined for each of the plurality of regions.

Images