Program, information processing method, and information processing device
The program analyzes cutting tool blade images using a trained learning model to detect wear and chipping, addressing the inaccuracy and inefficiency of existing methods by employing multiple surface models for precise deterioration state assessment.
Patent Information
- Application Number
- PCT/JP2023/046539
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2023-12-26
- Publication Date
- 2024-12-05
AI Technical Summary
Current methods for determining the deterioration state of cutting tool edges in the manufacturing industry are inaccurate and labor-intensive, relying on visual inspections or image processing techniques that may fail to detect wear areas properly.
A program and information processing method that uses a trained learning model to analyze captured images of cutting tool blades, employing multiple models for different surface areas (top, diagonal, outer periphery, and rake face) to detect wear and chipping with high accuracy by aggregating results from original and inverted images.
Accurately determines the deterioration state of cutting tool blades with reduced human error and increased efficiency, enabling timely replacement and improving machining accuracy.
Smart Images

Figure JP2023046539_05122024_PF_FP_ABST
Abstract
Description
Program, information processing method, and information processing device
[0001] The present invention relates to a program, an information processing method, and an information processing device.
[0002] At manufacturing sites, managing the tools used in the machining process is an important task. For example, in cutting tools used in cutting processes, the chipping and wear of the cutting edge affect the machining accuracy, so the deterioration state of the cutting edge must be appropriately determined and replaced with a new one as necessary. Currently, in many sites, skilled workers visually check the deterioration state of the cutting edge of the cutting tool and determine whether replacement with a new one is necessary. Furthermore, Patent Document 1 discloses a technology for analyzing the wear state of a tool by photographing the cutting edge of the tool at different angles after cutting and analyzing the image using image processing.
[0003] JP 2023-9980 A
[0004] In the technology disclosed in Patent Document 1, multiple image files are captured during one rotation around the tool, and each image file is binarized to detect wear areas from the images, and the amount of wear is measured from the detected wear areas. In Patent Document 1, the coating at the worn area peels off, exposing the internal metal and causing it to glow white, and the white areas in the binarized images are detected as wear areas. However, depending on the tool, there are cases where the wear area cannot be detected properly.
[0005] In one aspect, an object is to provide a program or the like that can accurately detect the deterioration state of a tool.
[0006] A program according to one aspect acquires an image of a cutting tool blade, inputs the acquired image into a learning model that has been trained to output information regarding the deterioration state of the cutting tool blade when the image of the cutting tool blade is input, and causes a computer to execute a process of acquiring information regarding the deterioration state of the cutting tool blade in the image.
[0007] In one aspect, the deterioration state of a tool can be detected with high accuracy.
[0008] 1 is a block diagram showing an example of the configuration of a tool management system. FIG. 1 is an explanatory diagram showing an example of the appearance of a tool to be photographed. FIG. 2 is an explanatory diagram showing an example of the appearance of a tool to be photographed. FIG. 3 is an explanatory diagram of the photographing direction. FIG. 4 is an explanatory diagram of the photographing direction. FIG. 5 is an explanatory diagram of the photographing direction. FIG. 6 is an explanatory diagram of an example of a record layout of a tool DB. FIG. 7 is an explanatory diagram of an example of the configuration of a top surface model. FIG. 8 is an explanatory diagram of an example of the configuration of a diagonal model. FIG. 9 is an explanatory diagram of an example of the configuration of a periphery model. FIG. 10 is an explanatory diagram of an example of the configuration of a rake face model. A flowchart showing an example of a deterioration state determination processing procedure. A flowchart showing an example of a deterioration state determination processing procedure. A diagram showing an example of a screen. A diagram showing an example of the appearance of a photographing device. A diagram showing an example of the configuration of a photographing device. A diagram showing an example of the configuration of a photographing device. A diagram showing an example of a record layout of a photographing condition DB. A flowchart showing an example of a photographing processing procedure by a photographing device. A flowchart showing an example of a photographing processing procedure by a photographing device of a third embodiment. A flowchart showing an example of a photographing condition adjustment processing. A flowchart showing an example of a photographing condition registration processing procedure. A diagram showing an example of a photographing condition registration processing. A flowchart showing an example of a deteriorated area estimation processing procedure. A diagram showing an explanatory diagram of a deteriorated area size estimation processing. A diagram showing a deteriorated area size estimation processing. 13 is an explanatory diagram illustrating an example of a record layout of a discrimination threshold DB. FIG. 14 is a flowchart illustrating an example of a degradation area estimation process procedure according to a sixth embodiment.
[0009] The program, information processing method, and information processing device of the present disclosure will be specifically described below with reference to drawings illustrating embodiments thereof. In the following embodiments, a process for determining the deterioration state of the blade of an end mill, which is a cutting tool used in cutting processing, will be described as an example. However, the object to be determined is not limited to an end mill, and may be a cutting tool such as a drill, a tap, or a milling cutter, or may be a grinding tool used in grinding processing. Furthermore, among cutting tools, an indexable tool (throw-away tool) whose cutting edge (tip) is detachable from the body (handle) may also be the object to be determined. Note that when an indexable tool is the object to be determined, the captured image used to determine the deterioration state of the cutting edge may be an image captured with the cutting edge detached from the body, or may be an image captured with the cutting edge attached to the body.
[0010] First Embodiment FIG. 1 is a block diagram showing an example of the configuration of a tool management system. In this embodiment, a tool management system is described that determines the deterioration state of a cutting tool blade based on an image of the cutting tool blade captured by an imaging device 20. The tool management system of this embodiment includes an imaging device 20 that captures images of the cutting tool blade, and a server 10, and the imaging device 20 and the server 10 are communicatively connected. The imaging device 20 and the server 10 may be configured to communicate directly via wired communication via a cable or wireless communication, or may be configured to communicate via a network. The network here may be the Internet or a public telephone network, or may be a local area network (LAN) constructed within the facility where the imaging device 20 and the server 10 are installed.
[0011] The server 10 is an information processing device capable of various information processing and information transmission / reception, and is composed of a server computer, a personal computer, or the like. The server 10 includes a control unit 11, a memory unit 12, a communication unit 13, an input unit 14, a display unit 15, a reading unit 16, and the like, which are interconnected via a bus. The control unit 11 includes one or more processors, such as a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), a general-purpose computing on graphics processing unit (GPGPU), a tensor processing unit (TPU), or an AI chip (AI semiconductor). The control unit 11 executes a program P stored in the memory unit 12 as appropriate to perform information processing and control processing to be performed by the server 10. Note that if the control unit 11 includes multiple processors, each process may be executed by a different processor.
[0012] The storage unit 12 includes RAM (Random Access Memory), flash memory, a hard disk, an SSD (Solid State Drive), etc. The storage unit 12 stores a program P (program product) executed by the control unit 11 and various data. The storage unit 12 also temporarily stores data generated when the control unit 11 executes the program P. The storage unit 12 also stores a learning model that has learned training data through, for example, machine learning. The learning model is expected to be used as a program module constituting artificial intelligence software. The learning model performs a predetermined calculation on input values and outputs the calculation result. The storage unit 12 stores data such as coefficients and thresholds of functions that define this calculation as the learning model. In this embodiment, the storage unit 12 stores the following learning models: an upper surface model M1, an oblique model M2, a periphery model M3, and a rake face model M4 (described later). The storage unit 12 also stores a tool DB 12a (described later). A part of the storage unit 12 may be another storage device connected to the server 10 or another storage device with which the server 10 can communicate.
[0013] The communication unit 13 is a communication module for performing processes related to wired or wireless communication, and transmits and receives information to and from other devices. In this embodiment, the communication unit 13 may be configured to communicate directly with the imaging device 20, or may be configured to communicate via a network. The input unit 14 accepts operation input by a user and sends a control signal corresponding to the operation content to the control unit 11. The display unit 15 is a liquid crystal display, an organic EL display, or the like, and displays various information in accordance with instructions from the control unit 11. The input unit 14 and the display unit 15 may be a touch panel configured as an integrated unit.
[0014] The reading unit 16 reads information stored in a portable storage medium 10a such as a CD (Compact Disc), a DVD (Digital Versatile Disc), a USB (Universal Serial Bus) memory, or an SD (Secure Digital) card. The program P and various data stored in the storage unit 12 may be read by the control unit 11 from the portable storage medium 10a via the reading unit 16 and stored in the storage unit 12. The program P and various data may also be written to the storage unit 12 during the manufacturing stage of the server 10, or may be downloaded by the control unit 11 from another device via the communication unit 13 and stored in the storage unit 12.
[0015] In this embodiment, the server 10 may be a multi-computer consisting of multiple computers, a virtual machine virtually constructed by software within a single device, or a cloud server. The input unit 14 and the display unit 15 are not essential for the server 10, and the server 10 may be configured to accept operations via a connected computer or to output information to be displayed to an external display device. The program P may be executed on a single computer or on multiple computers interconnected via a network.
[0016] The photographing device 20 is an information processing device having a photographing function, and includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, a display unit 25, a photographing unit 26, a photographing direction control unit 27, etc., and these units are connected via a bus. The control unit 21, storage unit 22, communication unit 23, input unit 24, and display unit 25 of the photographing device 20 each have the same configuration as the control unit 11, storage unit 12, communication unit 13, input unit 14, and display unit 15 of the server 10, and therefore a description of the configuration will be omitted.
[0017] The photographing unit 26 includes an imaging element such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) and a lens unit. The photographing unit 26 is a so-called industrial camera, and is a high-resolution camera with an imaging element of approximately 2 million pixels (e.g., 1600 pixels x 1200 pixels) for monochrome photography and approximately 12 million pixels (4024 pixels x 3036 pixels x 3 colors) for color photography. The photographing unit 26 also includes a light source unit including a light source such as an LED (light-emitting diode) and an illumination lens. The photographing unit 26 photographs the tool to be photographed using the illumination light emitted by the light source unit to acquire the photographed image. The light emitted by the light source unit is, for example, white light, but may also be light of a predetermined wavelength, such as green light or near-infrared light. The image captured by the photographing unit 26 may be a color image or a black-and-white image. The photographing unit 26 photographs the tool according to instructions from the control unit 21 and acquires, for example, one piece of image data (still image). The image data (photographed image) acquired by the photographing unit 26 is stored in the storage unit 22 .
[0018] 2A and 2B are explanatory diagrams showing an example of the appearance of a tool to be photographed. FIG. 2A shows a side view of the tool 30 from the lateral direction, and FIG. 2B shows the tip surface of the cutting portion 31 of the tool 30 as viewed from the tip side (top in FIG. 2A ). In this embodiment, an end mill as shown in FIGS. 2A and 2B is the object to be photographed by the photographing device 20. While FIGS. 2A and 2B show an example of a square end mill, the object to be photographed may also be a ball end mill, a radius end mill, a tapered end mill, or the like. The tool 30 has, for example, a cutting portion 31 and a shank 32 (handle portion) integrally formed, and the cutting portion 31 has multiple cutting edges 31a. While the tool 30 in FIGS. 2A and 2B has four cutting edges 31a, the tool 30 may have two to six cutting edges 31a.
[0019] The imaging direction control unit 27 controls the imaging direction of the imaging unit 26 relative to the tool to be imaged in accordance with instructions from the control unit 21. FIGS. 3A to 3C are explanatory diagrams of imaging directions. In this embodiment, as shown in FIGS. 3A to 3C, the tool 30 is fixed to a base installed in a predetermined position with the cutting portion 31 facing upward, and the position of the imaging unit 26 relative to the cutting portion 31 of the tool 30 is changed to image the cutting portion 31 of the tool 30 from multiple imaging directions. In this embodiment, as shown in FIG. 3A, the tip surface of the cutting portion 31 is imaged from above to obtain a top surface image; as shown in FIG. 3B, each cutting edge 31a is imaged from an obliquely upward direction to obtain an oblique image; and as shown in FIG. 3C, the side surface of each cutting edge 31a is imaged from the side to obtain an outer periphery image and a rake face image. The top surface image includes an entire top surface image (image of the tip surface of the cutting edge 31) captured in a state including all cutting edges 31a, and individual top surface images (images of the tip surface of each cutting edge 31a) captured individually. The oblique image is captured, for example, from a direction tilted by approximately 5 to 30 degrees from the tip of the blade 31a to be captured, with the top of the tool 30 as the reference point. The angle of the capturing direction can be changed by an operation input via the input unit 24, and is set depending on the type of tool 30 to be captured and the shape of the blade 31a, etc. The outer periphery image is an image captured with the focus adjusted on the flank face of the blade 31a to be captured (the area surrounded by the dashed line in FIG. 3C), and the rake face image is an image captured with the focus adjusted on the rake face of the blade 31a to be captured (the area surrounded by the dashed line in FIG. 3C). The rake face of the blade 31a is the surface that comes into contact with the chip, and the flank face of the blade 31a is the surface of the tool 30 opposite the rake face. The outer peripheral image and the cutting face image can be photographed from the same photographing direction relative to the tool 30, but in the outer peripheral image, the blade 31a photographed at the center of the image (the blade surrounded by the dashed line in Figure 3C) is the blade to be photographed, and in the cutting face image, the blade 31a photographed on the left side of the image (the blade surrounded by the dotted line in Figure 3C) is the blade to be photographed.
[0020] The base is configured to rotate around the axial direction of the tool 30 (the up-down direction in FIGS. 3A to 3C ). When capturing individual top surface images, oblique images, peripheral images, and rake face images, the imaging direction control unit 27 rotates the base by a predetermined angle. This allows each blade 31 a of the tool 30 to be moved within the imaging range of the imaging unit 26, allowing imaging of each blade 31 a. When the imaging target is a four-blade tool 30 as shown in FIGS. 3A to 3C , the imaging direction control unit 27 rotates the base by 90 degrees. Note that in addition to changing the imaging direction relative to the tool 30 (blade 31 a) to be imaged by rotating the base or the tool 30 mounted on the base, the blade 31 a to be imaged may also be changed by moving the imaging unit 26 around the tool 30 (blade 31 a).
[0021] In the tool management system configured as described above, the photographing direction of the tool to be photographed is pre-registered in the photographing device 20. When the photographing device 20 starts photographing a tool, the control unit 21 executes photographing by the photographing unit 26 while sequentially switching the photographing direction of the photographing unit 26 to the registered photographing directions using the photographing direction control unit 27. For example, the control unit 21 photographs one overall top surface image from above the tool as shown in FIG. 3A, then photographs individual top surface images of each blade while rotating the base as shown in FIG. 3B, then photographs oblique images of each blade from diagonally above the tool while rotating the base as shown in FIG. 3C, and finally photographs the outer periphery image and rake face image of each blade from the side of the tool while rotating the base and further adjusting the focus as shown in FIG. 3C. This allows multiple photographed images of the tool blade portion and each blade to be obtained from multiple pre-registered photographing directions.
[0022] The control unit 21 can identify information about the blade being photographed and the photographing direction relative to the blade for the captured image obtained by the photographing unit 26 based on the result of photographing direction control by the photographing direction control unit 27. Therefore, the control unit 21 adds information indicating the identified content to the captured image. For example, the control unit 21 may assign a file name reflecting the identified content to the captured image. This allows the blade being photographed and the photographing direction for each captured image to be identified by the file name. The control unit 21 may also add the date and time of photographing to the captured image as the photographing date and time. For example, the blade being photographed may be, for example, a blade at a predetermined position on a tool fixed to a base, designated as the first blade, and the blades being photographed according to the rotation direction of the tool relative to the base may be designated as the second to fourth blades. Furthermore, when the tool is fixed in a predetermined orientation relative to the base, the assignment of the first to fourth blades to each blade is always the same. The photographing device 20 transmits the photographed image, to which a file name reflecting information about the blade being photographed and the photographing direction relative to the blade, to the server 10 for registration. The server 10 of this embodiment performs a process of determining the deterioration state of each blade of a tool based on an image of the tool captured by the imaging device 20. The server 10 of this embodiment uses models M1 to M4 when determining the deterioration state of each blade of the tool in the captured image from the captured image of the tool.
[0023] In the tool management system of this embodiment, the photographing device 20 having the above-described configuration can be equipped with an industrial camera capable of taking high-resolution photographs, like a general tool presetter, and can use a device that can photograph the blade of the tool being photographed by switching between multiple photographing directions.
[0024] FIG. 4 is an explanatory diagram showing an example of a record layout of the tool DB 12a. The tool DB 12a is a database that stores information about tools managed by the tool management system. The tool DB 12a includes, for example, a tool ID column, a tool information column, a photographing date and time column, a photographed object column, a top surface image column, a slant image column, a periphery image column, a rake face image column, and a comprehensive judgment result column. The tool DB 12a stores information about each tool and images photographed by the photographing device 20 in association with a tool ID uniquely assigned to each tool. The tool information column stores information about the tool, including, for example, the tool type, manufacturer, model number, and usage history. The photographing date and time column stores the date and time when the photograph was taken by the photographing device 20. The photographed object column stores information about the blade of the photographed object. In this embodiment, a four-blade tool 30 is photographed, so the information about the blade of the photographed object indicates the entire tool and the first to fourth blades. The top surface image sequence stores information about top surface images (top surface overall images and top surface individual images) captured by the imaging device 20, associated with the date and time of capture and the subject being captured. Specifically, the top surface image sequence includes an original image sequence, a label image sequence, and a determination result sequence. The original image sequence stores top surface images (top surface overall images and top surface individual images) that are original images captured by the imaging device 20, and the label image sequence stores label images obtained by a degradation area determination process for the original images (top surface images). In addition to being stored in the tool DB 12a, the image data of the original images and label images may be stored in a predetermined area of the storage unit 12 or an external storage device connected to the server 10. In this case, the original image sequence and the label image sequence store information for reading the image data of the original images and label images (e.g., folder names and file names indicating the storage locations of the data). Note that the label image sequence may store information indicating the degradation area determination results (context, which is output data of models M1 to M4) instead of label images to which information indicating the degradation area determination results for the original images has been added. The determination result column stores the determination result of whether or not there is a deteriorated area based on the label image. In this embodiment, the deterioration state of the blade is determined by determining whether or not there is a worn area or a chipped area, so the determination result column stores "worn," "chipped," or "normal."
[0025] Like the top surface image sequence, the oblique image sequence, the outer periphery image sequence, and the rake face image sequence each include an original image sequence, a label image sequence, and a determination result sequence, and store information on the oblique images, outer periphery images, and rake face images captured by the imaging device 20 in association with the date and time of capture and the subject of capture. Note that the subjects of the top surface images are the entire blade portion and the first to fourth blades, so the top surface image sequence stores information on the top surface images (original images, label images, and determination results) of the entire blade portion and the first to fourth blades. Also, the subjects of the oblique images, outer periphery images, and rake face images are the first to fourth blades, so the oblique image sequence, outer periphery image sequence, and rake face image sequence store information on the oblique images, outer periphery images, and rake face images of the first to fourth blades. The overall judgment result column stores the judgment result of the deterioration state of each blade of the tool, which is judged comprehensively from the judgment results based on each photographed image of each blade of the tool, in association with the photographing date and time and the photographed object. The tool DB 12a is not limited to the configuration shown in Fig. 4, and various information related to the tool and various information related to the photographed images of the tool may be stored.
[0026] FIG. 5A is an explanatory diagram showing an example of the configuration of the top surface model M1, FIG. 5B is an explanatory diagram showing an example of the configuration of the oblique model M2, FIG. 5C is an explanatory diagram showing an example of the configuration of the outer periphery model M3, and FIG. 5D is an explanatory diagram showing an example of the configuration of the rake face model M4. The top surface model M1 is trained to input top surface images (top surface overall image and top surface individual images) of a tool, perform calculations based on the input top surface image to recognize deteriorated areas of the blades in the top surface image as objects to be detected, and output the recognition results. The deteriorated areas of the blades, which are objects to be detected, include worn areas and chipped (missing) areas on each blade. The top surface model M1 is a model that realizes multi-label classification to recognize multiple types of objects (wear areas and chipped areas) in the top surface image. The top surface model M1 is configured, for example, using EfficientDet. The upper surface model M1 is not limited to EfficientDet and can be configured using object detection algorithms such as YOLO (You Only Look Once), R-CNN (Regions with Convolution Neural Network), and SSD (Single Shot Multibox Detector), or can be configured by combining multiple algorithms. The upper surface model M1 may also be a model that recognizes objects (worn areas, missing areas) in the input upper surface image on a pixel-by-pixel basis using semantic segmentation. In this case, the upper surface model M1 can be configured using algorithms such as U-Net, FCN (Fully Convolutional Network), and SegNet.
[0027] The top surface model M1 has an input layer to which a top surface image is input, a middle layer that extracts features from the input top surface image, and an output layer that outputs a context indicating a deteriorated area of the blade detected in the top surface image based on the calculation results of the middle layer. The middle layer calculates an output value based on the top surface image input from the input layer using various functions, thresholds, etc. The output layer outputs a context including a discrimination label indicating the type of detected object (worn area or chipped area), coordinate information indicating the area of the detected object (deteriorated area), and a confidence level for the discrimination label. With this configuration, when a top surface image is input, the top surface model M1 outputs a context (information regarding the deterioration state of the blade) for the object (deteriorated area of the blade) detected in the top surface image, including coordinate information indicating the area of the object, the object name (worn or chipped), and a confidence level. Therefore, by using the upper surface model M1, it is possible to obtain an image (label image) in which a bounding box (a rectangle indicated by a dashed line in FIG. 5A) indicating the area of the object (a deteriorated area of the blade) detected in the upper surface image, the name of the object in the recognition result (worn or chipped), and the confidence level for the recognition result are added. In FIGS. 5A to 5D, the labeled images are shown as output data of models M1 to M4.
[0028] The top surface model M1 can be generated by machine learning using training data including a training top surface image (plain top surface image) and a correct context including coordinate information indicating the area of an object (degraded area) in the top surface image and the object name (correct label). The training data is generated by associating a context (correct context) generated from the result of annotator adding a bounding box and object name (correct label) to the object (degraded area) in the top surface image with the top surface image.
[0029] The top surface model M1 learns to output the correct context included in the training data when a top-view image included in the training data is input. During the learning process, the top surface model M1 performs calculations in the intermediate layer and output layer based on the input top-view image, and calculates the context to be output from the output layer. The top surface model M1 compares the calculated context with the correct context and optimizes parameters used in the calculation processes in the intermediate layer and output layer so that the two approximate each other. Specifically, the top surface model M1 compares the area (degraded area), discrimination label (type of degraded area), and confidence level of the detected object indicated by the calculated context with the area of the object indicated by the correct context and a value corresponding to the correct label (1 for the correct label, 0 for other labels), and optimizes the two so that they approximate each other. The parameters include weights (coupling coefficients) between nodes in the intermediate layer and output layer, and methods such as backpropagation and steepest descent can be used to optimize the parameters. This results in a top surface model M1 that, when a top surface image is input, detects degraded areas and the type of degraded area in the top surface image and outputs a context indicating the detection results (coordinate information indicating the degraded area, a discrimination label indicating the type of degraded area, and the confidence level for the discrimination label).
[0030] The top surface model M1 can be further optimized by repeatedly performing the learning process using the training data as described above. In addition, there are multiple types of tools and tool cutting parts, and the top surface model M1 has been trained using training data of top surface images of one or multiple types of tools, so that the top surface model M1 for each type of tool can be generated by fine-tuning the top surface model M1 through the learning process described above using training data of top surface images of other types of tools.
[0031] The oblique model M2, the outer periphery model M3, and the rake face model M4 have the same configuration as the top surface model M1 and are generated by the same learning process. The oblique model M2 is trained to receive an oblique image of a tool as input, perform a calculation to recognize the deteriorated area of the cutting edge (the object to be detected) in the oblique image based on the input oblique image, and output the recognition result. The outer periphery model M3 is trained to receive an outer periphery image of a tool as input, perform a calculation to recognize the deteriorated area of the cutting edge (the object to be detected) in the outer periphery image based on the input outer periphery image, and output the recognition result. The rake face model M4 is trained to receive an image of a tool's cutting face as input, perform a calculation to recognize the deteriorated area of the cutting edge (the object to be detected) in the rake face image based on the input rake face image, and output the recognition result. The oblique model M2, the outer periphery model M3, and the rake face model M4 are also models that realize multi-label classification that recognizes multiple types of objects (wear areas and chipped areas).
[0032] The models M1 to M4 configured as described above may be trained on another training device. The trained models M1 to M4 generated by training on the other training device are downloaded from the training device to the server 10 via a network or a portable storage medium 10a, for example, and stored in the storage unit 12.
[0033] The following describes the process of determining the deterioration state of a tool cutting portion based on an image of the tool captured by the imaging device 20 in the tool management system of this embodiment. Figures 6 and 7 are flowcharts showing an example of the deterioration state determination process procedure, and Figure 8 is an explanatory diagram showing an example screen. The following process is executed by the control unit 11 of the server 10 in accordance with the program P stored in the storage unit 12. The deterioration area estimation process shown in Figure 7 is the process of step S16 in Figure 6.
[0034] The control unit 11 of the server 10 acquires images of one tool taken from at least one of the imaging directions shown in Figures 3A to 3C (S11). For example, the control unit 11 acquires from the imaging device 20 one overall top surface image, individual top surface images, oblique images, peripheral images, and rake face images taken for each cutting edge. The control unit 11 acquires images taken from imaging directions corresponding to the tool being imaged. The imaging device 20 may start imaging the tool in response to instructions from a user via the input unit 24, in response to instructions from the server 10 via the communication unit 23, or automatically when the tool is attached to a predetermined position on the imaging device 20. Images captured by the imaging device 20 may be temporarily stored in a separate server or storage device. In this case, the control unit 11 acquires the images from the separate server or storage device. The control unit 11 stores the acquired images in the storage unit 12.
[0035] The control unit 11 acquires the photographed date and time of the photographed image together with the photographed image, and stores the acquired photographed date and time in the tool DB 12a in association with the tool ID of the tool to be photographed (S12). Note that if the control unit 11 cannot acquire the photographed date and time together with the photographed image, it may store the date and time at this time as the photographed date and time in the tool DB 12a.
[0036] The control unit 11 reads one of the acquired captured images from the storage unit 12 (S13). The control unit 11 identifies the blade of the captured image and the capture direction for the read captured image (S14). Here, if the file name of the captured image includes information about the blade of the captured image and the capture direction relative to the blade, the control unit 11 identifies the blade of the captured image and the capture direction from the file name. Note that if the file name of the captured image does not include information about the blade of the captured image and the capture direction relative to the blade, the control unit 11 may identify the blade of the captured image and the capture direction based on the captured image. For example, if there are four individual top surface images, the control unit 11 may identify the captured image of each image as the first to fourth blades. The same applies to the oblique image, outer periphery image, and rake face image. In this case, the blades of the captured image identified as the first to fourth blades do not have to match in each of the individual top surface images, oblique image, outer periphery image, and rake face image. Furthermore, if the correspondence between the blades of the subject in each image can be identified based on the shape of the blade portion or each blade, it is possible to match each of the blades of the subject in each image, which are designated as the first blade to the fourth blade.
[0037] The shooting direction of the captured image may be determined by template matching using a template image prepared in advance, which indicates image features of the captured image of the tool taken from each shooting direction. Furthermore, the process of determining the shooting direction of the captured image may be performed by image recognition using a learning model, in addition to template matching. For example, when a captured image of a tool is input, a trained learning model may be used to perform a calculation to determine the shooting direction of the tool in the input captured image and output the calculation result. In this case, the control unit 11 inputs the captured image to the learning model and can determine the shooting direction of the tool in the captured image based on output information from the learning model. With this configuration, even if the blade of the captured object and the shooting direction relative to the blade cannot be determined from the file name of the captured image, etc., the blade of the captured object and the shooting direction of the blade can be identified in the captured image.
[0038] The control unit 11 identifies (selects) a model from the models M1 to M4 that corresponds to the identified shooting direction (S15). Here, if the identified shooting direction is the direction of the top image, the control unit 11 identifies the top surface model M1. If the identified shooting direction is the direction of the oblique image, the control unit 11 identifies the oblique model M2. Furthermore, if the identified shooting direction is the direction of the outer periphery image, the control unit 11 identifies the outer periphery model M3. If the identified shooting direction is the direction of the rake face image, the control unit 11 identifies the rake face model M4. The control unit 11 reads the identified model from the storage unit 12, and performs a process of estimating a deteriorated area on the blade of the subject captured in the captured image based on the read model and the captured image read in step S13 (S16).
[0039] The deteriorated area estimation process shown in FIG. 7 uses TTA (Test Time Augmentation). Specifically, the control unit 11 generates a horizontally inverted image by horizontally inverting (flipping left and right) each pixel of the captured image (S31). The control unit 11 inputs the captured image, which is the original image, into the model identified in step S15 and acquires the context (information regarding the deterioration state of the blade or each blade) output from the model (S32). The context includes coordinate information indicating the position of the deteriorated area detected in the original image (captured image), a discrimination label indicating the type of the deteriorated area (worn or chipped), and a confidence level for the discrimination label. Similarly, the control unit 11 inputs the generated horizontally inverted image into the model identified in step S15 and acquires the context output from the model (S33).
[0040] The control unit 11 generates a labeled image that aggregates the estimation results for the two images (the original image and the horizontally flipped image) based on the context generated from the original image and the context generated from the horizontally flipped image (S34). For example, the control unit 11 generates a labeled image (hereinafter referred to as a first labeled image) for the original image based on the context generated from the original image, in which a bounding box surrounding a degraded area detected in the original image, a discrimination label indicating the type of the degraded area (wear or chip), and a confidence level for the discrimination label are assigned. The control unit 11 also generates a labeled image for the horizontally flipped image based on the context generated from the horizontally flipped image, in which a bounding box surrounding a degraded area detected in the horizontally flipped image, a discrimination label indicating the type of the degraded area, and a confidence level for the discrimination label are assigned. The control unit 11 then horizontally flips (left-right flips) the labeled image generated from the horizontally flipped image to generate a second labeled image, and generates a labeled image that aggregates the first labeled image and the second labeled image using WBF (Weighted Boxes Fusion). For example, the control unit 11 calculates the average value of the confidence level for the discrimination label in each of the bounding boxes in the first and second labeled images. In this case, a confidence level of 0 is assigned to each discrimination label in the areas of the labeled images other than the bounding box areas, so that a small confidence level is calculated for a bounding box (degraded area) that is detected in only a small number of labeled images.
[0041] Furthermore, if the size of a degraded area (bounding box) detected in each labeled image is equal to or larger than a predetermined value, the control unit 11 may exclude the bounding box from the detection results as a false positive. For example, if the horizontal length (number of pixels) of the bounding box is equal to or larger than one-third of the horizontal length (number of pixels) of the image, or if the vertical length (number of pixels) of the bounding box is equal to or larger than one-third of the vertical length (number of pixels) of the image, the control unit 11 may exclude the bounding box from the detection results as a false positive. Furthermore, as shown in FIG. 3C , the rake face of each blade is often captured in the left half of the captured image (rake face image). Therefore, if a degraded area (bounding box) detected in the rake face image is located in the right half of the image, the control unit 11 may exclude the bounding box from the detection results as a false positive. For example, in a coordinate system in which the upper left corner of the rake face image is the origin, the X axis is to the right, and the Y axis is to the bottom, the control unit 11 may identify the coordinate value (x, y) of the upper left pixel of the bounding box, and if the identified x coordinate value is to the right (larger) than the x coordinate value of the center of the image, the bounding box may be excluded from the detection result. Regarding captured images other than the rake face image, if the area in which the subject is captured is limited to a predetermined area, the bounding box detected in the captured image may be excluded from the detection result if it is outside the predetermined area. These processes may be configured to be performed appropriately taking into account the capture conditions, such as the capture direction of the subject.
[0042] The control unit 11 determines whether or not the captured image contains a degraded area based on the label image generated in step S34 (S35). For example, the control unit 11 determines whether or not the captured image contains a degraded area based on whether or not the generated label image contains a bounding box whose confidence level for the discrimination label is equal to or greater than a predetermined threshold. The control unit 11 then returns to the processing of FIG. 6. In the processing of FIG. 7, the control unit 11 uses TTA to perform estimation processing on two images: the captured image of the original image and a horizontally inverted image generated from the original screen, and combines the estimation results to obtain a highly accurate label image.
[0043] The control unit 11 stores the captured image (original image) read in step S13, the label image generated in step S34, and the determination result of the presence or absence of a deteriorated area determined in step S35 in the tool DB 12a in association with the capture date and time and the captured object identified in step S14 (S17). The control unit 11 determines whether or not there are any unprocessed captured images among the captured images acquired in step S11 that have not yet undergone the above-described degradation area estimation process (S18). If it is determined that there are unprocessed captured images (S18: YES), the control unit 11 returns to step S13 and performs the processes of steps S13 to S17 on the unprocessed captured images. The control unit 11 performs processing of steps S13 to S17 for each of the top surface overall image, top surface individual image, oblique image, outer periphery image, and cutting face image, and stores the original image for each image, the label image generated using models M1 to M4, and the judgment result based on the label image in the tool DB 12a, corresponding to the tool ID, shooting date and time, and photographed object.
[0044] If it is determined that there are no unprocessed captured images (S18: NO), the control unit 11 identifies a comprehensive assessment result for each blade that is the subject of the capture from the assessment results based on each captured image stored in the tool DB 12a (S19). For example, the control unit 11 calculates the percentage of captured images (individual top surface images, oblique images, outer periphery images, and rake face images) that are determined to have a deteriorated area (wear area or chipped area) for each blade that is the subject of the capture, and if the calculated percentage is equal to or greater than a predetermined value, determines that the blade that is the subject of the capture is in a deteriorated state. Note that the control unit 11 may also determine that the blade that is the subject of the capture is in a deteriorated state if it determines that a deteriorated area is present in at least one image.
[0045] The control unit 11 stores the comprehensive evaluation results determined for each blade in the tool DB 12a and displays them on the display unit 15 (S20). The control unit 11 displays an evaluation result screen, such as that shown in FIG. 8 . The screen shown in FIG. 8 displays the captured image in which a deteriorated area of the blade of the captured tool has been detected, in association with the tool ID of the captured tool, the capture date and time, and the blade information of the captured tool. The deteriorated areas (wear areas and chipped areas) detected in the captured image are indicated by rectangles. While the screen shown in FIG. 8 displays a rake face image as the captured image, all captured images in which a deteriorated area of the blade of the captured tool has been detected may also be displayed. Furthermore, the screen shown in FIG. 8 indicates chipped areas by dashed rectangles and worn areas by dashed rectangles. In this way, the detected deteriorated areas may be displayed in a manner corresponding to the type of deterioration state (wear or chipping). This allows the user to easily determine the deterioration state of the deteriorated areas detected in the captured image. 8 displays an enlarged view of the detected deteriorated area, and in the enlarged view, the chipped area and the worn area are indicated by a dashed rectangle and a dashed-dotted rectangle, respectively. Note that step S19 does not necessarily have to be executed in the above-described process, and in step S20, the control unit 11 may display a determination result screen showing the deteriorated area (wear area and chipped area) detected in the captured image for which it has been determined that a deteriorated area is present based on the determination result stored in the tool DB 12a in step S17.
[0046] In this embodiment, the deterioration of each blade is detected by image recognition using models M1 to M4 in the captured image of the tool blade captured by the imaging device 20, thereby reducing the workload of assessing the deterioration state of each blade. Conventionally, the deterioration state of tool blades is often visually inspected by skilled craftsmen, who view the tool from various angles to assess the deterioration state, which requires time and can lead to individual differences in the assessment of worn and chipped areas. Furthermore, some tools have blades with a diameter of approximately 6 mm, making visual inspection difficult. However, in this embodiment, the deterioration state of the photographed blade is automatically detected by image recognition using models M1 to M4, allowing for accurate assessment of the deterioration state in a short time. Furthermore, tools and blades determined to be deteriorated by image recognition using models M1 to M4 may be inspected by skilled craftsmen. In this case, the workload of the skilled craftsmen can be reduced and the skilled craftsmen can perform the deterioration state assessment process. Accurate assessment of the deterioration state of tool blades allows for timely tool replacement and smooth cutting.
[0047] In this embodiment, the deterioration state of each cutting edge of a tool is determined based on the captured images (top surface overall image, top surface individual image, oblique image, outer periphery image, and rake face image) obtained by capturing each cutting edge from multiple imaging directions. Therefore, depending on the location of a deteriorated area, it may not be captured in one of the images. However, by making a determination based on these multiple captured images, it is possible to prevent overlooking of the deteriorated area. Furthermore, in this embodiment, models M1 to M4 (top surface model M1, oblique model M2, outer periphery model M3, and rake face model M4) are prepared for each imaging direction, allowing for highly accurate detection of deteriorated areas in images captured from each imaging direction. In addition to a configuration in which models M1 to M4 are generated by training images captured from each imaging direction, a configuration in which a single model is generated by training images captured from all imaging directions may also be used. In this case, a single model can be used to detect deteriorated areas in images captured from all imaging directions.
[0048] In this embodiment, the process of detecting a degraded area in a captured image by image recognition using the models M1 to M4 is not limited to being performed by the server 10, but may be performed locally by, for example, the image capture device 20. In this case, the models M1 to M4 may be downloaded to the storage unit 22 of the image capture device 20, and after the image capture device 20 captures an image of a tool, the image capture device 20 may perform a process of detecting a degraded area in the captured image using the models M1 to M4. Even with such a configuration, it is possible to perform the same process as in the above-described embodiment, and the same effects can be obtained.
[0049] Second Embodiment A specific example of the photographing unit 26 and photographing direction control unit 27 of the photographing device 20 in the tool management system of the first embodiment will be described. FIG. 9 is an explanatory diagram showing an example of the appearance of the photographing device 20, and FIGS. 10A and 10B are explanatory diagrams showing an example of the configuration of the photographing device 20. Note that FIG. 9 shows an example of the appearance of the photographing unit 26 and photographing direction control unit 27, FIG. 10A shows the photographing unit 26 and photographing unit holding member 27c as viewed from the left side in the Y-axis direction in FIG. 9, and FIG. 10B shows the photographing unit 26 as viewed from the right side in the X-axis direction in FIG. 9. The photographing device 20 of this embodiment shown in FIGS. 9 to 10B is Tool Collage (registered trademark), an imaging unit manufactured by Mori Machinery Corporation.
[0050] The imaging direction control unit 27 has a base 27a that holds the tool 30 to be imaged with the blade portion 31 facing upward, and a base holding member 27b that holds the base 27a, on a top plate 27g of the housing of the imaging device 20. The base holding member 27b has a mechanism for moving the base 27a in the Y-axis direction indicated by arrow B on a horizontal plane (on the top plate 27g) and a mechanism for rotating or pivoting the base 27a in the θ-axis direction indicated by arrow E. The θ-axis direction is a rotation direction centered on the central axis (vertical direction) of the tool 30 fixed to the base 27a. The imaging direction control unit 27 also has an imaging unit holding member 27c that holds the imaging unit 26, a vertical member 27d that extends vertically from the upper surface of the top plate 27g, and a connecting member 27e that connects the vertical member 27d and the imaging unit holding member 27c. One end of the connecting member 27e is engaged with the vertical member 27d, and the vertical member 27d has a mechanism for moving the connecting member 27e up and down in the Z-axis direction (vertical direction) indicated by arrow C when engaged. The photographing unit holding member 27c is an L-shaped plate member, and the photographing unit 26 is attached to one end thereof, while the other end thereof is connected to the other end of the connecting member 27e. The other end of the connecting member 27e has a rotating member 27f that rotates the photographing unit holding member 27c when connected about an axis (rotation axis) parallel to the Y-axis. The connecting member 27e also has a mechanism for extending and retracting from its engagement position with the vertical member 27d in the X-axis direction indicated by arrow A, which is perpendicular to the Y-axis direction on a horizontal plane. This extension and retraction mechanism allows the rotating member 27f to move away from or towards the engagement position with the vertical member 27d.
[0051] With the above-described configuration, the imaging direction control unit 27 can move the imaging unit holding member 27c connected to the rotating member 27f along the X-axis direction using the extension / contraction mechanism of the connecting member 27e, thereby moving the imaging unit 26 held by the imaging unit holding member 27c along the X-axis direction. The imaging direction control unit 27 can also move the connecting member 27e along the Z-axis direction using the vertical member 27d, thereby moving the imaging unit 26 held by the imaging unit holding member 27c along the Z-axis direction. The imaging direction control unit 27 can also move the base 27a along the Y-axis direction using the base holding member 27b, and rotate or pivot it along the θ-axis direction. As shown in FIG. 10A , the imaging direction control unit 27 can also pivot the imaging unit holding member 27c around a pivot axis using the pivot member 27f. Accordingly, the photographing unit 26 held by the photographing unit holding member 27c can be moved (rotated) along the α-axis direction indicated by arrow D to any position between a position where the photographing direction is parallel to the X-axis direction (to the right of the X-axis) and a position where the photographing direction is parallel to the Z-axis direction (vertically downward). Fig. 10A shows a state in which the photographing unit 26 is disposed at a photographing position where the photographing direction is to the right, a photographing position where the photographing direction is diagonally downward to the right, and a photographing position where the photographing direction is directly downward.
[0052] As described above, the imaging direction control unit 27 can move the imaging unit 26 along the X-axis direction, the Z-axis direction, and the α-axis direction, and can also move the tool 30 fixed to the base 27a along the Y-axis direction and rotate or turn along the θ-axis direction. Since the positions of the imaging unit 26 and the tool 30 can be moved in this way, the imaging direction control unit 27 can appropriately change the imaging position and imaging direction of the imaging unit 26 with respect to the imaging target (tool 30).
[0053] 10B , the photographing unit 26 includes a camera 26 a and eight light source units 26 b arranged to surround the camera 26 a. The light source units 26 b are, for example, LEDs, and the eight light source units 26 b are configured to be individually controlled to be turned on and off and to control the amount of light (brightness). The number of light source units 26 b is not limited to eight.
[0054] In the photographing device 20 of this embodiment, it is possible to set in advance the camera conditions for the camera 26a and the lighting conditions for the light source unit 26b as photographing conditions for photographing each of the top surface image, oblique image, outer periphery image, and rake face image for each type of tool 30 to be photographed (e.g., model number of the tool 30). Once the photographing conditions are set, photographing processing is performed for the same type of tool (tool with the same model number) under the set camera conditions and lighting conditions, and the set images (top surface image, oblique image, outer periphery image, and rake face image) can be photographed. The camera conditions include information on the photographing position, photographing direction, and focus adjustment of the camera 26a with respect to the tool 30 fixed to the pedestal 27a, and the lighting conditions include the light intensity of each light source unit 26b.
[0055] FIG. 11 is an explanatory diagram showing an example of a record layout of the photographing condition DB. The photographing condition DB is a database that stores camera conditions and lighting conditions when photographing tools managed by the tool management system using the photographing device 20, and is stored, for example, in the storage unit 22. The photographing condition DB includes, for example, a model number column, a manufacturer name column, a type column, a top surface image column, a diagonal image column, a periphery image column, and a rake face image column, and stores information about each tool and the camera conditions and lighting conditions when photographing using the photographing device 20, in association with the model number uniquely assigned to each tool. The top surface image column stores photographing conditions when photographing top surface images (top surface overall image and top surface individual image) using the photographing device 20. Specifically, the top surface image column includes a photographing target column, a camera condition column, and a lighting condition column, and stores the camera conditions and lighting conditions when photographing each photographing target. The camera conditions include the position and shooting direction of the camera 26a relative to the cutting edge 31 of the tool 30 fixed to the base 27a. Specifically, the camera conditions include the X-axis, Z-axis, and α-axis positions of the shooting unit 26 (camera 26a) relative to a preset initial position, and the Y-axis and θ-axis positions of the shooting unit 26 relative to the base 27a relative to a preset initial position. The camera conditions also include information regarding focus adjustment of the camera 26a. The lighting conditions include the light intensity (including on and off) for each of the eight light sources 26b. Like the top surface image sequence, the oblique image sequence, the outer periphery image sequence, and the rake face image sequence each include a shooting target sequence, a camera condition sequence, and a lighting condition sequence, and store the camera conditions and lighting conditions used when shooting each target. The shooting condition DB is not limited to the configuration shown in FIG. 11 and may store various information related to the tool and various information related to shooting the tool.
[0056] Each piece of information registered in the photographing condition DB is stored, for example, in accordance with an input operation via the input unit 24. Specifically, when registering photographing conditions (camera conditions and lighting conditions) for an unregistered tool 30, the registration staff controls the photographing direction control unit 27 and the photographing unit 26 by operating the input unit 24, photographs the tool 30 fixed to the pedestal 27a using the photographing unit 26, checks the photographed image, and stores the conditions under which the tool 30 was photographed under appropriate photographing conditions in the photographing condition DB. The appropriate photographing conditions refer to image quality that suppresses halation occurring in the photographed image and allows the deterioration state of the cutting edge 31 in the photographed image (specifically, the presence or absence of worn areas and chipped areas) to be accurately determined using the models M1 to M4. By registering the photographing conditions for photographing each tool 30 in the photographing condition DB, the photographer can photograph various images of a registered tool 30 simply by fixing the tool 30 to the pedestal 27a.
[0057] FIG. 12 is a flowchart showing an example of a photographing process procedure by the photographing device 20. The control unit 21 of the photographing device 20 executes the following process according to a program stored in the storage unit 22. After the tool 30 to be photographed is fixed to the base 27a, the photographing device 20 executes the following process when, for example, an instruction to start the photographing process is received via the input unit 24. The control unit 21 of the photographing device 20 first identifies the model number of the tool 30 to be photographed (S41). The model number of the tool 30 may be received, for example, via input by the photographer via the input unit 24, or may be identified by template matching using a template image that indicates image features of the photographed image of each tool, or may be identified by image recognition using a learning model. When identifying the model number by image recognition, for example, when a photographed image of a tool is input, a trained learning model can be used to perform a calculation to determine the model number of the tool in the input photographed image and output the calculation result. In this case, the control unit 11 inputs the photographed image into the learning model and can identify the model number of the tool in the photographed image based on output information from the learning model. In addition, a code (e.g., a barcode or a QR code (registered trademark)) encoding the model number of the tool 30 may be printed or attached to the tool 30 or a holder that holds the tool 30, and the model number of the tool 30 being photographed may be obtained by reading this code using a code reading unit. Note that the tool information identified in step S41 is not limited to the model number as long as it is information that can identify the shape, size, etc. of the tool.
[0058] The control unit 21 of the photographing device 20 reads the camera conditions and lighting conditions corresponding to the identified model number from the photographing condition DB (S42). Specifically, the control unit 21 reads the camera conditions and lighting conditions for each image (top view image, oblique image, outer periphery image, and rake face image) associated with the model number and registered in the photographing condition DB. The control unit 21 determines whether the photographing conditions corresponding to the model number have been registered based on whether the camera conditions and lighting conditions have been read (S43). Specifically, if the control unit 21 has been able to read the camera conditions and lighting conditions, it determines that the photographing conditions corresponding to the model number have been registered. If the control unit 21 has not been able to read the camera conditions and lighting conditions, it determines that the photographing conditions corresponding to the model number have not been registered. If it determines that the photographing conditions corresponding to the model number have not been registered (S43: NO), the control unit 21 executes a registration process for the photographing conditions corresponding to the model number identified in step S41 (S44). For example, the control unit 21 displays a message on the display unit 25 notifying the user that the photographing conditions for the tool to be photographed have not been registered, and prompts the user to execute the registration process for the photographing conditions. The control unit 21 then controls the operation of the photographing direction control unit 27 and the photographing unit 26 in accordance with operations via the input unit 24, photographs the tool to be photographed while changing the camera conditions and lighting conditions, and when photographing conditions that allow the tool to be photographed in appropriate photographing conditions are specified, registers the specified photographing conditions in the photographing condition DB. If it is determined that the photographing conditions corresponding to the model number have already been registered (S43: YES), the control unit 21 skips step S44.
[0059] The control unit 21 extracts one photographing condition from the photographing conditions read out in step S42 or the photographing conditions registered in the photographing condition DB in step S44 (S45). For example, the control unit 21 extracts camera conditions and lighting conditions corresponding to one photographing target in one photographed image (top view image, oblique image, outer peripheral image, or rake face image). The control unit 21 executes photographing processing under the extracted camera conditions and lighting conditions (S46). The control unit 21 adds the photographing date and time (e.g., the current date and time) and photographed image information to the photographed photographed image and stores the image in the storage unit 22 (S47). For example, the control unit 21 stores the photographing date and time and photographed image information as tag information (additional information) of the photographed image. The photographed image information includes, for example, information on the blade of the photographed target and information on the photographing direction.
[0060] The control unit 21 determines whether or not image capture under all of the image capture conditions read in step S42 or registered in the image capture condition DB in step S44 has been completed (S48). If it determines that image capture has not been completed (S48: NO), the control unit 21 returns to step S45 and repeats steps S45 to S47 for the image capture conditions that have not been completed. This allows the control unit 21 to execute image capture under the camera and lighting conditions registered in association with the model number of the tool 30, thereby obtaining images captured from the appropriate image capture direction for each tool. If it determines that image capture under all image capture conditions has been completed (S48: YES), the control unit 21 creates a folder in the storage unit 22 for the tool 30 to be captured and stores the images stored in the storage unit 22 in step S47 for the tool 30 in the created folder (S49). This allows images to be stored in a folder for each tool 30 to be captured, allowing the images in the folder to be treated as images of the same tool 30. The control unit 21 may not perform step S49.
[0061] By the above-described processing, in this embodiment, the positional relationship between the tool 30 to be photographed, which is fixed to the base 27a, and the photographing unit 26 (camera 26a) can be controlled in the X-axis, Y-axis, Z-axis, α-axis, and θ-axis directions, enabling photographing from the optimal photographing position for each blade to be photographed. Furthermore, in this embodiment, the focus of the camera 26a can be adjusted and the light intensity of each of the eight light sources 26b can be controlled individually, allowing high-precision photographed images (top view image, oblique image, outer peripheral image, rake face image) to be acquired under optimal camera and lighting conditions for each blade. By detecting the deteriorated area of each blade through image recognition using models M1 to M4 in these photographed images, it becomes possible to more accurately determine the deterioration state of each blade.
[0062] This embodiment also provides the same effects as those of the first embodiment. Furthermore, this embodiment can acquire highly accurate images using the imaging device 20, so that the deterioration state of each blade of the imaging target can be determined more accurately based on the captured images. The modifications described in the first embodiment can also be applied to this embodiment as appropriate.
[0063] (Embodiment 3) In the imaging device 20 of the first and second embodiments described above, the shank 32 of the tool 30 is held in a holder (not shown) and fixed to the base 27a for imaging, as well as for storing and managing the tool 30. Meanwhile, the tool 30 is removed from the holder and reassembled to install a tool 30 of the same type. The mounting position of the tool 30 relative to the holder may shift before and after reassembly. If the mounting position of the tool 30 shifts, even if the tool 30 is of the same type, it may not be photographed in the same state when photographed under the same imaging conditions. Therefore, in this embodiment, a tool management system is described in which, when photographing a tool 30, if imaging conditions for the tool 30 have already been registered, it is determined whether it is possible to photograph the tool in the same state as when the imaging conditions were registered, and if not, the imaging conditions are updated (re-registered). The tool management system of this embodiment can be implemented using devices similar to those of the tool management systems of the first and second embodiments, and therefore a description of the configuration of each device is omitted. In this embodiment, for example, an image of each tool 30 photographed under stored photographing conditions (camera conditions and lighting conditions) is stored as a reference image in a photographing condition DB (see FIG. 11 ) stored in the storage unit 22 of the photographing device 20. The reference image may be photographed and stored for each photographing direction, or only reference images photographed in some photographing directions, such as a periphery image or a rake face image, may be stored.
[0064] Fig. 13 is a flowchart showing an example of the photographing processing procedure by the photographing device 20 of the third embodiment, and Fig. 14 is an explanatory diagram of the photographing condition adjustment processing. The processing shown in Fig. 13 is the processing shown in Fig. 12 with steps S51 to S56 added between YES in step S43 and step S45. Explanation of the same steps as in Fig. 12 will be omitted.
[0065] In this embodiment, when the control unit 21 of the imaging device 20 determines that the imaging conditions corresponding to the model number of the tool 30 to be imaged have been registered (S43: YES), it acquires a reference image of the outer periphery image of this tool 30 (the tool 30 with the same model number) from the imaging condition DB (S51). An example of the reference image of the outer periphery image is shown on the left side of FIG. 14 . Then, the control unit 21 identifies the imaging conditions for the outer periphery image from the imaging conditions read in step S42, and acquires the outer periphery image of the tool 30 to be imaged by capturing the image under the identified imaging conditions (S52). An example of a captured outer periphery image is shown on the right side of FIG. 14 . From the two outer periphery images shown in FIG. 14 , it can be seen that the mounting position of the tool 30 to be imaged relative to the holder is shifted compared to when the imaging conditions were registered (when the reference image was captured). Specifically, the tool 30 in the captured image is shifted downward in the Z-axis direction (see FIG. 9 ) and leftward in the θ-axis direction (see FIG. 9 ) relative to the tool in the reference image.
[0066] Therefore, in this embodiment, when there is a discrepancy between the position of the tool in the reference image and the position of the tool 30 in the captured image, the image capturing device 20 automatically adjusts the image capturing conditions to register appropriate image capturing conditions for the tool 30 currently held in the holder. First, the control unit 21 identifies the position of the blade 31a of the tool 30 being captured in the reference image acquired in step S51 and the outer periphery image (captured image) acquired in step S52 (S53). Specifically, the control unit 21 performs binarization processing on each of the reference image and the outer periphery image, and performs contour detection on the binary image, for example, by detecting the contour of a region where pixels with a pixel value of 1 are continuous, thereby identifying the edge position of the cutting edge of the blade 31a captured in the center of each image. In the reference image of FIG. 14, edge position E1 is identified, and in the outer periphery image, edge position E2 is identified. Furthermore, the control unit 21 identifies the top end surface of the tool in each image based on the reference image and the outer periphery image (captured image). The top end surface indicated by the dashed line in FIG. 14 is identified. The edge position E1 and the position of the upper end surface of the tool in the reference image may be detected when the photographing conditions are registered and stored in the photographing condition DB.
[0067] The control unit 21 determines whether the detected edge positions E1, E2 and the tool top surface position match in the reference image and the outer periphery image (captured image) (S54). Specifically, the control unit 21 determines whether the deviation between the edge position E1 in the reference image and the edge position E2 in the outer periphery image is less than a predetermined amount, and whether the deviation between the tool top surface in the reference image and the tool top surface in the outer periphery image is less than a predetermined amount. The control unit 21 then determines that the two match if both deviations are less than the predetermined amount, and determines that the two do not match if at least one deviation is equal to or greater than the predetermined amount. If the control unit 21 determines that the two match (S54: YES), there is no deviation between the tool position in the reference image and the tool 30 in the captured image. Therefore, capturing under the registered capturing conditions is possible, and the control unit 21 proceeds to step S45. The control unit 21 then extracts one capturing condition from the capturing conditions read out in step S42 (S45) and executes a capturing process under the extracted capturing condition (camera conditions and lighting conditions) (S46).
[0068] If it is determined that the edge positions E1, E2 or the tool upper end surface position do not match (S54: NO), a discrepancy exists between the tool position in the reference image and the tool 30 position in the captured image, and therefore adjustment (updating) of the registered capture conditions is necessary. Therefore, the control unit 21 calculates an adjustment amount for the capture conditions read out in step S42 (S55). Here, the control unit 21 calculates an adjustment amount so that the blade position in the captured image approaches the blade position in the reference image. Specifically, the control unit 21 calculates a rotation amount (adjustment amount) for rotating the base 27a holding the tool 30 in the θ-axis direction based on the edge position E1 in the reference image and the edge position E2 in the captured image. The adjustment amount in the θ-axis direction is the rotation amount required to match (bring closer) the edge position E2 to the edge position E1 in the captured image shown in FIG. 14 . This can be determined, for example, by pre-registering rotation amounts corresponding to the distances in the captured image. The adjustment amount in the θ-axis direction may be determined by, for example, rotating the base 27a a predetermined amount in the θ-axis direction, repeatedly photographing the tool 30 and comparing it with the edge position E1 in the reference image, and summing the rotation amounts until the edge position E2 in the photographed image matches the edge position E1 in the reference image. Furthermore, the control unit 21 calculates an adjustment amount for lowering the photographing unit 26 a predetermined amount downward in the Z-axis direction based on the top surface position in the reference image and the top surface position in the photographed image. The adjustment amount here is the amount of movement required to match (bring closer to) the top surface position of the photographed image to the top surface position of the reference image in the image of FIG. 14 .
[0069] The control unit 21 executes an update process for the imaging conditions stored in the imaging condition DB corresponding to the model number identified in step S41 based on the calculated adjustment amounts for the imaging conditions (S56). For example, if the adjustment amounts for the imaging conditions for the outer periphery image are calculated as described above by rotating the base 27a in the θ-axis direction and lowering the imaging unit 26 in the Z-axis direction, the control unit 21 updates (changes) the camera conditions for the outer periphery image stored in the imaging condition DB based on the calculated adjustment amounts. Note that the adjustment amount for lowering the imaging unit 26 in the Z-axis direction affects not only the imaging conditions for the outer periphery image but also the imaging conditions for other images. Therefore, the camera conditions for other images (the position of the imaging unit 26 in the Z-axis direction) may also be updated based on the calculated adjustment amount. This allows the imaging conditions stored in the imaging condition DB to be updated to imaging conditions that allow the tool 30 to be captured in the reference image, even if the mounting position of the tool 30 relative to the holder is shifted from the mounting position at the time of registering the imaging conditions.
[0070] The control unit 21 then proceeds to step S45, extracts one of the updated photographing conditions (S45), and executes photographing processing under the extracted photographing conditions (camera conditions and lighting conditions) (S46). By the above-described processing, even if the attachment position of the tool 30 relative to the holder is shifted from the attachment position at the time of registering the photographing conditions, by updating the photographing conditions, a photographed image of the tool 30 to be photographed in the same state as the photographed state of the reference image can be obtained. In this embodiment, based on the reference image and the photographed image of the outer periphery image, it is determined whether the attachment position of the tool 30 is shifted, and the adjustment amount of the photographing conditions is calculated. Alternatively, it may be configured to determine whether the attachment position of the tool 30 is shifted and calculate the adjustment amount of the photographing conditions based on the reference image and the photographed image of each image (top view image, oblique image, outer periphery image, rake face image). In the above-described processing, if the control unit 21 determines that the shooting conditions have been registered (S43: YES), it may be configured to determine whether the tool 30 has been replaced in the holder, and if a replacement has been made, to perform processing of steps S51 to S56.
[0071] In this embodiment, the process of calculating the adjustment amount for the registered photographing conditions based on the reference image and the captured image of the outer periphery image is not limited to being performed locally by the photographing device 20, but may also be performed by, for example, the server 10. In this case, the photographing device 20 can be configured to transmit a photographed image captured under the registered photographing conditions, such as a photographed image of the outer periphery image, to the server 10, and receive the result determined by the server 10 (information indicating whether or not there is a discrepancy between the position of the tool in the reference image and the position of the tool 30 in the captured image) and the adjustment amount calculated by the server 10 (the adjustment amount for the registered photographing conditions). In this configuration, the server 10 identifies the position of the blade (edge position E2 of the cutting edge) in the captured image based on the photographed image (outer periphery image) captured by the photographing device 20. The server 10 then determines whether the edge position E2 of the cutting edge matches a pre-registered position (edge position E1 in the reference image). If it determines that they do not match, it calculates an adjustment amount to match the edge position E2 in the captured image with the edge position E1 in the reference image. The server 10 instructs the imaging device 20 to adjust the imaging conditions set in the imaging device 20 based on the calculated adjustment amount. The imaging device 20 updates (adjusts) the registered imaging conditions in accordance with the instruction from the server 10, thereby registering, in the imaging condition DB, imaging conditions that allow the tool 30 to be imaged to be photographed in the imaging state of the reference image.
[0072] The configuration of this embodiment can be applied to the tool management systems of the first and second embodiments described above, and similar processes can be executed and similar effects can be obtained even when applied to the tool management systems of the first and second embodiments. Furthermore, the modified examples described in the first and second embodiments described above can also be applied to this embodiment.
[0073] (Embodiment 4) In the tool management systems of the above-described embodiments 1 to 3, a tool management system will be described that automatically determines whether an image is in focus when captured under each imaging condition and registers the imaging condition under which the image is in focus when capturing an image under each imaging condition when capturing an image under each imaging condition when capturing an image under each imaging condition, and registers the imaging condition under which the image is in focus. The tool management system of this embodiment can be realized by the same devices as those of the tool management systems of embodiments 1 to 3, and therefore a description of the configuration of each device will be omitted.
[0074] FIG. 15 is a flowchart showing an example of the procedure for registering the photographing conditions, and FIG. 16 is an explanatory diagram of the photographing condition registration process. The process shown in FIG. 15 corresponds to step S44 in FIG. 12. In this embodiment, the photographing device 20 photographs each image (top image, oblique image, outer periphery image, and rake face image) using default photographing conditions, determines whether the obtained photographed image is in focus, and if it determines that the image is out of focus, adjusts the photographing conditions and photographs again, repeating this process until the image is in focus. Then, the photographing device 20 registers the photographing conditions for the photographed image that is determined to be in focus as the photographing conditions for that image (photographing direction). The determination of whether the image is in focus is performed for a focus area previously set for each image. This enables efficient determination of whether each image is in focus.
[0075] In the process of registering the photographing conditions, the control unit 21 of the photographing device 20 first photographs, for example, an entire top surface image (S61). The control unit 21 extracts a focus area in the entire top surface image (S62). The focus area of the entire top surface image is the entire image, as indicated by the dashed line in FIG. 16 . The control unit 21 then determines whether the extracted focus area is in focus (S63). Here, the control unit 21 performs filtering on the focus area using a Laplacian filter to extract edges in the image, calculates variance based on each pixel value in the filtered image (focus area), and determines whether the image is in focus based on the calculated variance value. The variance can be calculated, for example, as the average value of the squares of the deviations (the differences between each pixel value and its average value). The control unit 21 determines that the image is out of focus if the variance value is equal to or greater than a predetermined value, and determines that the image is in focus if the variance value is less than the predetermined value. Note that the filtering process is not limited to using a Laplacian filter; various differential filters for extracting contours (edges) from an image may also be used.
[0076] If the control unit 21 determines that the image is not in focus (S63: NO), it adjusts (changes) the photographing conditions used in step S61 (S64). For example, the control unit 21 adjusts the photographing unit 26 by moving the photographing unit 26 upward in the Z-axis direction by a predetermined amount, adjusts the focus of the camera 26a, or performs other adjustments. This allows the photographing unit 26 to be moved upward in the Z-axis direction by a predetermined distance from the tool 30 being photographed, or adjusts the focus of the camera 26a. The control unit 21 then returns to step S61, photographs an entire top surface image under the adjusted photographing conditions (S61), and repeats the processes of steps S61 to S64 until it determines that the image is in focus. If the control unit 21 determines that the image is in focus (S63: YES), the control unit 21 registers the photographing conditions used to photograph the most recent entire top surface image in the photographing condition DB (S65).
[0077] Next, the control unit 21 performs the same processes as steps S61 to S65 on the oblique image (S66 to S70). The focus area of the oblique image is set to the horizontal center of the image, as indicated by the dashed line in FIG. 16 . Adjustments to focus the oblique image can be, for example, adjusting the photographing unit 26 a predetermined distance away from the tool 30 in the X-axis direction, adjusting the focus of the camera 26 a, etc. As a result, the photographing conditions for focusing on the focus area for the oblique image are also registered in the photographing condition DB. Next, the control unit 21 performs the same processes as steps S61 to S65 on the outer periphery image (S71 to S75). The focus area of the outer periphery image is set to the horizontal center of the image, as indicated by the dashed line in FIG. 16 . Again, adjustments to focus the oblique image can be, for example, adjusting the photographing unit 26 a predetermined distance away from the tool 30 in the X-axis direction, adjusting the focus of the camera 26 a, etc. As a result, the photographing conditions for focusing on the focus area for the outer periphery image are also registered in the photographing condition DB. Finally, the control unit 21 performs the same processing as steps S61 to S65 on the rake face image (S76 to S80). When the photographing conditions are such that the tool 30 is photographed at the center of the image, as shown in the upper part of FIG. 16, the focus area of the rake face image can be the area on the left side of the image, as indicated by the dashed line. Also, when the photographing conditions are such that the rake face of the blade 31a is photographed at the center of the image, as shown in the lower part of FIG. 16, such as for a tool with a large cutting edge diameter, the focus area may be the center of the image in the horizontal direction, as indicated by the dashed line. As a result, the photographing conditions for the rake face image, in which the focus area is in focus, are registered in the photographing condition DB. The control unit 21 may also perform similar processing on the individual top surface images.
[0078] After processing step S80, the control unit 21 returns to the processing of FIG. 12 and photographs the tool 30 to be photographed from multiple photographing directions according to the photographing conditions registered by the above-described processing. The photographing conditions for each image that bring the focus to the focus area are registered by the above-described processing. The focus area is the area in which the blade portion 31 (blade 31a) is photographed. By photographing under photographing conditions that bring this area into focus, high-quality photographed images can be obtained that can appropriately detect the deteriorated area of each blade by image recognition using the learning models M1 to M4, making it possible to more accurately determine the deterioration state of each blade.
[0079] In this embodiment, the registration process of the shooting conditions for each image is not limited to being performed locally by the image capture device 20, but may also be performed by, for example, the server 10. In this case, the image capture device 20 can be configured to transmit each captured image to the server 10 and receive the results determined by the server 10 (the shooting conditions determined to be in focus). In this configuration, the server 10 determines whether a predetermined area (focus area) of each captured image is in focus based on each captured image captured by the image capture device 20. If the server 10 determines that the image is not in focus, it calculates an adjustment amount to bring the predetermined area into focus and instructs the image capture device 20 to adjust the shooting conditions set in the image capture device 20 based on the calculated adjustment amount. The image capture device 20 can update (adjust) the registered shooting conditions in accordance with the instructions from the server 10 to bring the shooting conditions closer to those in which the focus area is in focus. The photographing device 20 and the server 10 repeat the above-described process until the focus area of each image is brought into focus, thereby registering the photographing conditions for bringing the focus area of each image into focus in the photographing condition DB.
[0080] The configuration of this embodiment can be applied to the tool management systems of the above-mentioned embodiments 1 to 3, and similar processes can be executed and similar effects can be obtained even when applied to the tool management systems of the above-mentioned embodiments 1 to 3. Furthermore, the modified examples described in the above-mentioned embodiments 1 to 3 can also be applied to this embodiment as appropriate.
[0081] (Embodiment 5) This embodiment will describe a tool management system that estimates the size of a deteriorated area when it is determined that a deteriorated area is present in a blade portion (blade) in a captured image through the deteriorated area estimation process in the tool management systems of the above-described embodiments 1 to 4. The tool management system of this embodiment can be realized by devices similar to those of the tool management systems of embodiments 1 to 4, and therefore a description of the configuration of each device will be omitted.
[0082] Fig. 17 is a flowchart showing an example of the procedure for estimating a degraded area, and Figs. 18A and 18B are explanatory diagrams of the size estimation process for a degraded area. The process shown in Fig. 17 is the process shown in Fig. 7 with steps S91 and S92 added after step S35. Explanations of the same steps as in Fig. 7 will be omitted.
[0083] In the process of estimating a deteriorated area, the control unit 11 of the server 10 of this embodiment performs the same processes as steps S31 to S35 in FIG. 7 . This allows the control unit 11 to determine whether a deteriorated area is present in the blade in the captured image read in step S13 in FIG. 6 . The control unit 11 then determines whether a deteriorated area is present based on the determination result of step S35 (S91). If it determines that a deteriorated area is present (S91: YES), the control unit 11 calculates the size of the deteriorated area (S92). For example, as shown in FIG. 18A , the control unit 11 acquires a label image for the captured outer peripheral image, which includes a bounding box surrounding the deteriorated area detected in the image, a discrimination label indicating the type of the deteriorated area (worn or chipped), and a confidence level for the discrimination label. After acquiring such a label image, the control unit 11 extracts the bounding box area from the captured image and performs a binarization process on the extracted area (see FIG. 18A (1)). The control unit 11 adds one diagonal line L1 (a diagonal line that slopes upward to the right) to the bounding box of the binary image and identifies a midpoint P1 of the diagonal line L1 (see FIG. 18A(2)). Note that the tool 30 shown in FIG. 18A has a blade edge that slopes upward to the right, so the diagonal line L1 is added here. However, a diagonal line that slopes downward to the right may also be added depending on the condition of the blade of the tool 30. Next, the control unit 11 identifies a perpendicular line L2 that passes through the midpoint P1 and intersects with the diagonal line L1 (see FIG. 18A(3)). The control unit 11 then identifies two intersections P2 and P3 between the perpendicular line L2 and the contour of the degraded area (see FIG. 18A(4)). The control unit 11 then calculates the distance between the intersections P2 and P3 and determines the calculated distance as the size (width) of the degraded area.
[0084] The bounding box is often added with the detected degraded area positioned at the center, and the midpoint P1 of the diagonal line L1 of the bounding box is likely to be near the center of the degraded area. Therefore, the size of the degraded area can be calculated with high accuracy by calculating the size of the degraded area based on the perpendicular line L2 passing through the midpoint P1 of the diagonal line L1 of the bounding box. The size of the degraded area can also be calculated using similar processing for the oblique image and the rake face image. Note that if the edge position of the cutting edge 31a can be detected with high accuracy, the size of the degraded area in the direction perpendicular (intersecting) to the edge position may also be calculated.
[0085] Furthermore, as shown in FIG. 18B , the control unit 11 acquires a label image for the captured top surface image (top surface overall image and top surface individual image) that includes a bounding box surrounding the deteriorated region, a discrimination label indicating the type of the deteriorated region (worn or chipped), and a confidence level for the discrimination label. When such a label image is acquired, the control unit 11 extracts the bounding box region from the captured image and performs a binarization process on the extracted region (see FIG. 18B (1)). The control unit 11 estimates the pre-deterioration shape (original shape) of the blade based on the binary image. For example, assuming the pre-deterioration shape of the blade is rectangular, the control unit 11 estimates a rectangle R1 encompassing the post-deterioration blade region (white pixel region) as the pre-deterioration shape of the blade, as shown in FIG. 18B (2), and adds the estimated rectangle R1 to the binary image. Next, the control unit 11 identifies the area on the tip side of the blade (the upper gray area in FIG. 18B (3)), which is the difference area between the blade area (area of white pixels) in the binary image and the area of the rectangle R1. The tip side of the blade is at the top of the bounding box in FIG. 18B , but by pre-setting the tip side of the blade in each of the entire top surface image and the individual top surface image, the difference area on the tip side of the blade can be identified. The difference area on the tip side of the blade indicates the deteriorated area of the blade. Therefore, the control unit 11 calculates the size (area) of the identified difference area and sets the calculated size as the size of the deteriorated area. Note that the control unit 11 may count the number of pixels in the deteriorated area as the size of the deteriorated area, or may calculate the length of the deteriorated area represented by a right-angled triangle in the left-right direction of the image (the circumferential direction of the blade portion 31 of the tool 30) and the length of the up-down direction of the image (the radial direction of the blade portion 31) as shown in FIG. 18B (4). In the top view image, each blade is captured in a substantially rectangular shape, so by comparing the area of the blade 31a in the image with the area of the rectangle R1, the size of the deteriorated area of the blade 31a can be estimated with high accuracy.
[0086] If the control unit 11 determines that there is no deteriorated area (S91: NO), it skips step S92 and returns to the processing of FIG. 6 . In the processing of FIG. 6 , in step S17, the control unit 11 associates the captured image (original image) read in step S13, the label image generated in step S34, the determination result of whether or not there is a deteriorated area in step S35, and the size of the deteriorated area calculated in step S92 if it is determined that there is a deteriorated area, with the date and time of capture and the captured object identified in step S14, and stores them in the tool DB 12a. Furthermore, in step S20, the control unit 11 may display the size of each deteriorated area (worn area and chipped area) detected in the captured image on the determination result screen shown in FIG. 8 in association with the deteriorated area. With this configuration, the determination result screen can display which part of the tool 30 is estimated to be a deteriorated area, the confidence level of the estimated deteriorated area, and the size of the deteriorated area all together.
[0087] In this embodiment, when a deteriorated area is estimated to exist on a tool 30 based on a captured image of the tool 30, the size of the estimated deteriorated area can be automatically detected. Therefore, not only can the size of the automatically detected deteriorated area be notified, but the size of the deteriorated area to be notified can be switched depending on the size of the deteriorated area. That is, a threshold for determining whether to issue an alert based on the size (deterioration width) of the deteriorated area can be set. For example, a tool 30 in which a deteriorated area with a size (width) of 0.5 mm or more is detected can be set as an alert target. In this case, by not setting a tool 30 with a deteriorated area with a size less than 0.5 mm as an alert target, unnecessary alert output can be suppressed. The configuration of this embodiment is applicable to the tool management systems of the first to fourth embodiments described above. Even when applied to the tool management systems of the first to fourth embodiments, similar processing can be performed and similar effects can be obtained. Furthermore, the modified examples described in the first to fourth embodiments can also be applied to this embodiment.
[0088] Sixth Embodiment A tool management system will be described in which the threshold for the confidence level of a discrimination label, which is used when determining whether or not a photographed tool has a deteriorated area based on a label image, can be switched according to the tool (tool model number). The tool management system of this embodiment can be realized by devices similar to those of the tool management systems of embodiments 1 to 5, so a description of the configuration of each device will be omitted. In this embodiment, the server 10 stores a discrimination threshold DB in the storage unit 12 in addition to the configuration shown in FIG. 1.
[0089] FIG. 19 is an explanatory diagram showing an example of a record layout of the discrimination threshold DB. The discrimination threshold DB is a database that stores thresholds used to determine the presence or absence of a deteriorated area using models M1 to M4, in association with each tool managed by the tool management system. The discrimination threshold DB includes, for example, a model number column, a manufacturer name column, a type column, and a discrimination threshold column, and stores information about each tool and a discrimination threshold for each tool in association with the model number of each tool. The discrimination threshold may be a preset numerical value or may be an arbitrarily changed numerical value. The discrimination threshold DB is not limited to the configuration shown in FIG. 19 .
[0090] Fig. 20 is a flowchart showing an example of a degradation region estimation process procedure according to embodiment 6. The process shown in Fig. 20 is the process shown in Fig. 7 with step S101 added between steps S34 and S35. Explanation of the same steps as in Fig. 7 will be omitted.
[0091] In the degradation area estimation process of this embodiment, the control unit 11 of the server 10 performs the same processes as steps S31 to S34 in FIG. 7 . As a result, the control unit 11 can generate a label image by adding a bounding box surrounding the degradation area, a discrimination label indicating the type of the degradation area, and a confidence level for the discrimination label (the likelihood that the degradation state indicated by the discrimination label) to the original image (captured image). The control unit 11 acquires a discrimination threshold for the tool 30 to be processed from the discrimination threshold DB (S101). Specifically, the control unit 11 identifies the model number of the tool 30 to be processed and reads the discrimination threshold corresponding to the identified model number from the discrimination threshold DB. For example, a code (e.g., a barcode or QR code) encoding information such as the model number of the tool 30 is printed or affixed to a holder that holds the tool 30. The control unit 11 can acquire the model number of the tool 30 to be processed by reading the code using the code reader. The model number of the tool 30 may be received, for example, via the input unit 14, or may be identified by template matching using a template image that indicates the image features of the captured image of each tool, or may be identified by image recognition using a learning model.
[0092] The control unit 11 uses the discrimination threshold acquired in step S101 to determine whether or not a deteriorated area exists in the tool in the captured image based on the label image generated in step S34 (S35). For example, the control unit 11 determines whether or not a deteriorated area exists depending on whether or not there is a bounding box in the label image whose confidence level for the discrimination label is equal to or greater than the discrimination threshold acquired in step S101. Specifically, if the control unit 11 determines that there is a bounding box whose confidence level is equal to or greater than the discrimination threshold, it determines that the area indicated by the bounding box is a deteriorated area. The control unit 11 then returns to the processing of FIG. 6. Note that, if the control unit 11 identifies the model number of the tool 30 being photographed in addition to the blade information of the photographed tool in step S14 of FIG. 6, the control unit 11 may acquire the discrimination threshold for the tool 30 in step S101 based on the model number of the tool 30 identified in step S14.
[0093] By the above-described processing, in this embodiment, it is possible to switch the threshold for determining whether or not a deteriorated area should be determined based on the certainty of the discrimination label in the label image for each tool (each tool model number). For example, the smaller the diameter of the cutting portion 31, the more affected by deterioration even a small deteriorated area is. Therefore, by setting a small discrimination threshold for a tool with a small diameter of the cutting portion 31, it is possible to accurately detect deteriorated areas with a low certainty. On the other hand, by setting a large discrimination threshold for a tool with a large diameter of the cutting portion 31, it is possible to exclude deteriorated areas with a certainty below the discrimination threshold from detection targets and reliably detect deteriorated areas with a certainty equal to or greater than the discrimination threshold.
[0094] The configuration of this embodiment can be applied to the tool management systems of the above-mentioned embodiments 1 to 5, and similar processes can be executed and similar effects can be obtained even when applied to the tool management systems of the above-mentioned embodiments 1 to 5. Furthermore, the modified examples described in the above-mentioned embodiments 1 to 5 can also be applied to this embodiment as appropriate.
[0095] The matters described in each embodiment can be combined with each other. Furthermore, the independent claims and dependent claims described in the claims can be combined with each other in any combination, regardless of the reference format. Furthermore, the claims use a format in which a claim references two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used.
[0096] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.
[0097] REFERENCE SIGNS LIST 10 Server 11 Control unit 12 Storage unit 13 Communication unit 20 Photography device 21 Control unit 22 Storage unit 23 Communication unit 26 Photography unit 27 Photography direction control unit 30 Tool 31 Cutting portion 31a Cutting edge M1 Top surface model M2 Oblique model M3 Outer periphery model M4 Rake face model 12a Tool DB
Claims
1. A program that causes a computer to execute a process of acquiring an image of a cutting edge provided on a cutting tool, inputting the acquired image into a learning model that has been trained to output information regarding the deterioration state of the cutting edge when an image of the cutting edge of the cutting tool is input, and acquiring information regarding the deterioration state of the cutting edge of the cutting tool in the image.
2. The program according to claim 1, which causes the computer to execute the following processes: the captured images include a plurality of captured images of the blade portion from a plurality of shooting directions; the learning model includes a plurality of learning models trained for each of the shooting directions; the captured images are identified in the captured images; a learning model is selected according to the identified shooting direction; the captured images are input into the selected learning model to obtain information regarding the deterioration state of the blade portion in the captured images; and a deterioration state of the blade portion is determined based on the information regarding the deterioration state obtained for each of the plurality of shooting directions.
3. The program according to claim 2, wherein the blade portion has a plurality of blades, and the captured images include a plurality of captured images of each blade taken from the plurality of shooting directions.
4. The program according to claim 3, which causes the computer to execute a process in which the captured images include an image of the tip surface of the blade portion and an image of the tip surface of each blade of the blade portion, and the captured images of the tip surface of the blade portion and the captured images of the tip surface of each blade are input into a learning model corresponding to the photographing direction of the tip surfaces to obtain information regarding the deterioration state of the blade portion or each blade.
5. The program according to claim 3 or 4, which causes the computer to execute a process in which the captured images include captured images of the side of each blade of the blade portion, the captured images of the side including an image captured with the focus on the rake face of the blade and an image captured with the focus on the flank of the blade, and the learning model includes a learning model for images captured with the focus on the rake face of the blade and a learning model for images captured with the focus on the flank of the blade, and the image captured with the focus on the rake face of the blade and the image captured with the focus on the flank of the blade are input into the learning model for images captured with the focus on the rake face of the blade or the learning model for images captured with the focus on the flank of the blade, respectively, to obtain information regarding the deterioration state of the blade.
6. The program of claim 5, which causes the computer to execute the following process: the information relating to the deteriorated state includes information indicating the area of the blade in the input photographed image that is in a deteriorated state; the photographed image focused on the cutting face of the blade is input into a learning model for photographed images focused on the cutting face of the blade to obtain the area of the deteriorated blade in the photographed image; and if the area of the deteriorated blade in the photographed image is outside a specified area in the photographed image, the program does not determine that the blade is a deteriorated blade.
7. The program according to claim 1 or 2, which causes the computer to execute a process in which the information relating to the deterioration state includes information indicating the area of the blade portion in the input photographed image that is in a deteriorated state and which of multiple types of deterioration states the blade portion is in, and outputs a screen that displays the area of the blade portion in the input photographed image that is in a deteriorated state in a manner corresponding to the type of deterioration state.
8. A program as described in claim 1 or 2, which causes the computer to execute the following process: based on the acquired captured image, identify the position of the blade in the captured image; determine whether the identified blade position matches a position registered in advance for the cutting tool; and if it is determined that they do not match, change the capturing position relative to the cutting tool so that the blade position in the captured image approaches the registered position in advance.
9. A program as claimed in claim 1 or 2, which causes the computer to execute a process of determining whether or not a specific area of the captured image is in focus based on the acquired captured image, and, if it is determined that the area is not in focus, changing the capturing position of the cutting tool or adjusting the focus.
10. The program of claim 1 or 2, which causes the computer to execute the following processes: the information relating to the deteriorated state includes a bounding box indicating the area of the blade in a deteriorated state in the input captured image; identifying a line segment that passes through the midpoint of one diagonal in the bounding box and intersects the diagonal; and calculating the distance between two intersections of the identified line segment and the contour of the area of the blade in a deteriorated state as the size of the area of the blade in a deteriorated state.
11. A program as described in claim 1 or 2, which causes the computer to execute the following process: the information regarding the deteriorated state includes a bounding box indicating the area of the blade in a deteriorated state in the input captured image; estimating the shape of the blade before deterioration based on the area of the blade in a deteriorated state within the bounding box; and calculating the difference between the estimated shape before deterioration and the area of the blade in a deteriorated state as the size of the area of the blade in a deteriorated state.
12. The program according to claim 1 or 2, which causes the computer to execute the following processes: the information on the deterioration state includes the area of the blade part that is in a deteriorated state in the input captured image and the possibility that the area of the blade part is in a deteriorated state; depending on the type of the cutting tool, a threshold is obtained for determining that the blade part is in a deteriorated state based on the possibility contained in the information on the deterioration state obtained from the learning model; and depending on whether the possibility contained in the information on the deterioration state obtained from the learning model is equal to or greater than the obtained threshold, the program causes the computer to execute the following processes.
13. An information processing method in which a computer executes a process of acquiring an image of a cutting edge provided on a cutting tool, inputting the acquired image into a learning model that has been trained to output information regarding the deterioration state of the cutting edge when an image of the cutting edge of the cutting tool is input, and acquiring information regarding the deterioration state of the cutting edge of the cutting tool in the image.
14. An information processing device having a control unit, wherein the control unit acquires an image of a blade portion provided on a cutting tool, and inputs the acquired image into a learning model that has been trained to output information regarding the deterioration state of the blade portion when the image of the blade portion of the cutting tool is input, thereby acquiring information regarding the deterioration state of the blade portion of the cutting tool in the captured image.
Citation Information
Patent Citations
Machining device and method for driving the same
JP2011206899A
Image output device and image output method
JP2018132962A
Tool wear prediction system
JP2021070114A
Cutting work monitoring system
JP2022143752A
Management device, display processing method, and display processing program
JP7099646B2
