Press machine and image monitoring method for press machine

Abstract

To provide a press machine capable of monitoring an image only with image data.SOLUTION: A press machine includes a storage unit for storing moving image data imaged by an imaging part arranged on a position capable of imaging a press die area, an image extraction unit for extracting a monitoring object image based on a slide position in the image or on the state of a workpiece, from a static image group constituting the moving image data, and an image monitoring unit for performing image monitoring based on the monitoring object image, and outputting a monitoring result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a press machine and an image monitoring method for a press machine. [Background technology]

[0002] As an example of image monitoring in a press, Patent Document 1 discloses an invention in which a gaze image is generated by cutting out a gaze object from a captured image based on distance data generated by a distance measuring device, and the image is output to a device that the user can use. Also, Patent Document 2 discloses an invention in which an image is captured in an injection molding machine when the mold is opened or when the molded product is removed, and the captured image is compared with a reference image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2022-51155 [Patent Document 2] Patent Publication No. 2021-41454 Summary of the Invention [Problem to be solved by the invention]

[0004] The invention described in Patent Document 1 combines a distance measuring device with an imaging device because there is a lot of disturbance when using only an imaging device, and does not perform monitoring using only captured images. Similarly, the invention described in Patent Document 2 also performs photography based on information from a controller that opens the mold in the injection molding machine, and does not perform monitoring using only captured images.

[0005] The present invention has been made in consideration of the above-mentioned problems, and its object is to provide a press machine and an image monitoring method for a press machine that are capable of image monitoring using only image data. [Means for solving the problem]

[0006] (1) The press machine according to the present invention is a press machine that converts the rotation of a motor into the reciprocating linear motion of a slide to press a workpiece, Thailand This press machine is characterized by including a memory unit that stores moving image data captured by an imaging unit positioned at a position where it can capture an area, an image extraction unit that extracts an image to be monitored from a group of still images that make up the moving image data based on the position of the slide in the image or the state of the processed material, and an image monitoring unit that performs image monitoring based on the image to be monitored and outputs the monitoring results.

[0007] The image monitoring method for a press machine according to the present invention is an image monitoring method for a press machine that converts the rotation of a motor into a reciprocating linear motion of a slide to perform press working on a workpiece, the method comprising the steps of: Thailand This is an image monitoring method for a press machine, characterized by including: a storage step for storing moving image data captured by an imaging unit arranged at a position where it can capture an area; an image extraction step for extracting an image to be monitored from a group of still images constituting the moving image data, based on the position of the slide in the image or the state of the processed material; and an image monitoring step for performing image monitoring based on the image to be monitored and outputting the monitoring results.

[0008] According to the present invention, images to be monitored are extracted from a group of still images constituting the moving image data based on the position of the slide in the image and the state of the processed material, so that images of the same timing in each cycle during press production can be extracted using only the image data (without using controller information) to perform image monitoring.

[0009] (2) In the press machine according to the present invention, the image extraction unit may extract, from the group of still images, still images in which the slide is in the same position as the monitoring target images.

[0010] In the image monitoring method for a press machine according to the present invention, in the image extraction step, still images in which the slide is in the same position may be extracted as the monitoring target images from the group of still images.

[0011] (3) In the press machine according to the present invention, the image extraction unit may extract, from the group of still images, an image at a time when the workpiece is stationary as the monitoring target image.

[0012] In the image monitoring method for a press machine according to the present invention, the image extraction step may extract, from the group of still images, an image at a time when the workpiece material is stationary as the monitored image.

[0013] (4) In the press machine of the present invention, the image monitoring unit may set a monitoring area in the monitored image corresponding to the area where the workpiece material is processed, and perform image monitoring of the monitoring area.

[0014] In the image monitoring method for a press machine of the present invention, the image monitoring step may set a monitored area in the monitored image corresponding to an area where the workpiece material is processed, and perform image monitoring of the monitored area.

[0015] According to the present invention, by performing image monitoring on a partial area (monitoring target area) of the monitoring target image, it is possible to reduce the processing load related to image monitoring.

[0016] (5) In the press machine of the present invention, the image monitoring unit may set the monitored area by comparing the monitored image extracted from the group of still images of the moving image data captured in a state where the workpiece material is present with the monitored image extracted from the group of still images of the moving image data captured in a state where the workpiece material is not present.

[0017] In the image monitoring method for a press machine of the present invention, the image monitoring step may set the monitored area by comparing the monitored image extracted from the group of still images of the moving image data captured in a state where the workpiece is present with the monitored image extracted from the group of still images of the moving image data captured in a state where the workpiece is not present.

[0018] According to the present invention, the monitoring area, which is the area where the workpiece material is processed, can be automatically set.

[0019] (6) In the press machine according to the present invention, the image monitoring unit may use a learning model generated by machine learning the image to be monitored that has been acquired in advance during normal operation to output the similarity of the image to be monitored acquired during monitoring with the image to be monitored at the time of learning as the monitoring result.

[0020] In the image monitoring method for a press machine according to the present invention, the image monitoring step may use a learning model generated by machine learning the image to be monitored that was previously acquired during normal operation, and output the similarity of the image to be monitored acquired during monitoring with the image at the time of learning as the monitoring result.

[0021] (7) In the press machine according to the present invention, the image monitoring unit performs learning of still images of frames before and after the monitoring target image acquired during monitoring, using each learning model generated by machine learning of still images of frames before and after the monitoring target image acquired in advance during normal operation. The degree of similarity with the time may be output as the monitoring result.

[0022] In the image monitoring method for a press machine according to the present invention, the image monitoring step may use each learning model generated by machine learning each of still images of frames before and after the image to be monitored that have been acquired in advance during normal operation, and output the similarity of the still images of frames before and after the image to be monitored that have been acquired during monitoring with those at the time of learning as the monitoring result. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a press machine according to an embodiment of the present invention. [Figure 2] 10 is a flowchart showing the flow of processing by the monitoring device. [Figure 3] FIG. 10 is a diagram showing an example of a monitoring target image selected from a group of still images. [Figure 4]FIG. 10 is a diagram showing an example of setting a monitoring target area. [Figure 5] FIG. 10 is a diagram showing an example of a display of monitoring results. [Figure 6] FIG. 10 is a diagram showing an example of setting a reference monitoring area. [Figure 7] 10A and 10B are diagrams showing examples of display of monitoring results when images to be monitored are extracted based on the state of the processed material. [Figure 8] FIG. 10 is a diagram showing an example of the display of monitoring results when learning and monitoring are performed using still images of frames before and after the image to be monitored. DETAILED DESCRIPTION OF THE INVENTION

[0024] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0025] FIG. 1 is a diagram showing an example of the configuration of a press machine (servo press machine) according to this embodiment. The press machine 1 converts the rotation of a servo motor 10 into vertical reciprocating motion (reciprocating linear motion, lifting motion) of a slide 17 using an eccentric mechanism that converts rotational motion into linear motion. The vertical reciprocating motion of the slide 17 is used to press a workpiece. The press machine 1 includes a servo motor 10, an encoder 11, a drive shaft 12, a drive gear 13, a main gear 14, a crankshaft 15, a connecting rod 16, the slide 17, a bolster 18, an imaging unit 22, a control device 30, a monitoring device 100, a user interface 110 (operation unit), a display 120 (display unit), and a memory unit 130. The press machine is not limited to a servo press machine. For example, it may be a mechanical press using a flywheel or a linear-acting press using a ball screw. In this case, the encoder may be provided on the shaft end of the crankshaft 15 or the shaft end of the ball screw.

[0026] A drive shaft 12 is connected to the rotating shaft of the servo motor 10, and a drive gear 13 is connected to the drive shaft 12. A main gear 14 is engaged with the drive gear 13, and a crankshaft 15 is connected to the main gear 14, and a connecting rod 16 is connected to the crankshaft 15. Rotating shafts such as the drive shaft 12 and the crankshaft 15 are supported by appropriately provided bearings (not shown). The crankshaft 15 and the connecting rod 16 form an eccentric mechanism. This eccentric mechanism enables a slide 17 connected to the connecting rod 16 to move up and down relative to a stationary bolster 18. One or more upper dies 20 are attached to the slide 17, and one or more lower dies 21 are attached to the bolster 18.

[0027] The imaging unit 22 is a digital camera capable of taking moving images, which is composed of an optical lens and a CCD / CMOS image sensor. Thailand Eli A It is placed in a position where it can be photographed. The press die area is the area where the upper die 20 and lower die 21, which are provided between the slide 17 and the bolster 18, are mounted, and the objects to be filmed in the video include the slide 17, bolster 18, upper die 20, lower die 21, the workpiece, the conveying device, the processing area, etc. The moving images captured by the imaging unit 22 are output to the monitoring device 100 and stored in the storage unit 130 as moving image data.

[0028] The control device 30 controls the slide 1 based on a predetermined slide motion stored in the storage unit. The control device 30 controls the lifting and lowering movement of the slide 17. The control device 30 is composed of control devices such as a PLC, servo controller, and servo amplifier, for example. More specifically, the control device 30 calculates a motor rotation position command value from commands for the position and speed of the slide 17 defined by the slide motion, and controls the rotation position, rotation speed, and current of the servo motor 10 based on the rotation position command. The rotation position of the servo motor 10 is detected by an encoder 11 attached to the servo motor 10.

[0029] The control device 30 also controls the operation of a transport device (not shown) that transports the workpiece to the next process. The transport device is, for example, a transfer feeder and includes a pair of transfer bars (not shown) driven by a servo motor (not shown). The transfer bars are provided with fingers (not shown) for clamping the workpiece. The transfer bars are driven by the servo motor to perform a clamping operation (moving in the Y-axis direction to attach the fingers to the workpiece), a lifting operation (moving up in the Z-axis direction), an advance operation (moving forward in the X-axis direction), a down operation (moving down in the Z-axis direction), an unclamping operation (moving in the Y-axis direction to detach the fingers from the workpiece), and a return operation (moving back in the X-axis direction). The transfer bars may perform the clamping operation and lifting operation, the lifting operation and advance operation, the advance operation and down operation, the down operation and unclamping operation, the unclamping operation and return operation, and the return operation and clamping operation, with some overlapping between the clamping operation and lifting operation, the lifting operation and advance operation, the advance operation and down operation, the down operation and unclamping operation, the unclamping operation and return operation, and the clamping operation and return operation. The conveying device may be a roll feeder (feeding device) that sends out sheet-like processed material to the next process. The control device 30 controls the operation of the slide 17 based on the press individual phase signal synchronized with the master phase signal, and also controls the operation of the transfer feeder based on the transfer individual phase signal synchronized with the master phase signal. In other words, the transfer feeder operates in synchronization with the operation of the slide 17.

[0030] An example of the monitoring device 100 is an industrial PC (IPC) equipped with a processor. The image extracting unit 101 may be a computer (PC) or a control board configured with a microcomputer and ICs, etc., and includes an image extracting unit 101 and an image monitoring unit 102. The image extracting unit 101 extracts monitoring target images from a group of still images (frame-by-frame images) constituting the moving image data stored in the storage unit 130, based on the position of the slide 17 in the image or the state of the material to be processed. The image extracting unit 101 may extract, from the group of still images, still images in which the slide 17 is in the same position as the image to be monitored, or may extract, from the group of still images, still images taken at the time the material to be processed comes to rest as the image to be monitored.

[0031] The image monitoring unit 102 performs image monitoring based on the images of the monitoring target, and outputs the monitoring results (such as images showing the monitoring results) to the display 120. For example, the image monitoring unit 102 uses a learning model generated by machine learning the images of the monitoring target extracted from the group of still images constituting the video data acquired during monitoring to output the similarity between the images of the monitoring target extracted from the group of still images constituting the video data acquired during monitoring and the images at the time of learning (during normal operation) as the monitoring results. The image monitoring unit 102 may also set a monitoring target area corresponding to the area in the monitoring target image where the workpiece material is processed, and perform image monitoring of the set monitoring target area.

[0032] 2 is a flowchart showing the processing flow of the monitoring device 100. After the start of production operation, the monitoring device 100 acquires moving image data (moving image data for learning) captured by the imaging unit 22 and stored in the storage unit 130 (step S10). Next, the monitoring device 100 displays a part of the group of still images constituting the acquired moving image data on the display 120 (step S11). Here, at least one cycle of still images is displayed. One cycle of still images is a group of still images captured while the crank rotates once (while the slide moves back and forth once). Next, the monitoring device 100 checks the still images displayed on the display 120. An operation to select one still image from the group of images (an operation by the user on the user interface 110) is accepted, and the still image selected by this operation is set as the image to be monitored (step S12).

[0033] FIG. 3 shows an example of a monitoring target image MI selected from the still image group SI. ThailandThis shows an example in which three processing zones within the area are positioned so that they can be photographed from diagonally above the rear side of the press. The monitored image MI shows slide 17, upper mold 20a and lower mold 21a in the first processing zone, upper mold 20b and lower mold 21b in the second processing zone, upper mold 20c and lower mold 21c in the third processing zone, and material M being processed in each of the first to third processing zones. The monitored image MI also shows a pair of transfer bars 40 that constitute a transfer feeder, which is a conveying device, and fingers 41a, 41b, and 41c provided on transfer bar 40. Before processing, the disk-shaped material M is transported from the upstream side to a first processing area by the finger 41a, processed by the upper mold 20a and the lower mold 21a, transported to a second processing area by the finger 41b, processed by the upper mold 20b and the lower mold 21b, transported to a third processing area by the finger 41c, and processed by the upper mold 20c and the lower mold 21c, transported downstream by a finger (not shown). In this example, from the still image group SI, a still image captured at the timing when the slide 17 is descending and the transport by the transfer feeder is completed (when the down operation is completed and the material M is placed on the lower mold, and the fingers are separated from the material M by the unclamping operation) is selected as the monitoring target image MI.

[0034] Next, the monitoring device 100 receives an operation (a user operation on the user interface 110) to specify one or more monitoring target areas in the monitoring target image, and sets the areas specified by the operation as the monitoring target areas (step S13). The user can arbitrarily specify the number of monitoring target areas and the position and size of each monitoring target area. In the example shown in FIG. 4, in the monitoring target image MI, the first machining area (the area including the material M on the lower mold 21a) is set as the monitoring target area MA1, the second machining area (the area including the material M on the lower mold 21b) is set as the monitoring target area MA2, and the third machining area (the area including the material M on the lower mold 21c) is set as the monitoring target area MA3.

[0035] The monitoring device 100 detects the position of the slide 17 in the monitored image MI set in step S12 and stores it in the storage unit 130. For example, the position of the bottom edge of the slide at the left edge of the monitored image MI (slide position SP) is detected, and the coordinates are stored in the storage unit 130 as the slide position of the monitored image MI. Because the slide 17 is painted in a specific color, the slide position SP can be detected by detecting the boundary between the slide paint color and the background. In addition, the movement direction of the slide 17 is determined from the slide position in still images of frames before and after the monitored image MI, and the determination result is stored in the storage unit 130 as the slide movement direction of the monitored image MI. If the slide position in the still image of the frame before the monitored image MI is higher than the slide position SP in the monitored image MI, and the slide position in the still image of the frame after the monitored image MI is lower than the slide position SP in the monitored image MI, the slide movement direction can be determined to be downward (descending).Also, if the slide position in the still image of the frame before the monitored image MI is lower than the slide position SP in the monitored image MI, and the slide position in the still image of the frame after the monitored image MI is higher than the slide position SP in the monitored image MI, the slide movement direction can be determined to be upward (ascending).The slide position SP and slide movement direction are used as criteria for extracting monitored images during learning and monitoring.

[0036] Furthermore, the monitoring device 100 stores the coordinates of each monitoring target area set in step S13 in the storage unit 130. Here, the setting of the monitoring target area in the monitoring target image MI is performed by the user's operation. The monitoring target image MI may be set automatically without relying on the operation. For example, after setting the monitoring target image MI, a blank beating operation is performed without transporting the material to be processed, and still images having the same slide position and slide movement direction as the monitoring target image MI are extracted from a group of still images constituting the video data captured at that time, and the extracted still images (monitoring target images extracted from a group of still images of the video data captured when the material to be processed is not present) are compared with the monitoring target image MI (monitoring target images extracted from a group of still images of the video data captured when the material to be processed is present). The area where the extracted still image and the monitoring target image MI differ (where the degree of difference is greater than or equal to a predetermined threshold, or the degree of similarity is less than or equal to a predetermined threshold) is the area where the material is placed (the processing area), and therefore this area can be set as the monitoring target area.

[0037] Next, the image extraction unit 101 detects the slide position and slide movement direction in each still image of the group of still images constituting the learning moving image data, and extracts from the group of still images a still image having the same slide position and slide movement direction as the monitoring target image MI set in step S12 as the monitoring target image for learning (step S14). In the example of setting the monitoring target image shown in Fig. 3, a still image of each cycle when the slide 17 is descending and transport by the transfer feeder is completed is extracted as the monitoring target image for learning.

[0038] Next, the monitoring device 100 performs machine learning on each monitoring target area in the monitoring target image extracted in step S14 to generate a learning model for each monitoring target area, and stores each learning model in the storage unit 130 (step S15). In the example of setting the monitoring target area shown in FIG. 4, machine learning is performed on the monitoring target area MA1 in each extracted monitoring target image for training to generate a first learning model, machine learning is performed on the monitoring target area MA2 in each extracted monitoring target image to generate a second learning model, and machine learning is performed on the monitoring target area MA3 in each extracted monitoring target image to generate a third learning model. AI techniques such as neural networks and deep learning can be used as machine learning algorithms. This learning model is designed, for example, to output a value closer to 100 if the input image is similar to the monitoring target area in the training monitoring target image (high similarity), and to output a value closer to 0 if the input image is different from the training monitoring target image (low similarity).

[0039] Next, the monitoring device 100 sequentially acquires video data (monitoring video data) captured by the imaging unit 22 and stored in the storage unit 130 (step S16). The image extraction unit 101 detects the slide position and slide movement direction for each still image in the group of still images constituting the acquired video data, and extracts from the group of still images a still image having the same slide position and slide movement direction as the monitoring target image set in step S12 as a monitoring target image for monitoring (step S17). Because the transfer feeder operates in synchronization with the operation of the slide 17, if there are no problems with the transfer feeder's transport or the condition of the processed material or mold, the extracted monitoring target image should be the same as the monitoring target image for learning. In step S17, the number of times (cumulative number) that a monitoring image has been extracted is counted, and this number is stored as the production number in association with the extracted monitoring target image.

[0040] Next, the image monitoring unit 102 inputs each of the monitored areas in the monitored image extracted in step S17 into a trained learning model corresponding to each monitored area, and outputs the similarity of each monitored area with respect to the time of learning as the monitoring result to the display 120 (step S18). In the example of setting the monitored area shown in FIG. 4, the monitored area MA1 in the extracted monitored image for monitoring is input into the first learning model and the similarity of the monitored area MA1 with respect to the time of learning is output, the monitored area MA2 in the extracted monitored image is input into the second learning model and the similarity of the monitored area MA2 with respect to the time of learning is output, and the monitored area MA3 in the extracted monitored image is input into the third learning model and the similarity of the monitored area MA3 with respect to the time of learning is output. The monitoring result of the extracted monitored image (the similarity of each monitored area) is The monitored image is output along with the associated production number.

[0041] Next, the monitoring device 100 determines whether to continue monitoring (step S19). If monitoring is to be continued (Y in step S19), the process proceeds to step S16, where the process repeats the extraction of the monitoring target image for each cycle and the output of the monitoring results until monitoring is terminated. Note that in step S19, it may be determined whether the accuracy of the monitoring results is sufficiently ensured, and if the determination is negative, the process proceeds to step S11, where the monitoring target image can be reset (reselected). For example, if the similarity of each monitoring target area with the learning time is equal to or less than a predetermined threshold, it may be determined that the accuracy of the monitoring results is not ensured.

[0042] FIG. 5 shows an example of a display of the monitoring results. In the example shown in FIG. 5, the display 120 displays the product name, production quantity (number of times the monitored image was extracted), and the setting status of the monitored area (monitoring location), and the monitoring results display the similarity of each monitored area MA1 to MA3 with the learning time. In this example, the similarity of monitored area MA1 with the learning time is low, suggesting a problem with the first processing area (either the transport by the finger 41a, the state of the workpiece on the lower mold 21a, or the state of the lower mold 21a). Note that instead of displaying the similarity of each monitored area with the learning time in real time as shown in FIG. 5, the average value (or maximum or minimum value) of the similarity for the most recent predetermined number of cycles (production quantity) may be displayed, or a graph may be displayed showing the temporal change in similarity for each cycle from the start of monitoring to the present.

[0043] According to this embodiment, the press Thailand By extracting images to be monitored from a group of still images constituting video data of an area based on the slide position (and slide movement direction) in the image, it is possible to extract images from the same timing in each cycle during production operation using only the image data (without using information from the control device 30) and perform image monitoring (outputting the similarity with the learning time). This makes it possible to detect misfeeds in materials / products, cracks and breaks in materials, scratches and broken pins in molds, etc. It also offers numerous benefits, such as eliminating the need for sensors used to detect misfeeds, providing opportunities for finger adjustments and reviewing the feed range by quantifying the stability of conveyance as similarity, simplifying product inspection work, and making it possible to determine the need for mold maintenance without stopping the press. Furthermore, because there is no need to input press angle signals or production number counter signals from the control device 30, the monitoring device 100 can be a device independent of the control device 30 in terms of control, making it easy to retrofit the monitoring device 100 to an existing press. Furthermore, according to this embodiment, by setting a partial area of ​​the monitored image as the monitored area and performing learning and monitoring, the processing load can be reduced compared to when learning and monitoring are performed for the entire monitored image.

[0044] In the above example, during learning and monitoring, a case was described in which still images with the same slide position are extracted as images to be monitored from a group of still images constituting video data, using the slide position as a reference. However, a still image at the moment when the processed material comes to a standstill may also be extracted as an image to be monitored from a group of still images constituting video data, using the state of the processed material in the image as a reference. A still image at the moment when the processed material comes to a standstill is a still image taken when transport by the transfer feeder is completed (when the down operation is completed, the material is placed on the lower die, and the fingers are released from the material M by the unclamping operation).

[0045] In the example shown in FIG. 6, the user sets the first processing area in the still image as the monitoring target area MA1, the second processing area as the monitoring target area MA2, and the third processing area as the reference monitoring area RA. The reference monitoring area RA is an area that serves as a reference when extracting the monitoring target image. In this case, the reference monitoring area RA is set as the monitoring target area MA1 from the still image group. A still image in which the area RA is the same as the reference monitoring area RA in the still image of the next frame (a still image captured when the material transported to the third processing area comes to rest) is extracted as the monitoring target image. For example, if the reference monitoring area RA in the still image of the nth frame and the reference monitoring area RA in the still image of the n+1th frame are different (low similarity), and the reference monitoring area RA in the still image of the n+1th frame and the reference monitoring area RA in the still image of the n+2th frame are the same (high similarity), the still image of the n+1th frame is extracted as the monitoring target image. During learning, the monitoring target area MA1 in each monitoring target image extracted for learning is machine-learned to generate a first learning model, the monitoring target area MA2 in each extracted monitoring target image is machine-learned to generate a second learning model, and the slide position in each extracted monitoring target image (the position SP1 of the slide's bottom edge at the left edge of the monitoring target image and the position SP2 of the slide's bottom edge at the right edge or top edge of the monitoring target image) is machine-learned to generate a third learning model. Also, during monitoring, the monitored area MA1 in each monitored image extracted for monitoring is input into a first learning model to output the similarity of the monitored area MA1 with respect to the time of learning, the monitored area MA2 in each extracted monitored image is input into a second learning model to output the similarity of the monitored area MA2 with respect to the time of learning, and the slide positions SP1 and SP2 in each extracted monitored image are input into a third learning model to output the similarity of the slide positions SP1 and SP2 with respect to the time of learning.

[0046] Fig. 7 shows an example of the display of the monitoring results when the monitored image is extracted based on the state of the workpiece material. In the example shown in Fig. 7, the monitoring results display the degree of similarity of each of the monitored areas MA1 and MA2 with the learning time, and the degree of similarity of the slide position with the learning time. In this example, the degree of similarity between the monitored area MA2 and the slide position is high, and the degree of similarity between the monitored area MA1 and the slide position is low, so it is inferred that there is a problem with the first processing area (either the transport by the fingers 41a, the state of the workpiece material on the lower die 21a, or the state of the lower die 21a). Furthermore, if the similarity between the slide position and the monitored area MA1 is high and the similarity between the monitored area MA2 is low, it is inferred that there is a problem in the second processing area (either the transport by the finger 41b, the state of the workpiece on the lower mold 21b, or the state of the lower mold 21b), and if the similarity between either the monitored areas MA1, MA2 and the slide position is low, it is inferred that there is a problem in the third processing area (either the transport by the finger 41c, the state of the workpiece on the lower mold 21c, or the state of the lower mold 21c).

[0047] Furthermore, in the above example, a case where learning and image monitoring are performed using only the monitoring target image extracted from the still image group has been described, but learning and image monitoring may also be performed using still images of n frames (n is a natural number) before and after the monitoring target image in addition to the monitoring target image extracted from the still image group. For example, when monitoring target areas MA1, MA2, and MA3 are set, learning models LM1, LM2, and LM3 are generated by machine learning the monitoring target areas MA1, MA2, and MA3 in the monitoring target image extracted for learning, respectively, and learning models LM1, LM2, and LM3 are generated by machine learning the monitoring target areas MA1, MA2, and MA3 in the still image of the frame before the monitoring target image, respectively. 1p ,LM 2p ,LM 3p and machine learning the monitored areas MA1, MA2, and MA3 in the still images of the frames after the monitored image to generate a learning model LM 1n ,LM 2n ,LM 3nThen, the monitoring target areas MA1, MA2, MA3 in the monitoring target image extracted for monitoring are input to the learning models LM1, LM2, LM3, and the similarity of the monitoring target areas MA1, MA2, MA3 in the monitoring target image with the learning time is output, and the monitoring target areas MA1, MA2, MA3 in the still image of the frame immediately before the monitoring target image are input to the learning models LM 1p ,LM 2p ,LM 3p and outputs the similarity of the monitored areas MA1, MA2, and MA3 in the still image with the learning time, and the monitored areas MA1, MA2, and MA3 in the still image of the frame after the monitored image are input to the learning model LM 1n ,LM 2n ,LM 3n and outputs the similarity between the still image and the time of learning for the monitored areas MA1, MA2, and MA3.

[0048] Figure 8 is a bubble chart showing an example of the monitoring results when learning and image monitoring are performed using still images from n frames before and after the monitored image. In the example in Figure 8, a still image taken when the workpiece is stationary is extracted as the monitored image. For each of the three monitored areas (monitored locations), the similarity of the monitored image to the learning time, the average similarity of the still images from the n frames before the monitored image to the learning time, and the average similarity of the still images from the n frames after the monitored image to the learning time are shown as circles (with a larger diameter as the similarity increases). In this example, for monitored area "1," the similarity of the images from the n frames before the monitored image is low, indicating poor material placement in the first processing area (e.g., the material is being placed while shaking). This suggests a problem with poor allocation angle settings, such as a long overlap between the transfer feeder's advance and down operations or the down and unclamping operations (large overlap amount). Furthermore, for the monitored area "3," the similarity of the images in the subsequent n frames of the monitored image is low, indicating that the material became unstable after being placed in the third processing area, suggesting problems such as a poor structure or condition of the lower mold 21c or a poor position or condition of the fingers 41c. By monitoring images using still images in frames before and after the monitored image (a still image taken when the material to be processed is stationary), it is possible to provide specific improvements to stabilize the transfer. In addition to directly displaying a bubble chart to encourage inference, the image monitoring unit 102 can also calculate a bubble chart and output a judgment result such as "the material is not properly placed."

[0049] Also, the press while transporting the material Thailand The images of the material to be monitored are extracted from the moving image data of the area, and the images of the material are monitored. Thailand The image of the mold may be monitored by extracting an image of the object to be monitored from video image data of the area. Also, an imaging unit may be further provided that images the upper mold from diagonally below, and the image of the object to be monitored may be extracted from video image data captured by the imaging unit to monitor the image of the upper mold.

[0050] Although the embodiments of the present invention have been described in detail above, it will be readily apparent to those skilled in the art that many modifications are possible without substantially departing from the novel features and effects of the present invention. [Explanation of symbols]

[0051] 1...press machine, 10...servo motor, 11...encoder, 12...drive shaft, 13...drive gear, 14...main gear, 15...crankshaft, 16...connecting rod, 17...slide, 18...bolster, 20...upper die, 21...lower die, 22...imaging unit, 30...control device, 100...monitoring device, 101...image extraction unit, 102...image monitoring unit, 110...user interface, 120...display, 130...storage unit

Claims

1. A press machine converts the rotation of a motor into the reciprocating linear motion of a slide to press a workpiece, a storage unit that stores moving image data captured by an imaging unit disposed at a position capable of capturing an image of the press die area; an image extraction unit that extracts a monitoring target image from a group of still images constituting the moving image data, based on the position and movement direction of the slide in the image detected from the still images, or the state of the workpiece in the image detected from the still images; an image monitoring unit that performs image monitoring based on the image of the object to be monitored and outputs the monitoring results.

2. In claim 1, The image extraction unit A press machine, characterized in that a still image in which the slide is in the same position is extracted as the monitoring target image from the group of still images.

3. In claim 1, The image extraction unit A press machine characterized in that a still image at a timing when the workpiece is stationary is extracted from the group of still images as the monitoring target image.

4. In any one of claims 1 to 3, The image monitoring unit A press machine characterized in that a monitoring target area corresponding to an area where the workpiece is processed is set in the monitoring target image, and the image of the monitoring target area is monitored.

5. In claim 4, The image monitoring unit extracted from the still image group of the moving image data captured in the presence of the workpiece material and setting the monitoring target area by comparing the image of the monitoring target extracted from the group of still images of the moving image data captured in a state in which the workpiece is not present.

6. In any one of claims 1 to 3, The image monitoring unit a learning model generated by machine learning the image to be monitored acquired in advance during normal operation, the learning model being used to output, as the monitoring result, the degree of similarity between the image to be monitored acquired during monitoring and the image to be monitored acquired at the time of learning.

7. In any one of claims 1 to 3, The image monitoring unit a press machine, characterized in that the press machine uses each learning model generated by machine learning each of still images of frames before and after the monitored image acquired in advance during normal operation, and outputs as the monitoring result the degree of similarity between still images of frames before and after the monitored image acquired during monitoring and those at the time of learning.

8. 1. An image monitoring method for a press machine that converts the rotation of a motor into a reciprocating linear motion of a slide to press a workpiece, comprising: a storage step of storing moving image data captured by an imaging unit disposed at a position capable of capturing an image of the press die area; an image extraction step of extracting a monitoring target image from a group of still images constituting the moving image data, based on the position and movement direction of the slide in the images detected from the still images, or the state of the workpiece in the images detected from the still images; an image monitoring step of performing image monitoring based on the image to be monitored and outputting a monitoring result.

Images