Press brake, image output device, and image output method

The press brake system addresses visibility issues by using an imaging and ranging device to generate a focused image of the gazed object, enhancing user efficiency in recognizing target objects.

JP7591375B2Active Publication Date: 2024-11-28AMADA CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2020157478
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-09-18
Publication Date
2024-11-28
Estimated Expiration
2040-09-18

Smart Images

  • Figure 0007591375000001
    Figure 0007591375000001
  • Figure 0007591375000002
    Figure 0007591375000002
  • Figure 0007591375000003
    Figure 0007591375000003
Patent Text Reader

Abstract

To provide a press brake by which a gaze object can be efficiently viewed from a captured image.SOLUTION: A camera 50 captures an image of a work area where a bending process is performed on an upper table 26 and a lower table 22 in a press brake body 10, and outputs a captured image ID. A distance measuring device 55 detects a distance from an object displayed on the captured image ID and generates distance data DD in which the object and the distance are associated. Based on the distance data DD, an image processing device 60 generates a gaze image CD in which a gaze object, which is an object to be gazed in the work area, is clipped from the captured image ID, and outputs the gaze image CD to a target device usable by a user.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a press brake, an image output device, and an image output method. [Background technology]

[0002] Patent Document 1 discloses that a working area of ​​a press brake is imaged by a plurality of imaging devices, and the images captured by the plurality of imaging devices are displayed on a display means. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2019 / 0176201 Summary of the Invention [Problem to be solved by the invention]

[0004] However, since the structure of the press brake is complicated, various objects are reflected in the captured image captured by the imaging device in addition to the gaze object that the user wants to gaze at. In addition, the press brake is generally made of similar colors. Therefore, when the user checks the display means, it is difficult for the user to efficiently gaze at the gaze object in the captured image.

[0005] The present invention has been made in consideration of the above problems, and has an object to provide a press brake, an image output device, and an image output method that enable efficient visual recognition of a focused object from within a captured image.

[0006] According to a first aspect of one or more embodiments, a press brake is provided that includes a press brake body, an imaging device, a distance measuring device, and a video processing device. The press brake body includes an upper table that holds an upper die and a lower table that holds a lower die, and the upper table moves up and down relative to the lower table to perform bending processing on a plate-shaped workpiece. The imaging device captures an image of a working area in the press brake body where the upper table and the lower table perform bending processing, and outputs the captured image. The distance measuring device detects a distance to an object shown in the captured image, and generates distance data in which the object and the distance are associated. The video processing device generates a gaze image by cutting out a gaze object, which is an object to be gazed at in the working area, from the captured image based on the distance data, and outputs the gaze image to a target device available to the user.

[0007] According to a second aspect of one or more embodiments, there is provided an image output device having an imaging device, a distance measuring device, and a video processing device. The imaging device captures an image of a work area where a plate-shaped workpiece is bent by an upper table that holds an upper die moving up and down relative to a lower table that holds a lower die, and outputs the captured image. The distance measuring device detects a distance to an object shown in the captured image, and generates distance data in which the object and the distance are associated with each other. The video processing device generates a gaze image by cutting out a gaze object, which is an object to be gazed at within the work area, from the captured image based on the distance data, and outputs the gaze image to a target device available to the user.

[0008] According to a third aspect of one or more embodiments, there is provided an image output method that executes the following steps. In a first step, an image capture device captures an image of a working area in a press brake body where an upper table and a lower table perform bending processing, and outputs the captured image. In a second step, a distance to an object shown in the captured image is detected, and distance data is generated in which the object and the distance are associated with each other. In a third step, a gaze image is generated by cutting out a gaze object, which is an object to be gazed at within the working area, from the captured image based on the distance data, and in a fourth step, the gaze image is output to a target device available to the user. Effect of the Invention

[0009] According to the present invention, a gaze object can be efficiently visually recognized from within a captured image. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a front view showing a schematic overall configuration of a press brake according to a first embodiment. [Diagram 2] FIG. 2 is a side view illustrating a schematic diagram of the overall configuration of the press brake according to the first embodiment. [Diagram 3] FIG. 3 is a block diagram showing the configuration of a control system of the press brake according to the first embodiment. [Figure 4] FIG. 4 is an explanatory diagram showing the distance from the distance measuring device to the upper end surface and the lower end surface of the gaze area. [Diagram 5] FIG. 5 is an explanatory diagram showing the distance distribution on the upper end surface and the distance distribution on the lower end surface. [Figure 6] FIG. 6 is an explanatory diagram of definition information that defines the fixation area. [Figure 7] FIG. 7 is a flowchart showing the operation of the press brake according to the first embodiment. [Figure 8] FIG. 8 is an explanatory diagram showing the concept of alignment between a captured image and distance data. [Figure 9] FIG. 9 is an explanatory diagram showing a captured image and a gaze image generated from the captured image by the clipping process. [Figure 10] FIG. 10 is an explanatory diagram showing another form of the gaze image. [Figure 11] FIG. 11 is a side view illustrating a schematic diagram of an overall configuration of a press brake according to the second embodiment. [Figure 12] FIG. 12 is an explanatory diagram showing definition information that defines the gaze area. [Figure 13] FIG. 13 is an explanatory diagram showing a concept of correcting definition information. [Figure 14]FIG. 14 is an explanatory diagram showing a concept of correcting definition information. [Figure 15] FIG. 15 is a block diagram showing the configuration of a control system for a press brake in which a captured image and distance data are obtained by a single sensor unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] (First embodiment) The press brake according to the present embodiment will be described with reference to the drawings. The press brake includes a press brake body 10, an NC device 40, a camera 50, a distance measuring device 55, an image processing device 60, and a display device 70.

[0012] The configuration of the press brake body 10 will be described with reference to Figures 1 to 3. 、 "U" and "D" stands for forward, backward, left, right , upward direction and downward, respectively.

[0013] 1 or 2, a press brake body 10 is a processing machine that performs bending processing on a plate-shaped workpiece (sheet metal) W using a pair of dies. The press brake body 10 includes a lower table 22 and an upper table 26.

[0014] The lower table 22 is provided at the bottom of the main body frame 16 and extends in the left-right direction. The lower table 22 holds the die 14, which is a lower metal mold. A lower metal mold holder 24 is attached to the upper end side of the lower table 22, and the die 14 is attached to the lower metal mold holder 24.

[0015] The upper table 26 is provided on the upper part of the main body frame 16 and extends in the left-right direction. The upper table 26 is provided above the lower table 22 so as to face the lower table 22. The upper table 26 holds the punch 12, which is an upper die. An upper die holder 28 is attached to the lower end side of the upper table 26, and the punch 12 is attached to the upper die holder 28.

[0016] The upper table 26 is configured to move up and down relative to the lower table 22 by raising and lowering a pair of hydraulic cylinders 30 provided on the left and right. Each hydraulic cylinder 30 is raised and lowered by operating an actuator mainly composed of a pump and a motor. The vertical position of the upper table 26 is detected by a position detection sensor such as a linear encoder (not shown). Position information detected by the position detection sensor is supplied to the NC device 40.

[0017] The press brake body 10 may be configured to move the lower table 22 up and down, instead of the upper table 26 up and down. In other words, the upper table 26 may be configured to move up and down relative to the lower table 22.

[0018] A table cover (not shown) that covers the upper table 26 is fixedly attached to the main body frame 16. Even if the upper table 26 moves up and down, the table cover does not move up and down but remains stationary.

[0019] In the press brake body 10, the workpiece W is placed, for example, on the die 14. When the upper table 26 is lowered, the workpiece W is sandwiched between the punch 12 and the die 14 and bent.

[0020] A foot switch 36 is provided in front of the lower table 22 and is operated by the worker M. When the worker M operates the foot switch 36, the foot switch 36 outputs an activation signal. The activation signal is a signal for starting the lowering operation of the upper table 26.

[0021] A back gauge 38 is provided behind the lower table 22 to position the workpiece W in the front-rear direction relative to the die 14. The back gauge 38 has an abutment member 39 against which the end face of the workpiece W can abut. The abutment member 39 protrudes forward from the back gauge 38. The position of the abutment member 39 in the front-rear direction is adjustable.

[0022] In the press brake body 10, the three-dimensional space including the lower table 22 and its surroundings, and the upper table 26 and its surroundings corresponds to a working area in which the lower table 22 and the upper table 26 perform bending. Within this working area, a gaze area GR that should be gazed upon within the working area is defined.

[0023] The fixation area GR is in the left-right direction, top and bottom The gaze area GR is a three-dimensional space of a substantially cube extending in the forward and backward directions. As shown in Fig. 1 and Fig. 2, the upper end face Fa1 of the gaze area GR is set to a size smaller than the lower end face Fa2 of the gaze area GR, and the gaze area GR has a shape that narrows toward the top. The reason that the upper end face Fa1 of the gaze area GR is set to a size smaller than the lower end face Fa2 of the gaze area GR is because it conforms to the angle of view of the camera 50 and the distance measuring device 55.

[0024] For example, the left-right range of the gaze area GR is set so as to include the die 14 attached to the lower die holder 24 and the punch 12 attached to the upper die holder 28. The up-down range of the gaze area GR is set so as to include the upper end side of the lower table 22 and the lower end side of the upper table 26. The up-down range is based on when the upper table 26 is in the highest position (fully open position).

[0025] The range of the gaze area GR in the front-rear direction is set so that a predetermined distance is secured in both the front and rear directions from the center of the lower table 22 and the upper table 26. The predetermined distance is determined in consideration of the length of the workpiece W in the front-rear direction, the distance from the lower table 22 to the backgage 38, etc.

[0026] The gaze area GR thus set includes gaze objects that are objects to be gazed at within the working area. The gaze objects are, for example, the punch 12, the die 14, the back gauge 38, the workpiece W placed on the press brake body 10, and the hands and arms of the worker M.

[0027] The gaze area GR may be set to a certain range and position regardless of the size of the workpiece W, the layout of the punch 12 and the die 14, and the position of the abutting member 39. However, the gaze area GR may be set to a range and position according to the layout of the punch 12 and the die 14 and the position of the abutting member 39.

[0028] The NC device 40 is a control device that controls the press brake body 10. The NC device 40 drives the pair of hydraulic cylinders 30 to move up and down to control the vertical movement of the upper table 26. The NC device 40 controls the vertical position of the upper table 26 based on position information detected by a position detection unit.

[0029] The camera 50 is an imaging device that captures an image of the work area centered on the gaze area GR and outputs a captured image ID. The camera 50 is attached to the table cover of the upper table 26 and is disposed behind the upper table 26. The camera 50 captures the gaze area GR and the surrounding area from above the gaze area GR. The camera 50 attached to the table cover does not move up and down even if the upper table 26 moves up and down, but maintains the same position.

[0030] When the worker M works, the worker M stands in front of the press brake body 10 so as to face the press brake body 10. The line of sight of the worker M is obstructed by the upper table 26, punch 12, lower table 22, die 14, etc., so the visibility of the work area behind the upper table 26 and the lower table 22 is reduced. The work area behind the upper table 26 and the lower table 22 includes the backsides of the punch 12 and the die 14, the back gauge 38, and the rear area of ​​the work W that is abutted against the back gauge 38. By capturing an image of the work area from above and behind the upper table 26 with the camera 50, the rear work area where the visibility of the worker M is low can be covered within the imaging range of the camera 50.

[0031] Here, the situation in which the camera 50 captures an image of the work area may be such that the gaze area GR is included within the imaging range, i.e., the angle of view, of the camera 50. Therefore, when observed from the camera 50, the situation may be such that part of the gaze area GR is blocked by structures of the press brake body 10, such as the upper table 26, the upper die holder 28, the punch 12, the lower table 22, the lower die holder 24, and the die 14.

[0032] The camera 50 includes an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 50 may be equipped with a wide-angle lens or a fisheye lens so as to capture a wide range. The camera 50 captures an image of the working area in response to a control signal from the video processing device 60, and acquires a captured image ID. The camera 50 outputs the acquired captured image ID to the video processing device 60.

[0033] As shown in FIG. 3, the distance measuring device 55 is a distance measuring sensor that detects the distance to an object included in the observation range and generates distance data DD in which the object and the distance are associated. The distance measuring device 55 includes, for example, a light projector such as an LED that projects light onto the observation range, and an image sensor. The distance measuring device 55 measures the TOF (Time Of Flight) for each pixel, which is the time from when the light projector emits light until the light is reflected by the object and received by the image sensor. The distance measuring device 55 detects the distance to the object for each pixel based on the measurement result for each pixel. The distance data DD generated by the distance measuring device 55 corresponds to a two-dimensional distribution of distances in the observation range.

[0034] figure 2 2, the distance measuring device 55, like the camera 50, is attached to the table cover of the upper table 26 and is disposed behind the upper table 26. The distance measuring device 55 is disposed in approximately the same position as the camera 50, and the observation range of the distance measuring device 55 also approximately coincides with the imaging range of the camera 50. In this way, the distance measuring device 55 detects the distance from above the gaze area GR to an object present in the gaze area GR and its surrounding area. In other words, the distance measuring device 55 detects the distance to an object shown in the captured image ID of the camera 50, and generates distance data DD.

[0035] Similar to the camera 50 , the distance measuring device 55 acquires distance data DD in response to a control signal from the video processing device 60 , and outputs the acquired distance data DD to the video processing device 60 .

[0036] The resolution (number of pixels) of the distance data DD generated by the distance measuring device 55 is the same as the resolution (number of pixels) of the captured image ID output from the camera 50. However, the resolution (number of pixels) of the distance data DD may be different from the resolution (number of pixels) of the captured image ID.

[0037] 3, the video processing device 60 displays the captured image ID captured by the camera 50 on the display device 70. The video processing device 60 does not directly display the captured image ID capturing the state in which the press brake body 10 performs bending processing, but displays a gaze image CD obtained by cutting out a gaze object from the captured image ID.

[0038] The video processing device 60 outputs control signals to the camera 50 and the distance measuring device 55 at a predetermined cycle, periodically acquires captured image ID from the camera 50, and periodically acquires distance data DD from the video processing device 60. The control signals output from the video processing device 60 to the camera 50 and the distance measuring device 55 are synchronized. Therefore, the timing of imaging by the camera 50 and the timing of distance measurement by the distance measuring device 55 are synchronized.

[0039] The video processing device 60 is composed of a microcomputer mainly composed of a CPU, a ROM, an OBM, an I / O interface, etc. The CPU of the video processing device 60 reads various programs and data corresponding to the processing contents from the ROM, etc., expands them in the RAM, and executes the various expanded programs, so that the microcomputer functions as multiple information processing circuits provided in the video processing device 60. In this embodiment, an example is shown in which the multiple information processing circuits provided in the video processing device 60 are realized by software, but dedicated hardware for executing each information processing circuit may also be provided.

[0040] The video processing device 60 includes a storage unit 61, an object identification unit 62, an image extraction unit 63, and an image output unit 64 as a plurality of information processing circuits.

[0041] The memory unit 61 stores definition information that defines the range of the gaze region GR. The definition information defines the range of the gaze region GR according to the distance from the distance measuring device 55. As shown in FIGS. 1 and 2, the upper end surface Fa1 of the gaze region GR is a horizontal plane corresponding to the upper end of the gaze region GR. When the vertical direction is used as a reference, the distance between the distance measuring device 55 and the upper end surface Fa1 is a distance D1 at any location on the upper end surface Fa1. On the other hand, the lower end surface Fa2 of the gaze region GR is a horizontal plane corresponding to the lower end of the gaze region GR. When the vertical direction is used as a reference, the distance between the distance measuring device 55 and the lower end surface Fa2 is a distance D2 at any location on the lower end surface Fa2.

[0042] As shown in Fig. 4, a reference plate of the same size as the upper end surface Fa1 is placed on the upper end surface Fa1, and the distance to the reference plate is measured by the distance measuring device 55. The distance measuring device 55 receives light reflected at an observation point OBij on the reference plate for each pixel constituting the distance data DD, and measures the linear distance Dij between the observation point OBij and the distance measuring device 55. By performing such measurements, the distance distribution of the upper end surface Fa1 is measured by the distance measuring device 55. The distance distribution of the upper end surface Fa1 has a unique distance Dij for each position of the upper end surface Fa1, i.e., for each observation point OBij. The same is true for the distance distribution of the lower end surface Fa2.

[0043] The information stored in the storage unit 61 includes a distance distribution of the upper end face Fa1 and a distance distribution of the lower end face Fa2. As shown in Fig. 5, the distance distribution of the upper end face Fa1 is information indicating a two-dimensional distance distribution when the upper end face Fa1 of the gaze region GR is observed from the distance measuring device 55, and a distance Dij is associated with each pixel. The distance distribution of the lower end face Fa2 is information indicating a two-dimensional distance distribution when the lower end face Fa2 of the gaze region GR is observed from the distance measuring device 55, and a distance Dij is associated with each pixel.

[0044] In the memory unit 61 memoryFrom the obtained information, the distance to the upper end surface Fa1 of the gaze region GR and the distance to the lower end surface Fa2 of the gaze region GR can be recognized. Therefore, as shown in Fig. 6, the range from the distance to the upper end surface Fa1 of the gaze region GR to the distance to the lower end surface Fa2 of the gaze region GR functions as definition information indicating the range of the gaze region GR.

[0045] The method of acquiring the distance distribution of the upper end face Fa1 and the distance distribution of the lower end face Fa2 may be other than placing a reference plate on the upper end face Fa1 and the lower end face Fa2 and actually measuring the distance to the reference plate. For example, the distance distribution of the upper end face Fa1 and the distance distribution of the lower end face Fa2 may be obtained by a geometric calculation by taking into account the positional relationship between the gaze area GR and the distance measuring device 55.

[0046] The object identification unit 62 identifies the gazed object based on the definition information and the distance data DD output from the distance measuring device 55. The object identification unit 62 identifies, as the gazed object, an object whose distance Dij to the object is within the gazed region GR among the objects shown in the captured image ID.

[0047] The image cutout unit 63 generates a gaze image CD by cutting out the gaze object from the captured image ID. The gaze image CD generated by the image cutout unit 63 is output to the image output unit 64.

[0048] The image output unit 64 outputs the gaze image CD to the display device 70. The image output unit 64 may also output the gaze image CD to the image storage device 80. The image storage device 80 can store a predetermined capacity of gaze images CD. Users including the worker M can use the gaze images CD stored in the image storage device 80 via a computer.

[0049] The display device 70 includes a display panel 71 that displays information. The display device 70 is disposed at a position that is visible to the operator M who operates the foot switch 36. The gaze image CD output from the image output unit 64 is displayed on the display panel 71 of the display device 70. The display device 70 may switch between displaying the gaze image CD output from the image output unit 64 and the captured image ID in response to an operation by the operator M. In addition to the gaze image CD, the display panel 71 of the display device 70 may also display information output from the NC device 40.

[0050] An image output method according to this embodiment will be described with reference to the flowchart shown in Fig. 7. The process shown in this flowchart is executed by the video processing device 60 at a predetermined cycle.

[0051] In step S10, the image cropping unit 63 outputs a control signal to the camera 50 and acquires the captured image ID from the camera 50.

[0052] In step S 11 , the object identification unit 62 outputs a control signal to the distance measuring device 55 and acquires distance data DD from the distance measuring device 55 .

[0053] For ease of explanation, the processing in step S10 and the processing in step S11 are separated. However, it is desirable that the processing in step S10 and the processing in step S11 are executed so that the timing of capturing the captured image ID and the timing of measuring the distance to the object are synchronized.

[0054] The distance data DD output from the distance measuring device 55 is pre-processed to correspond to the captured image ID. When the camera 50 and the distance measuring device 55 are offset in position, the position (coordinates) of pixel PAij of the captured image ID corresponding to the same object may be misaligned with the position (coordinates) of pixel PBij of the distance data DD, as shown in Fig. 8. The object identification unit 62 corrects the distance data DD using a known image processing technique such as coordinate conversion. As a result, the coordinates of pixel PBij of the distance data DD are corrected to the coordinates of pixel PCij corresponding to the coordinates of pixel PAij of the captured image ID.

[0055] The correction process of the distance data DD is performed not only on a specific pixel but on all pixels constituting the distance data DD. This causes the coordinates of the distance data DD for the same object to match the coordinates of the captured image ID. Therefore, the distance data DD corresponds to data in which the object reflected in the pixel PCij and the distance are associated with each pixel PCij corresponding to the captured image ID. The correction process of the distance data DD does not need to be performed by the distance measuring device 55, and may be performed by the object identification unit 62 that has acquired the distance data DD.

[0056] In step S12, the object identification unit 62 extracts a target pixel PCij to be processed from among the pixels PCij constituting the distance data DD. When the process of step S12 is executed for the first time, the object identification unit 62 extracts a preset pixel PCij, such as the pixel PCij located at the upper left in the distance data DD, as the target pixel PCij. On the other hand, when the process of step S12 is executed for the second or subsequent time, the object identification unit 62 extracts the target pixel PCij according to processing coordinate information updated in step S17, which will be described later.

[0057] In step S13, the object identification unit 62 refers to the definition information stored in the storage unit 61, and identifies the range of the fixation region GR that corresponds to the coordinates of the target pixel PCij.

[0058] In step S14, the object identification unit 62 judges whether or not the distance Dij of the target pixel PCij is within the fixation region GR. Specifically, the object identification unit 62 judges whether or not the distance Dij of the target pixel PCij is equal to or greater than the distance to the upper end face Fa1 of the fixation region GR and equal to or less than the distance to the lower end face Fa2 of the fixation region GR. If the distance of the target pixel PCij is within the fixation region GR, the result in step S14 is affirmative, and the process proceeds to step S15. On the other hand, if the distance of the target pixel PCij is outside the fixation region GR, the result in step S14 is negative, and the process skips step S15 and proceeds to step S16.

[0059] In step S15, the object identifying unit 62 identifies the target pixel PCij as the gazed object, and outputs the coordinate information of the target pixel PCij to the image clipping unit 63 as coordinate information of the gazed object.

[0060] In step S16, the object identification unit 62 judges whether the processing is complete. If all the pixels PCij constituting the distance data DD have not been extracted as the target pixel PCij, the object identification unit 62 judges that the processing is incomplete. In this case, the result in step S16 is negative, so the process proceeds to step S17. On the other hand, if all the pixels PCij constituting the distance data DD have been extracted as the target pixel PCij, the object identification unit 62 judges that the processing is complete. In this case, the result in step S16 is positive, so the process proceeds to step S18.

[0061] In step S17, the object identification unit 62 updates the processing coordinate information. The processing coordinate information is information that specifies the pixel PCij to be extracted as the target pixel PCij in the processing of step S12. The object identification unit 62 updates the processing coordinate information so that the pixel PCij that has not yet been extracted in the distance data DD becomes the target pixel PCij.

[0062] In step S18, the image cropping unit 63 performs a process of cropping the gaze target from the captured image ID as shown in FIG. 9, and generates the cropped image as a gaze image CD. The image cropping unit 63 refers to the coordinate information of the gaze object identified by the object identification unit 62, and extracts pixels PAij corresponding to the information of the gaze object from the captured image ID. The image cropping unit 63 generates an image composed of the extracted pixels PAij as a gaze image CD. The image cropping unit 63 outputs the gaze image CD to the image output unit 64.

[0063] In step S19, the image output unit 64 outputs the gaze image CD to the display device 70. The display device 70, to which the gaze image CD has been input, displays the gaze image CD on the display panel 71.

[0064] In this manner, the press brake of this embodiment is equipped with a video processing device 60 that generates a gaze image CD by extracting a gaze object, which is an object to be gazed at within the work area, from a captured image ID based on the distance data DD, and outputs the gaze image CD to a target device that can be used by the user.

[0065] According to this configuration, by generating the distance data DD, it is possible to recognize the distance to an object shown in the captured image ID. Since the distance data DD is data in which an object is associated with a distance, the video processing device 60 can distinguish, among the objects captured by the camera 50, a gazed object that exists within the gaze region GR from an object that exists outside the gaze region GR by referring to the distance data DD. This allows the video processing device 60 to Captured image ID A gaze image CD can be generated by cutting out the gaze object from the image. Then, the video processing device 60 outputs the gaze image CD to the target device, so that the user can use the gaze image CD via the target device. Since the gaze image CD is an image cut out of the gaze object, the gaze object can be more easily visually recognized than when the captured image ID is directly viewed. As a result, the gaze object can be efficiently visually recognized when observing the image.

[0066] In this embodiment, the target device is a display device 70 that is visually recognized by an operator M of the press brake body 10. According to this configuration, by using the display device 70, the operator M can check the gaze image CD in real time. Then, when observing the image, the operator M can efficiently visually recognize the gaze object.

[0067] However, the target device may be an image storage device 80 that can be used by a user such as a worker M via a computer. According to this configuration, by using the image storage device 80 via a computer, the user can check the gaze image CD at the required timing. Then, when observing the image, the gaze object can be efficiently visually recognized.

[0068] In this embodiment, the gaze area GR is a three-dimensional space set between the lower end side of the upper table 26 and the upper end side of the lower table 22.

[0069] According to this configuration, the object involved in the bending process is included in the gaze region GR. This allows the video processing device 60 to appropriately generate a gaze image CD from which the object involved in the bending process is cut out. Since the gaze image CD is an image from which the gaze object is cut out, the gaze object can be more easily visually recognized than when the captured image ID is directly viewed. As a result, the gaze object can be efficiently visually recognized when observing the image.

[0070] In this embodiment, the viewing area GR is set so as to include the punch 12, the die 14, the back gauge 38 against which the workpiece W is abutted, and the workpiece W.

[0071] According to this configuration, the video processing device 60 can appropriately generate a gaze image CD in which a gaze object such as the punch 12, the die 14, the back gauge 38, and the workpiece W is cut out. Since the gaze image CD is an image in which the gaze object is cut out, the gaze object can be more easily visually recognized than when the captured image ID is directly viewed. As a result, the gaze object can be efficiently visually recognized when observing the image.

[0072] The gaze area GR may be set to include the hands and arms of the worker M holding the workpiece W. With this configuration, the worker M can easily grasp the positional relationship between the worker M's own body and the workpiece W by grasping the hands and arms displayed on the display device 70. This can improve workability.

[0073] The distance data DD is data in which an object (i.e., a pixel PCij corresponding to an object shown in the captured image ID) is associated with a distance for each of a plurality of pixels PCij corresponding to the captured image ID. According to this configuration, the distance data DD associates an object with a distance for each pixel PCij. The video processing device 60 can extract the gazed object in pixel units by extracting the captured image ID based on the distance data DD. This allows the video processing device 60 to extract the gazed image CD corresponding to the gazed object by simple processing.

[0074] In this embodiment, the distance data DD associates an object with a distance for each pixel PCij. However, the distance data DD is not limited to a single pixel PCij, and may be data in which an object is associated with a distance for each pixel block consisting of a plurality of adjacent pixels PCij. The distance data DD may also be data in which an object is associated with a distance for each pixel block grouped for each object recognized by an image processing technique. In addition, the distance data DD does not need to correspond to all pixels PAij constituting the captured image ID. The distance data DD may be data in which an object is associated with a distance for each pixel, for only a specific pixel PAij selected from all pixels PAij constituting the captured image ID.

[0075] In this embodiment, the image processing device 60 identifies the gazed object among the objects shown in the captured image ID based on the definition information and the distance data DD. With this configuration, the image processing device 60 can recognize the range of the gazed area GR by the distance from the distance measuring device 55 by referring to the definition information. Therefore, the image processing device 60 can identify the gazed object by comparing the distance to the object with reference to the definition information and the distance data DD. This allows the gazed object to be appropriately identified.

[0076] In this embodiment, the definition information includes a distance distribution of the upper end face Fa1 of the gaze region GR and a distance distribution of the lower end face Fa2 of the gaze region GR. With this configuration, the image processing device 60 can recognize an object that is at a distance equal to or greater than the distance recognized from the distance distribution of the upper end face Fa1 and equal to or less than the distance recognized from the distance distribution of the lower end face Fa2 as a gazed object. This allows the image processing device 60 to appropriately identify the gazed object.

[0077] In this embodiment, an object present in the gaze region GR is recognized as a gaze object, and a gaze image CD corresponding to the gaze object is generated. However, the image cropping unit 63 may generate an image by cropping only a more characteristic portion from the gaze image CD. For example, as shown in FIG. 10, the image cropping unit 63 may generate cropped images CD1 and CD2 focusing on the butting position between the workpiece W and the butting member 39 of the back gauge 38, and a cropped image CD3 focusing on the contact position of the punch 12 with the workpiece W. In this case, the image cropping unit 63 may hold information on the coordinates corresponding to the position to be noted, and may further crop an image from the gaze image CD corresponding to the gaze object.

[0078] Second Embodiment A press brake according to the second embodiment will be described with reference to Fig. 11. The press brake according to the second embodiment includes a camera 50 and a distance measuring device 55 arranged behind the upper table 26, as well as a camera 51 and a distance measuring device 56 arranged in front of the upper table 26. Hereinafter, the camera 50 and the distance measuring device 55 arranged behind the upper table 26 will be referred to as the first camera 50 and the first distance measuring device 55, and the camera 51 and the distance measuring device 56 arranged in front of the upper table 26 will be referred to as the second camera 51 and the second distance measuring device 56.

[0079] The first camera 50 and the first distance measuring device 55 are disposed behind the upper table 26. Therefore, in the gaze area GR, an object located in front of the lower table 22 and the upper table 26 may be blocked by structures such as the upper table 26, the punch 12, the lower table 22, and the die 14, and may not be captured by the first camera 50 and the first distance measuring device 55. Also, depending on the angle of view of the first camera 50 and the first distance measuring device 55, it may not be possible to cover the entire gaze area GR. Therefore, the second camera 51 and the second distance measuring device 56 are disposed in front of the upper table 26 to cover the entire gaze area GR.

[0080] 12, the first distance measuring device 55 detects the upper end surface Fa1 of the gaze region GR, thereby acquiring a distance distribution of the upper end surface Fa1 corresponding to the observation range of the first distance measuring device 55. The first distance measuring device 55 detects the lower end surface Fa2 of the gaze region GR, thereby generating a distance distribution of the lower end surface Fa2 corresponding to the observation range of the first distance measuring device 55. Then, from the two distance distributions, first definition information, which is the range of the gaze region GR corresponding to the observation range of the first distance measuring device 55, is defined.

[0081] Similarly, by detecting the upper end surface Fa1 of the gaze region GR by the second distance measuring device 56, the distance distribution of the upper end surface Fa1 corresponding to the observation range of the second distance measuring device 56 is obtained. 1 distance measuring device 55 By detecting the lower end surface Fa2 of the gaze region GR using the distance distribution, a distance distribution of the lower end surface Fa2 corresponding to the observation range of the second distance measuring device 56 is generated. Then, from the two distance distributions, second definition information, which is the range of the gaze region GR corresponding to the observation range of the first distance measuring device 55, is defined.

[0082] In the press brake having such a configuration, the object identification unit 62 of the video processing device 60 identifies a gaze object existing in the gaze area GR based on the first and second definition information and the distance data DD of the first distance measuring device 55 and the second distance measuring device 56. The image clipping unit 63 of the video processing device 60 generates a gaze image CD by clipping the gaze object from the captured image ID.

[0083] The process of identifying the gaze object and the process of generating the gaze image CD can be, for example, the following method: That is, the first camera 50 and the first distance measuring device 55, and the second camera 51 and the second distance measuring device 56 perform processing separately, and finally synthesize the gaze image CD cut out from each.

[0084] Specifically, the object identification unit 62 identifies the gaze object based on the first definition information and the distance data DD of the first distance measuring device 55. Then, the image clipping unit 63 generates a gaze image CD by clipping the gaze object identified from the first distance measuring device 55 from the captured image ID of the first camera 50. Similarly, the object identification unit 62 identifies the gaze object based on the second definition information and the distance data DD of the second distance measuring device 56. Then, the image clipping unit 63 generates a gaze image CD by clipping the gaze object identified from the second distance measuring device 56 from the captured image ID of the second camera 51. Finally, the image clipping unit 63 synthesizes the gaze image CD clipped from the captured image ID of the first camera 50 and the gaze image CD clipped from the captured image ID of the second camera 51 so that the images overlap in the areas where the imaging ranges of the first camera 50 and the second camera 51 overlap. In this way, the image clipping unit 63 generates the gaze image CD.

[0085] According to this configuration, by using the multiple cameras 50, 51 and the multiple distance measuring devices 55, 56 in combination, it is possible to efficiently cover the entire gaze area GR.

[0086] (Third embodiment) A press brake according to a third embodiment will be described with reference to Fig. 13 and Fig. 14. In the press brake according to the third embodiment, the image processing device 60 corrects the range of the gaze area GR in accordance with the operation of the press brake main body 10.

[0087] 13, consider a configuration in which distance measuring device 55 is attached to table cover 32 of upper table 26. In this case, distance measuring device 55 attached to table cover 32 maintains the same position without moving up and down even if upper table 26 moves up and down.

[0088] When the upper table 26 moves up and down, the area to be watched changes substantially because the area to be watched GR only needs to include the objects to be watched, such as the punch 12, the die 14, the backgauge 38, the workpiece W, and the hands of the worker M. Therefore, the object identification unit 62 corrects the range of the area to be watched GR by correcting the definition information.

[0089] If the distance between distance measuring device 55 and the lower end side of upper table 26 is "D1", the distance distribution of upper end surface Fa1 stored in memory unit 61 is set based on the distance D1 when upper table 26 is in the fully open position. Therefore, object identification unit 62 corrects the distance distribution of upper end surface Fa1 read from memory unit 61 in accordance with the vertical movement of upper table 26. Specifically, when upper table 26 moves up and down, object identification unit 62 corrects the distance distribution of upper end surface Fa1 assuming that distance D1 varies according to the amount of movement of upper table 26, that is, upper end surface Fa1 moves up and down.

[0090] 14, consider a configuration in which distance measuring device 55 is attached to upper table 26. In this case, distance measuring device 55 attached to upper table 26 moves up and down in accordance with the up and down movement of upper table 26.

[0091] When the distance measuring device 55 moves up and down, the area to be watched changes substantially because the area to be watched GR only needs to include the objects to be watched, such as the punch 12, the die 14, the backgauge 38, the workpiece W, and the hands of the worker M. Therefore, the object identifying unit 62 corrects the range of the area to be watched GR by correcting the definition information.

[0092] If the distance between the distance measuring device 55 and the upper end side of the lower table 22 is "D2", the distance distribution of the lower end surface Fa2 stored in the memory unit 61 is set based on the distance D2 when the upper table 26 is in the fully open position. TaThe distance distribution of the lower end surface Fa2 is corrected. Specifically, the object identification unit 62 corrects the distance distribution of the lower end surface Fa2 assuming that when the distance measuring device 55 moves up and down, the distance D2 varies in accordance with the amount of movement of the distance measuring device 55, that is, that the lower end surface Fa2 moves up and down.

[0093] In this manner, in this embodiment, the image processing device 60 can correct the distance distribution of the lower end surface Fa2 in response to the vertical movement of the upper table 26. According to this configuration, the image processing device 60 corrects the distance distribution of the lower end surface Fa2 in response to the vertical movement of the distance measuring device 55 linked to the upper table 26. This makes it possible to optimize the range of the gaze region GR in response to the vertical movement of the distance measuring device 55, so that only the necessary gaze object can be cut out as a gaze image.

[0094] Furthermore, in this embodiment, the image processing device 60 can correct the distance distribution of the upper end surface Fa1 in response to the vertical movement of the upper table 26. According to this configuration, the image processing device 60 corrects the distance distribution of the upper end surface Fa1 in response to the vertical movement of the upper table 26. This makes it possible to optimize the range of the gaze region GR in response to the vertical movement of the upper table 26, so that only the necessary gaze object can be cut out as a gaze image.

[0095] In the above-described embodiment, the camera 50 and the distance measuring device 55 are configured as separate devices. As shown in Fig. 15, the camera 50 and the distance measuring device 55 may be configured as a single sensor 52 unit.

[0096] For example, the sensor 52 includes an image sensor 52a and a light projector 52b. The sensor 52 includes an image output unit 52c that receives a luminance signal indicating the intensity of the reflected light received by the image sensor 52a and outputs an image ID having a luminance indicating the intensity of the reflected signal based on the luminance signal. The sensor 52 also measures the delay time of the light reception timing relative to the light emission timing for each pixel based on the luminance signal, and generates distance data DD indicating the distance to the object.

[0097] According to this configuration, the camera 50 and the distance measuring device 55 can be shared, so that the device can be simplified.

[0098] As described above, the embodiment of the present invention has been described, but the description and drawings forming a part of this disclosure should not be understood as limiting this invention. From this disclosure, various alternative embodiments, examples and operating techniques will become apparent to those skilled in the art.

[0099] For example, the distance measuring device is not limited to a configuration using an imaging element, but may be a configuration that generates a two-dimensional distance distribution using a laser radar or an ultrasonic sensor, etc. Also, the press brake body 10 is configured so that the worker M places the workpiece W thereon, but it may also be configured so that the workpiece W is placed thereon by a transport robot.

[0100] Further, not only the above-mentioned press brake, but also an image output device and an image output method for outputting a gaze image from a captured image and distance data function as part of the present invention. [Explanation of symbols]

[0101] 10 Press brake body 12 Punch (upper die) 14 Die (lower die) 22 Lower Table 26 Upper Table 38 Backgauge 50, 51 Camera (imaging device) 55, 56 Distance measuring device 60 Image Processing Device 61 Storage section 62 Object identification part 63 Image extraction section 64 Image output section 70 Display device 80 Image storage device ID Captured image DD Distance Data CD Gazing Image GR gaze area

Claims

1. A press brake body including an upper table for holding an upper die and a lower table for holding a lower die, the upper table being moved up and down relative to the lower table to perform bending processing on a plate-shaped workpiece; an imaging device that images a working area in the press brake body where the upper table and the lower table perform the bending process and outputs a captured image; a distance measuring device that detects a distance to an object shown in the captured image and generates distance data in which the object and the distance are associated with each other; a video processing device that generates a gaze image by cutting out a gaze object, which is an object to be gazed at within the working area, from the captured image based on the distance data, and outputs the gaze image to a target device that a user can use; The image processing device includes: Identifying an object within a predetermined gaze area as the gaze object among the objects shown in the captured image; the gaze area is a three-dimensional space set between a lower end side of the upper table and an upper end side of the lower table, The image processing device includes: A definition information is stored that defines the range of the gaze area according to the distance from the distance measuring device; Identifying, as the gaze object, an object whose distance in the captured image is within the gaze area based on the definition information and the distance data; The imaging device includes: imaging the gaze area from above the gaze area; The distance measuring device is Detecting a distance from above the gaze area to the object to generate the distance data; The definition information is a distance distribution of an upper end surface, which shows a two-dimensional distance distribution when an upper end surface of the gaze area is observed from the distance measuring device; and a distance distribution of a lower end surface that indicates a two-dimensional distance distribution when a lower end surface of the gaze area is observed from the distance measuring device. Press brake.

2. A press brake body including an upper table for holding an upper die and a lower table for holding a lower die, the upper table moving up and down relative to the lower table to perform bending processing on a plate-shaped workpiece; an imaging device that images a working area in the press brake body where the upper table and the lower table perform the bending process and outputs a captured image; a distance measuring device that detects a distance to an object shown in the captured image and generates distance data in which the object and the distance are associated with each other; a video processing device that generates a gaze image by cutting out a gaze object, which is an object to be gazed at within the working area, from the captured image based on the distance data, and outputs the gaze image to a target device that a user can use; the imaging device outputs a luminance image in which a luminance of an object is detected for each pixel as the captured image; The distance measuring device outputs a distance image in which the time it takes for light reflected by an object to be received is measured for each pixel as the distance data. Press brake.

3. The image processing device identifies an object that is within a predetermined gaze area as the gaze object among the objects shown in the captured image, The gaze area is a three-dimensional space set between the lower end side of the upper table and the upper end side of the lower table.

3. The press brake according to claim 2.

4. The observation area is set to include the upper die, the lower die, a back gauge against which the workpiece is abutted, and the workpiece placed on the press brake body.

4. The press brake according to claim 1 or 3.

5. The distance data is The data is data in which the object and the distance are associated with each of one or more pixels corresponding to the captured image.

5. A press brake according to any one of claims 1 to 4.

6. The image processing device includes: A definition information is stored that defines the range of the gaze area according to the distance from the distance measuring device; Based on the definition information and the distance data, the object shown in the captured image is identified as the gazed object if the distance of the object is within the gazed area.

4. The press brake according to claim 3.

7. The imaging device includes: imaging the gaze area from above the gaze area; The distance measuring device is Detecting a distance from above the gaze area to the object to generate the distance data; The definition information is a distance distribution of an upper end surface, which shows a two-dimensional distance distribution when an upper end surface of the gaze area is observed from the distance measuring device; and a distance distribution of a lower end surface that indicates a two-dimensional distance distribution when a lower end surface of the gaze area is observed from the distance measuring device.

7. The press brake according to claim 6.

8. The upper table is configured to be movable up and down, the distance measuring device is fixed to the upper table and moves up and down in response to the up and down movement of the upper table; The image processing device includes: The distance distribution of the lower end surface is corrected in response to the vertical movement of the upper table.

8. A press brake according to claim 1 or 7.

9. The upper table is configured to be movable up and down, The distance measuring device is fixed immovably regardless of the vertical movement of the upper table, The image processing device includes: The distance distribution of the upper end surface is corrected in response to the vertical movement of the upper table.

8. A press brake according to claim 1 or 7.

10. The target device is: A display device that is visually recognized by the user, i.e., the operator of the press brake body, or a storage device that is available to the user via a computer. A press brake according to any one of claims 1 to 9.

11. An imaging device that images a work area in which a plate-shaped workpiece is bent by relatively moving an upper table and a lower table of the press brake body up and down, and outputs the captured image; a distance measuring device that detects a distance to an object shown in the captured image and generates distance data in which the object and the distance are associated with each other; a video processing device that generates a gaze image by cutting out a gaze object, which is an object to be gazed at within the working area, from the captured image based on the distance data, and outputs the gaze image to a target device that a user can use; The image processing device includes: Identifying an object within a predetermined gaze area as the gaze object among the objects shown in the captured image; the gaze area is a three-dimensional space set between a lower end side of the upper table and an upper end side of the lower table, The image processing device includes: A definition information is stored that defines the range of the gaze area according to the distance from the distance measuring device; Identifying, as the gaze object, an object whose distance in the captured image is within the gaze area based on the definition information and the distance data; The imaging device includes: imaging the gaze area from above the gaze area; The distance measuring device is Detecting a distance from above the gaze area to the object to generate the distance data; The definition information is a distance distribution of an upper end surface, which shows a two-dimensional distance distribution when an upper end surface of the gaze area is observed from the distance measuring device; and a distance distribution of a lower end surface that indicates a two-dimensional distance distribution when a lower end surface of the gaze area is observed from the distance measuring device. Image output device.

12. An imaging device that captures an image of a work area where a plate-shaped workpiece is bent by relatively moving up and down an upper table and a lower table of a press brake body, and outputs the captured image; a distance measuring device that detects a distance to an object shown in the captured image and generates distance data in which the object and the distance are associated with each other; a video processing device that generates a gaze image by cutting out a gaze object, which is an object to be gazed at within the working area, from the captured image based on the distance data, and outputs the gaze image to a target device that a user can use; the imaging device outputs a luminance image in which a luminance of an object is detected for each pixel as the captured image; The distance measuring device outputs a distance image in which the time it takes for light reflected by an object to be received is measured for each pixel as the distance data. Image output device.

13. The imaging device captures an image of a work area in which a plate-shaped workpiece is bent by the upper table and the lower table of the press brake body moving up and down relative to each other, and outputs the captured image. a distance measuring device detects a distance to an object shown in the captured image and generates distance data in which the object and the distance are associated with each other; generating a gaze image by cutting out a gaze object, which is an object to be gazed at within the working area, from the captured image based on the distance data; outputting the gaze image to a target device available to the user; The generating of the gaze image includes: Identifying an object within a predetermined gaze area as the gaze object among the objects shown in the captured image, the gaze area is a three-dimensional space set between a lower end side of the upper table and an upper end side of the lower table, Identifying the gaze object includes: Based on definition information that defines the range of the gaze area by the distance from the distance measuring device and the distance data, an object that is within the gaze area and is displayed in the captured image is identified as the gaze object; The imaging device includes: imaging the gaze area from above the gaze area; The distance measuring device is Detecting a distance from above the gaze area to the object to generate the distance data; The definition information is a distance distribution of an upper end surface, which shows a two-dimensional distance distribution when an upper end surface of the gaze area is observed from the distance measuring device; and a distance distribution of a lower end surface that indicates a two-dimensional distance distribution when a lower end surface of the gaze area is observed from the distance measuring device. Image output method.

14. An imaging device captures an image of a work area in which a plate-shaped workpiece is bent by relatively moving up and down an upper table and a lower table of a press brake body, and outputs the captured image; a distance measuring device detects a distance to an object shown in the captured image and generates distance data in which the object and the distance are associated with each other; generating a gaze image by cutting out a gaze object, which is an object to be gazed at within the working area, from the captured image based on the distance data; outputting the gaze image to a target device available to the user; the imaging device outputs a luminance image in which a luminance of an object is detected for each pixel as the captured image; The distance measuring device outputs a distance image in which the time it takes for light reflected by an object to be received is measured for each pixel as the distance data. Image output method.

Citation Information

Patent Citations

  • Controller, forging pressure machine, and controlling method thereof

    JP2017213593A

  • Press machine and method for controlling the same

    JP2019042766A

  • Safety device of processing machine

    JP2019166529A

  • Bending machine comprising a work area image detecting device and method for representing a work area

    US20190176201A1