Gain adjustment method, information processing device, and imaging system

Abstract

To provide an image of brightness that can be permitted by a user even when there is a change in an ambient environment of industrial equipment having a part that periodically operates.SOLUTION: An information processing device stores the allowable range of evaluation values of brightness about an output image of a camera for imaging a part included in industrial equipment, which periodically operates. The information processing device acquires the output image and data showing a periodical change in the evaluation values of the brightness of the output image on the basis of imaging by the camera after storing the allowable range. The information processing device acquires at least the value of one of the maximum value and the minimum value of the evaluation values of brightness during a prescribed period of one period or more of the most recent to operation from the data. The information processing device adjusts the gain of the camera on condition that the acquired value is not included within the allowable range.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present disclosure relates to a camera gain adjustment method, an information processing device that adjusts the gain of a camera, and an imaging system including a camera and an information processing device. [Background technology]

[0002] Conventionally, as disclosed in JP 2016-39473 A, a device is known that, upon receiving information indicating the occurrence of an event from outside, stores image data captured by an imaging means at a predetermined time before and after the time the event occurred in a storage means.

[0003] As disclosed in International Publication No. 2019 / 008855, a system is also known that uses a camera to capture the operation of industrial machinery (production machinery) such as a press machine and protects video data for a predetermined set period of time, including the time when an abnormality occurred.

[0004] Furthermore, cameras equipped with a function called automatic gain control (AGC) that automatically adjusts the gain during video capture in order to output an image with appropriate brightness are also known. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2016-39473 A [Patent Document 2] International Publication No. 2019 / 008855 Summary of the Invention [Problem to be solved by the invention]

[0006] For example, in a press machine, the slide on which the die is attached moves up and down at a constant cycle. When capturing images of the operation of an industrial machine that has parts that move periodically like this with a camera, the amount of light incident on the camera's image sensor changes continuously in a short cycle.

[0007] However, auto gain control cannot adjust the gain to follow such short-term changes in the amount of light (changes in brightness).In such a situation, when an abnormality occurs in industrial machinery, even if the user checks the image (video) captured by the camera later, the user may not be able to accurately determine the cause of the abnormality due to "crushed black" or "blown white" in the image.

[0008] The present disclosure has been made in consideration of the above problems, and its purpose is to provide a gain adjustment method, an information processing device, and an imaging system that are capable of obtaining an image with a brightness that is acceptable to a user even when there is a change in the surrounding environment of industrial equipment that has parts that operate periodically. [Means for solving the problem]

[0009] According to an aspect of the present disclosure, a method for adjusting a gain of a camera by an information processing device includes a step of storing an allowable range of a brightness evaluation value for an output image of the camera by the information processing device. The camera captures an image of a periodically operating part included in an industrial machine. The gain adjustment method further includes a step of the information processing device acquiring an output image and data indicating a periodic change in the brightness evaluation value of the output image based on the image captured by the camera after the allowable range has been stored, a step of the information processing device acquiring at least one of a maximum value and a minimum value of the brightness evaluation value in a predetermined period of one or more most recent cycles of the operation from the data, and a step of the information processing device adjusting a gain of the camera on condition that the acquired value is not within the allowable range.

[0010] According to another aspect of the present disclosure, an information processing device that adjusts a gain of a camera includes a processor and a memory that stores an allowable range of a brightness evaluation value for an output image of the camera. The camera captures an image of a periodically operating part included in an industrial machine. The processor acquires an output image and data indicating a periodic change in the brightness evaluation value of the output image based on the image captured by the camera after the allowable range has been stored. The processor acquires, from the data, at least one of a maximum value and a minimum value of the brightness evaluation value during a predetermined period of one or more most recent cycles of the operation. The processor adjusts the gain of the camera on condition that the acquired value is not within the allowable range.

[0011] According to yet another aspect of the present disclosure, an imaging system includes a camera that captures an image of a periodically operating part included in an industrial machine, and an information processing device that adjusts a gain of the camera. The information processing device stores an acceptable range of a brightness evaluation value for an output image of the camera. The information processing device acquires an output image and data indicating a periodic change in the brightness evaluation value of the output image based on imaging by the camera after the acceptable range has been stored. The information processing device acquires, from the data, at least one of a maximum value and a minimum value of the brightness evaluation value during a predetermined period of one or more most recent cycles of the operation. The information processing device adjusts the gain of the camera on condition that the acquired value is not within the acceptable range. Effect of the Invention

[0012] According to the above disclosure, even when there is a change in the surrounding environment of an industrial device having a part that operates periodically, it is possible to obtain an image with a brightness that is acceptable to the user. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a press system. [Diagram 2] FIG. 2 is a diagram for explaining the connection relationship of devices constituting the press system. [Diagram 3] FIG. 2 is a block diagram showing the functional configuration of the camera. [Figure 4] FIG. 1 illustrates an example of a hardware configuration of an information processing device. [Diagram 5] 11 is a flowchart illustrating a process of adjusting gain of a camera performed by an information processing device. FIG. [Figure 6] 11 is a diagram for explaining a method of calculating an evaluation value of brightness of an output image. FIG. [Figure 7] 13 is a diagram showing an image captured by a camera while a slide of a press machine body is periodically operating, the image being displayed on a display of an information processing device. FIG. [Figure 8] 11 is a graph for explaining a change in the brightness evaluation value. [Figure 9] 13 is a graph illustrating a change in the brightness evaluation value after the allowable range is set. [Figure 10] 13 is another graph illustrating the change in the brightness evaluation value after the allowable range is set. [Figure 11] 11 is a graph for explaining a change in the brightness evaluation value after a transition from the state of FIG. 10. [Figure 12] FIG. 6 is a flowchart for explaining details of the process flow of step S3 in FIG. 5. [Figure 13] FIG. 2 is a block diagram illustrating a functional configuration of an information processing device. [Figure 14] FIG. 2 is a diagram showing each RGB pixel of the camera's imaging element. [Figure 15] FIG. 13 is a diagram for explaining data interpolation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. The names and functions of the components are also the same. Therefore, detailed description thereof will not be repeated.

[0015] In the following embodiments, as an example of an industrial machine (production machine), a press system having a press machine, which is a type of forging machine, will be described as an example. Note that the industrial machine is not limited to the press system, and any machine having a periodically operating part, such as a slide of a press machine, may be used.

[0016] <A. System Configuration> FIG. 1 is a diagram showing a schematic configuration of a press system 1 according to the present embodiment.

[0017] As shown in FIG. 1, the press system 1 includes a coil holder 10, a level feeder 20, a press machine 30, a transfer conveyor 40, and cameras 61 and 62. The press machine 30 includes a press machine main body 31 and a press controller 32. The press machine main body 31 includes a main body frame 311, a bed 312, a bolster 313, a slide 314, and a warning lamp 319.

[0018] A coil is wound around the coil holder 10, and the coil W is conveyed to the press machine main body 31 via the level feeder 20. In this example, the case of press-working the coil W as a workpiece (material, workpiece) will be described.

[0019] The level feeder 20 adjusts the feeding height position of the coil conveyed from the coil holder 10 to the press machine main body 31 and conveys the coil W to the press machine main body 31 at a predetermined timing.

[0020] The slide 314 is supported so as to be vertically movable at a substantially central portion of the main body frame 311 of the press machine main body 31. Below the slide 314, a bolster 313 attached to the bed 312 is disposed. An upper die 316 is mounted on the lower surface of the slide 314. A lower die 315 is mounted on the upper surface of the bolster 313.

[0021] The press machine body 31 performs press working on the coil W transported from the straightener feeder 20 according to a processing pattern. The press machine body 31 positions the coil W in the lower die 315, which corresponds to a die consisting of an upper die 316 and a lower die 315, and then performs press working by lowering the upper die 316 together with the slide 314.

[0022] The press controller 32 is a device that controls the operation of the press machine main body 31. The press controller 32 periodically moves the slide 314 up and down. As a result, the upper die 316 also periodically moves up and down in accordance with the movement of the slide 314.

[0023] The press controller 32 is used to input various data necessary for controlling the press machine main body 31, and has switches and a numeric keypad for inputting data, as well as a display for displaying a setting screen and data output from the press machine 30.

[0024] The press controller 32 is mainly composed of a CPU (Central Processing Unit), a high-speed numerical calculation processor, a memory, etc., and is equipped with a computer device that performs arithmetic operations and logical operations on input data according to a predetermined procedure, and an input / output interface that inputs and outputs command currents.

[0025] The memory of the press controller 32 is configured to include appropriate storage media such as a ROM (Read Only Memory) and a RAM (Random Access Memory). This memory stores programs for implementing various functions of the press controller 32. The memory is also used as a work area for executing various arithmetic processing.

[0026] The transport conveyor 40 transports the molded product P formed by press processing in the press machine body 31 to a product box 50. However, without being limited to this, it is also possible to transport the molded product P to the next press machine using a transport robot or the like.

[0027] The warning light 319 is connected to the press controller 32. The warning light 319 is configured to light up based on a signal (a bit indicating an abnormality) sent from the press controller 32 when an abnormality occurs in the press machine main body 31. For example, the warning light 319 flashes a red light when an abnormality occurs in the processing step. Examples of abnormalities include a stop of operation of the press machine main body 31, misfeeding of the workpiece, etc.

[0028] The cameras 61, 62 capture moving images of the machining process of the coil W (workpiece) by the press machine body 31. Moving image data obtained by the cameras 61, 62 is stored in the memory of the information processing device 80 (see FIG. 2).

[0029] The cameras 61 and 62 are, for example, CCD (Charge Coupled Device) cameras. The camera 61 is disposed in the external space of the press machine body 31. The camera 62 is disposed, for example, in the internal space of the press machine body 31 (workpiece machining area).

[0030] The camera 61 is disposed so as to capture as its subjects the coil W and molded product P being transported, and a warning light 319 in addition to the lower mold 315, the upper mold 316, and the slide 314.

[0031] The camera 62 is disposed so as to capture as objects the lower die 315, the upper die 316, and the slide 314. The camera 62 can capture an image of the state of the press working by the lower die 315 and the upper die 316.

[0032] In the press system 1, the camera 61 captures images of the entire state of the press process, and the camera 62 captures images of the details of the processing state.

[0033] The number of cameras in the press system 1 is not limited to two. The arrangement of the cameras is also not limited to the above arrangement. It is preferable that the camera is configured to be capable of capturing an image of the state of press processing using a die. It is also preferable that the camera is configured to be capable of capturing an image of the coil W and the molded product P before and after press processing. It is also preferable that the camera is configured to be capable of capturing an image of the warning light 319.

[0034] Each part of the press system 1 is synchronized, and a series of operations are executed in sequence. The coil W is transported from the coil holder 10 to the press machine body 31 via the straightener feeder 20. Then, the coil W is pressed in the press machine body 31, and the processed workpiece (molded product P) is transported by the transport conveyor 40. The above series of processes is repeated.

[0035] The above-described configuration of the press system 1 is merely an example, and the press system 1 is not limited to this configuration.

[0036] FIG. 2 is a diagram for explaining the connections of devices constituting the press system 1. As shown in FIG. As shown in FIG. 2, the press system 1 includes, in addition to the cameras 61, 62, the press machine body 31, and the press controller 32, a hub 70, and an information processing device 80.

[0037] The cameras 61 and 62 capture moving images. The cameras 61 and 62 are communicatively connected to an information processing device 80 via a hub 70. The cameras 61 and 62 are wired and connected to the hub 70 by LAN (Local Area Network) cables L1 and L2, such as Ethernet (registered trademark) cables.

[0038] The cameras 61 and 62 sequentially transmit moving image data obtained by capturing images to the information processing device 80 via the hub 70. The moving image data is made up of a plurality of image frames. Each image frame includes a time stamp.

[0039] The hub 70 is, for example, a switching hub compatible with PoE (Power over Ethernet).

[0040] The press controller 32 is connected to the information processing device 80 via two communication paths: a communication path via the hub 70 and a communication path that does not pass through the hub 70. The press controller 32 is further connected to a warning light 319 via the communication path that does not pass through the hub 70. The press controller 32 is configured to be able to communicate with the server device 120 via the network NW.

[0041] More specifically, the press controller 32 is wired-connected to the hub 70 by a LAN cable L4. The press controller 32 is also connected to the information processing device 80 and the warning light 319 by an I / O (Input / Output) connection cable L9, which has high real-time performance, without going through the hub 70.

[0042] The press controller 32 sequentially transmits operation information indicating the operation state of the press machine body 31 to the information processing device 80 via the hub 70 using the LAN cables L3, L4. Part of the operation state is detected, for example, by a sensor 318 provided in the press machine body 31.

[0043] The operating information is information associated with time and includes at least various types of waveform data, various types of input / output signal data (on / off signals), and basic information.

[0044] Furthermore, if an abnormality occurs in the press machine body 31, the press controller 32 transmits a bit indicating the abnormality to the information processing device 80 and the warning light 319 via the I / O connection cable L9, without going through the hub 70.

[0045] In this example, the information processing device 80 includes a tablet terminal 801 and a cradle 802. The tablet terminal 801 is placed on the cradle 802. The tablet terminal 801 is communicatively connected to the hub 70 and the press controller 32 via the cradle 802. The cradle 802 is wired to the hub 70 by a LAN cable L3. The information processing device 80 can access the server device 120 via the hub 70 and the network NW.

[0046] When the information processing device 80 receives a bit indicating an abnormality from the press controller 32, it protects the video data and operation information for a predetermined set time (hereinafter referred to as the "set time Ts") that includes the time when the bit was received (the time when the abnormality occurred). As one example, the set time Ts can be 5 seconds immediately before the time when the abnormality occurred and 15 seconds immediately after that time.

[0047] By checking the protected video data and driving information on the information processing device 80, the user can check the video (image) at the time the abnormality occurred as well as the details of the abnormality.

[0048] Instead of the tablet terminal 801 and the cradle 802, the information processing device 80 may be a laptop or desktop personal computer.

[0049] The press machine 30 is an example of an "industrial machine" in the present invention. The information processing device 80 is an example of an "information processing device" in the present invention. The cameras 61, 62 are an example of a "camera" in the present invention. A system configuration consisting of the cameras 61, 62, the information processing device 80, and the hub 70 is an example of an "imaging system" in the present invention.

[0050] Slide 314 and the upper die 316 are an example of the "periodically operating part included in industrial machinery" in the present invention. When the upper die 316 is not attached to the slide 314, only the slide 314 out of the slide 314 and the upper die 316 is an example of the "periodically operating part included in industrial machinery" in the present invention.

[0051] <B. Configuration of Camera> FIG. 3 is a block diagram showing the functional configuration of the cameras 61 and 62.

[0052] As shown in FIG. 3, the cameras 61 and 62 include a band-pass filter 604, a color filter 605, an imaging device 606, an analog signal processing unit 607, an A / D (Analog Digital) converter 608, a digital signal processing unit 609, a communication unit 610, a zoom drive unit 611, a focus drive unit 612, a diaphragm drive unit 613, an imaging device drive unit 614, a control unit 615, and an imaging optical system 630. The analog signal processing unit 607 includes an AGC (Automatic Gain Control) circuit 670.

[0053] The imaging optical system 630 includes a zoom lens 601, a focus lens 602, and a diaphragm unit 603. The zoom lens 601 moves in the optical axis direction to change the focal length. The focus lens 602 moves in the optical axis direction to perform focus adjustment. The diaphragm unit 603 adjusts the amount of light.

[0054] The light that has passed through the imaging optical system 630 forms a subject image on the imaging device 606 via the band-pass filter 604 and the color filter 605. The subject image is photoelectrically converted by the imaging device 606.

[0055] The analog signal output from the imaging device 606 is adjusted in gain by the analog signal processing unit 607. The analog signal whose gain has been adjusted is converted into a digital signal by the A / D converter 608 and then input to the digital signal processing unit 609.

[0056] The digital signal processing unit 609 performs various image processing on the digital imaging signal to generate a video signal. The video signal is output to the information processing device 80 via the communication unit 610. The information processing device 80 sends an instruction to the control unit 615 via the communication unit 610, the details of which will be described later.

[0057] The control unit 615 calculates the setting values ​​for focus control in autofocus and the setting values ​​for exposure control. The control unit 615 instructs the setting values ​​to the analog signal processing unit 607, the zoom driving unit 611, the focus driving unit 612, the aperture driving unit 613, and the image sensor driving unit 614.

[0058] The zoom driver 611 controls the position of the zoom lens 601 based on instructions from the controller 615. The focus driver 612 controls the position of the focus lens 602 based on instructions from the controller 615. The aperture driver 613 controls the aperture of the aperture unit 603 based on instructions from the controller 615. The image sensor driver 614 tilts the image sensor 606 with respect to the optical axis of the imaging optical system 630 based on instructions from the controller 615.

[0059] The analog signal processing unit 607 will now be described in more detail. The analog signal processing unit 607 has a mode in which the AGC circuit 670 is made to function and a mode in which the AGC circuit 670 is made to stop functioning.

[0060] The analog signal processing unit 607 activates or deactivates the AGC circuit 670 in response to an instruction from the control unit 615. An instruction to switch such a function on or off can be given from an operation unit (not shown) of the cameras 61 and 62 or the information processing device 80.

[0061] When the function of the AGC circuit 670 is stopped, the analog signal processing unit 607 amplifies or attenuates the input signal with a gain designated in advance. The gain can be designated from the information processing device 80. The gain can also be set by a user's operation on the operation unit of the camera 61.

[0062] On the other hand, when the AGC circuit 670 is functioning (hereinafter also referred to as "AGC mode"), the AGC circuit 670 automatically adjusts the gain based on the input signal. The AGC circuit 670 amplifies the input signal by increasing the gain. As a result, the AGC circuit 670 brightens the image (video) to be output. The AGC circuit 670 attenuates the input signal by decreasing the gain. As a result, the AGC circuit 670 darkens the image to be output.

[0063] The AGC circuit 670 automatically adjusts the gain to provide an appropriate image brightness. In this manner, the AGC circuit 670 has a function of automatically adjusting the brightness of an image (hereinafter, also referred to as "auto gain control").

[0064] In the press machine 30, the slide 314 moves up and down at a constant cycle. When the operation of the press machine 30 having parts that operate cyclically in this way is captured by the camera 61, the amount of light incident on the imaging element 606 of the camera 61 changes continuously in a short cycle. However, the auto gain control cannot adjust the gain to follow such a short-term change in the amount of light (change in brightness). For this reason, in this embodiment, the auto gain control by the AGC circuit 670 is kept in the off state.

[0065] Furthermore, the user pre-sets (specifies) an appropriate gain value in consideration of the actual operating environment (brightness, etc.) of the press machine 30. Therefore, the analog signal processing unit 607 amplifies or attenuates the input signal at the specified gain (fixed value).

[0066] Details will be described later. In this example, an instruction for gain setting including a gain value is sent from the information processing apparatus 80 to the control unit 615 via the communication unit 610. The control unit 615 sets the gain (value) instructed from the information processing apparatus 80 in the analog signal processing unit 607. Thereby, the analog signal processing unit 607 performs amplification processing or attenuation processing of the input signal with the gain instructed from the information processing apparatus 80. As described above, in this example, the gain in the analog signal processing unit 607 is set by the information processing apparatus 80.

[0067] By the way, when the gain is set (fixed) as described above, the following problems occur. For example, when the operator exits the building where the press machine 30 is installed and the lighting in the building is turned off during the automatic operation of the press machine 30, "black crush" may occur in the images output from the cameras 61 and 62. Further, when the curtain of the building is closed although it is normally opened in the actual operation environment, "black crush" may occur in the images output from the cameras 61 and 62. As a reverse aspect, when the curtain of the building is opened although it is normally closed in the actual operation environment, "white jump" may occur in the images output from the cameras 61 and 62.

[0068] Hereinafter, mainly, the configuration and processing for suppressing the occurrence of such phenomena as "black crush" and "white jump" will be described.

[0069] <C. Hardware Configuration of Information Processing Apparatus> FIG. 4 is a diagram showing an example of the hardware configuration of the information processing apparatus 80.

[0070] As shown in FIG. 4, the information processing apparatus 80 includes a processor 81, a memory 82, communication IFs (Interfaces) 83 and 84, a display 85, and a touch panel 89. Note that the display 85 and the touch panel 89 constitute a touch screen.

[0071] The memory 82 includes, for example, a read only memory (ROM) 821, random access memories (RAMs) 822a and 822b, and a flash memory 823. Note that the memory 82 may include a hard disk drive (HDD) instead of or in addition to the flash memory 823.

[0072] The communication IF 83 is an interface for communicating with the hub 70. The communication IF 84 is an interface for communicating with the press controller 32.

[0073] The processor 81 executes an operating system and various programs stored in the memory 82. The processor 81 causes a display 85 to display various information (see FIG. 6).

[0074] The RAMs 822a and 822b are memories also called cyclic memories. The processor 81 stores the moving image data in the RAM 822a. The processor 81 stores the above-mentioned driving information in the RAM 822b.

[0075] The processor 81 sequentially stores in the RAM 822a moving image data obtained by shooting moving images with the cameras 61 and 62 by overwriting the oldest data with the latest data when the free space in the RAM 822a is insufficient. The processor 81 typically divides the memory area of ​​the RAM 822a into two areas, storing the moving image data from the camera 61 in one area and storing the moving image data from the camera 62 in the other area.

[0076] In addition, the processor 81 sequentially stores the driving information in the RAM 822b by overwriting the oldest data with the latest data when the free space of the RAM 822b is insufficient. Typically, the processor 81 divides the memory area of ​​the RAM 822b into a plurality of areas and stores various data included in the driving information in a divided manner for each area.

[0077] Note that, in the above description, the configuration includes the RAM822a and the RAM822b, but it is not limited thereto. In the case of a configuration in which the processor 81 divides and manages the area of the memory space in one RAM, one RAM may be sufficient. Also, a RAM may be provided for each data type.

[0078] Note that the processor 81 is an example of the "processor" in the present invention. <D. Gain adjustment process> (d1. Overall process flow) Hereinafter, among the cameras 61 and 62, the gain adjustment of the camera 61 will be taken as an example for explanation. Note that since the same gain adjustment as that of the camera 61 is performed in the camera 62, the gain adjustment of the camera 62 will not be repeatedly described.

[0079] FIG. 5 is a flowchart for explaining the processing flow of the gain adjustment of the camera 61 performed by the information processing apparatus 80.

[0080] As shown in FIG. 5, in step S1, the initial value of the gain is set. Although the details will be described later, in step S1, based on a user operation via the information processing apparatus 80, the initial value of the gain of the camera 61 is set. The initial value of the gain in the analog signal processing unit 607 is set. Note that the processing in step S1 corresponds to the pre-setting process of the gain by the user in consideration of the actual operation environment (brightness, etc.) of the press machine 30.

[0081] In step S2, in the information processing apparatus 80, the allowable range of the brightness evaluation value for the output image of the camera 61 is set. Typically, in the information processing apparatus 80, the upper limit value and the lower limit value of the brightness evaluation value for the output image of the camera 61 are set.

[0082] The "brightness evaluation value of the output image" is a value indicating the brightness of the image output from the digital signal processing unit 609 of the camera 61. For example, the brightness evaluation value of the output image is defined by pixel values.

[0083] When the camera 61 converts a color image into a monochrome image (typically a 256-level grayscale image) and outputs the monochrome image to the information processing device 80, the "evaluation value of brightness of the output image" is defined based on the pixel values ​​of each effective pixel of the camera 61. In this case, the "evaluation value of brightness of the output image" is a value within the range of 0 to 255, since it is the average of the pixel values ​​of each effective pixel in this example. Note that if the camera 61 does not have a color filter 605, the above conversion process is not necessary.

[0084] When the camera 61 outputs a color image (typically, an RGB color image), the "evaluation value of the brightness of the output image" is defined based on pixel values ​​after the interpolation process is performed.

[0085] For convenience of explanation, the following description will be given taking as an example a case where the camera 61 outputs a monochrome image to the information processing device 80. A case where the camera 61 outputs a color image will be described in a modified example.

[0086] In step S3, a gain update process is performed. More specifically, the information processing device 80 adjusts the gain and sends an instruction to the camera 61 to set the adjusted gain. The process in step S3 corresponds to a process for suppressing the occurrence of "blackout" and "whiteout."

[0087] The following describes the details of the processes in steps S1 and S2 with reference to Figures 6 to 8. Furthermore, the details of the process in step S3 with reference to Figures 9 to 12.

[0088] (d2. Details of steps S1 and S2) 6 is a diagram for explaining a method for calculating an evaluation value of brightness of an output image. Note that, hereinafter, for convenience, the "evaluation value of brightness of an output image" is also referred to as the "evaluation value M of brightness."

[0089] As shown in Fig. 6, the number of effective pixels in the vertical direction (Y direction) of the camera 61 is H, and the number of effective pixels in the horizontal direction (X direction) is W. In this case, there are H x W effective pixels in the imaging element 606 of the camera 61. In Fig. 6, the pixel value of each pixel is represented by d(i, j) (0 ≤ i ≤ H-1, 0 ≤ j ≤ W-1).

[0090] In this example, the brightness evaluation value M is calculated using the following formula (1). M = ΣΣd(i,j)÷(H×W) … (1) In addition, the ΣΣ term in equation (1) is calculated by taking the sum of all combinations of i and j. In other words, the ΣΣ term is calculated by taking the sum of the pixel values ​​of H×W pixels (more specifically, d(0,0)+d(0,1)+...+d(H-1,W-1)).

[0091] In this way, equation (1) calculates the average value of the pixel values ​​of all valid pixels. In the case of an 8-bit display, the pixel value of each pixel is expressed as a natural number between 0 and 255. Therefore, the value of M is a value between 0 and 255.

[0092] Note that formula (1) is an example of a method for calculating the brightness evaluation value M, and the method for calculating the evaluation value M is not limited to formula (1).

[0093] Figure 7 is a diagram showing an image captured by the camera 61 while the slide 314 of the press machine body 31 is operating periodically, displayed on the display 85 of the information processing device 80 (in this example, the display of the tablet terminal 801).

[0094] 7, the information processing device 80 displays a screen 890 for adjusting the gain of the camera 61 based on a user operation. The information processing device 80 displays a plurality of images 860, 870, and 880 in a state where the images are superimposed on an image (video) captured by the camera 61.

[0095] Image 860 includes a pull-down menu 861 for turning on or off automatic gain control by the AGC circuit 670 (see FIG. 3) of the camera 61, and a slider 862 and a spin button 863 for adjusting the gain.

[0096] As described above, in this example, since it is necessary to turn off the auto gain control, "Off" indicating that the function is off is selected in the pull-down menu 861.

[0097] When the gain value is changed by the slider 862 and the spin button 863, the information processing device 80 adjusts the gain of the camera 61. The information processing device 80 sends an instruction to the camera 61 so that the gain of the camera 61 becomes the value changed by the slider 862 and the spin button 863.

[0098] The information processing device 80 displays and stores the maximum and minimum values ​​of the evaluation value M of the brightness of the output image at the changed gain. In this example, the maximum and minimum values ​​are displayed in an image 870. The maximum and minimum values ​​at the gain are continuously updated until the gain is changed to another value by the user.

[0099] The user of the information processing device 80 adjusts the gain while viewing the screen 890. More specifically, the user adjusts the gain while checking the change in brightness of the image caused by the up and down movement of the slide 314. When the user determines that the brightness of the image displayed on the screen 890 is appropriate, the user selects the image 880 indicating a setting button. This causes the information processing device 80 to set the gain value at the time the image 880 was selected to the initial gain value (step S1 in FIG. 5).

[0100] Furthermore, information processing device 80 sets the maximum value of the gain when image 880 is selected to the upper limit of the allowable range of brightness evaluation value M, and sets the minimum value to the upper limit of the allowable range (step S2 in FIG. 5).

[0101] For example, when the maximum value of the range of the brightness evaluation value M when the user sets the gain to Gp is ​​M1 and the minimum value is M2, if the user selects image 880 with the gain set to Gp, the information processing device 80 sets M1 to the upper limit value M_UL of the allowable range of the brightness evaluation value M and sets M2 to the lower limit value M_LL of the allowable range.

[0102] FIG. 8 is a graph for explaining changes in the brightness evaluation value M. 8, when an image is captured by the camera 61 with the gain kept constant (for example, with the gain maintained at Gp), the information processing device 80 generates data D1 indicating periodic changes in the brightness evaluation value M based on the above formula (1). Waveform 901 is a graph representing the data D1. The data D1 is a record of the evaluation value M in chronological order.

[0103] The reason that the evaluation value M changes periodically as shown by the waveform 901 is that the slide 314 moves up and down periodically.

[0104] When such data D1 is obtained, if the user selects an image 880 (FIG. 7), the maximum value of the data D1 is set as the upper limit M_UL of the allowable range R, and the minimum value of the data D1 is set as the lower limit M_LL of the allowable range R. The allowable range R ranges from the lower limit M_LL to the upper limit M_UL.

[0105] It is preferable to set the allowable range R based on the gain adjustment via the user interface every time the mold consisting of the upper mold 316 and the lower mold 315 is replaced. This is because the appropriate allowable range R differs depending on the shape of the mold.

[0106] (d3. Details of step S3) FIG. 9 is a graph for explaining the change in the brightness evaluation value M after the allowable range R is set.

[0107] 9, the information processing device 80 generates data D2 indicating periodic changes in the brightness evaluation value M based on the above formula (1). A waveform 902 is a graph representing the data D2.

[0108] As shown in waveform 902, the maximum value M_MAX of the data D2 is below the upper limit M_UL of the set tolerance range R. However, the minimum value M_MIN of the data D2 is also below the lower limit M_LL of the tolerance range R.

[0109] The minimum value M_MIN falls below the lower limit value M_LL of the allowable range R for the following reasons, for example:

[0110] (i) When the press system 1 is operating automatically, when the operator leaves the building in which the press system 1 is installed, the lighting in the building is turned off.

[0111] (ii) If the curtain of the building in which the press system 1 is installed is closed even though it is normally open in an actual operating environment.

[0112] When the information processing device 80 determines that the minimum value M_MIN of the data D2 is below the lower limit M_LL of the allowable range R, it sends an instruction to the camera 61 to increase the gain of the camera 61.

[0113] The information processing device 80, for example, sends an instruction to the camera 61 to lower the gain value by a predetermined value (for example, 1 db). Alternatively, the information processing device 80 may send the gain value itself to the camera 61. Moreover, the information processing device 80 may determine the predetermined value or the gain value itself based on the magnitude of the difference between the minimum value M_MIN and the lower limit value M_LL.

[0114] FIG. 10 is another graph for explaining the change in the brightness evaluation value M after the allowable range R is set.

[0115] 10, the information processing device 80 generates data D3 indicating periodic changes in the brightness evaluation value M based on the above formula (1). A waveform 903 is a graph representing the data D3.

[0116] As shown in waveform 903, the minimum value M_MIN of the data D3 exceeds the lower limit M_LL of the set tolerance range R. However, the maximum value M_MAX of the data D3 also exceeds the upper limit M_UL of the tolerance range R.

[0117] In this way, a reason why the maximum value M_MAX falls below the upper limit value M_UL of the tolerance range R may be, for example, when the curtains of the building in which the press system 1 is installed are opened even though they are normally closed in an actual operating environment.

[0118] When the information processing device 80 determines that the maximum value M_MAX of the data D3 exceeds the upper limit M_UL of the permissible range R, it sends an instruction to the camera 61 to lower the gain of the camera 61.

[0119] The information processing device 80 sends an instruction to the camera 61 to increase the gain value by a predetermined value (for example, 1 db). Alternatively, the information processing device 80 may send the gain value itself to the camera 61. Moreover, the information processing device 80 may determine the predetermined value or the gain value itself based on the magnitude of the difference between the maximum value M_MAX and the upper limit value M_UL.

[0120] Fig. 11 is a graph for explaining a change in the brightness evaluation value M after a transition from the state of Fig. 10. In detail, Fig. 11 is a graph for explaining a change in the brightness evaluation value M after an instruction to reduce the gain value is sent from the information processing device 80 to the camera 61 in the state of Fig. 10.

[0121] 11, the information processing device 80 generates data D4 indicating periodic changes in the brightness evaluation value M based on the above formula (1). A waveform 904 is a graph representing the data D4.

[0122] As shown in waveform 904, the minimum value M_MIN of data D4 is greater than the lower limit M_LL of the set allowable range R, similar to Fig. 10. The maximum value M_MAX of data D4 is less than the upper limit M_UL of allowable range R, unlike Fig. 10.

[0123] In this way, by controlling the gain of the camera 61 from the information processing device 80, it is possible to keep the minimum value M_MIN and maximum value M_MIN of the data D4 within the allowable range R of the evaluation value M of brightness.

[0124] For the sake of convenience, the above description has been given of an example in which the minimum value M_MIN and the maximum value M_MIN of data D4 fall within the allowable range R of the brightness evaluation value by a single gain change. However, if the minimum value M_MIN and the maximum value M_MIN do not fall within the allowable range R by a single gain change, the information processing device 80 may repeatedly transmit a gain change instruction to the camera 61 until the minimum value M_MIN and the maximum value M_MIN fall within the allowable range R.

[0125] Incidentally, the information processing device 80 performs the same process as described above in the case of FIG. 9 as well, thereby placing the minimum value M_MIN and maximum value M_MIN of the data within the permissible range R of the evaluation value M of brightness.

[0126] 11, the information processing device 80 continues data acquisition processing. The information processing device 80 monitors whether the minimum value M_MIN and maximum value M_MIN of the data are within the allowable range R of the brightness evaluation value M. If the information processing device 80 determines that they are not within the allowable range R, it again transmits an instruction to change the gain to the camera 61, as described above.

[0127] Through such processing, the brightness evaluation value M for the image (video) recorded in the information processing device 80 generally falls within the set allowable range R. Therefore, when the above occurs in the press machine body 31 and the image at the time of the abnormality is checked by the information processing device 80, it is possible to prevent the image from being in a state of crushed blacks or blown out highlights. Therefore, according to the information processing device 80, it is possible to promote the analysis processing of the abnormality occurrence conditions using the moving image at the time of the abnormality occurrence.

[0128] FIG. 12 is a flowchart illustrating the details of the process flow of step S3. 12, if the power supply of the press machine body 31 is turned off after step S2 (FIG. 5), in step S101, an operator turns on the power supply of the press machine body 31. In step S102, imaging by the camera 61 is started. Note that an instruction to the camera 61 to start imaging may be an automatic instruction from the press controller 32, or an instruction based on a user operation to the information processing device 80. In step S103, the press machine body 31 is operated. As a result, the slide 314 to which the upper die 316 is attached moves up and down periodically.

[0129] In step S104 , the information processing device 80 starts acquiring data indicating periodic changes in the brightness evaluation value M based on the images captured by the camera 61 .

[0130] In step S105, the information processing device 80 judges whether or not it is a judgment timing. For example, the information processing device 80 judges whether or not a predetermined time has elapsed since the data acquisition start time or the previous judgment timing. The information processing device 80 can use, for example, N times the period of the up and down movement of the slide 314 (for example, N is a natural number equal to or greater than 2) as the predetermined time. The information processing device 80 judges whether or not it is a judgment timing that arrives periodically.

[0131] If it is determined that the judgment timing has arrived (YES in step S105), in step S106, the maximum value M_MAX and minimum value M_MIN of the brightness evaluation value M for a predetermined period Tp that is at least one cycle (operation cycle) immediately preceding the judgment timing are obtained from the data showing the periodic change. In detail, the predetermined period Tp is the period immediately preceding the judgment timing and is at least one cycle of the up and down movement of the slide 314. The reason that the predetermined period Tp is at least one cycle of the up and down movement of the slide 314 is that it includes data of at least one bright state and one dark state.

[0132] If it is determined that the determination timing has not come (NO in step S105), information processing device 80 returns the process to step S105.

[0133] In step S107, the information processing device 80 determines whether or not the maximum value M_MAX of the evaluation value M exceeds the upper limit value M_UL of the allowable range R. If it is determined that the maximum value M_MAX exceeds the upper limit value M_UL of the allowable range R (YES in step S107), the information processing device 80 performs control to lower the gain of the camera 61 in step S108. After step S108, the information processing device 80 advances the process to step S109.

[0134] If it is determined that the maximum value M_MAX does not exceed the upper limit value M_UL of the acceptable range R (NO in step S107), in step S110, the information processing device 80 determines whether the minimum value M_MIN of the evaluation value M is below the lower limit value M_LL of the acceptable range R.

[0135] If it is determined that the minimum value M_MIN is below the lower limit value M_LL of the allowable range R (YES in step S110), the information processing device 80 performs control to increase the gain of the camera 61 in step S111. After step S111, the information processing device 80 advances the process to step S109. If it is determined that the minimum value M_MIN is not below the lower limit value M_LL of the allowable range R (NO in step S110), the information processing device 80 advances the process to step S109.

[0136] In step S109, the information processing device 80 determines whether or not the power supply to the press machine body 31 has been turned off. If it is determined that the power supply has not been turned off (NO in step S109), the information processing device 80 returns the process to step S105. If it is determined that the power supply has been turned off (YES in step S109), the information processing device 80 ends the process of step S3.

[0137] In the above, the example in which step S110 is executed after step S107 has been described, but the present invention is not limited to this example. The order of the determinations in steps S107 and S110 may be reversed.

[0138] (d4. Summary) Some of the processes executed by the press system 1 can be summarized as follows.

[0139] (1) Information processing device 80 stores an allowable range R of an evaluation value M of brightness for an output image of camera 61 (or camera 62). Camera 61 captures images of at least slide 314 and upper die 316 that are included in press machine body 31 and operate periodically.

[0140] The information processing apparatus 80 acquires an output image and data (hereinafter referred to as data D) indicating a periodic change in the evaluation value M of the brightness of the output image based on the imaging by the camera 61 after the allowable range R is stored. The information processing apparatus 80 acquires the maximum value M_MAX and the minimum value M_MIN of the evaluation value M of the brightness in a predetermined period Tp of one or more cycles of the most recent operation from the data D.

[0141] The information processing apparatus 80 adjusts the gain of the camera 61 on the condition that the maximum value M_MAX is not included in the allowable range R. The information processing apparatus 80 changes the gain of the camera 61 from the current set value to a value lower than the current set value on the condition that the maximum value M_MAX exceeds the allowable range R.

[0142] The information processing apparatus 80 adjusts the gain of the camera 61 on the condition that the minimum value M_MIN is not included in the allowable range R. The information processing apparatus 80 changes the gain of the camera 61 from the current set value to a value higher than the current set value on the condition that the minimum value M_MIN is lower than the allowable range R.

[0143] With such a configuration, it is possible to obtain an image with a brightness that the user can tolerate even when there is a change in the surrounding environment of the press system 1 (specifically, the press machine main body 31) having a slide that operates periodically.

[0144] (2) The information processing apparatus 80 sets the allowable range R of the evaluation value M of the brightness by the following processing. As shown in FIG. 7, the information processing apparatus 80 accepts gain adjustment based on a user operation in a state where the output image is displayed on the display 85. The information processing apparatus 80 acquires the upper limit value M_UL and the lower limit value M_LL of the evaluation value M of the brightness in a state where the gain adjustment based on the user operation has been performed. The information processing apparatus 80 sets the numerical range between the upper limit value M_UL and the lower limit value M_LL as the allowable range R.

[0145] <E. Functional Configuration of Information Processing Apparatus> FIG. 13 is a block diagram for explaining the functional configuration of the information processing apparatus 80.

[0146] As shown in FIG. 13, the information processing device 80 includes a control unit 810, a storage unit 820, a communication unit 830, an operation unit 840, and a display unit 850.

[0147] The control unit 810 includes an upper limit setting unit 811, an upper limit reading unit 812, a lower limit setting unit 813, a lower limit reading unit 814, a data generation unit 815, a maximum value acquisition unit 816, a minimum value acquisition unit 817, a gain change instruction unit 818, and a display control unit 819.

[0148] The control unit 810 controls various operations of the information processing device 80. The control unit 810 corresponds to the processor 81 (FIG. 4). In detail, the control unit 810 is realized by the processor 81 executing an operating system and various programs stored in the memory 82.

[0149] The storage unit 820 stores image data (moving image data) captured by the cameras 61 and 62. The storage unit 820 corresponds to the memory 82 (FIG. 4).

[0150] The operation unit 840 accepts user operations. For example, the operation unit 840 accepts an operation of moving a slider 862 (FIG. 7), an operation of changing a gain value with a spin button 863, etc. The operation unit 840 sends a signal according to the accepted operation to the control unit 810. The operation unit 840 corresponds to the touch panel 89 (FIG. 4).

[0151] The communication unit 830 is an interface for communicating with external devices such as the cameras 61 and 62. The communication unit 830 corresponds to the communication IFs 83 and 84 (FIG. 4).

[0152] The display unit 850 displays various information. The display unit 850 corresponds to the display 85 (FIG. 4).

[0153] In the following, a situation in which the allowable range R is set and a situation after the allowable range R has been set will be described separately. For the sake of convenience, the following description will also focus on the camera 61 out of the cameras 61 and 62.

[0154] (e1. Setting the tolerance R) The display control unit 819 causes the display unit 850 to display image data (output image) captured by the camera 61. In addition, the display control unit 819 causes the display unit 850 to display a plurality of images 860, 870, and 880 superimposed on the output image, as shown in FIG.

[0155] The operation unit 840 accepts a gain setting operation by the user. The control unit 810 sends an instruction to the camera 61 via the communication unit 830 so that the gain of the camera 61 becomes the value accepted by the operation unit 840.

[0156] The control unit 810 receives image data (output image) obtained by imaging from the camera 61 via the communication unit 830. The control unit 810 sequentially stores the image data in the storage unit 820. In this case, the display control unit 819 causes the display unit 850 to display the image data.

[0157] The data generation unit 815 of the control unit 810 sequentially calculates the brightness evaluation value M based on the image data stored in the storage unit 820, and generates data D indicating a periodic change in the brightness evaluation value M of the output image. The data generation unit 815 stores the generated data D in the storage unit 820.

[0158] Upper limit setting section 811 sets upper limit value M_UL based on data D stored in storage section 820. Upper limit setting section 811 sets the maximum value of data D to upper limit value M_UL. Upper limit setting section 811 stores the set upper limit value M_UL in storage section 820.

[0159] The lower limit setting unit 813 sets a lower limit M_LL based on the data D stored in the storage unit 820. The lower limit setting unit 813 sets the minimum value of the data D to the lower limit M_LL. The lower limit setting unit 813 stores the set lower limit M_LL in the storage unit 820.

[0160] Through the above process, the information processing device 80 sets an allowable range R defined by an upper limit value M_UL and a lower limit value M_LL.

[0161] (e2. The situation after the tolerance R is set) The control unit 810 receives image data (output image) obtained by capturing an image from the camera 61 via the communication unit 830. The control unit 810 causes the storage unit 820 to store the image data one by one.

[0162] The upper limit reading unit 812 reads the upper limit value M_UL from the storage unit 820. The upper limit reading unit 812 sends the read upper limit value M_UL to the gain change instruction unit 818.

[0163] The lower limit value reading unit 814 reads the lower limit value M_LL from the storage unit 820. The lower limit value reading unit 814 sends the read lower limit value M_LL to the gain change instruction unit 818.

[0164] The data generation unit 815 sequentially calculates the brightness evaluation value M based on the image data stored in the storage unit 820, and generates data D indicating a periodic change in the brightness evaluation value M of the output image. The data generation unit 815 stores the generated data D in the storage unit 820.

[0165] When the determination timing arrives, the maximum value acquiring unit 816 acquires the maximum value M_MAX of the brightness evaluation value M in the data D that is continually being updated under the current gain setting. The maximum value acquiring unit 816 acquires the maximum value M_MAX of the brightness evaluation value M in a predetermined period Tp that is equal to or greater than the most recent cycle (the operation cycle of the slide 314) from the data D. The maximum value acquiring unit 816 sends the acquired maximum value M_MAX to the gain change instructing unit 818.

[0166] When the determination timing arrives, the minimum value acquisition unit 817 acquires the minimum value M_MIN of the brightness evaluation value M in the continuously updated data D in the current gain setting. The minimum value acquisition unit 817 acquires the minimum value M_MIN of the brightness evaluation value M in the predetermined period Tp from the data D. The minimum value acquisition unit 817 sends the acquired minimum value M_MIN to the gain change instruction unit 818.

[0167] The gain change instruction unit 818 determines whether the maximum value M_MAX exceeds the upper limit value M_UL. Also, the gain change instruction unit 818 determines whether the minimum value M_MIN is lower than the lower limit value M_LL.

[0168] When the gain change instruction unit 818 determines that the maximum value M_MAX exceeds the upper limit value M_UL, it sends a gain change instruction to the camera 61 via the communication unit 830. Specifically, the gain change instruction unit 818 sends an instruction to lower the gain of the camera 61 to the camera 61.

[0169] When the gain change instruction unit 818 determines that the maximum value M_MIN is lower than the lower limit value M_LL, it sends a gain change instruction to the camera 61 via the communication unit 830. Specifically, the gain change instruction unit 818 sends an instruction to increase the gain of the camera 61 to the camera 61.

[0170] <F. Modified Example> (1) First Modified Example In the above embodiment, data D showing a periodic change in the brightness evaluation value M of the output image is acquired using the entire area of the output image, and the allowable range R is set based on the data D. However, it is not limited to this. For example, the information processing apparatus 80 may use only a partial area of the output image. Specifically, the information processing apparatus 80 may perform the following processing.

[0171] The information processing device 80 acquires data D indicating periodic changes in the evaluation value M of the image brightness of a partial region (hereinafter also referred to as a "specific region") including a periodically moving part (slide 314) among the output images from the cameras 61 and 62. The specific region is designated in advance based on an operator's operation or the like.

[0172] The information processing device 80 sets the maximum value of the evaluation value M of the specific region as the upper limit M_UL and the minimum value as the lower limit M_LL, thereby setting the allowable range R for the specific region.

[0173] Thereafter, in the process of step S3 shown in FIG. 5 and FIG. 12, the information processing device 80 may send an instruction to the camera 61 to increase or decrease the gain based on the evaluation value M calculated from the pixel values ​​of the specific region.

[0174] According to this process, it is possible to adjust the gain by focusing on important parts of the output image. Therefore, even if there is a change in the surrounding environment of the press machine body 31, it is possible to obtain an image with a brightness that is more preferable for the user.

[0175] (2) Second modified example Instead of the formula (1) shown in the above embodiment, the evaluation value M may be calculated using the following formula (2).

[0176] M=ΣΣ(d(i,j)×w(i,j))÷(H×W) … (2) w(i,j) is a weight. In the calculation of the ΣΣ term in equation (2), the sum of all combinations of i and j is taken, as in equation (1). In other words, in the calculation of Σ, the sum of H×W pixel values ​​(more specifically, d(0,0)×w(0,0)+d(0,1)×w(0,1)+...+d(H-1,W-1)×w(H-1,W-1)) is taken.

[0177] For example, by weighting the pixels in the central region greater than the pixels in the peripheral region, the brightness of the central region is taken into consideration more than the brightness of the peripheral region when calculating the evaluation value M. In particular, by weighting the above-mentioned specific region (a portion of the output image that includes a part that operates periodically) greater than the weighting of the remaining regions, it is possible to perform gain adjustment processing that focuses on important parts of the output image. Therefore, even when there is a change in the environment surrounding the press machine body 31, it is possible to obtain an image with a brightness that is more preferable for the user.

[0178] It should be noted that if the weight of the specific region is set to a constant value and the weight of the remaining regions is set to zero, the result will be the same processing as in the first modified example.

[0179] (3) Third modified example In the above embodiment, a method for calculating the evaluation value M in the case of a monochrome image has been described. Below, a method for calculating the evaluation value M in the case of a color image will be described. Specifically, a method for calculating the evaluation value M in the case where the output image is an RGB image will be described.

[0180] In the following, for convenience, pixels to which a red (R) filter is applied will be referred to as "R pixels," pixels to which a green (G) filter is applied will be referred to as "G pixels," and pixels to which a blue (B) filter is applied will be referred to as "B pixels."

[0181] 14 is a diagram showing each of the RGB pixels of the imaging element 606 of the camera 61. More specifically, it is a diagram showing the arrangement of colors (R, G, B) in the color filter 605.

[0182] As shown in Fig. 14, the RGB color arrangement is a Bayer type in which green (G) is arranged alternately. Note that the arrangement type is only an example and is not limited to the Bayer type. For example, the arrangement type may be one in which only green (G) columns are arranged alternately.

[0183] The pixel value of a red (R) pixel is represented by a symbol beginning with R, such as R(i,j+1). The pixel value of a G pixel is represented by a symbol beginning with G, such as G(i,j). The pixel value of a B pixel is represented by a symbol beginning with B, such as B(i+1,j). Since the human eye is highly sensitive to the green component, there are more G pixels than R pixels and B pixels. As mentioned above, i is a natural number between 0 and H-1. j is a natural number between 0 and W-1.

[0184] Fig. 15 is a diagram for explaining data interpolation. In detail, Fig. 15 is a diagram for explaining data interpolation of R pixels.

[0185] 15, the camera 61 performs data interpolation for pixel regions of G pixels and B pixels based on pixel values ​​of surrounding R pixels. For example, the camera 61 (more specifically, the digital signal processor 609) estimates the pixel value of a G pixel from the R pixels above and below it.

[0186] As a result, the pixel values ​​of R are associated with pixel regions of G pixels and B pixels. As a result, an R frame consisting of H×W R pixels is generated. By similar processing, the camera 61 generates a G frame consisting of H×W G pixels and a B frame consisting of H×W B pixels.

[0187] A single frame for display is generated by combining the above three types of frames. In this way, the camera 61 performs color reproduction by a pixel mixing method.

[0188] In this case, the information processing device 80 calculates the evaluation value M using, for example, the following formula (3).

[0189] M=(ΣΣR(i,j)÷(H×W)+ΣΣG(i,j)÷(H×W)+ΣΣB(i,j)÷(H×W))÷3 … (3) In addition, in the calculation of the ΣΣ term in equation (3), the sum of all combinations of i and j is taken, just like in equation (1). In equation (3), in the case of 8-bit representation, the value of M is between 0 and 255, just like in equation (1).

[0190] Note that formula (3) is an example of a method for calculating the brightness evaluation value M, and the method for calculating the evaluation value M is not limited to formula (3).

[0191] In the above embodiment, the camera 61 is described as having RGB filters (primary color filters) as the color filter 605, but the color filter 605 may be a filter (complementary color filter) consisting of four colors of cyan (Cy), magenta (Mg), yellow (Y), and green (G). In this case, the evaluation value M may be calculated by a calculation using Cy, Mg, Y, and G instead of R, G, and B in the formula (3).

[0192] (4) Fourth modified example In the above embodiment, as shown in FIG. 12, gain adjustment was performed both when the maximum value M_MAX of the evaluation value M exceeded the upper limit value M_UL and when the minimum value M_MIN of the evaluation value M fell below the lower limit value M_LL (steps S108, S111).

[0193] However, this is not necessarily limited to this, and the information processing device 80 may be configured to perform gain adjustment only when either the maximum value M_MAX of the evaluation value M exceeds the upper limit value M_UL, or the minimum value M_MIN of the evaluation value M falls below the lower limit value M_LL.

[0194] (5) Fifth Variation For example, a case may be assumed in which the maximum value M_MAX of the evaluation value M exceeds the upper limit value M_UL and the minimum value M_MIN of the evaluation value M falls below the lower limit value M_LL. Also, if the gain is lowered when the maximum value M_MAX of the evaluation value M exceeds the upper limit value M_UL, a situation may occur in which the minimum value M_MIN of the evaluation value M falls below the lower limit value M_LL. Conversely, if the gain is increased when the minimum value M_MIN of the evaluation value M falls below the lower limit value M_LL, a situation may occur in which the maximum value M_MAX of the evaluation value M exceeds the upper limit value M_UL.

[0195] To deal with such a case, the information processing device 80 may adjust the gain so that at least one of the maximum value M_MAX and the minimum value M_MIN after the gain adjustment, selected (specified) by a user operation, falls within the allowable range R.

[0196] For example, if it is desired to prevent whiteout rather than blackout, the operator may set the information processing device 80 to prioritize regulation on the upper limit so that the maximum value M_MAX after gain adjustment falls within the permissible range R. Conversely, if it is desired to prevent blackout rather than whiteout, the operator may set the information processing device 80 to prioritize regulation on the lower limit so that the minimum value M_MIN after gain adjustment falls within the permissible range R. Such operator operations may be realized via a user interface provided by the information processing device 80.

[0197] (6) Sixth Modification In the above embodiment, the slide 314 has been taken as an example of a part that moves periodically, but the present invention is not limited to this.

[0198] For example, a conveying device that conveys a workpiece (typically, a work in progress) between press bodies (typically, an arm-type robot that delivers the workpiece between an upstream press body and a downstream press body) operates periodically, and therefore the above-described gain adjustment process can be applied to a situation where the conveying device is imaged by a camera.

[0199] The embodiments disclosed herein are merely examples and are not limited to the above. The scope of the present invention is defined by the claims, and it is intended to include all modifications within the scope and meaning equivalent to the claims. [Explanation of symbols]

[0200] 1 Press system, 10 Coil holder, 20 Straightener feeder, 30 Press machine, 31 Press machine body, 32 Press controller, 40 Transport conveyor, 50 Product box, 61, 62 Camera, 70 Hub, 80 Information processing device, 85 Display, 120 Server device, 311 Body frame, 312 Bed, 313 Bolster, 314 Slide, 315 Lower die, 316 Upper die, 318 Sensor, 319 Warning light, 601 Zoom lens, 602 Focus lens, 603 Unit, 604 Band pass filter, 605 Color filter, 606 Imaging element, 630 Imaging optical system, 801 Tablet terminal, 802 Cradle, 860, 870, 880 Image, 861 Pull-down menu, 862 Slider, 863 Spin button, 890 Screen, 901, 902, 903, 904 waveforms, L1, L2, L3, L4 LAN cables, L9 connection cable, NW network, P molded product, W coil.

Claims

1. A camera gain adjustment method using an information processing device, comprising: a step of storing an allowable range of an evaluation value of brightness for an output image of the camera by the information processing device, the camera capturing an image of a periodically operating part included in an industrial machine, a step in which the information processing device acquires the output image and data indicating a periodic change in an evaluation value of brightness of the output image based on an image captured by the camera after the allowable range has been stored; The information processing device acquires, from the data, at least one of a maximum value and a minimum value of the brightness evaluation value during a predetermined period of one or more cycles immediately preceding the operation; The gain adjustment method further comprises a step of adjusting a gain of the camera by the information processing device on condition that the acquired value is not within the allowable range.

2. the one value being the minimum value, 2. The gain adjustment method according to claim 1, wherein in the step of adjusting the gain of the camera, the gain is changed from a current setting value to a value higher than the current setting value on condition that the minimum value is below the allowable range.

3. the one value being the maximum value, 2. The gain adjustment method according to claim 1, wherein in the step of adjusting the gain of the camera, the gain is changed from a currently set value to a value lower than the currently set value on condition that the maximum value is greater than the allowable range.

4. The method further includes a step of selecting one of the maximum value and the minimum value based on a first user operation, In the step of obtaining at least one of the maximum value and the minimum value, obtaining at least the selected value; 2. The gain adjustment method according to claim 1, wherein in the step of adjusting the gain of the camera, on condition that the selected value is not included in the acceptable range, the information processing device adjusts the gain of the camera so that the selected value is included in the acceptable range.

5. the acceptable range is a range of evaluation values ​​of brightness of an image of a partial region of the output image that includes the part, 5. A gain adjustment method according to claim 1, wherein in the step of acquiring the output image and data indicating a periodic change in an evaluation value of the brightness of the output image, data indicating a periodic change in an evaluation value of the brightness of the image of the partial area is acquired as the data.

6. The step of storing an allowable range of the brightness evaluation value includes: receiving a gain adjustment based on a second user operation while the output image is displayed on a display; acquiring an upper limit value and a lower limit value of the brightness evaluation value in a state in which gain adjustment based on the second user operation has been performed; The gain adjustment method according to claim 1 , further comprising the step of: setting a numerical range between the upper limit value and the lower limit value as the allowable range.

7. the industrial machine is a press machine, The part is a die and a slide to which the die is attached, The gain adjustment method according to claim 6 , wherein in the step of storing the allowable range of the brightness evaluation value, the gain adjustment based on the second user operation is accepted every time the mold is replaced.

8. An information processing device that adjusts a gain of a camera, A processor; a memory that stores an allowable range of an evaluation value of brightness for an output image of the camera, the camera capturing an image of a periodically operating part included in an industrial machine, The processor, acquiring the output image and data indicating a periodic change in an evaluation value of brightness of the output image based on an image captured by the camera after the allowable range has been stored; From the data, at least one of a maximum value and a minimum value of the evaluation value of the brightness during a predetermined period of at least one cycle immediately preceding the operation is obtained; The information processing device adjusts a gain of the camera on condition that the acquired value is not within the allowable range.

9. the one value being the minimum value, The information processing apparatus according to claim 8 , wherein the processor changes the gain from a currently set value to a value higher than the currently set value on condition that the minimum value is lower than the allowable range.

10. the one value being the maximum value, The information processing apparatus according to claim 8 , wherein the processor changes the gain from a current setting value to a value lower than the current setting value on condition that the maximum value is greater than the allowable range.

11. The processor, selecting one of the maximum value and the minimum value based on a first user operation; obtaining at least the selected value of the maximum value and the minimum value; The information processing apparatus according to claim 8 , further comprising: adjusting a gain of the camera such that the selected value is included in the acceptable range on condition that the selected value is not included in the acceptable range.

12. the acceptable range is a range of evaluation values ​​of brightness of an image of a partial region of the output image that includes the part, The information processing apparatus according to claim 8 , wherein the processor acquires, as the data, data indicating a periodic change in an evaluation value of brightness of the image of the partial region.

13. Further equipped with a display, the information processing device accepts a second user operation for gain adjustment while the output image is displayed on the display; The processor, obtaining an upper limit value and a lower limit value of the brightness evaluation value in a state in which the gain adjustment based on the second user operation has been performed; The information processing apparatus according to claim 8 , wherein a numerical range between the upper limit value and the lower limit value is set as the allowable range.

14. 1. An imaging system, comprising: A camera that captures an image of a periodically operating part included in an industrial machine; an information processing device for adjusting a gain of the camera; The information processing device includes: A tolerance range for an evaluation value of brightness of an output image of the camera is stored, acquiring the output image and data indicating a periodic change in an evaluation value of brightness of the output image based on an image captured by the camera after the allowable range has been stored; From the data, at least one of a maximum value and a minimum value of the evaluation value of the brightness during a predetermined period of at least one cycle immediately preceding the operation is obtained; and adjusting a gain of the camera on condition that the acquired value is not within the tolerance range.

Images