Industrial system control device, industrial system, and image acquisition method

Abstract

An industrial system control device according to one embodiment of the present disclosure controls an imaging device having an imaging element that captures an image of an object imaged on an imaging surface and generates a captured image, and one or a plurality of industrial machines having a plurality of drive shafts that generate relative movement between the object and the imaging device. The industrial system control device is provided with: an imaging number determination unit that determines the number of images captured by the imaging device; a photographing instruction unit that instructs the photographing device to photograph; a displacement amount determination unit that determines a required displacement amount for an imaging position of the object on the imaging surface; a drive amount calculation unit that calculates a drive amount of the drive shaft that displaces the imaging position of the object by the displacement amount; a drive instruction unit that instructs the drive shaft to drive by the drive amount; an operation adjustment unit for inserting an instruction based on the driving of the driving instruction unit and repeatedly executing an instruction based on the imaging of the imaging instruction unit until the number of images is reached; and an image synthesis unit that synthesizes a plurality of captured images captured by the imaging device to generate one synthesized image.

Description

Technical Field

[0001] The present invention relates to an industrial system control device, an industrial system and an image acquisition method. Background Art

[0002] In industrial systems such as machine tools that process workpieces, images of objects such as workpieces are sometimes obtained by photographing them with a camera, and various judgments are made based on the images. In order to make high-precision judgments, it is sometimes desirable to obtain images with high resolution (number of pixels). Generally speaking, the resolution of an image depends on the camera element, and a camera that can take images with high resolution is relatively expensive.

[0003] In addition, in a general imaging device, an imaging element is used in which each pixel detects the intensity of any one of RGB (three primary colors of light), and the intensities of the remaining two colors of light are supplemented based on the detection results of the surrounding pixels. By performing such interpolation, a false color is generated in which the color of the pixel is different from the actual color, and the correct color information cannot be obtained.

[0004] As a method for obtaining an image with correct color information and high resolution, the following technology has been proposed: a high-quality image is obtained by synthesizing multiple images captured by moving the imaging element by, for example, half the pixel pitch and multiple images captured by moving the imaging element by the amount of the pixel pitch (for example, refer to Patent Document 1).

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2017-11329 Summary of the invention

[0008] Problems to be solved by the invention

[0009] An imaging device that moves an imaging element to synthesize a high-quality image is also relatively expensive. Therefore, a technology for obtaining a high-quality image at a relatively low cost in an industrial system is desired.

[0010] Means for solving problems

[0011] An industrial system control device according to one embodiment of the present disclosure controls a photographing device and one or more industrial machines, wherein the photographing device has a photographing element for photographing an image of an object formed on a photographing surface to generate a photographic image, and the industrial machine has a plurality of drive shafts for generating relative movement between the object and the photographing device, and the industrial system control device comprises: a photographing number determination unit for determining the number of photographs of the photographing device; a photographing instruction unit for instructing the photographing device to photograph; a displacement amount determination unit for determining the required displacement amount of the imaging position of the object on the photographing surface; a driving amount calculation unit for calculating the driving amount of the driving shaft for displacing the imaging position of the object by the displacement amount; a driving instruction unit for instructing the driving shaft to drive by the driving amount; an action adjustment unit for inserting a driving instruction based on the driving instruction unit and repeatedly executing the photographing instruction based on the photographing instruction unit until the number of photographs is reached; and an image synthesis unit for synthesizing a plurality of photographic images captured by the photographing device to generate a synthetic image.

[0012] Effects of the Invention

[0013] According to the present disclosure, high-quality images can be obtained in an industrial system at a relatively low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram showing the configuration of an industrial system according to the first embodiment of the present disclosure.

[0015] Figure 2 Yes means Figure 1 A flowchart of a process of an embodiment of an image acquisition method in an industrial system.

[0016] Figure 3 It is a schematic diagram showing the configuration of an industrial system according to a second embodiment of the present disclosure.

[0017] Figure 4 Yes means Figure 3 A flowchart of a process of an embodiment of an image acquisition method in an industrial system. DETAILED DESCRIPTION

[0018] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Figure 1 It is a schematic diagram showing the configuration of an industrial system 1 according to the first embodiment of the present disclosure.

[0019] The industrial system 1 includes: a camera 10 that captures an object W to generate a photographic image; a machine tool 20 that holds the object W in a movable manner; and a numerical control device 100 that controls the camera 10 and the machine tool 20. The machine tool 20 is a type of industrial machinery that can generate relative movement between the object W and the camera 10. The numerical control device 100 itself is an embodiment of the industrial system control device disclosed in the present invention. In addition, the numerical control device 100 is also a device that automatically implements an embodiment of the image acquisition method disclosed in the present invention in the industrial system 1.

[0020] The imaging device 10 is a digital camera having an imaging element, an optical system, and an electronic circuit. The imaging element captures the image of the object formed on the imaging surface to generate a photographic image. The optical system forms an image of the light from the object on the imaging element. The electronic circuit outputs the image signal generated by the imaging element as the data of the photographic image. In this embodiment, the imaging device 10 can be fixed to a main part of the machine tool 20, that is, it can be fixed to a position where the object W arranged on the machine tool 20 can be photographed, for example, above the machine tool 20.

[0021] In the present embodiment, the machine tool 20 is a machine tool having a table 21 that holds the object W in a positionable manner and a processing head 22 that processes the object W by rotating the tool T. The machine tool 20 of the present embodiment has a plurality of drive shafts for positioning the table 21 and the object W at least in the horizontal direction, a plurality of drive shafts for positioning the rotating tool T, and a main shaft for driving the rotating tool T.

[0022] The numerical control device 100 controls the machine tool 20 according to the machining program. The numerical control device 100 can be realized by one or more computer devices having a memory, a processor, an input / output interface, etc., and executing an appropriate control program.

[0023] The numerical control device 100 includes a storage unit 101, a program reading unit 102, an analyzing unit 103, an interpolation control unit 104, a servo control unit 105, an imaging quantity determination unit 106, a displacement determination unit 107, a drive amount calculation unit 108, an imaging instruction unit 109, a drive instruction unit 110, an operation adjustment unit 111, and an image synthesis unit 112. Each of these components is a component in which the functions of the numerical control device 100 are classified, and may not be clearly distinguished in terms of physical structure and program structure.

[0024] The storage unit 101 stores a processing program that specifies the operation of the machine tool 20 for processing the object W, specifications of the imaging device 10 (pixel configuration information, focal length, etc.), an axis structure of the machine tool 20, etc. As a well-known numerical control program, the processing program can be described as determining a plurality of instruction points, each of which indicates the coordinates of a path through which the rotating tool T should move relative to the object W.

[0025] The program reading unit 102 reads the machining program from the storage unit 101 and inputs the machining program to the analyzing unit 103 in a processable form, for example, in units of program blocks.

[0026] The analyzing unit 103 analyzes the input machining program and calculates the position or speed of the driving axis for realizing the position or speed required for the table 21 holding the object W and the machining head 22 .

[0027] The interpolation control unit 104 calculates the position or speed of each drive axis between command points described in the machining program.

[0028] The servo control unit 105 adjusts the power supplied to the servo motor of each drive axis so that the position or speed of each drive axis matches the position or speed calculated by the interpolation control unit 104 .

[0029] The number of shots determining unit 106 determines the number of shots required to obtain an image of the object W having the image quality required by the processing program. As for the number of shots, it is preferred that the number of shots is set to a number determined from an integer multiple of 4 based on the pixel arrangement information of the imaging device 10, and is typically set to 4 or 16. The number of shots is set to the number of photographic images required for performing known image synthesis for high image quality, and may be a pre-fixed number or a number pre-set to correspond to a mode selected by the processing program or the user's input. That is, the number of shots determining unit 106 may be configured to obtain a pre-set number of shots.

[0030] The displacement determination unit 107 determines the displacement of the imaging position of the object W on the imaging surface required between the photographic images used for image synthesis based on information such as the pixel arrangement of the imaging element. The displacement can be determined according to the number of shots determined by the number of shots determination unit 106, based on the selected mode, etc. Specifically, preferably, the displacement for obtaining an image with improved resolution is set to a distance of 0.5 pixels in the imaging element, and the displacement for obtaining an image without false color is set to a distance of 1.0 pixels in the imaging element, that is, it is set to 0.5 times (half pixel pitch) or 1.0 times the pixel pitch of the imaging element, or an integer multiple thereof. Therefore, the displacement determination unit 107 can also be configured to obtain a preset displacement.

[0031] The drive amount calculation unit 108 calculates the drive amount of the drive shaft of the machine tool 20 that moves the object W in a plane perpendicular to the optical axis of the imaging device 10 so that the imaging position of the object W is displaced by the displacement amount determined by the displacement amount determination unit 107. In the case where the coordinate system (imaging surface direction and optical axis direction) of the imaging device 10 is inconsistent with the coordinate system of the machine tool 20, preferably, the drive amount calculation unit 108 converts the displacement amount in the coordinate system of the imaging device 10 into the displacement amount in the coordinate system of the machine tool 20 by coordinate conversion, calculates the movement amount of the object W equivalent to the displacement amount, and calculates the drive amount of each drive shaft based on the movement amount. In addition, even if the displacement amount is fixed, the greater the distance between the object W and the imaging device 10, the greater the drive amount required to move the imaging position of the object W by the displacement amount, that is, the actual relative movement amount of the object W relative to the imaging device 10. Therefore, preferably, the drive amount calculation unit 108 is configured to calculate the drive amount in consideration of the axis structure of the machine tool 20 and the distance between the object W and the imaging device 10 calculated based on the position of each drive axis. In addition, even when the distance between the object W and the imaging device 10 is the same, the displacement amount still changes according to the optical system information at the focus position of the imaging device 10 relative to the object W, and specifically, it is inversely proportional to the focal length or the photographic magnification of the optical system. Therefore, preferably, it is configured to calculate the drive amount based on the focal length or the photographic magnification information of the optical system. Furthermore, even when the displacement amount is the same, since the drive amount sometimes changes according to the position and posture of the object W and the imaging device 10, that is, it changes according to the current position of each drive axis, the drive amount calculation unit 108 may also be configured to calculate the drive amount in consideration of these. In addition, when the imaging device 10 is fixed and the position of the optical system in the imaging device is not changed, and the axis structure of the machine tool 20 is such that the object can only be moved in the direction perpendicular to the optical axis, a drive amount corresponding to the displacement amount may be preset in advance.

[0032] The imaging instruction unit 109 instructs the imaging device 10 to perform imaging. That is, the imaging instruction unit 109 outputs a command signal instructing imaging.

[0033] The drive instruction unit 110 inputs a command signal to the servo control unit 105 so as to drive the drive shaft at the drive amount calculated by the drive amount calculation unit 108 .

[0034] The action adjustment unit 111 adjusts the output timing of the signals of the photographing instruction unit 109 and the drive instruction unit 110. Specifically, the action adjustment unit 111 inserts the driving instruction based on the drive instruction unit 110 to repeatedly execute the photographing instruction based on the photographing instruction unit 109 until the number of photographing is reached. That is, the action adjustment unit 111 controls the photographing instruction unit 109 and the drive instruction unit 110 so as to repeat the photographing by the photographing device 10 and the movement of the imaging position of the object W by the machine tool 20 by the displacement amount.

[0035] The image synthesis unit 112 obtains the photographic images of the number of shots from the imaging device 10 and synthesizes the photographic images of the number of shots to generate an image with correct color information for each pixel, a high resolution (a large number of recorded pixels), or a synthesized image with correct color information and high resolution. The combination of the number of shots and the displacement amount and the synthesis of the images are well-known techniques, so detailed descriptions are omitted.

[0036] like Figure 2 As shown, an embodiment of the image acquisition method of the present disclosure implemented by the numerical control device 100 in the industrial system 1 includes: a process of configuring the object W at the shooting start position by the machine tool 20 (step S01); a process of obtaining the shooting device information (step S02); a process of determining the displacement amount in the coordinate system of the shooting device 10 (step S03); a process of converting the displacement amount coordinates in the coordinate system of the shooting device 10 into the displacement amount in the coordinate system of the machine tool 20 (step S04); a process of calculating the driving amount of the driving axis that causes the imaging position of the object W to be displaced by the displacement amount (step S05); a process of determining the number of shots (step S06); a process of positioning the object W at the shooting position by the machine tool 20 (step S07); a process of causing the shooting device 10 to shoot (step S08); a process of confirming whether the number of shots has been reached (step S09); and a process of synthesizing multiple photographic images (step S10).

[0037] In the image acquisition method of the present embodiment, if the number of shots in step S09 has reached the number of shots, the process proceeds to step S10, but if the number of shots has not been reached, the process returns to step S07. That is, the image acquisition method of the present embodiment inserts a process (step S07) of driving the drive shaft so as to move the imaging position of the object W on the imaging surface, and repeats the process (step S08) of causing the imaging device 10 to perform the shooting until the number of shots is reached. The driving amount of the driving shaft of the machine tool 20 in the process of driving the driving shaft in step S07 is determined by a method including a process (step S03) that determines the required displacement amount of the imaging position of the object W on the imaging surface, and a process (step S05) that calculates the driving amount of the driving shaft that causes the imaging position of the object W to be displaced by the displacement amount.

[0038] In the industrial system 1, the numerical control device 100 can be used to move the object W in the machine tool 20, and a plurality of photographic images whose positions are shifted by a certain amount can be obtained using the imaging device 10 that does not have the function of moving the imaging element. By synthesizing the plurality of photographic images, the industrial system 1 can be a relatively low-cost structure, and a high-quality synthesized image with high color accuracy can be obtained.

[0039] Next, a second embodiment of the present disclosure will be described. Figure 3 It is a schematic diagram showing the configuration of an industrial system 1A according to a third embodiment of the present disclosure. In the description of this embodiment, the same reference numerals are given to the same components as those of the first embodiment, and duplicate descriptions may be omitted.

[0040] The industrial system 1A includes: a camera 10 that captures an object W and generates a photographic image; a machine tool 20 that is a first industrial machine capable of generating a relative movement between the object W and the camera 10; a robot 30 that is a second industrial machine that replaces the object W and holds the camera 10 in a positionable manner; a numerical controller 100A that controls the camera 10 and the machine tool 20; and a robot controller 200 that controls the camera 10 and the robot 30. The numerical controller 100A and the robot controller 200 are other embodiments of the industrial system controller of the present disclosure. In addition, the numerical controller 100A and the robot controller 200 are also devices that automatically implement other embodiments of the image acquisition method of the present disclosure in the industrial system 1A.

[0041] Although the robot 30 can Figure 3As shown in the example, the robot 30 is a vertical articulated robot, but it is not limited thereto, and may be, for example, an orthogonal coordinate robot, a horizontal articulated robot, a parallel link robot, etc. The robot 30 is provided with a hand 31 for holding the object W and the camera 10 at the front end. That is, the robot 30 can generate a relative movement between the object W and the camera 10 by positioning the camera 10.

[0042] The numerical controller 100A can be realized by a computer device similar to the numerical controller 100 of the first embodiment. The numerical controller 100A includes a storage unit 101, a program reading unit 102, an analysis unit 103, an interpolation control unit 104, a servo control unit 105, a drive instruction unit 110, and an operation adjustment unit 111. That is, the numerical controller 100A omits a part of the functions of the numerical controller 100 of the first embodiment.

[0043] The robot control device 200 can be implemented by one or more computer devices, which have a memory, a processor, an input / output interface, etc., and execute an appropriate control program. The robot control device 200 includes a storage unit 201, an analysis unit 202, a trajectory control unit 203, a servo control unit 204, a shooting instruction unit 205, and an image synthesis unit 206. The structural elements of the robot control device 200 are also classified by function, and they may not be clearly distinguished.

[0044] The storage unit 201 of the robot control device 200 stores a work program for executing the operation of the robot 30 for performing the replacement operation of the object W, and the axis structure information of the robot 30. The analysis unit 202 analyzes the work program and determines the operation of the robot 30. The trajectory control unit 203 supplements the operation described in the work program as needed and calculates the position or speed of each drive axis of the robot 30 at each moment. The servo control unit 204 controls the servo motors of each drive axis of the robot 30 so as to achieve the position or speed calculated by the trajectory control unit 203.

[0045] The imaging instruction unit 205 and the image synthesis unit 206 of the robot control device 200 are functionally the same as the imaging instruction unit 109 and the image synthesis unit 112 of the numerical controller 100 of the first embodiment. These components implement the following image acquisition method: by exchanging various data with the drive instruction unit 110 and the motion adjustment unit 111 of the numerical controller 100A, it is possible to acquire a high-quality image similar to that of the numerical controller 100 of the first embodiment.

[0046] like Figure 4As shown, in the industrial system 1A, the image acquisition method implemented in collaboration between the numerical controller 100A and the robot controller 200 includes a machine tool control program for controlling the machine tool 20 by the numerical controller 100A, and a robot control program for controlling the robot 30 and the imaging device 10 by the robot controller 200.

[0047] The machine tool control program includes a step of receiving a notification of arranging the imaging device 10 to the imaging start position from the robot controller 200 (step S101), a step of arranging the object W at the predetermined imaging start position (step S102), a step of acquiring imaging device information (step S103), a step of determining the displacement amount of the imaging position (step S104), a step of performing coordinate conversion of the displacement amount (step S105), a step of calculating the driving amount of the driving axis (step S106), a step of determining the number of shots (step S107), a step of positioning the object W at the imaging position (step S108), a step of instructing the robot controller 200 to perform imaging (step S109), a step of receiving a notification of the end of imaging from the robot controller 200 (step S110), a step of confirming whether the number of shots has been reached (step S111), and a step of instructing the robot controller 200 to perform image synthesis (step S112). In step S111, if the number of shots has not been reached, the process returns to step S108.

[0048] The robot control program includes: a process of configuring the imaging device 10 at a predetermined imaging start position by the robot 30 (step S201), a process of notifying the numerical controller 100A of the configuration to the imaging start position performed by the robot 30 (step S202), a process of receiving an instruction from the numerical controller 100A (step S203), a process of confirming whether the instruction from the numerical controller 100A is an instruction for image synthesis (step S204), a process of causing the imaging device 10 to perform imaging when the instruction from the numerical controller 100A is not an instruction for image synthesis (step S205), a process of notifying the numerical controller 100A of imaging (step S206), and a process of synthesizing a plurality of photographic images when the instruction from the numerical controller 100A is an instruction for image synthesis (step S207).

[0049] In the industrial system 1A, the imaging device 10 is positioned using the robot 30 that processes the object W, so that the object W can be imaged from any direction. In addition, in the industrial system 1A, a plurality of photographic images can be acquired while the relative position of the object W with respect to the imaging device 10 is finely adjusted by the machine tool 20, so that high-quality images can be acquired.

[0050] Although the embodiments of the present disclosure have been described above, the present invention is not limited to the embodiments described above. In addition, the effects described in the embodiments described above are merely examples of the ideal effects produced by the present invention, and the effects of the present invention are not limited to the effects described in the embodiments described above.

[0051] In the present invention, the industrial machinery for positioning the photographing device and the object may be any machine, for example, the processing head of a machine tool may be made to hold the photographing device in a movable manner, and a robot may be made to hold the object. In addition, in the present invention, the photographing device may be moved to move the imaging position of the object by a displacement amount. As an example, the industrial system of the present invention may also include a first industrial machine (such as a machine tool) that holds the object in a locatable manner, and a second industrial machine (such as a robot) that holds the photographing device in a locatable manner, and the drive amount calculation unit is configured to drive the drive shaft of the first industrial machine and the second industrial machine with respect to the side with a higher resolution of the displacement amount.

[0052] Description of Reference Numerals

[0053] 1.1A Industrial System

[0054] 10. Filming Device

[0055] 20 Machine Tools

[0056] 21 Workbench

[0057] 22 Processing head

[0058] 30 Robot

[0059] 31 Hands

[0060] 100, 100A numerical control device

[0061] 101 Storage

[0062] 102 Program reading unit

[0063] 103 Analysis Department

[0064] 104 Interpolation control unit

[0065] 105 Servo Control Unit

[0066] 106 Shooting Quantity Determination Department

[0067] 107 Displacement determination unit

[0068] 108 Driving quantity calculation unit

[0069] 109 Shooting Instruction Department

[0070] 110 drive indicator

[0071] 111 Action Adjustment Department

[0072] 112 Image synthesis unit

[0073] 201 Storage

[0074] 202 Analysis Department

[0075] 203 Trajectory Control Department

[0076] 204 Servo Control Department

[0077] 205 Shooting Instruction Department

[0078] 206 Image Synthesis Department

[0079] T Rotary Tool

[0080] W object.

Claims

1. An industrial system control device, which controls a camera and one or more industrial machines, wherein the camera has a camera element that captures an image of an object formed on a camera surface to generate a photographic image, and the industrial machine has a plurality of drive shafts that generate relative movement between the object and the camera, characterized in that: The industrial system control device comprises: a shooting quantity determination unit, which determines the shooting quantity of the shooting device; a shooting instruction unit for instructing the shooting device to shoot; a displacement determination unit that determines a required displacement of an imaging position of the object on the imaging plane; a driving amount calculation unit that calculates a driving amount of the driving axis for displacing an imaging position of the object by the displacement amount; a drive instruction unit for instructing the drive shaft to be driven at the drive amount; an action adjustment unit that inserts the driving instruction based on the driving instruction unit and repeatedly executes the shooting instruction based on the shooting instruction unit until the shooting number is reached; The image synthesis unit synthesizes a plurality of images captured by the imaging device to generate a single synthesized image.

2. The industrial system control device according to claim 1, characterized in that: The drive amount calculation unit calculates the drive amount in consideration of a distance between the object and the imaging device calculated based on positions of the plurality of drive shafts of the industrial machine.

3. The industrial system control device according to claim 1 or 2, characterized in that: The displacement amount is an integer multiple of a half pixel pitch of the imaging element or an integer multiple of a pixel pitch.

4. The industrial system control device according to any one of claims 1 to 3, characterized in that: The imaging number determination unit determines the imaging number from a number that is an integer multiple of 4 based on pixel arrangement information of the imaging device.

5. An industrial system, characterized in that: have: A photographing device and an object to be photographed; One or more industrial machines that cause relative movement between the object and the camera; The industrial system control device according to any one of claims 1 to 4.

6. The industrial system according to claim 5, characterized in that: The industrial machine includes a machine tool that holds the object in a positionable manner and processes the object.

7. The industrial system according to claim 5 or 6, characterized in that: The industrial machine includes a robot that holds the imaging device in a positionable manner.

8. The industrial system according to claim 5, characterized in that: The industrial machinery comprises: a first industrial machine that holds the object in a positionable manner; A second industrial machine that holds the imaging device in a positionable manner, The drive amount calculation unit calculates the drive amount by driving one of the first industrial machine and the second industrial machine having a higher resolution of the drive shaft with respect to the displacement amount.

9. An image acquisition method for acquiring an image of an object in an industrial system including an imaging device and one or more industrial machines, wherein the imaging device has an imaging element for capturing an image of the object formed on a photographic surface to generate a photographic image, and the industrial machine has a plurality of drive shafts for generating relative movement between the object and the imaging device, characterized in that: The image acquisition method comprises: A step of determining the number of shots taken by the shooting device; A step of determining a required displacement amount of an imaging position of the object on the imaging plane; a step of calculating a driving amount of the driving shaft for causing the object to be displaced by the displacement amount; A step of driving the driving shaft in such a manner as to move the imaging position of the object on the imaging surface, and repeating the step of causing the imaging device to take images until the number of images is reached; A step of synthesizing a plurality of photographic images captured by the imaging device to generate a single synthesized image.

Images