Model image registration method, image processing method, and container transport apparatus

Abstract

To provide a model image registration method for enabling even a worker who is not familiar with software for image processing to easily register a model image to be referred to in image processing at a work site without spending cost, an image processing method using the model image, and a container conveying device for executing the image processing method.SOLUTION: A method includes: a first step AST of detecting an edge E of a workpiece from an image of the workpiece and calculating at least X-Y coordinate width for specifying a rectangular image region including the edge E for generating a model image and a numerical value of start point coordinates serving as a reference thereof; and a second step BST of cutting out an image region MA indicated by the numerical value from the image of the workpiece and registering the image region MA in a storage unit 8 as a model image corresponding to a type of the workpiece.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a method for registering a model image, an image processing method using the model image, and a container transport device that captures an image of a workpiece using an imaging device, recognizes an image during the image capture that is determined to be identical to a model image of a container as the workpiece that has been registered in advance, identifies the container, and transports it. [Background technology]

[0002] Through image processing of the workpiece, processing such as specific pick-up and quality inspection may be performed. For example, in a product manufacturing line, an image of a workpiece to be a product is captured by an imaging device such as a camera along a conveyance path, and an image determined to be identical to a pre-registered model image of the workpiece is recognized during the image capture to identify the workpiece, and the workpiece being conveyed along the conveyance path is picked up by a pickup robot according to position information related to the identified workpiece.

[0003] In addition, products may be classified as good or bad by determining the identity, similarity, or difference between a model image of the workpiece and an image of the actual workpiece.

[0004] In this way, in order to recognize a specific workpiece by comparing an image of the workpiece with a pre-registered model image, it is necessary to register the model image in the image processing device prior to the work / processing.

[0005] In other words, first, a model image is registered (model image registration process), and then when a specific task is to be performed, the registered model image is identified and the specific task is performed based on image processing using the model image (actual task using image processing). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2021-039457 [Patent Document 2] Japanese Patent Application Publication No. 9-12836 Summary of the Invention [Problem to be solved by the invention]

[0007] However, conventional model image registration processing uses dedicated software for model image registration, which requires the operator to become proficient with the software, and requires time and money to train the operators at the production site.

[0008] In addition, the environment for registering model images requires dedicated software, which means it cannot be run on a general PLC (control device) or touch panel; it is necessary to add a dedicated control terminal or modify the PLC itself, which is costly.

[0009] Therefore, the present invention aims to provide a model image registration method that allows even workers who are not familiar with image processing software to register model images to be referenced in image processing easily and inexpensively at the work site, an image processing method that uses the model image, and a container conveying device that executes the image processing method. [Means for solving the problem]

[0010] In order to achieve the above-mentioned object, the model image registration method of the present application is a method for identifying a target workpiece using a model image from an image of the workpiece for identification, and for registering the model image in a memory unit to be referenced when performing a specified work process on the identified workpiece, and is characterized by having: a first step of detecting the edge of the workpiece from the image of the workpiece, and calculating at least the XY coordinate width and the numerical value of the starting point coordinate that serves as its reference to identify a rectangular image area including the edge for generating the model image; and a second step of cutting out an image of the area indicated by the numerical values ​​from the image of the workpiece, and registering it in the memory unit as a model image corresponding to the type of the workpiece.

[0011] The first step comprises a first imaging step of imaging the workpiece, a step of detecting the edges of the workpiece from the image obtained in the first imaging step, and a model area calculation step of calculating at least numerical values ​​of X and Y coordinate widths that identify a rectangular image area including the edges for generating the model image, and the second step comprises a second imaging step of imaging the workpiece, and a model registration step of cutting out an image of the area indicated by the numerical values ​​from the image obtained in the second imaging step and registering it in the memory as a model image corresponding to the type of the workpiece.

[0012] According to the model image registration method of the present invention, it is possible to easily register a model image by simply setting the workpiece to be registered within the imaging field of view of the imaging device and sending a model image registration command.

[0013] Furthermore, the image processing method of the present invention is an image processing method that is executed when identifying a target workpiece from an identification image of the workpiece captured by the imaging device using a registered model image corresponding to the workpiece to be identified, and performing a predetermined work process on the identified workpiece, wherein the model image is a two-dimensional image of the workpiece to be identified in a horizontal state, registered by the above-mentioned model image registration method, the identification image is an image of the workpiece being transported in a horizontal state on a transport path, and the image processing unit of the imaging device performs image processing to detect the position and orientation of the workpiece in the identification image based on the model image when the work process is performed.

[0014] According to the image processing method of the present invention, a model image can be easily registered by simply setting the workpiece to be registered within the imaging field of view of the imaging device and sending a model image registration command from an existing PLC, etc., and the target workpiece can be identified based on the model image, and a specified work process can be performed on the identified workpiece.

[0015] The container conveying device of the present invention is a container conveying device as a production device that executes an alignment and conveying program that identifies a target work from an identification image of the work using a registered model image corresponding to the work to be identified, and aligns the identified work in a predetermined posture and sends it out to the next process, and is characterized by comprising: an input means for instructing the execution of the alignment and conveying program; a conveying device that conveys the container as the work along a conveying path; an imaging device that is arranged so as to be able to acquire the identification image of the container conveyed along the conveying path and has an image processing unit that executes the model image registration method and / or the image processing method; a pick-up robot that captures the container identified in the image processing unit, aligns it in a predetermined posture and sends it out to the next process; and a control device that controls the operation of the conveying device, the imaging device, and the pick-up robot when the alignment and conveying program is executed.

[0016] With the container conveying device of the present invention having this configuration, the aforementioned model image registration method and image processing method are executed in response to an instruction to execute the alignment conveying program from the input means, and the identified workpiece is securely held, aligned in a predetermined posture, and sent out to the next process. [Effects of the Invention]

[0017] Thus, according to the model image registration method and image processing method of the present invention, the model image used by the image processing unit can be registered easily and at low cost at the production site, the target work can be identified based on the model image, and the identified work can be subjected to specified work processing.In the container conveying device, the identified work can be securely held, adjusted to a specified posture, and sent out to the next process. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is an explanatory diagram showing a schematic configuration of an embodiment of a container transport device of the present invention; [Figure 2] 1 is a flowchart illustrating an embodiment of a model image registration method according to the present invention. [Figure 3]FIG. 1 is an explanatory diagram showing the positional relationship of a bottle-shaped container as a workpiece with respect to an imaging area when performing the first step of the model image registration method. [Figure 4] FIG. 1 is an explanatory diagram of edge detection of an imaged bottle-shaped container when performing the first step of the model image registration method. [Figure 5] An explanatory diagram of how to determine the Y coordinate (y1) of the top side of the rectangular image area MA [Figure 6] An explanatory diagram of how to determine the Y coordinate (y2) of the bottom side of the rectangular image area MA [Figure 7] An explanatory diagram of how to determine the X coordinate (x1) of the left side and the X coordinate (x2) of the right side of the rectangular image area MA [Figure 8] An explanatory diagram of how to determine the coordinates (x1, y1) of the starting point that serves as the reference for the rectangular image area MA, and the values ​​of the area length width (x2-x1) and area height width (y2-y1) DETAILED DESCRIPTION OF THE INVENTION

[0019] Below, we will explain embodiments of a container conveying device 1 as a production device that executes an alignment and conveying program that identifies a target workpiece from an image taken for workpiece identification using a registered model image corresponding to the workpiece, and then aligns the identified workpiece in a specified posture and sends it out to the next step, as well as a method for registering the model image and an image processing method using the model image that are executed in the alignment and conveying program.

[0020] 1 shows the main components of the container conveying device 1. This container conveying device 1 has a conveying device 2 that conveys bottle-shaped containers B in a sideways position (the orientation of the mouth and bottom does not matter) with the bottle axis aligned along the running direction of the conveyor belt 12. The conveying device 2 has a conveyor belt 12 that is stretched between a drive pulley connected to a motor as a drive source and a driven pulley, and that circulates along a conveying path for conveying the bottle-shaped containers B and a return conveying path. In this embodiment, the conveyor belt 12 is made of a flexible, light-transmitting sheet material.

[0021] The conveying path is provided with a supply section where transparent or translucent bottle-shaped containers B are supplied from a container supply device 3 from the upstream side in the running direction of the conveyor belt 12 with the bottle axis aligned with the running direction of the conveyor belt 12 and placed on the conveyor belt 12, an imaging area where the posture of the bottle-shaped containers B being conveyed, more specifically the orientation of the mouth and bottom of the bottle-shaped containers B, and a transfer area where the bottle-shaped containers B that have been imaged are conveyed, picked up by a pick-up robot 4 and transferred to another location.

[0022] The imaging area arranged on the conveying path is provided with an imaging device 6 having a camera 5 that images transparent or translucent bottle-shaped containers B being conveyed on a conveyor belt 12 from above the conveying path, and a backlight 11 that illuminates the bottle-shaped containers B in the imaging area from the back side of the conveying path through the conveyor belt 12.

[0023] In this embodiment, the imaging device 6 has an image processing unit 7 that registers a model image, which will be described later, and performs image processing using the model image. In this embodiment, the model image is stored in a storage unit 8 of the image processing unit 7, but the storage unit 8 may be located anywhere in the container conveying device 21 as long as it can be referenced in the image processing, which will be described later.

[0024] In a transfer area located downstream of the imaging area on the conveying path, a pickup robot 4 is provided which picks up a bottle-shaped container B based on the imaging results of the camera 5 and transfers it in a desired posture, such as upright, to a subsequent location, such as onto a conveyor of a filling device that fills the bottle-shaped container B with filling liquid.

[0025] The container conveying device 21 of this embodiment is equipped with a control device (PLC) 9 that controls the driving of the conveyor belt 12 of the conveying device 2 and the driving of the pickup robot 4, and also commands and controls the registration of model images in the image processing unit 7 and the execution of image processing using the model images, and an input device 10 for sending instruction commands to this control device 9, such as executing an alignment conveying program and registering model images.

[0026] Here, the registration of a model image in the image processing unit 7 of this embodiment and image processing using the model image will be described.

[0027] 2 is a flowchart illustrating the flow of model image registration (model image registration method) performed in the container conveying device 21 of this embodiment. This model registration is a process in which a model image is used to identify a target bottle-shaped container B from an identification image of the bottle-shaped container B as a work, and the model image is registered in the storage unit 8 as a reference in an alignment and conveyance program that performs a predetermined work process on the identified bottle-shaped container B, and is executed in response to receipt of an instruction command from the input device 10.

[0028] In this embodiment, the process of registering a model image can be broadly divided into a first process (model area calculation process) of detecting an edge E from an image of the bottle-shaped container B being transported and calculating a numerical value that identifies a rectangular image area MA that includes the edge E for generating a model image, and a second process (model image registration process) of cutting out an image of the area indicated by the numerical value from the image of the bottle-shaped container B and registering it in the memory unit 8 as a model image corresponding to the variety of the bottle-shaped container B.

[0029] More specifically, with the bottle-shaped container B placed within the container placement area SA in the imaging area PA on the conveying path of the conveying device 2, a command to register a model image is instructed from the input device 10. In the first step (AST), the control device 9 starts a job for calculating numerical values ​​(AST1) and operates the camera 5 to obtain an image of the bottle-shaped container B (AST2).

[0030] In this embodiment, as shown in Figure 3, in the container installation area SA within the imaging area PA with the backlight 11 turned on, the bottle-shaped container B is placed with its bottle axis aligned with the conveying direction of the conveyor belt 12, that is, in a horizontal position parallel to the X-axis.

[0031] Next, the edge E of the bottle container being imaged is detected (AST3). In this embodiment, as shown in Fig. 4, the edge E is detected by detecting the edges E of the mouth and bottom of the imaged bottle-shaped container B in the X-axis direction, and detecting the edges E of both sides of the bottle-shaped container B in the Y-axis direction. In Fig. 4, the arrows indicated by solid lines indicate positions where the edge E was detected, and the arrows indicated by dashed lines indicate positions where the edge E could not be detected.

[0032] Next, the XY coordinate width specifying the rectangular image area MA including the edge E for generating the model image and the numerical values ​​of the starting point coordinates serving as the reference are calculated (AST4).

[0033] Specifically, for the top side of the rectangular image area MA, as shown in Fig. 5, the smallest value (y1) of the Y coordinate of the edge E detected on the upper side surface of the bottle-shaped container B being imaged is found, and for the bottom side, as shown in Fig. 6, the largest value (y2) of the Y coordinate of the edge E detected on the lower side surface of the bottle-shaped container B being imaged is found. Also, for the left side of the rectangular image area, as shown in Fig. 7, the X coordinate (x1) of the edge E detected on the left side surface of the bottle container being imaged is found, and for the right side, the X coordinate (x2) of the edge E detected on the right side surface of the bottle container being imaged is found.

[0034] From this result, as shown in Figure 8, the coordinates of the starting point that serves as the reference for the rectangular image area MA that includes the edge E for generating the model image are set to the point with coordinates (x1, y1), and the area length and width are calculated as (x2-x1) and (y2-y1) for the area height and width. It is also possible to set a margin in the image area MA, and in that case the start point coordinates, area length and width, and area height are determined taking the margin width into account as appropriate.

[0035] Next, the process moves to the second step (model image registration step). First, to register the type of bottle container, a production job is read (BST1), and the coordinates (x1, y1) of the starting point, the coordinates of the area width (x2-x1), and the coordinates (y2-y1) of the area height are registered (BST2), which are the reference coordinates of the rectangular image area MA including the edge E for generating the previously obtained model image.

[0036] Next, the bottle-shaped container B placed in the imaging area PA with the backlight 7 turned on in the imaging field of view of the camera 5 is imaged again (BST3), and a rectangular image area MA including the edge E for generating the model image is cut out from the image (BST3), and the bottle-shaped container B as the workpiece this time is registered as the model image in the storage unit 8 (BST4).Then, the production job is saved (BST5), and the model image registration command is terminated.

[0037] In this way, in this embodiment, it is possible to register a model image without having to specify an area using a separately provided device such as a GUI.

[0038] The container conveying device 1 of this embodiment executes an alignment and conveyance program that uses the model image registered as described above to perform image processing to identify the target bottle-shaped container B, and causes the pickup robot 4 to grasp the identified bottle-shaped container B in a sideways position, change it to an upright position with its mouth facing upward, and send it onto a subsequent conveyor (not shown). The image processing referred to here is image processing that is known as a method of identifying workpieces using a model image, and is a process that calculates the rotation angle of the bottle-shaped container B to be identified and the XY coordinates of the gripping position by the pickup robot 4 based on the model image.

[0039] That is, bottle-shaped containers B are supplied from a container supply device 3 (not shown) to a supply section provided at the most upstream of the conveying path of the conveying device 2, with their bottle axes aligned along the running direction of the conveyor belt 12. In other words, the mouths of the bottle-shaped containers B are either facing in the conveying direction or facing in the opposite direction. By conveying the bottle-shaped containers B with their bottle axes oriented in two different directions in advance, it is possible to separate the bottle-shaped containers B lined up in the conveying direction, which makes it easier to detect the orientation of each bottle-shaped container B, as will be described later.

[0040] When the bottle-shaped container B reaches the imaging area, the image processing unit 7 of the imaging device 6 illuminates it from the back side using the backlight 11, while the camera 5 captures an image for identification that emphasizes the edge E of the bottle-shaped container B. The image processing unit 7 then determines the position of the bottle-shaped container B currently being transported to be identified (the position of the bottle container itself and the gripping position) and the orientation of the mouth by comparing it with the model image in terms of a rotation angle (0° or 180°).

[0041] The image capturing results are sent to a control unit (not shown) that controls the operation of the pickup robot 4, which is provided in a transfer area downstream of the conveyance path. Based on the image capturing results indicating the rolled state and position of the bottle-shaped container B being conveyed on the conveyor belt 12, which are provided by the image processing unit 7, the pickup robot 4 grasps the bottle-shaped container B at a predetermined grasping position, changes the posture to an upright position, and sends it out onto the subsequent conveyor (not shown).

[0042] In this way, the container conveying device of this embodiment executes the above-mentioned image processing method upon receiving an instruction from the input means to execute the alignment conveying program, and can securely hold the identified work, align it in a specified posture, and send it out to the next process.

[0043] The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, in the container conveying device and image processing method of this embodiment, the bottle-shaped container is conveyed with two different axial directions for identification image capture by a camera. However, the orientation of the workpiece placed on the conveyor during conveyance is not limited to the example of this embodiment. For example, the position and orientation of the workpiece oriented in various directions can also be detected by comparison with a model image. [Explanation of symbols]

[0044] 1 Container transport device 2. Conveyor equipment 3 Container feeding device 4. Pickup robot 5. Camera 6. Imaging device 7 Image processing section 8 Memory section 9 Control Device 10 Input Devices 11 Backlight 12 Conveyor Belt E-edge B. Bottle-shaped container PA imaging area SA container installation area MA (Model Image) Image Area

Claims

1. A method for identifying a target workpiece using a model image from an image of the workpiece for identification, and registering the model image in a storage unit to be referenced when performing a predetermined work process on the identified workpiece, a first step of detecting an edge of the workpiece from an image of the workpiece, and calculating at least the X-Y coordinates and the numerical values ​​of the starting point coordinates that serve as a reference for specifying a rectangular image area that includes the edge for generating the model image; a second step of cutting out an image of the area indicated by the numerical value from the image of the workpiece and registering it in the storage unit as a model image corresponding to the type of the workpiece; A model image registration method comprising:

2. The first step includes a first imaging step of imaging the workpiece, a step of detecting an edge of the workpiece from the image obtained in the first imaging step, and a model area calculation step of calculating at least X-Y coordinate values ​​that identify a rectangular image area including the edge for generating the model image, The second step includes a second imaging step of imaging the workpiece, and a model registration step of cutting out an image of the area indicated by the numerical value from the image obtained in the second imaging step and registering the image in the memory as a model image corresponding to the type of the workpiece.

2. The model image registration method according to claim 1, further comprising:

3. An image processing method that is executed when identifying a target workpiece from an image of the workpiece for identification taken by an imaging device using a registered model image corresponding to the workpiece to be identified, and performing a predetermined work process on the identified workpiece, the model image is a two-dimensional image of a workpiece to be identified in a horizontally lying state, which is registered by the model image registration method according to claim 2, and the identification image capture is an image of a workpiece being transported in a horizontally lying state on a transport path, An image processing method characterized in that an image processing unit of an imaging device performs image processing to detect the position and orientation of a workpiece in the identification image based on the model image when the work processing is being performed.

4. A container conveying device as a production device that executes an alignment conveying program that identifies a target workpiece from an identification image of the workpiece using a registered model image corresponding to the workpiece, aligns the identified workpiece in a predetermined posture, and sends it out to a subsequent device, an input means for instructing execution of the alignment and conveyance program; a conveying device that conveys the container as the workpiece on a conveying path; an imaging device that is arranged to be able to acquire the identification image of the container being transported along the transport path and that has an image processing unit that executes the image processing method according to claim 3; a pickup robot that captures the container identified by the image processing unit, arranges it in a predetermined posture, and sends it out to a subsequent location; a control device that controls driving of the transport device, the imaging device, and the pick-up robot when the alignment and transport program is executed; A container transport device comprising:

Images