Conveying system

Abstract

The conveying system of the present invention can implement the identification processing of the conveyed object at a high speed while reducing the errors in the identification processing caused by the posture change of the conveyed object during transportation; the conveying system of the present invention comprises: a conveying device that conveys the conveyed object along a conveying path, an image acquisition unit that sequentially acquires a plurality of images in a manner that all the conveyed objects passing through the conveying path are necessarily photographed in the two or more images of the measuring area, and a conveyed object identification unit, the conveyed object identification unit sequentially performing image processing on the two or more images photographing the conveyed object for each of the conveyed objects, thereby deriving an identification result based on the processing results of the two or more images.

Description

Technical Field

[0001] The present invention relates to a delivery system. Background Art

[0002] Conventionally, in various conveying systems, images generated by photographing conveyed objects are processed to count conveyed objects, determine whether they are good or bad, distinguish their postures, detect conveying states (speed, density, interval, etc.), etc. Examples of the above-mentioned conveying systems include the devices disclosed in the following Patent Documents 1 and 2.

[0003] [Prior art literature]

[0004] [Patent Literature]

[0005] Patent Document 1: Japanese Patent Application Publication No. 2017-121995

[0006] Patent Document 2: Japanese Patent Application Publication No. 2019-169010 Summary of the invention

[0007] However, in the above-mentioned conveying system, since the position or posture of the conveyed object sometimes changes irregularly during conveyance, there is a case where the image processing is affected by the change in position or posture and an erroneous identification result is obtained. In this case, the following first problem occurs, namely: conveyed objects without problems are excluded from the conveying path, or conveyed objects with problems flow directly to the downstream side, etc., resulting in poor conveying form due to errors in the identification results.

[0008] In addition, when performing image processing of a conveyed object, if the area of ​​the measurement area to be processed is too large, the conveyed object needs to be identified after the position of the conveyed object inside the measurement area is determined, so the amount of data to be processed increases, the load of image processing increases, and the time required for image processing increases. Therefore, there is also a second problem that high-speed identification processing cannot be performed and high-speed transportation cannot be coped with. Here, if the image processing speed is pursued, the identification accuracy for the change of the position or posture of the conveyed object is likely to decrease, and thus the dilemma of further increasing the misjudgment caused by the change of the position or posture of the conveyed object will occur.

[0009] Therefore, the present invention is an invention for solving the above-mentioned problems, and its subject is to reduce the identification processing errors caused by the change of the position or posture of the conveyed object during conveying in the conveying system. In addition, it is also to be able to implement the identification processing of the conveyed object at a high speed while reducing the identification processing errors of the conveyed object.

[0010] In order to solve the above-mentioned problem, the conveying system of the present invention comprises: a conveying device, which conveys a conveyed object along a conveying path; an image acquisition unit, which is a unit for acquiring an image of a measuring area through which the conveyed object passes on the conveying path, and sequentially acquires a plurality of the images in such a manner that all the conveyed objects passing through the conveying path are necessarily photographed in two or more of the images of the measuring area respectively; and a conveyed object identification unit, which is a unit for performing image processing on an image portion of the conveyed object in the image to identify the conveyed object, and, for each of the conveyed objects, image processing is performed sequentially on the two or more images in which the conveyed object is photographed, thereby deriving an identification result based on the processing results of the two or more images.

[0011] According to the present invention, since all conveyed objects are captured by the image capturing unit in two or more images respectively, for each conveyed object, the conveyed object identifying unit can derive an identification result for each conveyed object based on two or more processing results obtained by performing image processing on the image portion of the conveyed object in the two or more images, thereby reducing the possibility of an erroneous identification result caused by the influence of the processing result caused by changes in the position or posture of the conveyed object, etc.

[0012] In the present invention, it is preferred that the transported object identification unit derives the identification result corresponding to the most processing result among the processing results for the two or more images for each of the transported objects. Thus, it is possible to improve the identification accuracy by deriving the identification result corresponding to the most processing result among the processing results for the two or more images. In particular, it is preferred that the two or more images are three or more images.

[0013] In the present invention, the transported object identification unit preferably derives the identification result corresponding to the predetermined processing result for each of the transported objects when the predetermined processing results for the two or more images reach a set number. In particular, the two or more images are preferably three or more images, and the set number is two or more.

[0014] In the present invention, it is preferred that when the transport object identification unit performs image processing on the two or more images, the image processing is performed on the second and subsequent images within an observation area limited by the position information of the transport object obtained from the image processed last time and the predicted movement form of the transport object. Thus, although two or more images need to be processed for each transport object, since the image processing is performed on the second and subsequent images within the observation area limited by the position information of the transport object obtained as the image processing result in the image processed last time and the predicted movement form, the image processing burden can be reduced and the image processing speed can be increased.

[0015] In the present invention, it is preferable that the predicted movement form is a predicted movement amount in the conveying direction corresponding to a conveying speed of the conveyed object and a time difference between capturing an image to be processed this time and a next image to be processed.

[0016] In the present invention, it is preferred that the observation area for the second and subsequent images is set as an area formed by expanding the reference range in a manner that includes a peripheral area arranged around the reference range, wherein the reference range is a range formed by arranging the detection range of the conveyed object obtained in the image processing of the image processed last time at the predicted position after the movement. As a result, the conveyed object can be detected more reliably in the observation area for the next image, and the processing result of the identification processing can be derived. Here, it is sufficient to set the peripheral area on at least a part of the periphery of the reference range. However, from the perspective of improving the reliability of the derivation of the processing result, it is better to set the peripheral area around the entire periphery of the reference range. In this case, the degree of expansion of the observation area relative to the reference range is preferably increased or decreased according to the deviation of the past movement form of the conveyed object.

[0017] In the present invention, the predicted movement form is preferably obtained based on the past movement form of the conveyed object. In this case, the past movement form refers to the past movement speed, movement direction, position or posture change of the conveyed object. In addition, the method of obtaining the predicted movement form can adopt various methods according to the movement characteristics of the conveyed object. For example, the predicted movement form can be derived based on a representative value such as an average value or median value of a plurality of the past movement forms, or it can also be derived based on the latest past movement form.

[0018] In the present invention, it is preferred that the transported object identification unit performs image processing on the first image of the two or more images in a predetermined first observation area set in a manner in which the transported object can be placed, thereby obtaining the transported object identification information. Here, when the image is acquired by the image acquisition unit by photographing the passing part of the transported object, it is preferred to determine the first observation area in a manner corresponding to the passing detection position of the transported object. In addition, in this case, it is preferred that the observation area corresponding to the next image is set according to the predetermined first predicted movement form.

[0019] In the present invention, it is preferred that a conveyed object passage determination unit is further provided for detecting whether the conveyed object has passed through the limited area by performing image processing on the limited area within the image set in the measurement area, and when the conveyed object passage determination unit determines that the conveyed object has passed through the limited area, image processing is performed on the initial image in the initial observation area that is in a predetermined positional relationship with the limited area. Here, the limited area is preferably an area that is short in the conveying direction and long in a direction perpendicular to the conveying direction. In addition, the conveyed object passage determination unit is preferably capable of detecting a first state in which the conveyed object exists in the limited area and a second state in which the conveyed object does not exist in the limited area, and determines that the conveyed object has passed based on the switching from the first state to the second state.

[0020] In the present invention, it is preferred that the image acquisition unit acquires the images in sequence at a predetermined acquisition time regardless of the presence or absence of the conveyed object and without using a trigger signal, and provides the images to the conveyed object identification unit. Thus, since there is no need to use a trigger signal for determining the time to acquire the image, which is emitted by a conveyed object detection sensor or the like, it is possible to prevent detection errors of the conveyed object and to simply construct the image acquisition unit. In this case, it is preferred to set a measurement area in the image that can contain all the conveyed objects passing through multiple times based on the relationship between the maximum moving speed of the conveyed object and the time interval between the image acquisition times, and to process the initial image by the conveyed object identification unit using the entire measurement area as the observation area, thereby obtaining the initial conveyed object detection information. At this time, the observation area for the next image is preferably set based on the predetermined initial predicted movement form.

[0021] In the present invention, it is preferred that the transported object identification unit repeatedly performs identification processing on the observation area and sets the observation area for the next image according to the detection range of the transported object and the predicted movement form, thereby obtaining the transported object identification information by image processing in the observation area for the two or more images. In this case, for a plurality of transported objects, the transported object identification unit preferably performs the identification processing in the two or more images in parallel on each of the transported objects, and derives the identification result based on the plurality of processing results.

[0022] In the present invention, it is preferred that a conveying control unit for controlling the conveying device according to the above-mentioned conveying object identification information is further provided. Here, it can be cited that the conveying control unit controls a conveying object processing mechanism provided on the conveying device for screening or changing the posture of the conveying object according to the above-mentioned conveying object identification information. For example, as the conveying object processing mechanism, there are conveying object exclusion processing units for excluding conveying objects from the conveying path by airflow pressure, and posture reversal processing units for changing the posture by applying airflow pressure to the conveying objects. As control objects of these conveying object processing units, there are action time or action pressure of airflow pressure, etc.

[0023] (Effects of the Invention)

[0024] According to the present invention, it is possible to reduce the error of identification processing caused by the position or posture change of the conveyed object during transportation in the conveying system. In addition, for each conveyed object, the object portion for image processing after the second image is set within a limited observation area, thereby enabling the identification processing of the conveyed object to be performed at a high speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a schematic flowchart showing schematic steps of a conveyance object identification processing unit in an embodiment of the conveyance system according to the present invention.

[0026] Figure 2 This is a schematic configuration diagram schematically showing an example of the overall configuration of this embodiment.

[0027] Figure 3 (a) to (g) are image explanatory diagrams schematically showing an example of the process of the transported object identification processing according to this embodiment.

[0028] Figure 4 (a) is a schematic diagram schematically showing the overall structure of a function realization unit corresponding to a conveyed object detection process used in conveyed object identification processing of this embodiment, and (b) is an explanatory diagram for explaining the principle of the processing content implemented by this structure.

[0029] Figure 5 (a) to (e) are explanatory diagrams for explaining a process of deriving conveyed object identification information through image processing performed by the image processing unit in the conveyed object identification process of this embodiment.

[0030] Figure 6 (a) to (e) are diagrams showing examples of the results of executing the transport object identification process according to this embodiment.

[0031] Figure 7 (a) is a schematic diagram showing an example of a problem of a conventional example in a conveyed object identification process, and (b) is a diagram showing another example.

[0032] Figure 8 10P is a schematic flowchart showing a processing procedure of an example of the operating program 10P in this embodiment.

[0033] (Explanation of symbols)

[0034] 10 conveying system, 10P action program, 11 feeder, 110 conveyor, 111 conveyor path, 12 linear feeder, 120 conveyor, 121 conveyor path, OP jet port, CA, CA1~CA3 conveyed objects, CM1, CM2 cameras, CL11, CL12 controllers, DTU inspection processing unit, DP1, DP2 display devices, GP1, GP2 image processing devices, GM1, GM2 image processing memories, GPX captured images, GPY image area, MPU operation processing device, MM main storage device, SP1, SP2 operation input device, RAM memory for operation processing, θ main axis angle (angle posture), 100 conveyed object identification processing unit, 101 conveyed object detection processing procedure, A image acquisition unit, Ai image, B image processing unit, Bi conveyed object detection information, Bia conveyed object detection range, C movement form prediction unit, Ci predicted movement form, D observation area setting unit, Di observation area, di reference range, Ei conveyed object movement form, MS conveyed object movement status, Ji conveyed object processing result, K conveyed object identification result DETAILED DESCRIPTION

[0035] Next, an embodiment of the conveying system according to the present invention will be described in detail with reference to the accompanying drawings. Figures 1 to 5 , an overview of an embodiment of the transport system according to the present invention is described. Figure 1 This is a schematic flowchart showing a part of the operating program 10P executed by the computer of the conveyance system 10 according to the present embodiment, that is, a processing procedure of the conveyance object identification processing unit 100 . Figure 2 It is a schematic configuration diagram schematically showing the overall configuration of an embodiment of the transport system 10 . Figure 3 (a) to (g) are explanatory diagrams showing examples of conveyance forms of conveyance objects on the conveyance path 121 of the conveyance system 10 . Figure 4 (a) is a schematic diagram showing a tracking process of a transported object used in the transported object identification processing unit 100 by means of function realization units A to D, and (b) is an explanatory diagram for explaining the concept of the tracking process. Figure 5 (a) to (e) are explanatory diagrams schematically showing various image processing performed in the process.

[0036] First, refer to Figure 2 The overall structure of the conveying system 10 is described. Figure 2As shown, the conveying system 10 is a conveying system for conveying a conveying object CA along a predetermined conveying path. The conveying system 10 is a vibrating conveying device having a feeder 11 and a linear feeder 12, wherein the feeder 11 is provided with a bowl-shaped conveying body 110 having a spiral conveying path 111, and the linear feeder 12 is provided with a conveying body 120 having a linear conveying path 121, and the linear conveying path 121 is provided with an inlet configured to receive the conveying object from the outlet of the above-mentioned conveying path 111 of the feeder 11. In addition, the conveying device has a conveying management function, and the conveying management function checks and judges the conveying object on the conveying path 121 of the conveying body 120 of the linear feeder 12, that is, the conveying object CA according to the captured image GPX. In addition, in the present invention, the structure not limited to the vibrating conveying device can be used in various conveying devices that convey the conveying object CA along the conveying path. In addition, even if it is a vibrating conveying device, it is not limited to the combination of the above-mentioned feeder 11 and the linear feeder 12, and can be used in other forms of conveying devices such as a circulating feeder. Furthermore, even in the above combination, the inspection is not limited to the inspection of the conveyed objects CA on the conveyance path 121 of the linear feeder 12 , and the inspection may be performed on the conveyance path 111 of the feeder 11 .

[0037] The feeder 11 is driven and controlled by the controller CL11. In addition, the linear feeder 12 is driven and controlled by the controller CL12. The controllers CL11 and CL12 drive the excitation mechanism (including an electromagnetic driver or a piezoelectric driver, etc.) of the feeder 11 or the linear feeder 12 by AC, so that the conveying bodies 110 and 120 vibrate in such a way that the conveyed objects (conveyed objects CA) on the conveying paths 111 and 121 move toward a predetermined conveying direction F. In addition, the controllers CL11 and CL12 are connected to the main body of the conveying management system, that is, the inspection processing unit DTU having an image processing function, via an input / output circuit (I / O).

[0038] In addition, when a predetermined operation input (debugging operation) is performed on the operation processing unit MPU that executes the above-mentioned conveyed object identification processing unit 100 via the operation input device SP1, SP2, etc. described later, such as a mouse, the controller CL11, CL12 stops the driving of the conveying device of the conveying system 10 according to the above-mentioned action program 10P. At this time, according to the above-mentioned action program 10P, for example, the image measurement processing in the inspection processing unit DTU is also stopped. The debugging operation and the operation of each part corresponding to the operation will be described in detail later.

[0039] The inspection processing unit DTU is composed of a processing unit MPU (microprocessing unit) of a personal computer or the like as a core. In the example shown in the figure, the processing unit MPU is composed of central processing units CPU1, CPU2, cache memory CCM, memory controller MCL, chipset CHS, etc. In addition, the inspection processing unit DTU is provided with image processing circuits GP1, GP2 which are connected to cameras CM1, CM2 as shooting units and are used to perform image processing. The image processing circuits GP1, GP2 are connected to image processing memories GM1, GM2, respectively. The outputs of the image processing circuits GP1, GP2 are also connected to the processing unit MPU, so that the image data of the captured image GPX obtained from the cameras CM1, CM2 is processed, and the appropriate processed image (for example, the image data in the image area GPY described later) is transferred to the processing unit MPU. The main storage device MM pre-stores the action program 10P of the transport management system. When the inspection processing unit DTU is started, the action program 10P is read out and executed by the processing unit MPU. In addition, the main memory device MM stores image data of the captured image GPX or the image area GPY which is the object of image measurement processing to be described later by the arithmetic processing unit MPU.

[0040] In addition, the inspection processing unit DTU is connected to the display devices DP1, DP2 or operation input devices SP1, SP2 such as liquid crystal monitors via input / output circuits (I / O). The display devices DP1, DP2 display the image data of the captured image GPX or the image area GPY processed by the above-mentioned operation processing device MPU, the result of the image measurement processing, i.e., the result of the conveyed object identification processing or the conveying behavior detection processing, etc. in a predetermined display form. In addition, this display function is not limited to the actual conveying of the conveyed object CA, and as described later, it also works when reading past data and reproducing it. In addition, by operating the operation input devices SP1, SP2 while viewing the screens of the display devices DP1, DP2, various operation instructions, set values, and other processing conditions can be input into the above-mentioned operation processing device MPU.

[0041] In the present embodiment, the cameras CM1 and CM2 continuously take pictures at a predetermined shooting interval, and perform image measurement processing on the image data in the measurement area, wherein the measurement area has a range of the conveying direction F that is pre-set in a manner to always include all conveyed objects (conveyed objects CA) passing through the conveying path according to the relationship between the conveying speed Vs of the conveyed object and the shooting interval Ts. As a result, all conveyed objects CA can be detected in the captured images of any measurement area, so that there is no need to generate a trigger signal for detecting the position of each conveyed object as in the prior art. In addition, by processing the image data of the conveyed object CA contained in the image, it is possible to reliably extract the information required for conveyed object identification processing, conveyed object detection processing, conveying motion detection processing, etc. In addition, in the conveying system of the present invention, without adopting the triggerless shooting method as described above, it is also possible to obtain an image at a shooting time corresponding to the detection time of the conveyed object determined by a normal sensor or the like.

[0042] In this embodiment, the image measurement process is further configured to make the following conditions meet. That is, all the transported objects CA passing through the measurement area are captured in two or more images Ai within the specified range of the measurement area. Here, the setting within the specified range is assumed to be the following situation, that is, the identification process is performed on the upstream side of the transported object processing unit (the position opposite to the air jet port OP) of the embodiment described later, and the transported object identification result is used to control the transported object processing unit. If it is not necessary to set the above-mentioned specified range in the measurement area as described above, it is sufficient to capture all the transported objects CA in two or more images Ai. Through the above-mentioned processing, the transported object identification process can be performed on each transported object CA in two or more images Ai, so that two or more transported object processing results Ji can be obtained as described later. Then, the transported object identification result K can be finally derived based on the two or more transported object processing results Ji.

[0043] In the conveying system 10, the operation program 10P (see Figure 8 ) in the above-mentioned transport object identification processing unit 100 (refer to Figure 1 ) is executed to perform a transport object identification process during transport, and the transport device is controlled according to the identification result of the process. In the transport object identification processing unit 100, the transport object identification processing unit 100 is used Figure 4 The transported object detection process 101 shown in (a) of FIG. 1 is used to execute the transported object detection process.

[0044] like Figure 4As shown in (a) of FIG. 1 , the transport object detection process 101 comprises: an image acquisition unit A, which sequentially acquires a plurality of images Ai including an area where a transport object CA may exist; an image processing unit B, which performs image processing in an observation area Di of these images Ai; a movement form prediction unit C, which obtains a predicted movement form Ci of the transport object CA involved in the object detection information Bi obtained by the image processing unit B; and an observation area setting unit D, which sets an observation area Di+1 related to the next image Ai+1 based on the predicted movement form Ci. Here, i is a natural number, representing a plurality of numbers from 1 to n (n is a natural number greater than or equal to 2).

[0045] The transported object detection information Bi includes a transported object detection range Bia indicating the configuration of the transported object CA, and may also include information on the type, appearance, presence or absence of defects, or type, quality of the transported object, position or posture of the transported object, etc. In addition, the above-mentioned predicted movement form Ci includes the amount and direction of movement of the transported object CA in the next image Ai+1 when the transported object CA in the current image Ai is used as a reference, for example, it can be expressed in the form of a movement vector. In addition, the predicted movement form Ci may also include a posture change of the transported object CA in the next image Ai+1 when the transported object CA in the current image Ai is used as a reference.

[0046] Figure 4 (b) in the figure shows the following situation: when the transport object detection range Bia of the transport object CA arranged inside the observation area Di is obtained in a certain image Ai, the observation area Di+1 in the next image Ai+1 is set according to the transport object detection range Bia and the predicted movement form Ci. The next observation area Di+1 is set by predicting the area including the next transport object detection range Bia+1, but the setting is based on the previous transport object detection information Bi. For example, in the example shown in the figure, the next observation area Di+1 is set by using the reference range di+1 formed by moving the previous transport object detection range Bia to the predicted position in the next image Ai+1 according to the predicted movement form Ci as a reference, and expanding the reference range di+1 according to a prescribed method.

[0047] Here, if we set (x i ,y i ) is the center coordinate of the transport detection range Bia, (v i x , v i y ) is the value of the predicted motion vector representing the predicted motion form Ci, (x i+1 ,y i+1) is the position coordinate of the center of the reference range di+1, Δt is the time interval (time difference) between the previous image and the next image, Δw is the increase in the width Wi of the observation area Di+1 from the width Wi of the reference range di+1 (the amount of expansion in the x direction), and Δh is the increase in the height Hi of the observation area Di+1 from the height Hi of the reference range di+1 (the amount of expansion in the y direction), then the following equations (1) to (4) hold. Here, the function σ(v i x ),σ(v i y ) represents the standard deviation of the x-component or y-component relative to the set of moving speeds so far. c is a proportional constant and can be set to any number such as 0.01, 0.1, etc. Δw and Δh can be set to values ​​corresponding to the amount of expansion of the surrounding area on the left and right sides of the diagram and on the upper and lower sides of the diagram of the reference range di+1, as shown in the example, or can be set to values ​​corresponding to the total amount of expansion of the surrounding area on the left and right sides of the diagram and on the upper and lower sides of the diagram. In the latter case, the ratio of the left to right of the diagram or the ratio of the upper to lower of the diagram of the amount of expansion on the left and right sides of the diagram and on the upper and lower sides of the diagram of the reference range di+1 can be 1:1, or can be set to a ratio corresponding to the information of other predicted movement forms Ci, such as the ratio of the movement probability on the left and right sides of the diagram and on the upper and lower sides of the diagram.

[0048] x i+1 =x i +v i x ·Δt …(1)

[0049] y i+1 =y i +v i y ·Δt …(2)

[0050] Δw i =c·σ(v i x )… (3)

[0051] Δh i =c·σ(v i y )… (4)

[0052] In the example shown in the figure, as shown in the above formulas (1) to (4), the predicted movement form Ci is expressed only by the amount and direction of movement of the predicted movement vector between the center point of the conveyed object detection range Bia and the center point of the reference range di+1. However, the predicted movement form Ci is not only the amount and direction of movement of the above-mentioned center position, but also generally includes information indicating the position and range of the conveyed object CA after movement that is equivalent to the conveyed object detection range Bia (width Wi, height Hi), that is, information that can derive the above-mentioned reference range di+1, and further, information that can derive the above-mentioned expansion degrees Δw and Δh. In addition, the predicted movement form Ci may further include information related to the posture change of the conveyed object CA.

[0053] The predicted movement form Ci can be obtained by various prediction methods according to the movement characteristics of the conveyed object CA. At this time, the predicted movement form C1, which is a prediction of the movement form of the conveyed object CA between the first image A1 and the next image A2, is preferably set in advance to an initial value determined in advance. For example, when the general movement speed, movement direction, and posture change of the conveyed object CA are known, the first predicted movement form C1 is set to the change of the movement speed, movement direction, and posture. The subsequent predicted movement form C2 and thereafter are preferably calculated by taking into account the previously derived movement form of the conveyed object CA. For example, the predicted movement form C2 is actually preferably determined by taking into account the initial actual movement form E1 of the conveyed object CA, wherein the initial actual movement form E1 of the conveyed object CA is derived using the conveyed object detection information B1 obtained by performing image processing on the initial observation area D1 and the conveyed object detection information B2 obtained by performing image processing on the next observation area D2. At this time, the next predicted movement form C2 can be determined by taking into account both the above-mentioned initial predicted movement form C1 and the actual movement form E1. In addition, the next predicted movement form C2 may be determined based only on the actual movement form E1. This includes setting the next predicted movement form Ci+1 (e.g., C2) to be the same as the previous actual movement form Ei (e.g., E1). Furthermore, in the case where there are multiple actual movement forms Ei in the past, the predicted movement form Ci+1 may be obtained based on the multiple movement forms Ei and using an appropriate method. As such an appropriate method, for example, a representative value such as an average value or a median value may be set, or a method of obtaining a representative value by giving a greater weight to movement forms that are closer together may be considered.

[0054] Since the initial observation area D1 needs to be given at a stage where the above-mentioned predicted movement form has not been obtained, the initial observation area D1 is pre-set to a pre-determined area according to the configuration characteristics of the conveyed object CA in the above-mentioned area, or is pre-set to a area determined according to a pre-determined rule according to the detected position of the conveyed object CA. For example, in the initial image A1, the initial observation area D1 can be set to the entire image A1, or it can be set to an area in the image A1 where the conveyed object CA is likely to be arranged. If it is difficult to predict in advance the position where the conveyed object CA can be arranged in the image A1, the initial observation area D1 is set within a relatively wide range. On the other hand, when the initial image A1 is acquired at the same time as or immediately after the position of the conveyed object CA is detected, the observation area D1 can be set to an area in the initial image A1 that reliably contains the conveyed object CA using the position information of the detected conveyed object CA.

[0055] The expansion amounts Δw and Δh from the reference range di of the observation area Di can be calculated, for example, according to the above-mentioned formulas (3) and (4). Preferably, the expansion degree or expansion rate Δw, Δh at this time is determined according to the movement characteristics of the conveyed object CA. For example, when the deviation of the moving form Ei of the conveyed object CA is large, it is necessary to increase the above-mentioned expansion degree or expansion rate, and when the deviation of the moving form Ei is small, the above-mentioned expansion degree or expansion rate can be reduced. In the case of obtaining multiple past actual moving forms Ei of the conveyed object CA, it is preferable to increase or decrease the above-mentioned expansion degree or expansion rate according to the deviation (for example, standard deviation) of the set of these moving forms Ei. By doing so, it is possible to more reliably obtain the conveyed object detection information Bi related to the conveyed object CA through image processing in the observation area Di.

[0056] In this embodiment, when the detection information of the transported object CA is analyzed in more detail, Figure 4 As shown by the dotted line in (a), additional information processing other than the above can also be performed. For example, a process of obtaining multiple movement forms Ei of the conveyed object CA and calculating the movement status MS of the conveyed object CA based on these movement forms Ei can be cited. The conveyed object movement status MS can include, for example, the form of the movement trajectory of the conveyed object CA, the change form of the movement direction, the change form of the conveying position or conveying posture, etc. These conveyed object movement status MS can be used as a detection amount for various controls or judgments on the conveyed object CA.

[0057] Next, refer to Figure 1The above-mentioned transport object identification processing unit 100 which can be used as an example of the above-mentioned transport object detection process will be described. In the transport object identification processing unit 100, the overall start setting conditions of the transport object identification processing unit 100, the initial conditions such as the setting value of the first observation area D1 or the first predicted movement form C1 in various processes for each transport object CA, the image processing routine of the observation area Di or the image processing conditions such as various thresholds, and the prediction processing conditions such as the processing routine or calculation constants used in the prediction of the movement form of the transport object CA are stored in a predetermined memory, and the program is executed according to the above-mentioned conditions that are appropriately read. In addition, the imaging device (camera) connected via the input and output circuit of the computer can be operated under the predetermined imaging conditions, thereby sequentially acquiring a plurality of images of the range including the measurement area where the transport object CA can be arranged. However, the transport object identification processing unit 100 can also be configured as a device that processes data formed by pre-saving a plurality of sequentially captured images. In this case, the device may be configured so as to sequentially store the data of the plurality of images in a computer according to a predetermined condition, or sequentially read the data of the plurality of images already stored in the computer for each image.

[0058] The transported object identification processing unit 100 starts working when a trigger signal is given, such as a signal when a predetermined start operation is completed, or a signal when a certain transported object CA to be detected is detected by a sensor or the like, or a signal when a shooting start signal is input to a shooting device. At this time, if the initial value of the initial condition is read, for example, the initial image A1 is obtained for the above-mentioned transported object CA, image processing is performed in the initial observation area D1 set to a predetermined range. Generally speaking, in this process, if Figure 5 As shown in (a) of FIG. 1 , the transport object detection process is implemented by performing image processing on the observation area Di of the i-th image Ai. In the transport object detection process, it is preferred that: first, Figure 5 As shown in (b), the image data of the observation area Di is processed by a predetermined filtering process, and the transport identification process is converted into an easy state. For example, when converting into an image that enhances the contour of the transport object CA, edge extraction processing or edge enhancement processing is performed. On the other hand, when the influence other than the contour is to be reduced, smoothing processing of other parts is performed. As examples of filtering (processing methods) that meet these requirements at the same time, guided filtering, bilateral filtering, etc. can be cited. In addition, as filtering processing, a binarization filter, a moving average filter, a Gaussian filter, a median filter, a Sobel filter, a Prewitt filter, etc. can be used.

[0059] Next, the filtered image is subjected to image processing for detecting the position, range, posture, pattern, color, and other states of the conveyed object CA. Figure 5 The range of the transported object CA is determined as shown in (c) or (d) in FIG. Generally, contour tracking processing, pattern detection processing, pattern fitting processing, etc. are performed in this process. Through these processing, the range of the transported object CA detected in the observation area Di can be determined, for example, the transported object detection range Bia can be obtained. The transported object detection range Bia can be determined, for example, by Figure 5 Bounding boxes such as the one shown in (c) Figure 5 The results can be expressed by a fitting graph such as an ellipse fitting as shown in (d).

[0060] The boundary graphics such as the above-mentioned bounding box form a region such as a rectangle surrounding the image of the conveyed object CA, and are obtained in a manner that makes the area (boundary graphics) of the specified conditions surrounding the conveyed object CA as small as possible (for example, as a circumscribed area). In addition, the fitting graphics such as the above-mentioned ellipse fitting fit the image of the conveyed object CA into various graphics such as a circle, an ellipse, a rectangle, a polygon, etc., to form the closest graphic. In addition, in the conveyed object detection process, shape detection technology, graphic detection technology, area extraction technology, etc. in image processing technology can be used, so that various information related to the conveyed object CA can be recognized, classified, and distinguished. The specific methods of the above-mentioned technologies are not limited, and template matching, blob detection method, various edge detection, various conversion processing, etc. can be used. In this embodiment, in order to form a bounding box, the following method can be used: using a rotation calipers algorithm, the minimum area rectangle in the circumscribed rectangle obtained for each angle angle by rotating the calipers relative to the point group of edge pixels is obtained. In addition, as an ellipse fitting, for example, a method of finding an ellipse that minimizes the square error of the point group of edge pixels can be used.

[0061] By using the transport object detection range Bia that can be derived as described above, the position and range of the transport object CA in the observation area Di can be known. In addition, the posture of the transport object CA can also be detected from the transport object detection range Bia. Figure 5As shown in (e) in the figure, by finding the directions of the edges of the boundary figure and the fitting figure and the direction of the main axis, the angular posture of the conveyed object CA can be obtained. In the example shown in the figure, the posture of the conveyed object CA can be easily found by calculating the angle (main axis angle) θ of the main axis of the fitted ellipse. In this embodiment, an example of distinguishing a two-dimensional posture is given, but the three-dimensional posture of the conveyed object CA can also be found by analyzing the appearance of the conveyed object CA. In addition, in general, when the conveyed object CA is far away from the camera or the lighting environment is uneven, as described later, it is sometimes difficult to find a boundary figure (such as a rectangle) that reflects the precise shape of the conveyed object CA with high accuracy, so a method of finding a compact fitting figure such as an ellipse or a circle is preferred.

[0062] In the present embodiment, when at least a part of the image portion of the transported object CA is not included in the observation area Di, an observation area correction unit for correcting the observation area Di in such a manner that the entire transported object CA is included in the observation area Di may be further provided. Thus, the observation area is corrected by the observation area correction unit in such a manner that the entire image portion of the transported object CA is included, and thus even when the prediction of the set observation area is insufficient, the object can be reliably detected. Here, as a method for detecting that at least a part of the image portion of the transported object CA is not included in the observation area Di, for example, a method for detecting that the transported object detection range Bia obtained by the above method is not included in the range of the preset image area of ​​the transported object CA (larger or smaller than the range), a method for detecting that the transported object detection range Bia is in contact with the boundary of the observation area Di, a method for detecting that the shape of the transported object detection range Bia is significantly deformed relative to the preset shape, or a combination of the above methods, etc. When it is determined by these methods that the transported object CA is not contained in the observation area Di, the observation area Di is reset by further expanding or moving it according to the detection state, and then image processing is further performed on the reset observation area Di.

[0063] Next, refer to Figure 3 (a) to (g) in the figure illustrate the situation of the transported object CA on the transport path 121 processed by the transported object identification processing unit 100. Here, an example of the process of identifying the transported object CA by the transported object identification processing unit 100 based on the image Ai on the transport path captured by the above-mentioned cameras CM1 and CM2 of the transport device of the transport system 10 is shown. In addition, Figure 3 The plurality of images A01, A02, and A1 to A5 shown are constituted by at least a part of the image data of the captured image GPX or the image area GPY.

[0064] First, if Figure 3As shown in (a) and (b) in the figure, in the images A01 and A02 in the image area GPY obtained by photographing the above-mentioned cameras CM1 and CM2, a limited area T with a shape that is short in the conveying direction F and long in the direction intersecting the conveying direction F is pre-set within the range where the conveyed object CA on the conveying path passes. Then, when the pixel group defined by the limited area T switches from the "present" state indicating the existence state of the conveyed object CA (the distribution of pixel values ​​such as brightness along the extension direction of the pixel group) as in the image A01 to the "absent" state indicating the non-existence state (gap) of the conveyed object CA as in the image A02, it is determined that the conveyed object CA has passed through the limited area T. When performing the conveyed object passing judgment, as shown in the figure, the image A02 is set as the initial image A1, or the next image of the image A02 is set as the initial image A1, and the initial observation area D1 is set on the downstream side of the passage detection position of the conveyed object CA, that is, the limited area T. In the illustrated case where the image A02 is set as the initial image A1, the initial observation area D1 is as shown. Figure 3 As shown in (c) in the figure, the image is set at a fixed position in the downstream area adjacent to the limited area T with a size that includes the conveyed object CA. On the other hand, when the next image of the image A02 is set as the initial image A1, the initial observation area D1 is set to a range that deviates from the limited area T to the downstream side by a distance corresponding to the movement amount, taking into account the movement amount of the conveyed object CA calculated based on the time (time difference) between the image A02 and the image A1 and the movement speed of the conveyed object CA in the conveying direction F. In these cases, the initial observation area D1 can be set to a narrow area close to the amount of one conveyed object CA, so it becomes easier to increase the speed and sophistication of image processing.

[0065] The above-mentioned transported object passing judgment is to distinguish whether the transported object CA is passing through according to the distribution of pixel values ​​of the limited area T, and the distinguishing process can be realized by various processing processes. As an example, the distinguishing process can also be performed by using a neural network, wherein the neural network inputs the pixel values ​​of the pixel column of the limited area T to an input layer having the same number of nodes, and through an appropriate intermediate layer (hidden layer), and through the output of an output layer having one or more nodes, for example, distinguishes whether the transported object CA is passing through the limited area T, or distinguishes whether the limited area T is in the gap between the transported objects CA. In this embodiment, for example, a fully coupled neural network is constructed, and a distinguishing process routine of the transported object CA is used to provide the fully coupled neural network with learning data of more than 29 frames (images) to make it learn, and distinguishes the passing and gap according to the output value of a single output layer, wherein the fully coupled neural network has: an input layer having nodes corresponding to 53 pixels, two intermediate layers, and an output layer having one node. In this process, for example, by setting the number of nodes in the output layer to be multiple, the direction of the conveyed object CA, the position of the appearance pattern (the logo portion described later), etc. can also be identified. In addition, the conveyed object passage determination unit that performs the conveyed object passage determination process is not limited to the above-mentioned configuration as long as it can detect that the conveyed object CA has passed through the specified limited area T. In addition, by making the initial observation area D1 in the initial image A1 when the conveyed object CA is detected to pass through the specified limited area T in a predetermined positional relationship, the range in which the conveyed object CA that has passed through the limited area T can be set.

[0066] In the example shown in the figure, in any case, the initial observation area D1 is set at the upstream side of the range (conveyed object processing section, the same below) where the conveyed object CA faces the air jet port OP opened on the conveying path for jetting the airflow. In addition, in the conveyed object processing section provided with the air jet port OP, a solenoid valve (not shown) is controlled based on the result of the determination of whether the airflow should be jetted to the conveyed object CA on the conveying path 121 by the conveyed object identification processing, thereby determining whether the airflow is jetted from the air jet port OP.

[0067] In the transport object identification process, the image processing is performed in the observation area D1 in the same manner as described above, and the transport object detection information Bi is obtained through the transport object detection process. Figure 3The conveyed object detection range B1a is depicted in (c) of FIG. In addition, the processing required for conveyed object identification other than the conveyed object detection processing is performed as needed. As described above, the observation area D1 is set at the upstream side of the range relative to the conveyed object CA and the air jet port OP. Here, the initial predicted movement form C1 is obtained for the conveyed object CA involved in the conveyed object detection information B1a obtained in the initial observation area D1. In this case, the initial predicted movement form C1 can be obtained based on the moving speed of the conveyed object CA in the conveying direction F (for example, the average conveying speed of objects on the conveying path) and the forward direction of the conveying direction F as the moving direction.

[0068] Here, in the initial predicted movement form C1, only the movement component in the x direction along the conveying direction F may be set, and the movement component in the y direction (vertical direction) may be set to 0 and not considered. However, when the conveyed object CA moves up and down on the conveying path 121 due to vibration or the like and the moving speed (including direction, the same below) of the conveyed object CA in the y direction is known in advance, there is a high possibility that the conveyed object CA continues to move at the same moving speed due to inertia, so the moving component in the y direction may also be considered. In addition, in the case where the up and down movement of the conveyed object CA during transportation is large, the expansion degree Δh may be set in advance to a certain degree value or more according to the up and down movement. Furthermore, in the case where the speed of the conveying direction F during transportation of the conveyed object CA increases and decreases sharply, the expansion degree Δw may also be set in advance to a certain degree value or more. Here, if a certain degree of expansion degree Δw, Δh is set in the observation area Di, the predicted movement form Ci may also be set to a fixed predicted movement amount, which is obtained by setting the moving direction to the forward direction of the conveying direction F and assuming that the conveying speed of the conveyed object CA is a constant speed. In this way, the observation area Di can be easily set, so the conveyed object detection process 101 can be executed simply and at high speed.

[0069] Then, if Figure 3As shown in (d) in FIG. 1 , a reference range (d2) (not shown) is obtained based on the transport object detection range B1a and the predicted movement form (C1) (not shown), and the observation area D2 in the next image A2 is set based on this. Here, since the transport object CA moves in the transport direction F on the transport path 121, the observation area D2 is offset by a predetermined movement amount in the transport direction F relative to the above-mentioned observation area D1. Here, the movement amount is equivalent to the product of the transport speed of the transport object CA (the actual speed can be measured, or the average transport speed can be set in advance and used) and the shooting time difference Δt between the images A1 and A2. When the transport object detection process is performed in the observation area D2, the transport object detection information Bi is obtained, and the transport object detection range B2a is also determined. Here, the past movement form E1 of the transport object CA, that is, the movement amount (movement speed and movement direction) between the actual images A1 and A2 of the transport object CA can be obtained based on the transport object detection ranges B1a and B2a. Generally speaking, the predicted movement forms Ci for setting the next observation area Di+1 and later can be obtained from these past movement forms Ei.

[0070] Then, if Figure 3 As shown in (e) in FIG. 1 , the observation area D3 in the next image A3 is set in the same manner as above. Figure 3 As shown in (f) in FIG. 1 , the observation area D4 in the next image A4 is set, such as Figure 3 As shown in (g) in FIG. 1 , the observation area D5 in the next image A5 is set. In this way, the transport condition of the transport object CA on the transport path 121 can be understood through the transport object detection information Bi obtained in the above process, especially the movement condition of the transport object detection range B1a to B5a, that is, the above movement form Ei (E1 to E3).

[0071] In the above Figure 3In the example shown, before the conveyed object CA reaches the conveyed object processing section provided with the above-mentioned air jet OP, the conveyed object CA can be distinguished by image processing in the observation areas D1 to D4 in the plurality of images A1 to A4. The conveyed object CA distinguishing process obtains conveyed object processing information G1 to G4, which is information necessary for controlling conveyed object processing such as screening or posture change in the conveyed object processing section located downstream of the arrangement position of the conveyed object CA in the above-mentioned plurality of images A1 to A4. The conveyed object processing information Gi can include the above-mentioned conveyed object detection information Bi. In addition, when the above-mentioned necessary information is sufficient to be the conveyed object detection information Bi obtained in the above-mentioned conveyed object detection process, the conveyed object CA is distinguished directly based on the conveyed object detection information Bi. In each image Ai, the conveyed object processing result Ji is derived from the above-mentioned conveyed object processing information Gi. The conveyed object processing result Ji is a result that directly reflects the above-mentioned necessary information.

[0072] When the transport object processing result Ji is obtained as described above, as long as the predetermined measurement processing end condition is not satisfied, the next transport object identification processing is transferred to after the prediction setting of the next observation area Di+1 is completed. On the other hand, when the measurement processing end condition is satisfied, the predetermined transport object identification result K is calculated based on the multiple transport object processing results Ji obtained as described above. Here, the measurement processing end condition is whether the predetermined maximum number of processing times m is reached, or, although the maximum number of processing times m is not reached, whether the transport object identification result K can be derived by combining the multiple transport object processing results Ji. The maximum number of processing times m is determined by the shooting time difference Δt of the image Ai, the conveying speed v of the transport object CA, and the conveying speed v of the transport object CA. x , the position of the first observation area D1, etc. In the case of the example shown in the figure, according to the above-mentioned various conditions, the maximum number of processing times m is 4. In addition, the relationship between the transported object processing result Ji and the transported object identification result K will be described later.

[0073] The method of the transported object detection process 101 used in the transported object identification processing unit 100 is not limited to the method shown in the above-mentioned equations (1) to (4). For example, the transported object CA basically moves along the transport direction F on the transport path 121, so a method of shifting the observation area Di along the transport direction F by a fixed movement amount corresponding to the time difference Δt between the images may also be used.

[0074] As examples of the transported object processing unit having the air jet port OP, in order to control the transported object CA for which the predetermined transported object identification result K has been derived, there can be cited a transported object rejecting unit for rejecting a part of the transported object CA from the transport path 121 and a posture changing unit for changing the posture of a part of the transported object CA by turning it over. In addition, a transported object distributing unit for distributing a part of the transported object CA to two or more downstream branch paths can also be considered.

[0075] Figure 6 (a) to (e) are explanatory diagrams showing the identification form of the transported object CA performed by the transported object identification processing unit 100 of the above-mentioned embodiment of the transport system 10. In addition, in these explanatory diagrams, the transport form of the transported object CA on the transport path 121 is shown from the image Ai taken by the display cameras CM1 and CM2. Figure 3 The conveying path 121 is depicted in a cross-sectional structure in the state observed from different directions. The conveying object processing unit with the air injection port OP is shown on the right side of the figure on the downstream side. Here, the multiple conveying objects CA shown in the figure are all the same specific conveying object CA, and the above-mentioned specific conveying object CA photographed in multiple images Ai is repeatedly displayed.

[0076] exist Figure 6 In the example shown in (a), the transported object CA is subjected to transported object identification processing in the observation areas D1 to D4 in sequence using a plurality of (four) images A1 to A4. Since the transported object processing results J1, J2 and J3 are OK, and the same processing results appear more than half three times, the fourth processing is terminated (not necessary) and the final transported object identification result K is set to OK. Therefore, the airflow does not flow from the air jet port OP, and the transported object CA passes through the transported object processing unit directly. In this example, if the fourth observation area D4 is processed, an erroneous processing result J4=NG may occur due to the change in the position or posture of the transported object CA. However, since the three processing results are OK, the transported object identification result K=OK is set based on these results, thereby avoiding the generation of erroneous identification results. In this example, as the measurement end condition, among the two processing results OK and NG with a maximum number of processing times of four, if the same transported object processing result Ji appears three or more times, which is the set number and the maximum number, the transported object identification result K is set to that result and the subsequent processing is terminated. Furthermore, when the processing is completed to the end and the processing results are the same number of OK and NG twice, if the transport capacity (transport efficiency) is prioritized, the OK of the transported object processing result Ji is prioritized as the transported object identification result K. On the other hand, if the screening accuracy (screening effect) is prioritized, the NG of the transported object processing result Ji is prioritized as the transported object identification result K.

[0077] exist Figure 6In the example shown in (b), the transport object processing result Ji or the transport object identification result K is Figure 6 The example shown in (a) is the same, but the time difference Δt when acquiring the multiple images A1 to A4 is shorter than the above, so although the conveying speed of the conveyed object CA is the same, the setting interval of the observation areas D1 to D4 is smaller than the above. In this way, the conveyed object CA is frequently subjected to identification processing at positions close to each other in the degree of overlap, so that it can be processed at a position close to the conveyed object processing unit that uses the conveyed object identification result K. Therefore, on a conveying path where the characteristics such as the posture of the conveyed object CA may frequently change during conveyance (such as when multiple conveyed object processing units are close to each other), it is easy to increase the maximum number of processing times m, thereby improving the identification accuracy.

[0078] exist Figure 6 In the example shown in (c), the following situation is assumed, that is, because the posture of the conveyed object CA is bad, the conveyed object processing results from the first to the third are all NG, but the fourth processing result becomes OK due to the change of the posture of the conveyed object CA. However, even in this case, since the conveyed object processing result Ji=NG appears in G1 to G3, the identification process is no longer performed, and the conveyed object identification result K=NG is set. As a result, the airflow is ejected from the air jet OP to remove the conveyed object CA from the conveying path 121 or reverse the posture. In this example, it is assumed that the posture of the conveyed object CA in the fourth observation area D4 is close to the good posture and an OK judgment is given, but if it does not finally become a good posture and passes directly to the downstream side in a bad posture, a bad situation will occur. Therefore, by controlling the conveyed object CA using the airflow from the air jet OP in the state of NG judgment, a bad situation caused by an erroneous identification result is avoided.

[0079] exist Figure 6 In the example shown in (d), the case where the transported object processing result Ji is unstable due to the change in the posture of the transported object CA is shown. In this case, the maximum number of processing times is four times, and NG appears continuously in the first two times (set number), so the subsequent processing is stopped and the transported object identification result K=NG is set. In this example, since the processing is performed with priority given to the screening accuracy (screening effect), it can be concluded that NG appears two or more times in four processings, so no further processing is required, and the transported object identification result K=NG is set.

[0080] exist Figure 6In the example shown in (e), there is an example in which the conveyed object processing result Ji is different due to the position change caused by the up and down movement of the conveyed object CA. In this example, when the up and down movement is too large, the accuracy of image processing decreases and the conveyed object processing result Ji=NG, but as long as it is a normal up and down movement, Ji=OK. In this example, the conveying capacity (conveying efficiency) is set to take priority, and it is set that when the conveyed object processing result Ji=OK appears twice (set number), no further processing is required, and the conveyed object identification result K=OK is set.

[0081] However, the conveyed object processing result Ji is usually not limited to the case where there are two processing results (OK and NG) as described above, and there may be more than three processing results. In these cases, generally speaking, when a specified number (set number) is generated for each processing result, its result can be set as the conveyed object identification result K. In this case, the above-mentioned set number may also be different between multiple processing results. This is because, for example, as mentioned above, sometimes it is considered from the perspective of emphasizing the conveying capacity, and sometimes it is considered from the perspective of emphasizing the conveying accuracy. It is usually preferred that when a processing result exceeding half (the maximum number) of the maximum processing number m is obtained, it is set as the conveyed object identification result K corresponding to the processing result Ji. However, it does not necessarily have to exceed half. When the minimum number is twice and the same processing result Ji is obtained, the conveyed object identification result K corresponding to the processing result Ji is derived, which can greatly improve the accuracy of the identification result compared to the prior art.

[0082] <Composition of action program 10P>

[0083] Next, refer to Figure 8 The flow of the overall operating program 10P according to each embodiment of the present invention will be described. Figure 8This is a schematic flow chart showing various processing procedures for conveying management executed by the arithmetic processing device MPU of the inspection processing unit DTU according to the action program 10P. When the action program 10P is started, the above-mentioned image capturing and image measurement processing are first started, and at the same time, the driving of the conveying device (feeder 11 and linear feeder 12) is started through the controllers CL11 and CL12. Then, if the debugging setting corresponding to the above-mentioned debugging operation is OFF, the image measurement processing is performed on the captured image GPX or the image area GPY, and if the final judgment result of the conveyed object identification processing is an "OK" judgment, the image measurement processing of the next captured image GPX or the image area GPY is directly implemented as long as the debugging operation is not performed. For example, in the conveyed object exclusion processing section, the airflow of the air jet OP is usually stopped, and if the judgment result is "NG" (defective product), the airflow flows out of the air jet OP. In this way, the defective conveyed object CA is excluded from the conveying path. In addition, in the posture reversal processing unit, the airflow from the air jet port OP is usually stopped, and if the judgment result is "NG" (defective product), the airflow is ejected from the air jet port OP to reverse the conveyed object CA on the conveying path. In addition, it is also possible to form the opposite of the above, so that the airflow is usually flowing, and if the judgment result is "OK" (good product), the airflow is stopped.

[0084] In this way, by identifying the transported object CA as a transported object on the transport path and performing processing according to the identification result, only good products are supplied to the downstream side in an orderly state. In this case, as long as no debugging operation is performed thereafter, the image measurement processing and the transported object identification processing are directly implemented in the measurement area of ​​the next captured image GPX or image area GPY. Here, in the transported object identification processing performed by the transported object identification processing unit 100, the processing for a certain transported object CA is performed as described above, but for multiple transported objects CA continuously transported to the measurement area, the same transported object identification processing is usually performed in parallel for each transported object CA. In addition, in parallel with the above-mentioned image measurement processing or transported object identification processing, for example, a transport behavior detection processing for tracking the transported object CA on the transport path in order to detect the transport behavior of the transported object CA when it is transported along the transport direction F on the transport path can also be performed. Moreover, according to the detection result of the transport behavior detection processing, the drive of the conveying device can be controlled to adjust the conveying form. The control of the conveying drive, for example, controls the driving conditions of the excitation element of the excitation mechanism of the conveying device, such as controlling the frequency or voltage of the piezoelectric driver, so as to adjust to the appropriate conveying form. In addition, the above-mentioned conveying behavior detection processing can also be executed in parallel with the conveying object detection processing process 101 used in the above-mentioned conveying object identification processing unit 100, or it can be completely executed by other image processing without being related to the above-mentioned conveying object detection processing. For example, the frequency and amplitude of the vibration can be controlled according to the size of the position change in the direction perpendicular to the conveying direction F of the conveying object CA when it moves along the conveying direction F on the conveying path 121, thereby preventing unnecessary turbulence of the conveying object CA on the conveying path 121.

[0085] When the debugging operation is performed in the middle of the above, and the debugging setting is "ON", the above routine (operation mode) is exited, the driving of the conveying device is stopped, and the image measurement processing, conveyed object identification processing, conveyed object detection processing, and conveying motion detection processing are stopped. Then, in this state, appropriate operations are performed, and the past image file can be selected. At this time, the image file selected for display is an image file containing multiple captured images GPX or image areas GPY recorded in the operation mode just now. Directly select the image file and perform appropriate operations to enter the re-execution mode. In this mode, based on the image file that records the processing results of the above-executed image measurement processing, conveyed object identification processing, conveyed object detection processing, and the above-mentioned conveying motion detection processing, it is possible to re-execute the image display, various processing or control, etc. In other words, when an unfavorable condition occurs in the control (elimination, reversal, conveyed object detection) of the conveying device's conveyed object, that is, the conveyed object CA, in order to eliminate the unfavorable condition, the image processing is firstly performed again based on the past image data, thereby exploring the problem parts of each processing or control, etc. If the problem area is identified, the setting contents (set values) of each process or control can be changed or adjusted according to the problem area, and the image measurement process can be performed again on the past image data, thereby confirming the results of the adjustment and improvement work. Then, by performing an appropriate reset operation, the debug setting returns to "OFF", restarting the image measurement process and restarting the drive of the conveyor device.

[0086] In addition, in each of the above-described embodiments, for each transported object CA on the transport path, two or more images Ai capturing the transported object CA are sequentially processed, and the transported object identification result K is derived based on a plurality of processing results Ji of the transported object CA obtained from each image Ai. This can reduce errors in the identification result caused by changes in the position or posture of the transported object CA.

[0087] In contrast, in the prior art example, it is formed as follows: Figure 7 As shown in (a) in FIG. 1 , a light sensor SA is provided on the conveying path 131 to detect the passage of the conveyed object CA, and an image SB is captured at the detection time, and the conveyed object CA is determined based on the image SB. However, in this case, the image processing may be affected by the change in the position or posture of the conveyed object CA, and the conveyed object CA may be erroneously determined to be a defective product NG. In this case, even if the conveyed object CA is originally a good product OK, it may be removed from the conveying path 131 by the airflow from the air jet port OP or its posture may be reversed.

[0088] In addition, if Figure 7As shown in (b) in the figure, even when the same image measurement processing based on triggerless shooting as in the present embodiment described above is performed, since the position of the conveyed object CA, which is the object of the conveyed object identification processing, is limited to a specified range SD within the measurement area SC, it is easy to make an erroneous judgment that the conveyed object CA is a defective product NG due to a change in the position or posture of the conveyed object CA, as described above.

[0089] Unlike the above-mentioned prior art, in the present embodiment, when two or more conveyed object processing results Ji are obtained for each conveyed object CA based on two or more images Ai, and a conveyed object identification result K is derived based on these conveyed object processing results Ji, an observation area Di for performing image processing in the two or more images Ai is further provided for each conveyed object CA, thereby reducing the load of image processing for each conveyed object CA. In addition, even if image processing for multiple conveyed objects CA is performed in parallel, high-speed processing can be performed.

[0090] In the present embodiment, an observation area Di is set for each conveyed object CA based on the position information of the conveyed object CA and the predicted movement form Ci of the conveyed object CA, and image processing is performed while tracking the conveyed object CA in the observation area Di. Therefore, even if a plurality of conveyed objects CA pass continuously through the measurement area, conveyed object identification processing and conveyed object detection processing can be performed on the plurality of conveyed objects CA in parallel without confusing each other.

[0091] In this embodiment, since the next observation area Di+1 corresponding to the transported object detection range Bia and the predicted movement form Ci is set, and the transported object CA is detected by the transported object detection processing based on the image processing in the observation area Di+1, thereby deriving the transported object detection information Bi+1, it is possible to track the transported object CA and reliably detect it while limiting the image data to be processed. Therefore, it is possible to achieve high-speed processing, and it is possible to perform more advanced processing than the prior art within a limited time. In addition, by repeatedly performing the above-mentioned prediction of the observation area and image processing, it is possible to easily grasp the complex movement form and action (transported object movement status MS) of the transported object CA.

[0092] In particular, the moving form and motion of the conveyed object can be quickly and reliably grasped through the conveyed object detection process 101. This can not only realize the high-speed conveying system, but also realize the high density of the conveyed objects CA on the conveying path and the high-precision arrangement of the conveyed objects CA.

[0093] In this embodiment, when setting the next observation area Di+1, the surrounding area is further added to the reference range di+1 obtained by the conveyed object detection information Bi and the predicted movement form Ci calculated based on the observation area Di, thereby setting an expanded area. Therefore, even if the conveyed object CA shows a movement form different from the predicted movement form Ci, the conveyed object detection information Bi+1 can be more reliably derived from the next image Ai+1.

[0094] In addition, the conveying system of the present invention is not limited to the above-mentioned example, and various changes can be added without departing from the scope of the present invention. For example, the present invention can be a conveying system that obtains a conveyed object identification result K based on a plurality of conveyed object processing results Ji for a plurality of images Ai, and the method of obtaining K from Ji can be set according to the purpose or usage of the conveying system.

Claims

1. A conveying system, characterized in that: have: A conveying device that conveys the conveyed object along the conveying path; an image acquisition unit that acquires an image of a measurement area through which the conveyed object passes on the conveying path, and sequentially acquires a plurality of the images in such a manner that all the conveyed objects passing through the conveying path are necessarily captured in two or more images of the measurement area; and The transport object identification unit is a unit that performs image processing on the image portion of the transport object in the image to identify the transport object, and for each of the transport objects, the image processing is sequentially performed on the two or more images in which the transport object is photographed, thereby deriving an identification result based on the processing results of the two or more images. In the transport object identification unit, a plurality of results as the processing results and a maximum number of processing times of the image processing for obtaining the processing result for each of the transport objects are preset. The transported object distinguishing unit derives the certain type of processing result as the distinguishing result when the number of times the certain type of processing result is obtained for each of the transported objects is greater than the number of times any other type of processing result is obtained among the processing results of the maximum processing number.

2. The conveying system according to claim 1, characterized in that The two or more images are three or more images.

3. The conveying system according to any one of claims 1 to 2, characterized in that The transported object identifying unit performs the image processing on the second and subsequent images among the two or more images within an observation area defined based on position information of the transported object obtained from the image processed last time and a predicted movement form of the transported object.

4. The conveying system according to claim 3, characterized in that The predicted movement form is a predicted movement amount in the conveying direction corresponding to the conveying speed of the conveyed object and a time difference between the image to be processed this time and the image to be processed next time.

5. The conveying system according to claim 3, characterized in that The observation area for the second and subsequent images is set as: an area formed by expanding a reference range in a manner that includes a peripheral area arranged around the reference range, wherein the reference range is a range formed by arranging the detection range of the conveyed object obtained in the image processing of the previous image at the predicted position after the movement.

6. The conveying system according to claim 5, characterized in that The degree of expansion of the observation area relative to the reference range is increased or decreased according to the magnitude of the deviation of the past movement pattern of the conveyed object.

7. The conveying system according to claim 3, characterized in that The predicted movement pattern is calculated based on the past movement patterns of the transported object.

8. The conveying system according to claim 3, characterized in that The transported object identifying unit performs image processing on a first image among the two or more images in a predetermined first observation area set so that the transported object can be arranged, and sets the observation area for a next image based on a predetermined first predicted movement form.

9. The conveying system according to any one of claims 1 to 2, characterized in that: further comprising a conveying control unit for controlling the conveying device; The transport control unit controls the transport device based on the identification result.

10. The conveying system according to claim 9, characterized in that The transport control unit controls a transport object processing mechanism provided in the transport device for sorting or changing the posture of the transport object based on the identification result.

11. A conveying system, characterized in that: have: A conveying device that conveys the conveyed object along the conveying path; an image acquisition unit that acquires an image of a measurement area through which the conveyed object passes on the conveying path, and sequentially acquires a plurality of the images in such a manner that all the conveyed objects passing through the conveying path are necessarily captured in two or more images of the measurement area; and The transport object identification unit is a unit that performs image processing on the image portion of the transport object in the image to identify the transport object, and for each of the transport objects, the image processing is sequentially performed on the two or more images in which the transport object is photographed, thereby deriving an identification result based on the processing results of the two or more images. In the transport object identification unit, a plurality of results as the processing results and a maximum number of processing times of the image processing for obtaining the processing result for each of the transport objects are preset. In the transport object identification unit, a setting number of two or more is defined for each type of the processing result. The transported object distinguishing unit derives a certain type of processing result as the distinguishing result for each of the transported objects when the number of times the certain type of processing result is obtained reaches the set number for the first time.

12. The delivery system according to claim 11, characterized in that The transported object identifying unit performs the image processing on the second and subsequent images among the two or more images within an observation area defined based on position information of the transported object obtained from the image processed last time and a predicted movement form of the transported object.

13. The delivery system according to claim 12, characterized in that The predicted movement form is a predicted movement amount in the conveying direction corresponding to the conveying speed of the conveyed object and a time difference between the image to be processed this time and the image to be processed next time.

14. The delivery system according to claim 12, characterized in that The observation area for the second and subsequent images is set as: an area formed by expanding a reference range in a manner that includes a peripheral area arranged around the reference range, wherein the reference range is a range formed by arranging the detection range of the conveyed object obtained in the image processing of the previous image at the predicted position after the movement.

15. The delivery system according to claim 14, characterized in that The degree of expansion of the observation area relative to the reference range is increased or decreased according to the magnitude of the deviation of the past movement pattern of the conveyed object.

16. The delivery system according to claim 12, characterized in that The predicted movement pattern is calculated based on the past movement patterns of the transported object.

17. The delivery system according to claim 12, characterized in that The transported object identifying unit performs image processing on a first image among the two or more images in a predetermined first observation area set so that the transported object can be arranged, and sets the observation area for a next image based on a predetermined first predicted movement form.

18. The delivery system according to claim 11, characterized in that further comprising a conveying control unit for controlling the conveying device; The transport control unit controls the transport device based on the identification result.

19. The delivery system according to claim 18, characterized in that The transport control unit controls a transport object processing mechanism provided in the transport device for sorting or changing the posture of the transport object based on the identification result.

Images