Detection device, method, program and transport system

Abstract

To improve separation performance on articles.SOLUTION: A detection device according to an embodiment comprises: an acquisition unit; a determination unit; and a detection unit. The acquisition unit acquires height information of a plurality of points on a subject. The determination unit determines whether or not there are one or more steps at a height equal to or greater than a predetermined height from a reference in the subject, on the basis of the height information. The detection unit detects that the subject is in a state in which a plurality of articles overlap, when the number of steps is more than or equal to a first threshold value.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to a detection device, a method, a program, and a transport system. [Background technology]

[0002] When delivering small parcels, such as postal parcels, randomly collected parcels are separated and fed into a downstream address reader. This separation process is currently performed manually, and a singulator, which automates this process, is effective in reducing the number of workers required. Inside the singulator, the parcels are separated by changing the speed of the conveyor or by creating steps, but there are cases where they cannot be completely separated. Furthermore, there is a method for detecting overlapping mail items that extracts the outline of an envelope and determines whether it is a duplicate. However, this method is only suitable for thin, rectangular parcels such as envelopes, and has the problem that it cannot handle the thick, irregularly shaped mail items that singulators target. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 4405918 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a detection device, method, program, and conveying system that can improve the separation performance of items. [Means for solving the problem]

[0005] The detection device according to this embodiment includes an acquisition unit, a determination unit, and a detection unit. The acquisition unit acquires height information for a plurality of points on an object. The determination unit determines, based on the height information, whether or not one or more steps exist on the object at a height equal to or greater than a predetermined height from a reference. If the number of steps is equal to or greater than a first threshold, the detection unit detects that the object is in a state where multiple items are overlapping each other. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a conceptual diagram showing a transport system. [Figure 2] 4 is a timing chart showing operation timings of the transport system. [Figure 3] FIG. 10 is a diagram showing an example of an image of cargo captured by an imaging device as viewed from the X-axis direction. [Figure 4] FIG. 10 is a diagram showing an example of an image of cargo captured by an imaging device as viewed from the Y-axis direction. [Figure 5] FIG. 4 is a diagram showing a height image obtained by the detection device. [Figure 6] FIG. 1 is a block diagram showing a detection device according to a first embodiment. [Figure 7] 6 is a flowchart showing a multifeed detection process of the detection device according to the first embodiment. [Figure 8] 10 is a flowchart showing details of a step determination process. [Figure 9] FIG. 10 is a block diagram showing a detection device according to a second embodiment. [Figure 10] 10 is a flowchart showing a step determination process according to the second embodiment. [Figure 11A] FIG. 10 is a diagram showing an example of a method for setting a determination region. [Figure 11B] FIG. 10 is a diagram showing an example of a method for setting a determination region. [Figure 11C] FIG. 10 is a diagram showing an example of a method for setting a determination region. [Figure 11D] FIG. 10 is a diagram showing an example of a method for setting a determination region. [Figure 12] 10A to 10C are views showing a specific example of surface detection processing by a surface detection unit according to the second embodiment. [Figure 13]13A to 13C are views showing a specific example of surface detection processing by a surface detection unit according to the third embodiment. [Figure 14] FIG. 10 is a diagram showing a case where the signs of the elevation differences are different. [Figure 15] FIG. 10 is a diagram showing an example in which a height image is used as a partial image. [Figure 16] FIG. 2 is a diagram illustrating an example of a hardware configuration of a detection apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0007] The detection device, method, program, and transport system according to the present embodiment will be described in detail below with reference to the drawings. In the following embodiments, parts with the same reference numerals perform similar operations, and redundant descriptions will be omitted as appropriate.

[0008] (First embodiment) The transport system according to this embodiment will be described with reference to the schematic diagram of FIG. The conveying system includes a detection device 10, an imaging device 11, an article evaluation device 12, a rejection mechanism 13, and a conveying mechanism 14. Note that, although the example in Fig. 1 shows a case where the detection device 10 and the imaging device 11 are separate entities, the detection device 10 and the imaging device 11 may be integrated.

[0009] The conveying mechanism 14 is, for example, a conveyor, and as the conveyor rotates, the items to be conveyed (also referred to as cargo) flow from the upstream side to the downstream side. It is assumed that cargo is automatically supplied to the upstream side of the conveying mechanism 14. The cargo comes in a variety of shapes and sizes, from large cargo such as cargo 151 to small cargo such as cargo 152. In addition, the posture of cargo on the conveying mechanism 14 also varies in various ways, such as cargo 153, whose longitudinal direction is not aligned with the conveying direction of the conveying mechanism 14. Furthermore, there is also a situation called double conveyance, in which multiple cargoes are conveyed overlapping each other, such as cargo 154 and cargo 155.

[0010] The product evaluation device 12 senses a certain evaluation area (or evaluation line) on the conveyance mechanism 14 and detects whether or not cargo has passed through that area. If the passage of cargo is detected, the time of passage is acquired.

[0011] The imaging device 11 captures or senses an imaging area (or imaging line) on the conveying mechanism 14, and obtains height information of cargo, which is a subject, passing through the area. The detector 10 receives height information from the imaging device 11 and detects whether the cargo is in a double-feed state.

[0012] When the exclusion mechanism 13 detects that the cargo is not a duplicate, i.e., that it is a single cargo, it advances the cargo downstream of the conveying mechanism 14, and when the detection device 10 detects that the cargo is a duplicate, i.e., that it is a stack of multiple cargoes, it controls the cargo to flow in a direction (also called the reprocessing side) different from the downstream side of the conveying mechanism 14. This prevents the duplicate cargo from flowing downstream. The exclusion mechanism 13 is assumed to be, for example, a sorting plate that can be driven by a control signal, but any structure is acceptable as long as the cargo conveying direction can be switched by a control signal.

[0013] Next, a timing chart relating to the operation timing of the transport system will be described with reference to FIG. 2 is a timing chart that is a graph of the calculated timing of cargo entering the product evaluation device 12, the imaging device 11, and the rejection mechanism 13 in that order. Based on the time when the cargo enters the evaluation line of the product evaluation device 12, the time when the cargo has completely passed through, the cargo's moving speed, and the distance between the evaluation line of the product evaluation device 12 and the imaging line of the imaging device 11, it is possible to calculate the time interval from when the cargo enters the product evaluation device 12 to when the cargo reaches the imaging line of the imaging device 11. Similarly, it is also possible to calculate the time interval from when the cargo moves from the imaging line of the imaging device 11 to when the cargo reaches the rejection mechanism 13. Therefore, based on the calculated times, the product evaluation device 12, the detection device, and the rejection mechanism 13 can each determine the timing at which the cargo will arrive, and the detection result of the detection device 10 and the operation of the rejection mechanism 13 can be correctly associated with the cargo.

[0014] Next, an example of capturing an image of cargo by the imaging device 11 will be described with reference to FIGS.

[0015] Fig. 3 is a diagram showing the arrangement of the conveying mechanism 14 and the imaging device 11 as viewed from the x-axis direction, where the cargo travels from left to right. Fig. 4 is a diagram showing the arrangement of the conveying mechanism 14 and the imaging device 11 of the detection device 10 as viewed from the y-axis direction, which is the upstream side.

[0016] The imaging device 11 is, for example, an optical cutting device, and irradiates light from an irradiation unit 301 toward the conveyance surface of the conveyance mechanism 14 on which cargo is placed. When viewed from the direction of FIG. 3, the light appears linear. When viewed from the direction of FIG. 4, the light spreads out in a fan shape from the irradiation unit 301 and is irradiated onto the conveyance surface. The irradiated light is reflected by the cargo or the conveyance surface and enters the light receiving unit 302 as reflected light. The image formed by the reflected light is linear if there is no cargo on the conveyance surface and there is nothing there. However, if cargo is present on the conveyance surface, the image changes from a straight line. In the example of FIG. 3, cargo 154 and cargo 155 are present, so the shape of the image formed by the reflected light changes from a straight line. Based on the changed shape of the image, the imaging device 11 obtains height information of the subject (here, cargo) from the conveyance surface at the irradiation position. A sequence of numbers representing the height along a straight line obtained in one measurement is called a profile.

[0017] Here, since light is emitted from one point of the irradiating unit 301, shadows 401 may occur where the light does not reach. In the example of Fig. 4, shadows 401 occur at the edge between cargo 154 and the conveyance surface, and at the edge between cargo 154 and cargo 155.

[0018] The irradiation unit 301 of the imaging device 11 does not have to be located directly above the conveying mechanism 14, but may be located from the end of the width of the flow path of the conveying mechanism 14 toward the center of the flow path, as shown by the dashed line in Fig. 4. This reduces the possibility of a shadow 401 being generated on the side of the cargo facing the imaging device. The imaging device 11 is not limited to an optical cutting device, and may be any device, such as a ToF (Time of Flight) camera or a stereo camera, as long as it can acquire at least height information of the subject.

[0019] Next, an example of a height image obtained by the detection device 10 will be described with reference to FIG. A fixed imaging device 11 captures multiple images of cargo moving on the conveying mechanism 14 over a short period of time, and a height image is generated from the resulting profiles. The height image is a two-dimensional array obtained by arranging the multiple profiles in chronological order. For example, Figure 5 shows a height image of the cargo 154 and 155 shown in Figures 3 and 4 captured as subjects. The conveying surface of the conveying mechanism 14 is displayed in black, and pixels corresponding to points higher than the conveying surface are displayed in white. When the imaging device 11 is an optical cutting device, a characteristic of the image obtained from the optical cutting device is that height information cannot be obtained in areas such as the shadow 401 in Figure 4, where the light from the optical cutting device cannot reach due to the shadow of the cargo. This can result in a missing region 501 consisting of pixels with no height. The missing region 501 depends on the position and shape of the subject and is difficult to predict in advance. The detection device 10 according to this embodiment can appropriately detect overlapping or multiple-conveyance of items, even when such an unpredictable missing region 501 exists.

[0020] Next, the components of the detection device 10 according to the first embodiment will be described with reference to the block diagram of FIG. The detection device 10 according to the first embodiment includes an acquisition unit 101, a determination unit 102, and an overlay detection unit 103.

[0021] The acquisition unit 101 receives a plurality of profiles relating to height information of a plurality of points on the subject from the image capture device 11, and obtains a height image relating to the subject by arranging the plurality of profiles in chronological order. The determination unit 102 receives the height information from the acquisition unit 101, and determines whether or not there is one or more steps in the subject at a height equal to or greater than a predetermined height from a reference based on the height information, and generates a determination result. The overlap detection unit 103 receives the step determination result from the determination unit 102, and if the number of steps is equal to or greater than a threshold, it detects that the subject is in a state where multiple cargoes are overlapping. That is, in the first embodiment, it detects that a double feed has occurred.

[0022] Next, the multifeed detection process of the detection device 10 according to the first embodiment will be described with reference to the flowchart of FIG. In step S701, the acquisition unit 101 acquires height information and generates a height image from the height information in multiple time series. In step S702, the determining unit 102 determines whether there is a step in the cargo.

[0023] In step S703, the determination unit 102 determines whether the number of steps is equal to or greater than a threshold value. If the number of steps is equal to or greater than the threshold value, the process proceeds to step S704, and if the number of steps is less than the threshold value, the process proceeds to step S705. In step S704, the overlap detection unit 103 determines that there is a multiple feed because there are multiple steps equal to or greater than the threshold value. In step S705, the overlap detection unit 103 determines that there is no overlap feeding.

[0024] Next, the step determination process in step S702 will be described in detail with reference to the flowchart in FIG. In step S801, the determining unit 102 selects one pixel pair corresponding to a predetermined pixel position in the height image. In step S802, the determination unit 102 determines whether the two pixel positions included in the pixel pair are at a height above a predetermined height from the reference, in this case, whether they are located above the conveying surface. Specifically, if the determination unit 102 determines, based on the height information of the two images and the height information of the conveying surface, that the heights of the two pixel positions are higher than the height of the conveying surface, the process proceeds to step S803, and if the height of at least one of the two pixel positions is the same as the conveying surface, the process proceeds to step S805.

[0025] In step S803, the determination unit 102 determines whether the difference in elevation between the two pixel positions is equal to or greater than a threshold. Specifically, the determination unit 102 calculates the difference in height between the two pixel positions as the difference in elevation. If the difference in elevation is equal to or greater than the threshold, the process proceeds to step S804, and if the difference in elevation is less than the threshold, the process proceeds to step S805. In step S804, the determining unit 102 determines that there is a step because the positions of the two pixels are higher than the conveyance surface and the difference in height is equal to or greater than the threshold.

[0026] In step S805, the determining unit 102 determines that there is no step in the positional relationship between the two pixels. In step S806, the determination unit 102 determines whether the determination processes from step S802 to step S805 have been performed for all pixels. If the determination processes have been performed for all pixels, the step determination process by the determination unit 102 ends and the process proceeds to step S703. If the determination processes have not been performed for all pixels, the process returns to step S801 and the same processes are repeated for unprocessed pixel pairs.

[0027] According to the first embodiment described above, a step is determined by comparing the height difference between predetermined pixel pairs based on the height image of the subject. If there are a number of steps equal to or greater than a threshold, it is determined that multiple items are stacked. If cargo is being transported, it is determined that the cargo is in a double-feed state. This improves the separation performance of items regardless of the size, shape, or orientation of the cargo.

[0028] (Second embodiment) In the first embodiment described above, there are cases where a multifeed cannot be correctly detected. For example, if noise is superimposed on the profile obtained from the imaging device 11, the height difference between the two pixels of a pixel pair may show a large value in an area other than the step due to the influence of the noise, or the height difference at the step may fall below the threshold. Also, if the area on the surface of the cargo where the two pixels correspond is small, small irregularities on the surface of the cargo may be erroneously detected as a step. Furthermore, if the pixels used to detect the step are pixels in a defective area, the step cannot be detected. Therefore, in the second embodiment, by making a determination based on two surfaces in the height image, it is possible to reduce the influence of noise and missing areas and improve the accuracy of determining whether cargo is being double-fed.

[0029] A detection device 10 according to a second embodiment will be described with reference to the block diagram of FIG. The detection device 10 according to the second embodiment includes an acquisition unit 101, a determination unit 102, an overlap detection unit 103, and a surface detection unit 901.

[0030] The plane detection unit 901 receives height information from the acquisition unit 101 and detects whether or not the first and second areas set on the subject form a plane based on the height information. That is, it detects whether or not a first plane is formed in the first area and whether or not a second plane is formed in the second area. When the surface detection unit 901 detects that a first surface is formed in the first area and a second surface is formed in the second area, and when the difference in height between the first surface and the second surface is greater than a threshold value, the determination unit 102 determines that a step exists.

[0031] Next, the step determination process of the detection device 10 according to the second embodiment will be described with reference to the flowchart of Fig. 10. Note that the processes in steps S804 and S805 are the same as those in the first embodiment. In step S1001, the surface detection unit 901 sets a plurality of determination regions for the height image. The method for setting the determination regions will be described later with reference to FIG.

[0032] In step S1002, the surface detection unit 901 selects one determination area. In step S1003, the surface detection unit 901 determines whether the first area included in the determination area is a plane. If the first area is a plane, that is, if a first surface is detected, the process proceeds to step S1004. If the first area is not a plane, the process proceeds to step S805, where the determination unit 102 determines that there is no step.

[0033] In step S1004, the surface detection unit 901 determines whether the second area included in the determination area is a plane. If the second area is a plane, that is, if a second surface is detected, the process proceeds to step S1005. If the second area is not a plane, the process proceeds to step S805, where the determination unit 102 determines that there is no step. In step S1005, the determination unit 102 determines whether the two surfaces are at a height above a predetermined level from the reference, in this case, whether they are higher than the conveying surface. That is, if the height information indicates that the first surface formed in the first area and the second surface formed in the second area are higher than the conveying surface, the process proceeds to step S1006. If at least one of the first surface and the second surface is at the same height as the conveying surface, the process proceeds to step S805, where the determination unit 102 determines that there is no step.

[0034] In step S1006, the determination unit 102 determines whether the difference in elevation between the two surfaces is equal to or greater than a threshold. That is, if the difference in elevation between the first surface and the second surface is equal to or greater than the threshold, the process proceeds to step S804, where the determination unit 102 determines that there is a step. If the difference in elevation between the first surface and the second surface is less than the threshold, the process proceeds to step S805, where the determination unit 102 determines that there is no step. In step S1007, the determination unit 102 determines whether or not processing has been completed for all determination regions. If processing has been completed for all determination regions, the process proceeds to step S703. If an unprocessed determination region exists, the process returns to step S1002 and the same processing is repeated.

[0035] Next, a method for setting a determination region for a height image will be described with reference to FIGS. 11A, 11B, and 11C. FIG. 11A shows an example in which a plurality of determination regions 1101 are arranged vertically on a height image 1100 as shown in FIG. 5. The determination region 1101 is divided into two regions, left and right, consisting of a first region 1102 and a second region 1103. The surface detection unit 901 determines whether the surface of the subject is a surface in each of the first region 1102 and the second region 1103. The determination regions 1101 are assumed to be arranged without gaps across the entire height image 1100, but may be arranged at predetermined intervals as long as the intervals are sufficiently small compared to the size of the determination regions 1101. Conversely, the determination regions 1101 may be arranged so that some of them overlap.

[0036] FIG. 11B shows an example in which a plurality of determination regions 1101 are arranged horizontally on a height image 1100, with a first region 1102 and a second region 1103 being divided into two regions, one above the other. FIG. 11C shows an example in which a plurality of determination regions 1101 are arranged in a diagonal direction that rises to the right on a height image 1100, and a first region 1102 and a second region 1103 are arranged in a diagonal direction that rises to the right. FIG. 11D shows an example in which a plurality of determination regions 1101 are arranged diagonally downward to the right on a height image 1100, and a first region 1102 and a second region 1103 are arranged diagonally downward to the right.

[0037] 11A to 11C, it is sufficient to use at least one of the arrangement patterns of FIGS. 11A to 11C in one measurement. That is, the determination unit 102 may determine a step using any one of the arrangement patterns, or a combination of multiple arrangement patterns. By determining a step using a combination of multiple arrangement patterns, the step is determined from various aspects, allowing for more accurate detection of the step. 11A to 11C show examples in which multiple determination regions 1101 are set, but a single determination region 1101 may also be set. The shape of the determination region 1101 is not limited to a rectangle; it may be any shape, such as a circle or a polygon, or a combination of different shapes. The size of the determination region 1101 is also arbitrary, and determination regions 1101 of different sizes may be combined. For example, in the conveying system of FIG. 1, the size of the determination region 1101 may be set based on the amount of calculation that can be calculated within the allowable processing time from when a step is detected and determined to be a double feed until the removal mechanism 13 switches the conveying direction of the double-fed cargo. Specifically, by increasing the size of the determination region 1101 and arranging them at a predetermined interval, the number of determination regions to be processed is reduced, thereby shortening the processing time required for step detection. However, care must be taken with this method because if the number of regions is reduced too much, for example, when a small step passes through the center of the minute region 1102, it may be overlooked and determined as not having a step in S1003 or S1004.

[0038] Next, a specific example of the face detection process by the face detection unit 901 in step S1003 will be described with reference to FIG. Fig. 12 is a diagram showing cargo 1250 overlapping cargo 1200 as viewed from the side of the cargo. The determination area 1101 is set to the area viewed from above the cargo as in Fig. 11, but the example in Fig. 12 assumes that one determination area is viewed from the side of the cargo.

[0039] As a specific method for detecting the surface, first, N1 (N1 is an integer equal to or greater than 2) different pixel positions 1201 (x 1i ,y 1i )(i=0,..., N1-1) 1i The average value μ 1Z and standard deviation σ 1Z and are calculated as shown in equation (1).

[0040]

number

[0041] Standard deviation σ 1Z is a given threshold T P , it is determined that the first area 1102 forms a first surface, since there is no variation in the height of the cargo contained in the first area 1102. Similarly, N2 (N2 is an integer equal to or greater than 2) different pixel positions 1202 (x 2i ,y 2i )(i=0,..., N2-1) 2i The average value μ 2Z and standard deviation σ 2Z Calculate the standard deviation σ 2Z is a given threshold T P If the height difference is smaller than 1102, it indicates that there is no variation in the height of the cargo contained in the second region 1103, and therefore it can be determined that the second region 1103 forms a second surface. In this way, even if there are defective pixels 1203 whose height could not be obtained in the profile, or even if noise is superimposed on the height value, the influence of the defective pixels and noise can be reduced by averaging the heights in the first region 1102 and the second region 1103.

[0042] Next, a specific example of the processing performed by the determination unit 102 in steps S1005 and S1006 will be described. In step S1005, the determination unit 102 determines the average value μ of the height of the first surface. 1Z and the average value of the height of the second surface μ 2Z is higher than the conveying surface. Specifically, if the height of the conveying surface is set to "0", then μ 1Z >0 and μ 2Z It is enough to determine whether μ > 0 is satisfied. 1Z >0 and μ 2Z > 0, the determination unit 102 determines that the first and second surfaces are higher than the conveying surface. In case noise is superimposed on the height measurement result, a threshold value that is a positive real number specified by the user may be used instead of 0. The height difference |μ between the first and second surfaces in step S1006 2z -μ 1Z Regarding the determination of |, the equation (2) is satisfied, that is, the height difference |μ 2z -μ1Z If | is greater than the threshold, it is determined that there is a step. The right-hand side of equation (2) is the threshold, and for example, parameters α and β, which are real numbers specified in advance by the user, are used. α is a coefficient that increases the threshold value for surfaces with a large standard deviation, i.e., surfaces with large unevenness, and β is the height value that serves as the base for the threshold.

[0043]

number

[0044] Note that by increasing the area of ​​the rectangle in the first region 1102 or the second region 1103, it is possible to prevent minute changes in unevenness from being mistakenly determined as a step. Also, by making the area of ​​the rectangle in the first region 1102 or the second region 1103 larger than the maximum width of the shadow calculated from the expected height of the cargo and the installation angle of the imaging device 11, and ignoring the defective pixel 1203 that is in the shadow and from which height information cannot be obtained when calculating the average and standard deviation of the cargo height, it is possible to determine a step even if a shadow is present.

[0045] According to the second embodiment described above, a determination area is set for the height image, and the first and second surfaces are detected for the first and second areas within the determination area, respectively. If it is determined that the heights of the first and second surfaces are higher than the conveyance surface and that the difference in height between the first and second surfaces is equal to or greater than a threshold, a step is detected. By determining a step using a surface that averages the height within the area in this way, it is possible to determine a step even if the profile contains noise or there are missing areas where the height cannot be obtained. Furthermore, by determining a step using a surface, it is less likely that minute irregularities will be mistakenly determined to be a step. In other words, it is possible to improve the accuracy of determining whether cargo is being transported multiple times while reducing the effects of noise and missing pixels.

[0046] (Third embodiment) In the second embodiment, if the surface of the cargo is inclined, the variance of the height values ​​in the determination area becomes large, and it may not be detected as a surface, and the step may be overlooked. Therefore, in the third embodiment, by detecting bending of the plane, it is possible to determine whether or not a double feed has occurred even on an inclined surface.

[0047] The configuration of the detection device according to the third embodiment is the same as that of the second embodiment, and therefore a detailed description thereof will be omitted.

[0048] The surface determination process of the surface detection unit 901 according to the third embodiment will be described with reference to FIG. The plane detection unit 901 sets a determination area in the height image, as in the second embodiment. The plane detection unit 901 models the surface of the cargo 1200 in the first area 1102 with a plane model z1=a1x+b1y+c1, and calculates a first regression plane 1301. The values ​​of the parameters (a1, b1, c1) of this plane model are determined by the height (x 1i ,y 1i ,z 1i ) (i=0,...,N1-1), it is the value that minimizes the regression error E1 calculated by equation (3).

[0049]

number

[0050] Using the same threshold as in the second embodiment, the regression error E1 is set to the threshold T P If the regression error E1 is smaller than the threshold T P If so, the first region 1102 is determined to not be a surface.

[0051] Similarly, for the second region 1103, the surface of the cargo 1250 in the second region 1103 is modeled using a plane model z2 = a2x + b2y + c2, and a second regression plane 1302 is calculated. The values ​​of the parameters (a2, b2, c2) of this plane model are values ​​that minimize the regression error E2 calculated in the same way as in equation (3). When the regression error E2 is smaller than the threshold T P If the regression error E2 is smaller than the threshold T PIf so, it is determined that the second region 1103 is not a surface. Note that, in calculating the regression plane and the regression error, it is assumed that height information for four or more points is acquired in each of the first region 1102 and the second region 1103.

[0052] Here, |μ in the second embodiment 2z -μ 1z The amount equivalent to the height difference, such as |, is calculated from the height difference from the center of gravity of the height information of one area to the height of the regression plane of the other area. Specifically, in the example of Figure 13, the height difference h from the regression plane of the second plane model to the center of gravity of the first plane is 12 and the height difference h from the regression plane of the first plane model to the center of gravity of the second plane. 21 The difference in height h is calculated. 12 is the center of gravity (x - 1,y - 1,z - 1) and the parameters (a2, b2, c2) of the plane model calculated in the second region 1103. - " indicates the value of the center of gravity of the coordinates from which the height information in the first region is obtained. Here, (x - 1,y - 1,z - The center of gravity of each of the above items (1) can be calculated from equation (4), and the height difference h 12 can be expressed by equation (5).

[0053]

number

[0054] Similarly, the height difference h 21 is the center of gravity (x - 2,y - 2,z - 2) and the parameters (a1, b1, c1) of the plane model calculated in the first region 1102, it is defined by equation (6).

[0055]

number

[0056] In the determination of step S1005 described above, whether the first and second surfaces are higher than the conveying surface or not, the determination unit 102 uses the center of gravity z - 1 and z - 2 is higher than the conveying surface, that is, z - 1>0 and z - It is sufficient to determine whether 2>0. Height difference h 12 ,h 21 The size of |h 12 | and |h 21 | is used to determine the magnitude of the height difference of the cargo according to the second embodiment when both the first plane and the second plane are parallel to the conveying plane. 2z -μ 1z Equals |. Height difference h 12 and h 21 If the signs are different, the height difference |h 12 | and |h 21 The harmonic mean H of | 12 Specifically, the harmonic mean H 12 is calculated using equation (7).

[0057]

number

[0058] The determination unit 102 determines the harmonic mean H 12 "H 12 >α(E1+E2)+β" is satisfied, it is determined that there is a step in the determination region. The parameters α and β are the same as those in the second embodiment. In addition, without using the harmonic mean, the height difference |h 12 | and |h 21 For example, the smaller of the height difference |h 12 | is better than |h 21 If it is smaller than |, then "h 12>α(E1+E2)+β" is satisfied, and if the condition is satisfied, it may be determined that there is a step in the determination region. This reduces the calculation cost.

[0059] Next, the height difference h 12 and h 21 The case where the signs are different will be described with reference to FIG. Height difference h 12 ,h 21 The sign of the cargo surface shape is given by the height difference h 12 ,h 21 can also be expressed as a vector. 12 ,h 21 If the sign of is negative, it is a downward vector, and the height difference h 12 ,h 21 If the sign of is positive, it can be said to be an upward vector. If both are positive numbers, or if both are negative numbers, the first area 1102 and the second area 1103 are likely not steps but bent or have uneven surfaces. 12 and h 21 If the signs are the same, it is determined that the determination area is not a step. Height difference h 12 The coordinates of the center of gravity of the first region (x - 1,y - 1) and the first vector going from a point on the second regression plane 1302 to the first regression plane 1301 in the shortest distance, and the height difference h 21 The plane coordinates of the center of gravity of the second regression plane corresponding to (x - 2,y - 2) If the second vectors pointing from a point on the first regression plane corresponding to the first regression plane to the second regression plane in the shortest distance are in the same direction, it is considered that the first regression plane and the second regression plane intersect. Therefore, the determining unit 102 determines that there is no step. In the example of Figure 14, since the first vector and the second vector are pointing downward and in the same direction, it is determined that there is no step, such as that the cargo related to the determination area is bent. On the other hand, as described above, when the first vector and the second vector are oriented in different directions, the determining unit 102 can determine that there is no step.

[0060] In the above example, the first regression plane of the first region and the second regression plane of the second region are calculated independently using different parameters, but the parameters representing the inclination of the surface may be common. For example, if we set a1 = a2 and b1 = b2, and use equation (8) to find the error E 12 It is sufficient to calculate the solution to the minimization problem of

[0061]

number

[0062] Whether the first and second regions are planar or not is determined by, for example, the determination unit 102 determining the regression error E 12 is the threshold T P If it is smaller than , it is determined to be a plane, and the regression error E 12 is the threshold T P If it is equal to or greater than this, it is determined that the surface is not flat. 12 When using the formula (2), the overlap detection unit 103 determines whether the step is large or small by using the formula (2). 12 +β may be used to determine the step.

[0063] Furthermore, the overlap detection unit 103 may determine whether or not there is a multifeed by taking into consideration the arrangement of the judgment areas, in addition to whether or not the number of judgment areas determined to have a step is equal to or greater than a threshold. For example, in Fig. 11A, if it is determined that there is a step in the uppermost judgment area 1101, and the adjacent judgment area 1101 below is determined to have no step, it may determine that there is no multifeed even if the step determination result for the uppermost judgment area is "step present." In this way, by narrowing the individual judgment areas and taking into consideration the relationship between adjacent judgment areas, it is possible to reduce the impact of cargo misalignment on the multifeed determination result.

[0064] According to the third embodiment described above, the first regression plane of the first region and the second regression plane of the second region are calculated in the determination region, and the height difference in each region is calculated. Taking into account the sign of the height difference, if the height difference vector in the first region and the height difference vector in the second region are in the same direction, it is determined that there is no step. This allows for detection of the surface and step even when the surface of the cargo is inclined, thereby enabling even more accurate detection of double feeds.

[0065] Although the height image has been described using an image that captures the entire cargo packaging, in each of the first to third embodiments, the height image may be a partial image of the cargo. An example of using a height image as a partial image will be described with reference to FIG. FIG. 15 shows the sampling timing of the profile along the time series from T2 to T6 for the height image shown in FIG.

[0066] For example, consider a case where a long piece of cargo is being transported. In order for the imaging device 11 to capture an image that captures the entire long piece of cargo, it must wait until the rear end of the cargo passes through the imaging area of ​​the imaging device. Furthermore, the number of pixels in the image of a long piece of cargo increases, which increases the calculation time required to determine whether the cargo is a double feed, i.e., the latency until the exclusion mechanism is activated. Therefore, if the height image can be divided into strips along the direction of travel of the cargo, as shown in FIG. 15, the latency from when the detection device 10 completes processing until the exclusion mechanism 13 is activated if the cargo is a double feed can be reduced, resulting in a shorter transport path length (conveyor length).

[0067] Here, the acquisition unit 101 divides the height image along the direction of travel of the cargo and acquires partial images. That is, rather than acquiring the entire range from time T2 to T6 in FIG. 15 and then performing processing by the determination unit 102 and the overlap detection unit 103, partial image 1501 is generated when a profile from time T2 to T4 is acquired, and processing by the determination unit 102 and the overlap detection unit 103 begins. While processing partial image 1501, profiles from time T4 to T5 are acquired in parallel to generate partial image 1502 from time T3 to T5. The determination unit 102 and the overlap detection unit 103 then process partial image 1502. By processing profiles from a predetermined range in this manner, it is possible to initiate double feed detection processing each time a portion of a long cargo passes through the detection device, even if the cargo is moving forward. Furthermore, if the start of double feed detection processing can be accelerated, the time from when the trailing end of the cargo passes until the removal mechanism 13 is activated can be shortened.

[0068] In the above-described embodiment, the detection device 10 is intended to be used to determine whether cargo is being sent multiple times in a conveying system, but it is not limited to this and can also be used in applications such as determining the overlapping state of items so that items can be picked up one by one during an item picking process.

[0069] Next, an example of the hardware configuration of the detection device 10 according to each of the above-described embodiments is shown in the block diagram of FIG. The detection device 10 includes a CPU (Central Processing Unit) 1601, a RAM (Random Access Memory) 1602, a ROM (Read Only Memory) 1603, storage 1604, a display device 1605, an input device 1606, and a communication device 1607, each of which is connected by a bus.

[0070] The CPU 1601 is a processor that executes arithmetic processing, control processing, etc. according to a program. The CPU 1601 uses a predetermined area of ​​the RAM 1602 as a working area and executes the processing of each part of the detection device 10 described above in cooperation with programs stored in the ROM 1603 and the storage 1604.

[0071] The RAM 1602 is a memory such as an SDRAM (Synchronous Dynamic Random Access Memory), and functions as a work area for the CPU 1601. The ROM 1603 is a memory that stores programs and various types of information in a non-rewritable manner.

[0072] The storage 1604 is a device that writes and reads data to a magnetic recording medium such as a hard disk drive (HDD), a semiconductor storage medium such as a flash memory, a magnetically recordable storage medium such as a HDD, an optically recordable storage medium, etc. The storage 1604 writes and reads data to and from the storage medium under the control of the CPU 1601.

[0073] The display device 1605 is a display device such as an LCD (Liquid Crystal Display), etc. The display device 1605 displays various information based on a display signal from the CPU 1601.

[0074] The input device 1606 is an input device such as a mouse, a keyboard, etc. The input device 1606 receives information input by a user as an instruction signal, and outputs the instruction signal to the CPU 1601.

[0075] The communication device 1607 communicates with external devices via a network under the control of the CPU 1601 .

[0076] The instructions shown in the processing procedures described in the above-described embodiments can be executed based on a software program. A general-purpose computer system can store this program in advance and, by loading this program, achieve effects similar to those achieved by the control operation of the detection device described above. The instructions described in the above-described embodiments can be recorded as a computer-executable program on a magnetic disk (such as a flexible disk or hard disk), an optical disk (such as a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, or Blu-ray Disc), a semiconductor memory, or a similar recording medium. The recording medium may take any storage format as long as it is readable by a computer or embedded system. A computer can load the program from the recording medium and execute the instructions described in the program on a CPU based on the program, thereby achieving operations similar to those of the detection device described in the above-described embodiments. Of course, the computer may acquire or load the program via a network. In addition, an OS (operating system), database management software, network middleware, etc. running on a computer may execute some of the processes required to realize this embodiment based on instructions from a program installed on the computer or embedded system from a recording medium. Furthermore, the recording medium in this embodiment is not limited to a medium independent of a computer or an embedded system, but also includes a recording medium that stores or temporarily stores a program downloaded via a LAN, the Internet, or the like. Furthermore, the number of recording media is not limited to one, and cases where the processing in this embodiment is executed from multiple media are also included in the recording media in this embodiment, and the media may have any configuration.

[0077] The computer or embedded system in this embodiment is for executing each process in this embodiment based on a program stored on a recording medium, and may be configured as either a device consisting of a single device such as a personal computer or a microcomputer, or a system in which multiple devices are connected to a network. Furthermore, the computer in this embodiment is not limited to a personal computer, but also includes a processing unit, a microcomputer, etc. included in a detection device, and is a general term for equipment or devices that can realize the functions in this embodiment by a program.

[0078] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0079] 10···Detection device, 11···Imaging device, 12···Item evaluation device, 13···Rejection mechanism, 14···Transport mechanism, 101···Acquisition unit, 102···Evaluation unit, 103···Overlap detection unit, 151 to 155, 1200, 1250···Cargo, 301···Irradiation unit, 302···Light receiving unit, 401···Shadow, 501···Defective area, 901···Surface detection unit, 1100···Height image, 1101···Evaluation area , 1102···First region, 1103···Second region, 1201, 1202···Pixel position, 1203···Defective pixel, 1301···First regression plane, 1302···Second regression plane, 1501, 1502···Partial image, 1601···CPU, 1602···RAM, 1603···ROM, 1604···Storage, 1605···Display device, 1606···Input device, 1607···Communication device.

Claims

1. an acquisition unit that acquires height information of a plurality of points on the subject; a determination unit that determines whether or not one or more steps exist in the subject at a height equal to or greater than a predetermined height from a reference based on the height information; a detection unit that detects that the subject is in a state where a plurality of articles are overlapped when the number of the steps is equal to or greater than a first threshold value; Equipped with a plane detection unit that detects whether the first area and the second area set on the subject form a plane based on the height information; The determination unit determines that the step exists when the surface detection unit detects that a first plane is formed in the first region and a second plane is formed in the second region, and when the difference in elevation between the first plane and the second plane is greater than a second threshold value.

2. The detection device according to claim 1 , wherein the determination unit determines that the step does not exist when the height difference between the first plane and the second plane is equal to or less than the second threshold value.

3. A detection device as described in Claim 1, wherein the determination unit determines that there is no step when the surface detection unit detects that at least one of the first region and the second region does not form the plane.

4. the plane detection unit calculates a first regression plane obtained by modeling the first plane with a first plane model and a second regression plane obtained by modeling the second plane with a second plane model different from the first plane model; 2. The detection device according to claim 1, wherein the determination unit determines that the step does not exist when, when the second regression plane is expanded into the first region, a first vector directing from a point on the second regression plane in the first region to the first regression plane by the shortest distance and, when the first regression plane is expanded into the second region, a second vector directing from a point on the first regression plane in the second region to the second regression plane by the shortest distance have the same direction.

5. The detection device according to claim 4 , wherein the determination unit determines that the step exists when the first vector and the second vector are oriented in different directions.

6. the plane detection unit calculates a first regression plane obtained by modeling the first plane with a first plane model and a second regression plane obtained by modeling the second plane with a second plane model different from the first plane model; 3. The detection device according to claim 1, wherein the determination unit determines that the step exists when a value obtained by subtracting a value obtained by multiplying a sum of a regression error of the first regression plane and a regression error of the second regression plane by a first value from a difference in elevation between the first regression plane and the second regression plane is greater than the second threshold value.

7. the first region and the second region are rectangular; The determination unit a first determination result in which the first region and the second region are aligned in a vertical direction and it is determined whether the plane exists in each region; a second determination result in which the first region and the second region are aligned in the horizontal direction and it is determined whether the plane exists in each region; 7. The detection device according to claim 1, wherein the first region and the second region are arranged diagonally side by side, and the presence or absence of the step is determined using at least one of a third determination result that determines whether the plane exists in each of the first region and the second region.

8. 8. The detection device according to claim 1, wherein the detection unit detects that the subject is in a state where multiple items are overlapping each other when the number of steps is equal to or greater than the first threshold and multiple steps are spatially adjacent to each other.

9. 9. The detection device according to claim 1, wherein the detection unit detects that the subject is a single item if the number of steps is less than the first threshold, or if the number of steps is equal to or greater than the first threshold and the steps are not spatially adjacent.

10. An acquisition unit that acquires height information of a plurality of points on a subject; a determination unit that determines whether or not one or more steps exist in the subject at a height equal to or greater than a predetermined height from a reference based on the height information; a detection unit that detects that the subject is in a state where a plurality of articles are overlapped when the number of the steps is equal to or greater than a first threshold value; Equipped with a plane detection unit that detects whether the first area and the second area set on the subject form a plane based on the height information; The determination unit determines that the step does not exist when the surface detection unit detects that at least one of the first region and the second region does not form the plane.

11. A conveyance system comprising the detection device according to any one of claims 1 to 10, a conveyance mechanism for conveying an article, and a removal mechanism, The exclusion mechanism includes: A conveying system in which, when a detection unit of the detection device detects that a single item is being conveyed, the single item is advanced downstream of the conveying mechanism, and, when the detection unit detects that a multiple item is being conveyed in an overlapping state, the multiple items being conveyed in a direction different from the downstream side of the conveying mechanism.

12. Obtaining height information of a plurality of points on the subject; determining whether or not one or more steps exist in the subject at a height equal to or greater than a predetermined height from a reference based on the height information; If the number of steps is equal to or greater than a first threshold, the object is detected as being in a state where a plurality of articles are overlapped. Equipped with The method further includes detecting whether or not a first area and a second area set on the subject form a plane based on the height information; The detection method includes determining that a step exists when it is detected that a first plane is formed in the first region and a second plane is formed in the second region, and when the difference in height between the first plane and the second plane is greater than a second threshold value.

13. The computer acquisition means for acquiring height information of a plurality of points on the subject; a determining means for determining whether or not one or more steps exist in the subject at a height equal to or greater than a predetermined height from a reference based on the height information; When the number of steps is equal to or greater than a first threshold, the object is made to function as a detection unit that detects that a plurality of articles are overlapped; further functioning as a plane detection unit that detects whether or not the first and second areas set on the subject form planes based on the height information; The determination means determines that a step exists when the surface detection means detects that a first plane is formed in the first region and a second plane is formed in the second region, and when the difference in height between the first plane and the second plane is greater than a second threshold value.

Images