Inspection system

Abstract

This inspection system radiates, towards a target region through which an article conveyed in a conveyance direction along a conveyance path passes, slit light from both sides intersecting the conveyance path, captures an image of the target region by a camera, and obtains shape information for the article, wherein the camera is installed at a position that is above the target region offset in the conveyance direction and above the target region offset in an intersection direction in which the slit light is radiated, and the camera obtains a composite image by using a corrected image in which the incline state of the camera is compensated and inclination of a line extending in the intersection direction is cancelled out.

Description

Inspection System

[0001] The present invention relates to an inspection system for acquiring shape information of an object conveyed along a conveyance path.

[0002] A distribution system for acquiring shape information of an article is known in the art, which irradiates an article conveyed in a predetermined direction with slit light beams from diagonally above on both sides to form a single slit light beam on the article, and captures an image of the slit light beam with a camera to acquire the shape information of the article (Patent Document 1). In particular, the method shown in Figures 8(a) and 8(b) of Patent Document 1 combines two light sources and two cameras.

[0003] A known food placement status management device determines the placement status of food by using a camera to capture images of food being transported in a specified direction from above, and then analyzing the captured images using machine learning to determine the placement status of the food (Patent Document 2).

[0004] The method of Patent Document 1 makes it possible to measure the state of an object even if there is an object above the object that obstructs observation, but requires a complex method for capturing images of the object with two cameras and synthesizing the shape information.The method of Patent Document 2 assumes that there is no object above the object that obstructs observation.

[0005] JP 2007-532929 A JP 2021-029677 A

[0006] In view of the above problems, the present invention aims to provide an inspection system that can easily measure the overall condition of an item even when there is an object above the item that obstructs observation.

[0007] In one form of the present invention, an inspection system irradiates a target area through which an item conveyed in a conveying direction along a conveying path with slit light from both sides intersecting the conveying path, captures an image of the target area with a camera, and obtains shape information of the item, wherein the camera is installed above the target area, offset from the conveying direction and offset from the intersecting direction in which the slit light is irradiated, and obtains a composite image using a corrected image that compensates for the tilted state of the camera and offsets the tilt of the line extending in the intersecting direction.

[0008] In the above-described inspection system, even if the camera must be positioned at a complex tilt to deal with the case where there is an object above the article that obstructs observation, a composite image is obtained using a corrected image that compensates for the tilt of the camera and offsets the tilt of the lines extending in the intersecting direction, making the process of generating the composite image easier.

[0009] According to another aspect of the invention, the camera is tilted about its optical axis, which simplifies the acquisition of the corrected image.

[0010] According to another aspect of the present invention, the camera is installed tilted about its optical axis so that the intersecting direction within the camera's screen coincides with the reference axis of the camera's screen, with the horizontal and vertical axes as references. In other words, an object extending in the horizontal intersecting direction on the transport path extends along the reference axis corresponding to the vertical and horizontal axes within the camera's screen, making it easy to obtain height information.

[0011] According to another aspect of the present invention, a control device is provided that performs image processing for rotation correction on the image captured by the camera, which eliminates the need to install the camera tilted around the optical axis.

[0012] According to another aspect of the present invention, a first slit light source device includes a first light source installed near a conveying path and emitting a first slit light along the conveying path, and a first mirror installed near a target area and reflecting the first slit light, a second light source installed near the conveying path opposite the first light source across the conveying path and emitting a second slit light along the conveying path, and a second mirror installed near the target area opposite the first mirror across the conveying path and reflecting the second slit light, and a composite slit light combining the first slit light and the second slit light is irradiated onto the target area. In this case, even if there is an object above the object that obstructs light irradiation to the object, the slit light can be irradiated toward the target area, i.e., the object, from both sides of the conveying path.

[0013] According to another aspect of the present invention, a control device is provided that generates, as a composite image, an image including height information of an article from a plurality of frame images obtained by photographing the article along the conveyance path, and the control device can generate a depth-added scanning line image having a brightness distribution that reflects the height information, for example, as a brightness coefficient.

[0014] According to another aspect of the invention, the camera captures color images.

[0015] According to another aspect of the invention, the article is a food product covered by a transparent cover member having an opaque area thereon.

[0016] Fig. 1A is a plan view of the inspection system, and Fig. 1B is a side cross-sectional view of the inspection system. Fig. 2A is a front view of the inspection system, and Fig. 2B is a front view from the cross section taken along the arrow AA in Fig. 1A. Fig. 3 is a conceptual diagram explaining the movement of the combined slit light on the transport path. Fig. 4A shows an example of a frame image captured by a camera, and Fig. 4B is a conceptual diagram explaining the combination of the frame images. Fig. 5 is a diagram explaining a modified example.

[0017] [Embodiment] An inspection system according to an embodiment will be described below with reference to the drawings.

[0018] An example of the configuration of an inspection system 100 will be described with reference to Figures 1A and 1B. Figure 1A is a plan view of the inspection system 100, and Figure 1B is a side cross-sectional view of the inspection system 100. Figure 2A is a front view of the inspection system 100, and Figure 2B is a front view from the cross section taken along the arrow AA in Figure 1A. However, for convenience of explanation, a camera 30 is included in Figure 2B.

[0019] The inspection system 100 is incorporated into a food handling apparatus 200, which is equipment that allows for the sorting and portioning of specific foods while moving food during, for example, food production or serving. The inspection system 100 optically inspects the condition of items GO, which are food FD, transported by a transport device 10. The food handling apparatus 200 includes the transport device 10 for the items GO, the inspection system 100 for the items GO, etc.

[0020] The transport device 10 includes a transport member 11 such as a conveyor that supports a container 61 containing an item GO and moves it in a transport direction D1, and a drive device (not shown) that operates the transport member 11. The transport member 11 that supports the container 61 forms a transport path TP along which the container 61 is transported.

[0021] The container 61 moving on the conveying member 11 has a tray 61a and a dome 61b. The item GO is placed on the tray 61a. The tray 61a is a circular light-shielding body and may be a two-tiered tray with the item GO placed higher. The dome 61b is fixed to the tray 61a via a rear hinge (not shown) and is openable on the front +Y side. The item GO can be removed by lifting the front portion of the dome 61b. The dome 61b is a transparent cover member that is light-transmitting overall, but the entire circumference or bottom edge of the dome 61b is provided with a light-scattering frosted portion 61d, and a tag 61e made of a light-shielding material is attached to the top. A three-dimensional tag 61f can also be added to the tag 61e. The frosted portion 61d and the tags 61e and 61f form a non-transparent area AZ that obstructs the view of the item GO.

[0022] The transparency of the dome 61b of the container 61 is hindered by the frosted portion 61d and the tag 61e, so that the item GO inside the dome 61b can be observed from the outside only through the intermediate inclined region 61z (the donut-shaped region shown by a number of dot patterns in Figure 1A) excluding the bottom and top of the dome 61b.

[0023] It is assumed that the container 61 does not rotate on the transport member 11. The container 61 or the transport member 11 has a structure (not shown) that prevents the container 61 from rotating when transporting the container 61.

[0024] The inspection system 100 basically uses a light section method, and includes an illumination device 20, a camera 30, a light shielding device 40, and a control device 80.

[0025] The illumination device 20 includes a first slit light source device 21 having a first light source 21a and a first mirror 21b, and a second slit light source device 22 having a second light source 22a and a second mirror 22b. The first light source 21a and the second light source 22a are arranged opposite each other across the transport path TP, and are arranged to avoid the space above the transport member 11 so as not to overlap with the transport path TP or the transport member 11 in a planar view. The first mirror 21b and the second mirror 22b are arranged opposite each other across the transport path TP, and are arranged to avoid the space above the transport member 11 so as not to overlap with the transport path TP or the transport member 11 in a planar view. The illumination device 20 forms a composite slit light CL that is thinly irradiated parallel to the YZ plane in the area sandwiched between the pair of mirrors 21b and 22b.

[0026] The first light source 21a and the second light source 22a emit, for example, red and white slit light beams L1 and L2. The light sources 21a and 22a are, for example, laser light sources, but may be replaced with LEDs. The light sources 21a and 22a may not only be light sources but also include lenses, slits, diaphragms, etc. that adjust the beam cross sections of the slit light beams L1 and L2 emitted from the light sources 21a and 22a. The first mirror 21b and the second mirror 22b are flat reflecting mirrors. The mirrors 21b and 22b are arranged in tilted positions by appropriately rotating them around the Z-axis, which is the vertical axis, and around the X-axis, which is the horizontal axis, based on a state parallel to the XZ plane.

[0027] The first light source 21a emits a first slit beam L1 that, when viewed from the side in the +X direction, spreads at a divergence angle of approximately α0 along a plane extending in a direction tilted counterclockwise at an inclination angle α1 (specifically, for example, 20°) relative to the +Z direction or the vertical XZ plane. The first slit beam L1 from the first light source 21a is reflected by the first mirror 21b. The first slit beam L1 reflected by the first mirror 21b spreads at a divergence angle of approximately α0 about a direction tilted counterclockwise at an inclination angle α2° (specifically, α2 = α1) relative to the horizontal -Y direction or the XY plane in a side view corresponding to FIG. 1B, while passing through the dome 61b and incident on the top surface of the item GO. The first slit beam L1 that is incident on the top surface of the item GO from the upper right spreads along the YZ plane at an angle α3 of 90° relative to the conveying direction D1 in a plan view corresponding to FIG. 1A. Here, the lower limit of the inclination angle α2 at which the first slit light L1 passes through the dome 61b is set so that the first slit light L1 enters the center of the top surface of the item GO without interfering with the tag 61e, and the upper limit of the inclination angle α2 at which the first slit light L1 passes through the dome 61b is set so that the first slit light L1 enters the center of the bottom surface of the item GO without being blocked by the frosted portion 61d.

[0028] The second light source 22a emits a second slit light beam L2 that, when viewed from the side in the +X direction, spreads at a divergence angle of approximately α0 along a plane extending in a direction tilted clockwise at an inclination angle α1 (specifically, for example, 20°) relative to the +Z direction or the vertical XZ plane. The second slit light beam L2 from the second light source 22a is reflected by the second mirror 22b. The second slit light beam L2 reflected by the second mirror 22b spreads at a divergence angle of approximately α0 about a direction tilted clockwise at an inclination angle α2° (specifically, α2 = α1) relative to the horizontal +Y direction or the XY plane in a side view corresponding to FIG. 1B, while passing through the dome 61b and incident on the top surface of the item GO. The second slit light beam L2 incident on the top surface of the item GO from the upper left spreads along the YZ plane at an angle α3 of 90° relative to the conveying direction D1 in a plan view corresponding to FIG. 1A. Here, the upper and lower limits of the inclination angle α2 at which the second slit light beam L2 passes through the dome 61b are the same as those of the first slit light beam L1 formed by the second light source 22a and the second mirror 22b. As a result, the slit light beams L1 and L2 pass through the intermediate inclined region 61z of the dome 61b, and a composite slit light beam CL formed by combining the slit light beams L1 and L2 is projected along the YZ plane within the dome 61b, forming a measurement zone MZ irradiated with the composite slit light beam CL that spreads parallel to the YZ plane. The measurement zone MZ is a target region OZ through which the object GO passes.

[0029] The camera 30 is a video camera equipped with a lens and an image sensor, and is capable of continuously capturing still images at high speed. The camera 30 is disposed diagonally above the transport path TP. The camera 30 is disposed to avoid the space above the transport member 11 so as not to overlap with the transport path TP or the transport member 11 in a plan view.

[0030] The optical axis AX of the camera 30, when viewed from the side (+X direction) corresponding to FIG. 1B, extends in a direction tilted clockwise at an angle β1 of approximately 30 to 60° relative to the upward +Z direction or the vertical XZ plane. When viewed from the planar view (+Z direction) corresponding to FIG. 1A, the optical axis AX of the camera 30 extends in a direction tilted clockwise at an angle β2 of approximately 20 to 30° relative to the rearward conveyance direction D1 or the +X direction. When viewed from the front (+Z direction) corresponding to FIG. 2A, the optical axis AX of the camera 30 extends in a direction tilted downward at an angle β3 of approximately 20 to 30° relative to the rearward conveyance direction D1 or the +X direction. By tilting the camera 30 in this manner, the item GO can be photographed through the intermediate inclined region 61z of the dome 61b, and the item GO can be photographed while avoiding the frosted portion 61d and the tag 61e. As a result, it is possible to capture an image of the object GO illuminated by the synthetic slit light CL in the measurement zone MZ. Note that the angles β1, β2, and β3 are not limited to the above-mentioned numerical ranges, and can be set appropriately depending on the configuration of the conveying device 10 and the container 61, etc.

[0031] From another perspective, the camera 30 is positioned above the center or its periphery of the inspection target area OZ as the base point, offset by an angle β3 with respect to the conveying direction D1 (see Figure 2B), and is also positioned above the cross direction CD in which the slit light L1 and L2 are irradiated by an angle 90°-β1 (see Figure 1B).

[0032] The posture of the camera 30 will be described with reference to FIG. 1B . The camera 30 is positioned tilted with respect to its optical axis AX and rotated about the optical axis AX, with respect to the horizontal and vertical axes (specifically, the Y and Z axes). In normal use, the horizontal reference line SL of the camera 30 is set parallel to the horizontal axis and perpendicular to the vertical axis. In contrast, in this embodiment, the horizontal reference line SL of the camera 30 is rotated by an angle θ with respect to a horizontal axis that is perpendicular to the optical axis AX and passes through the XY plane corresponding to the horizontal plane. As a result, objects on the conveying member 11 that extend perpendicular to the conveying path TP, i.e., those extending in the ±Y directions, are captured in a state where they coincide with the reference axis (parallel to the reference line SL) of the screen of the camera 30. The angle θ can be set to an appropriate value, for example, by placing a flat reference member in the target area OZ and adjusting the screen of the camera 30 so that the reference member extends horizontally within the screen while referring to the screen of the camera 30 that captures the reference member.

[0033] Referring to Figure 3, by moving the container 61 at a constant speed in the transport direction D1 (in this case, the -X direction) along the transport path TP, the composite slit light CL moves at a constant speed in the scanning direction SD, which is the +X direction in a planar view.

[0034] FIG. 4A is a conceptual diagram illustrating the screen 30a of the camera 30. Here, the axis x corresponds to the reference line SL of the camera 30 shown in FIG. 1B. The screen 30a or an image corresponding thereto is a corrected image 33 in which the tilt of the lines extending in the cross direction CD (i.e., lines extending parallel to the Y direction) on the conveying member 11 is offset. The corrected image 33 is equivalent to a state in which the camera 30 is positioned tilted above the center of the conveying member 11, and the lines of the object extending parallel to the Y direction are imaged as extending horizontally. In the corrected image 33, the line drawing portion RS represents an image of the top surface of the object GO illuminated by the synthetic slit light CL, and the position of the axis y reflects the height position of the object GO in the Z-axis direction shown in FIG. 1B, etc.

[0035] Returning to FIG. 1A etc., the control device 80 is a computer and includes a CPU 81, a storage device 82, and an input / output device 83. The CPU 81 of the control device 80 controls the camera 30 to capture images and stores the captured frame images in the storage device 82. Pixels with brightness levels below a predetermined threshold are discarded from each frame image. The CPU 81 of the control device 80 performs image processing on each captured frame image to create a composite image. For each frame image, height information for the surface of the object GO is obtained from the positional deviation δ in the y direction of the line drawing portion RS shown in FIG. 4A. Height information may be calculated based on the vertical deviation δ' from the reference position 11a corresponding to the top surface of the conveying member 11 instead of the positional deviation δ. This height information is used as a brightness coefficient to obtain a depth-added scanning line image by multiplying the brightness coefficient by a scanning line image in which the line drawing portion RS is linearly represented for each frame image. As the object GO is moved in the conveying direction D1, multiple depth-added scanning line images RR acquired from each frame image are vertically shifted in accordance with the conveying speed and frame interval, and then synthesized to obtain a depth-added image 34 (see FIG. 4B). The depth-added image 34 thus obtained, which is a composite image, becomes relatively darker as the surface becomes lower.

[0036] The depth-added scanning line image RR shown in FIG. 4B becomes a color scanning line image when the slit light beams L1 and L2 are white. The color scanning line image may not include height information. In this case, a composite image that resembles a planar color image of the item GO is obtained. The depth-added image 34 of the item GO and the planar color image of the item GO may be displayed side by side on a display that constitutes the input / output device 83 of the control device 80.

[0037] The depth-added scanning line image RR or its color scanning line image, or the depth-added image 34 obtained as described above can be used to check for defects in the item GO. In particular, when the depth-added image 34 is a color image or when it is displayed alongside a planar color image of the item GO, the operator or worker of the inspection system 100 can easily find the defective item GO. Furthermore, by observing the depth-added image 34 or its color image, it is possible to perform an appearance inspection regarding the type and condition of the item. If the item GO is a cake topped with fruit such as strawberries, it is possible to check not only for misalignment or falling of the topping, but also for the type and condition of the topping.

[0038] 4A has a width in the y direction, the control device 80 can determine the peak position or the midpoint in the y direction of the line drawing portion RS as the position in the y direction of the line drawing portion RS. This makes it possible to prevent a decrease in resolution even if the focus of the camera 30 is shifted or the width of the synthetic slit light CL in the X direction increases.

[0039] The control device 80 may synthesize the frame images captured by the camera 30 while shifting them vertically as they are. In this case, the synthesized image is a pseudo-three-dimensional display in which cross-sectional contours are superimposed.

[0040] The inspection system 100 according to the embodiment described above is an inspection system that irradiates a target area OZ through which an item GO conveyed in a conveying direction D1 along a conveying path TP with slit light beams L1 and L2 from both sides intersecting the conveying path TP, captures an image of the target area OZ with a camera 30, and obtains shape information of the item GO. The camera 30 is installed above the target area OZ, offset with respect to the conveying direction D1, and offset with respect to the cross direction CD in which the slit light beams L1 and L2 are irradiated, and obtains a composite image using a corrected image 33 that compensates for the tilted state of the camera 30 and offsets the tilt of the line extending in the cross direction CD.

[0041] In the above-described inspection system 100, even if the camera 30 must be positioned in a complex tilted state to deal with the case where there is an object above the item GO that obstructs observation, a composite image is obtained using a corrected image 33 that compensates for the tilted state of the camera 30 and offsets the tilt of the line extending in the cross direction CD, thereby facilitating the process of generating the composite image.

[0042] In the inspection system 100, the camera 30 is installed tilted about the optical axis with respect to the horizontal and vertical axes so that the cross direction CD extends in a direction that coincides with the axis x (corresponding to the reference line SL of the camera 30) that is the reference axis of the screen 30a of the camera 30 within the screen 30a of the camera 30. In other words, an object extending in the horizontal cross direction CD on the transport path TP extends along the reference axis corresponding to the vertical and horizontal axes within the screen 30a of the camera 30, making it easy to obtain height information and perform image processing.

[0043] The present invention is not limited to the above embodiment and includes the following modifications. For example, the measurement zone MZ or target area OZ is not limited to being perpendicular to the conveying direction D1, and the measurement zone MZ shown in FIG. 1A can be rotated around the Z axis and tilted. In this case, the positions and tilt angles of the first mirror 21b and the second mirror 22b are adjusted to adjust the passing positions and directions of the slit beams L1 and L2.

[0044] In the above description, the camera 30 is rotated by an angle θ around the optical axis AX and placed in an inclined state ( FIG. 1B ). However, the camera 30 may not be rotated, and the reference line SL beside the camera 30 may be parallel to the horizontal axis. In this case, the control device performs image processing for rotation correction on the image acquired corresponding to the screen 30a of the camera 30. In FIG. 5 , the dotted line drawing portion RS0 indicates the original image, and the solid line drawing portion RS indicates the image after rotation correction.

[0045] The illumination device 20 may split a slit light beam from a single light source using a beam splitter and direct the split slit light beams to the measurement zone MZ or the target area OZ using individual mirror systems. In this case, the slit light beam from the light source can be split into three or more beams, not just two.

[0046] The container 61 may rotate on the conveying path. In this case, however, in order to acquire an overall image of the item GO even when the container 61 rotates, additional cameras similar to the camera 30 may be placed symmetrically across the target area OZ. The cameras are placed both upstream and downstream of the target area OZ on the conveying path TP.

[0047] The item GO to be inspected by the inspection system 100 is not limited to cakes with fruit toppings, but may also be cakes without toppings, other desserts, or other confectioneries, and can be applied to various foods other than confectioneries (e.g., sushi) and items other than food.

[0048] This application claims priority based on Japanese Patent Application No. 2023-154139, filed on September 21, 2023, the entire disclosure of which is incorporated herein by reference.

Claims

1. An inspection system which irradiates a target area through which an object conveyed in a conveying direction along a conveying path with a slit of light from both sides intersecting the conveying path, captures an image of the target area with a camera, and obtains shape information of the object, wherein the camera is installed offset above the target area with respect to the conveying direction and offset above the intersecting direction in which the slit of light is irradiated, and obtains a composite image using a corrected image which compensates for the tilted state of the camera and offsets the tilt of a line extending in the intersecting direction.

2. The inspection system according to claim 1, wherein the camera is installed tilted about an optical axis of the camera.

3. The inspection system according to claim 2, wherein the camera is installed tilted about an optical axis based on a horizontal axis and a vertical axis so that the intersecting direction within the screen of the camera extends in a direction that coincides with the reference axis of the screen of the camera.

4. The inspection system according to claim 1, further comprising a control device that performs image processing for rotation correction on the image captured by the camera.

5. The inspection system described in claim 1, comprising: a first slit light source device having a first light source installed in the vicinity of the transport path and emitting a first slit light along the transport path, and a first mirror installed in the vicinity of the target area and reflecting the first slit light; a second slit light source device having a second light source installed in the vicinity of the transport path facing the first light source across the transport path and emitting a second slit light along the transport path, and a second mirror installed in the vicinity of the target area facing the first mirror across the transport path and reflecting the second slit light; and a composite slit light combining the first slit light and the second slit light is irradiated onto the target area.

6. The inspection system according to claim 1, further comprising a control device that generates, as the composite image, an image including height information of the object from a plurality of frame images taken of the object along the transport path.

7. The inspection system of claim 1, wherein the camera captures color images.

8. The inspection system of claim 1, wherein the article is a food product covered by a transparent cover member having a non-transparent area thereon.

Images