Object recognition device

Abstract

To properly calculate the position in which an object is to be arranged from an image of the object.SOLUTION: An object recognition device includes: an illumination apparatus 23 for illuminating an object 29; a camera 22 for taking a first image of the object 29 when the object 29 is illuminated by the illumination apparatus 23 under a first illumination condition and taking a second image of the object 29 when the object 29 is illuminated by the illumination apparatus 23 under a second illumination condition different from the first illumination condition; and a position calculation unit for calculating the position in which the object 29 is to be arranged by processing the first image and the second image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The technology of the present disclosure relates to an object recognition device and an object processing device. [Background technology]

[0002] An automatic recyclable waste sorting device that separates recyclable waste, such as glass bottles and PET bottles, by material is known. The automatic recyclable waste sorting device includes an object recognition device that determines the material and position of recyclable waste based on captured images of the recyclable waste, and a robot that moves recyclable waste of a predetermined material to a predetermined position. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2008-514915 [Patent Document 2] Japanese Patent Publication No. 59-161776 Summary of the Invention [Problem to be solved by the invention]

[0004] When recyclable waste is made of a light-transmitting material, the light that passes through the recyclable waste may reflect the background behind the recyclable waste in the image. When the recyclable waste is glossy, the light that is specularly reflected from the recyclable waste may cause blown-out highlights in the image. When such distracting images are reflected in the image of the recyclable waste, the object recognition device may not be able to properly extract the image of the recyclable waste from the image, and may not properly calculate the position of the recyclable waste. When the position of the recyclable waste cannot be properly calculated, the automatic recyclable waste sorting device may not properly sort the recyclable waste.

[0005] The disclosed technology has been made in consideration of the above points, and aims to provide an object recognition device and an object processing device that appropriately calculate the position of an object from an image in which the object appears. [Means for solving the problem]

[0006] An object recognition device according to one aspect of the present disclosure includes: thing A lighting device for illuminating the body; capturing a first image of the object when the object is illuminated by the lighting device under a first lighting condition, and capturing a second image of the object when the object is illuminated by the lighting device under a second lighting condition different from the first lighting condition; an imaging unit that captures an image of the The position where the object is located is calculated by performing image processing on the first image and the second image. Control device and 、 Equipped with The object is transported such that a first position where the object is placed at a first timing when the first image is captured is different from a second position where the object is placed at a second timing when the second image is captured. . [Effects of the Invention]

[0007] The disclosed object recognition device and object processing device can appropriately calculate the position of an object from an image in which the object appears. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a perspective view showing an automatic recyclable waste sorting apparatus provided with an object processing apparatus according to an embodiment. [Figure 2] FIG. 2 is a cross-sectional view showing the optical unit. [Figure 3] FIG. 3 is a block diagram showing the control device. [Figure 4] FIG. 4 is a flowchart showing the operation of the control device to control the robot unit and the optical unit. [Figure 5] FIG. 5 is a diagram showing an image of a high light intensity portion. [Figure 6] FIG. 6 is a diagram showing an image of a low-light portion. DETAILED DESCRIPTION OF THE INVENTION

[0009] An object recognition device and an object processing device according to embodiments of the present disclosure will be described below with reference to the drawings. Note that the following description does not limit the technology of the present disclosure. In addition, in the following description, the same components are given the same reference numerals, and duplicated descriptions will be omitted. [Example]

[0010] As shown in FIG. 1, an object processing device 1 of the embodiment is provided in an automatic recyclable waste sorting device 2. FIG. 1 is a perspective view showing the automatic recyclable waste sorting device 2 in which the object processing device 1 of the embodiment is provided. The automatic recyclable waste sorting device 2 includes the object processing device 1 and a conveying device 3. The conveying device 3 is formed from a so-called belt conveyor, and includes a belt conveyor frame 5, a belt 6, and a plurality of fixed pulleys 7, as well as a belt drive device (not shown). The belt conveyor frame 5 is placed on an installation surface on which the automatic recyclable waste sorting device 2 is installed. The belt 6 is made of a flexible material and is formed into a loop-shaped band.

[0011] Each of the multiple fixed pulleys 7 is formed in a cylindrical shape. The multiple fixed pulleys 7 are arranged along multiple rotation axes, respectively. The multiple rotation axes are parallel to the X-axis, which is parallel to the plane along which the installation surface lies, and overlap on a single plane parallel to the plane along which the installation surface lies. The multiple fixed pulleys 7 are each supported by the belt conveyor frame 5 so as to be rotatable around the multiple rotation axes. The belt 6 is stretched across the multiple fixed pulleys 7 and movably supported by the belt conveyor frame 5. The belt 6 has an upper portion arranged above the multiple fixed pulleys 7 and a lower portion arranged below the multiple fixed pulleys 7. The upper portion is arranged along another plane parallel to the plane along which the installation surface lies. The belt drive device rotates the multiple fixed pulleys 7 so that the upper portion of the belt 6 moves parallel to the Y-axis. The Y-axis is parallel to the plane along which the installation surface lies and perpendicular to the X-axis.

[0012] The object processing device 1 includes an object recognition device 10 of the embodiment and a robot unit 11. The object recognition device 10 includes an optical unit 12. The optical unit 12 is disposed above a portion of the upper portion of the belt 6. The robot unit 11 is disposed above another portion of the upper portion of the belt 6, and is disposed downstream of the object recognition device 10 in the conveying direction 14. The conveying direction 14 is parallel to the Y axis.

[0013] The robot unit 11 includes multiple picking robots 15 and a suction pump (not shown). One of the multiple picking robots 15 includes a suction pad 16, an X-axis actuator 17, a Z-axis actuator 18, a grip sensor 19, a dump case (not shown), and a solenoid valve. The dump case is placed on the installation surface beside the conveyor device 3. The suction pad 16 is supported on the belt conveyor frame 5 via the X-axis actuator 17 and the Z-axis actuator 18 so as to be translatable parallel to the X-axis or the Z-axis. The Z-axis is perpendicular to the plane along which the installation surface lies, i.e., perpendicular to the X-axis and the Y-axis. The range of motion of the suction pad 16 includes the initial position. When positioned in the initial position, the suction pad 16 is positioned above the dump case. An air intake port is formed on the lower surface of the suction pad 16 facing the installation surface.

[0014] The suction pump is connected to the suction pad 16 via a pipe (not shown) and sucks air through the air intake port of the suction pad 16. A solenoid valve is provided midway along the pipe connecting the suction pad 16 and the suction pump. When the solenoid valve is opened, it connects the suction pad 16 to the suction pump so that air is sucked through the air intake port of the suction pad 16. When the solenoid valve is closed, it cuts off the connection between the suction pad 16 and the suction pump so that air is not sucked through the air intake port of the suction pad 16.

[0015] X-axis actuator 17 moves suction pad 16 parallel to the X-axis. Z-axis actuator 18 moves suction pad 16 parallel to the Z-axis. Grip sensor 19 detects whether or not an object is being gripped by suction pad 16. The other picking robots among multiple picking robots 15 are also formed in the same manner as that picking robot, that is, equipped with suction pads, X-axis actuators, Z-axis actuators, grip sensors, a dump case, and solenoid valves.

[0016] FIG. 2 is a cross-sectional view of the optical unit 12. The optical unit 12 includes a housing 21, a camera 22, and a lighting device 23. The housing 21 is formed from a light-opaque material and has a box shape. An internal space 24 is formed inside the housing 21. The housing 21 is disposed above the belt 6 such that a portion of the upper portion of the belt 6 is disposed in the internal space 24 of the housing 21, and is fixed to the belt conveyor frame 5 of the conveying device 3. The housing 21 blocks external light so that light from outside the housing 21 does not enter the internal space 24. The housing 21 has an entrance and an exit. The entrance is formed in a portion of the housing 21 on the upstream side in the conveying direction 14, and the internal space 24 is connected to the outside of the housing 21 via the entrance. The exit is formed in a portion of the housing 21 on the downstream side in the conveying direction 14, and the internal space 24 is connected to the outside of the housing 21 via the exit.

[0017] The camera 22 is disposed above the housing 21. The camera 22 is fixed to the housing 21, i.e., fixed to the belt conveyor frame 5 via the housing 21. The camera 22 is a so-called digital camera, and uses visible light to capture an image of a subject 29 placed on a part of the upper part of the belt 6 that is disposed in the internal space 24. The image is formed from a plurality of pixels that are spread across the image. The plurality of pixels are associated with a plurality of pieces of color information. Each of the plurality of pieces of color information indicates, for example, a red gradation value, a green gradation value, and a blue gradation value. Note that the image may be a monochrome image, and in this case, the color information indicates one gradation value.

[0018] The lighting device 23 includes a reflecting member 25, a plurality of light sources 26, and an ultraviolet light source 27. The reflecting member 25 covers substantially the entire inner surface of the housing 21 facing the internal space 24 and is disposed so as to surround the camera 22, i.e., so as to surround the viewpoint of the image captured by the camera 22. The reflecting member 25 diffuses the light projected onto the reflecting member 25. The plurality of light sources 26 are disposed on the lower side of the housing 21 near the belt 6, and emit light to project low or high amounts of visible light onto the reflecting member 25. The ultraviolet light source 27 is disposed on the upper side of the housing 21 far from the belt 6, and emits light to project ultraviolet light toward the upper portion of the belt 6.

[0019] As shown in Fig. 3, the object recognition device 10 further includes a control device 31. Fig. 3 is a block diagram showing the control device 31. The control device 31 is a computer, and includes a storage device 32 and a CPU (Central Processing Unit) 33. The storage device 32 records computer programs installed in the control device 31 and records information used by the CPU 33. Examples of the storage device 32 include memory such as RAM and ROM, a fixed disk device such as a hard disk, and an SSD (Solid State Drive).

[0020] The CPU 33 processes information, controls the storage device 32, and controls the camera 22, the plurality of light sources 26, the X-axis actuator 17, the Z-axis actuator 18, the grip sensor 19, and the solenoid valve by executing a computer program installed in the control device 31. The computer program installed in the control device 31 includes a plurality of computer programs for causing the control device 31 to respectively realize a plurality of functions. The plurality of functions include a lighting control unit 34, a camera control unit 35, a position calculation unit 36, a discrimination unit 37, a grip position / grip timing calculation unit 38, and a grip control unit 39.

[0021] The lighting control unit 34 controls the lighting devices 23 so that the subject 29 placed in the internal space 24 is illuminated under a plurality of lighting conditions. That is, the lighting control unit 34 controls the plurality of light sources 26 so that the plurality of light sources 26 are turned on at a low light intensity or a high light intensity, or so that the plurality of light sources 26 are turned off. The lighting control unit 34 also controls the ultraviolet light source 27 so that the ultraviolet light source 27 is turned on or off. The camera control unit 35 controls the camera 22 so that an image of the subject placed in the internal space 24 of the housing 21 is captured using visible light. The camera control unit 35 also controls the storage device 32 so that data of the image captured by the camera 22 is recorded in the storage device 32 in association with the image capture time.

[0022] The position calculation unit 36 ​​processes the image captured by the camera control unit 35 to cut out partial images from the image. The position calculation unit 36 ​​processes the cut-out partial images to determine whether or not an object is captured in the partial images. When it is determined that an object is captured in the partial images, the position calculation unit 36 ​​calculates the position of the object's center of gravity by further image processing the partial images. When it is determined that an object is captured in the partial images, the discrimination unit 37 further processes the partial images to determine the material that makes up the object, and determines whether or not the object is a target to be grasped based on the determined material.

[0023] When it is determined that the grasp target is captured in the multiple partial images, the grasp position / grasp timing calculation unit 38 calculates the grasp position and grasp timing based on the image capture time when the image was captured by the camera control unit 35, the position calculated by the position calculation unit 36, and the transport speed. The grasp control unit 39 controls the X-axis actuator 17 so that the suction pad 16 is positioned at the grasp preparation position above the grasp position calculated by the grasp position / grasp timing calculation unit 38 before the grasp timing calculated by the grasp position / grasp timing calculation unit 38. The grasp control unit 39 controls the Z-axis actuator 18 so that the suction pad 16 is positioned at the grasp position calculated by the grasp position / grasp timing calculation unit 38 at the grasp timing calculated by the grasp position / grasp timing calculation unit 38. The grasp control unit 39 further controls the solenoid valve so that air is sucked through the opening of the suction pad 16 at the grasp timing calculated by the grasp position / grasp timing calculation unit 38.

[0024] The operation of the automatic recyclable waste sorting device 2 includes an operation in which the conveying device 3 conveys recyclable waste and an operation in which the control device 31 controls the robot unit 11 and the optical unit 12. In the operation in which the conveying device 3 conveys recyclable waste, a user first operates the conveying device 3 to start the conveying device 3. When the conveying device 3 is started, the belt drive unit of the conveying device 3 rotates the multiple fixed pulleys 7 at a predetermined rotational speed. The upper portion of the belt 6 moves in translation in the conveying direction 14 at a predetermined conveying speed as the multiple fixed pulleys 7 rotate at the predetermined rotational speed. The user then places multiple recyclable waste items on the upper portion of the belt 6, upstream of the optical unit 12 in the conveying direction 14. Examples of recyclable waste include PET bottles and glass bottles. The multiple recyclable waste items placed on the upper portion of the belt 6 are conveyed in the conveying direction 14 at the conveying speed as the upper portion of the belt 6 moves in translation in the conveying direction 14 at the conveying speed. The plurality of recyclable waste pieces move in parallel in the conveying direction 14, thereby entering the internal space 24 of the housing 21 through the entrance and leaving the internal space 24 of the housing 21 through the exit.

[0025] 4 is a flowchart showing the operation of the control device 31 to control the robot unit 11 and the optical unit 12. The operation of the control device 31 to control the robot unit 11 and the optical unit 12 is executed in parallel with the operation of the transport device 3 to transport recyclable waste. The control device 31 controls the multiple light sources 26 to turn on the multiple light sources 26 and cause the multiple light sources 26 to emit high-intensity visible light (step S1). The high-intensity visible light emitted from the multiple light sources 26 is diffusely reflected by the surface of the reflective member 25, and is thereby projected onto the multiple recyclable waste transported by the transport device 3. In other words, the lighting device 23 illuminates the multiple recyclable waste with high-intensity visible light emitted from a surface light source surrounding the camera 22.

[0026] When the plurality of recyclable waste items are illuminated with high-intensity visible light by the lighting device 23, the control device 31 controls the camera 22 to capture a high-light image showing the plurality of recyclable waste items using visible light (step S2). After the high-light image is captured, the control device 31 controls the plurality of light sources 26 to turn off the plurality of light sources 26 (step S3). The control device 31 records the high-light image in the storage device 32 in association with the time when the high-light image was captured. The control device 31 processes the high-light image to cut out a high-light partial image that appears in a predetermined region of the high-light image from the high-light image (step S4).

[0027] After the plurality of light sources 26 are turned off, the control device 31 controls the ultraviolet light source 27 to turn on the ultraviolet light source 27 and cause the ultraviolet light source 27 to emit ultraviolet light (step S5). The ultraviolet light emitted from the ultraviolet light source 27 is projected onto the plurality of recyclable wastes being transported by the transport device 3. That is, the lighting device 23 projects ultraviolet light onto the plurality of recyclable wastes that have entered the internal space 24, illuminating the plurality of recyclable wastes with the ultraviolet light.

[0028] When the plurality of recyclable waste items are illuminated with ultraviolet light by the lighting device 23, the control device 31 controls the camera 22 to capture a fluorescent image of the plurality of recyclable waste items using visible light (step S6). The timing at which the fluorescent image is captured is equal to the timing after a predetermined first elapsed time (e.g., several tens of milliseconds) has elapsed since the timing at which the high-light-intensity image was captured. After the fluorescent image is captured, the control device 31 controls the ultraviolet light source 27 to turn off the ultraviolet light source 27 (step S7). The control device 31 records the fluorescent image in the storage device 32 in association with the time at which the fluorescent image was captured.

[0029] The control device 31 processes the fluorescent image to extract from the fluorescent image a fluorescent partial image that appears in a region of the fluorescent image calculated based on the first elapsed time (step S8). The region of the upper portion of the belt 6 that appears in the fluorescent image differs from the region of the upper portion of the belt 6 that appears in the high-light-intensity image because the upper portion of the belt 6 translates in the conveying direction 14 at the conveying speed. The fluorescent partial image is extracted from the fluorescent image so that the region of the upper portion of the belt 6 that appears in the fluorescent partial image coincides with the region of the upper portion of the belt 6 that appears in the high-light-intensity image. In other words, the region of the fluorescent image that appears in the fluorescent partial image is calculated based on the first elapsed time so that the region of the upper portion of the belt 6 that appears in the fluorescent partial image coincides with the region of the upper portion of the belt 6 that appears in the high-light-intensity image.

[0030] After the ultraviolet light source 27 is turned off, the control device 31 controls the plurality of light sources 26 to turn on the plurality of light sources 26 and cause the plurality of light sources 26 to emit low-intensity visible light (step S9). The low-intensity visible light emitted from the plurality of light sources 26 is diffusely reflected on the surface of the reflective member 25, and is projected onto the plurality of recyclable wastes being transported by the transport device 3. In other words, the lighting device 23 illuminates the plurality of recyclable wastes with low-intensity visible light emitted from a surface light source surrounding the camera 22.

[0031] When the plurality of recyclable waste items are illuminated with low-intensity visible light by the lighting device 23, the control device 31 controls the camera 22 to capture a low-light image of the plurality of recyclable waste items using visible light (step S10). The timing at which the low-light image is captured is equal to the timing after a predetermined second elapsed time (e.g., several tens of milliseconds) has elapsed since the timing at which the fluorescent image was captured. After the low-light image is captured, the control device 31 controls the plurality of light sources 26 to turn off the plurality of light sources 26 (step S11). The control device 31 records the low-light image in the storage device 32 in association with the time at which the low-light image was captured.

[0032] The control device 31 processes the low-light image to extract from the low-light image a low-light partial image that appears in a region of the low-light image calculated based on the second elapsed time (step S12). The region of the upper portion of the belt 6 that appears in the low-light image differs from the region of the upper portion of the belt 6 that appears in the high-light image and differs from the region of the upper portion of the belt 6 that appears in the fluorescent image due to the translation of the upper portion of the belt 6. The low-light partial image is extracted from the low-light image such that the region of the upper portion of the belt 6 that appears in the low-light partial image matches the region of the upper portion of the belt 6 that appears in the high-light partial image and matches the region of the upper portion of the belt 6 that appears in the fluorescent image. That is, the region of the low-light image that appears in the low-light partial image is calculated based on the first elapsed time and the second elapsed time such that the region of the low-light partial image matches the region of the high-light partial image and the fluorescent partial image.

[0033] The control device 31 performs image processing on the plurality of partial images, including the high-light partial image, the low-light partial image, and the fluorescent partial image, to determine whether an object is captured in the plurality of partial images (step S13). When it is determined that an object is captured in the plurality of partial images, the control device 31 performs image processing on the plurality of partial images to calculate the position of the center of gravity of the object (step S14). When it is determined that an object is captured in the plurality of partial images, the control device 31 further performs image processing on the plurality of partial images to determine the material from which the object is made (step S15).

[0034] The control device 31 determines whether the object is a sorting target based on the material identified in step S15 (step S16). When the control device 31 determines that the object is a sorting target, it determines which of the multiple picking robots 15 the object is to be grasped by. When the control device 31 determines that the object is a target for the target picking robot, it calculates the grasp timing and grasp position (step S17). The grasp timing is calculated based on the image capture time when an image of the object is captured, the position where the center of gravity of the object is located at the image capture time, the transport speed, and the position of the target picking robot in the Y-axis direction. The grasp timing indicates the timing when the object passes through the movable range of the suction pad 16 of the target picking robot. The grasp position indicates the position where the center of gravity of the object is located at the grasp timing, i.e., the position where the object passes through the movable part of the suction pad 16 of the target picking robot.

[0035] The control device 31 controls the X-axis actuator 17 of the target picking robot to place the suction pad 16 of the target picking robot at the grip preparation position (step S18). The grip preparation position is above the grip position, and the X-axis position of the grip preparation position in the X-axis direction is equal to the X-axis position of the grip position in the X-axis direction. In other words, the figure obtained by orthogonally projecting the suction pad 16 placed at the grip preparation position onto the X-axis overlaps the figure obtained by orthogonally projecting the target to be gripped at the grip position onto the X-axis. After the suction pad 16 is placed at the grip preparation position, the control device 31 controls the solenoid valve to connect the suction pad 16 to a suction pump and sucks air through the opening of the suction pad 16 (step S19).

[0036] The control device 31 controls the Z-axis actuator 18 of the target picking robot to place the opening of the suction pad 16 of the target picking robot at the gripping position at the gripping timing (step S20). The suction pad 16 comes into contact with the target to be gripped by the positioning of the opening of the suction pad 16 at the gripping position at the gripping timing. When the target to be gripped comes into contact with the opening of the suction pad 16, air is sucked through the opening of the suction pad 16, so that the target is gripped by the suction pad 16. After the suction pad 16 is placed at the gripping position, the control device 31 controls the Z-axis actuator 18 to place the suction pad 16 at the gripping ready position (step S21). The target to be gripped is lifted from the belt 6 by the positioning of the suction pad 16 at the gripping ready position.

[0037] When the suction pad 16 is placed in the grip preparation position, the control device 31 controls the grip sensor 19 of the target picking robot to determine whether the grip target is properly gripped by the suction pad 16 (step S22). When the grip target is properly gripped by the suction pad 16 (step S22, successful), the control device 31 controls the X-axis actuator 17 to place the suction pad 16 in the initial position of the target picking robot (step S23).

[0038] After the suction pad 16 is placed in the initial position, the control device 31 controls the solenoid valve to disconnect the suction pad 16 from the suction pump, preventing air from being sucked through the opening of the suction pad 16 (step S24). When air is no longer sucked through the opening of the suction pad 16, the target object held by the suction pad 16 is released from the suction pad 16, drops, and is placed in the disposal case of the target picking robot. When the target object is not properly held by the suction pad 16 (step S22, failure), the control device 31 controls the solenoid valve to close the solenoid valve, preventing air from being sucked through the opening of the suction pad 16 (step S24). When multiple targets are captured in the captured image, the control device 31 repeatedly executes the processes of steps S18 to S24.

[0039] As shown in FIG. 5, a high-light-intensity partial image 41 cut out from a high-light image captured when multiple recyclable waste items are illuminated with high-intensity visible light shows, for example, an image 42 of subject 29. FIG. 5 is a diagram illustrating high-light-intensity partial image 41. Image 42 includes a blown-out highlight region 43. The blown-out highlight region 43 is an area of ​​image 42 where blown-out highlights have occurred, and is an area filled with solid white. That is, the red, green, and blue gradation values ​​indicated by each pixel included in blown-out highlight region 43 are each at their upper limit values. Such blown-out highlights occur when light projected from lighting device 23 onto subject 29 is specularly reflected off the surface of subject 29 when the surface of subject 29 is glossy.

[0040] The ratio of the area of ​​the blown-out highlight region 43 to the area of ​​the image 42 becomes larger than a predetermined value when light emitted from the surface light source of the lighting device 23 is projected onto the subject 29. That is, the reflecting member 25 of the lighting device 23 is formed so that the ratio of the area of ​​the blown-out highlight region 43 to the area of ​​the image 42 becomes larger than a predetermined value. Furthermore, the multiple light sources 26 of the lighting device 23 are set so that when a high amount of visible light is emitted, the amount of visible light emitted from the multiple light sources 26 becomes larger than a predetermined value so that the image 42 includes the blown-out highlight region 43.

[0041] The image 42 may contain an interfering image that prevents the image 42 from being extracted from the high-light-intensity partial image 41. For example, if a film is attached to the surface of the subject 29 or if an image such as text, a picture, or a photograph is printed on the surface of the subject 29, the image 42 may contain an image of the film or image. Furthermore, if the subject 29 is made of a light-transmitting material, the background behind the subject 29 may be reflected in the image 42 due to light passing through the subject 29. Examples of light-transmitting materials include polyethylene terephthalate (PET) and glass. When an interfering image is projected on the image 42, the control device 31 may erroneously extract the image 42 containing the subject 29 due to the background image. When the image of the subject 29 is erroneously extracted from the high-light-intensity partial image 41, the control device 31 may not properly calculate the position of the center of gravity of the subject 29. When the position of the center of gravity of the subject 29 is not calculated appropriately, the object processing device 1 may not grip the subject 29 appropriately.

[0042] The control device 31 can relatively reduce the ratio of the area of ​​the distracting image to the area of ​​the image 42 by increasing the ratio of the area of ​​the blown-out highlight region 43 to the area of ​​the image 42. By reducing the area of ​​the distracting image, the control device 31 can improve the probability of properly extracting the image 42 from the high-light-intensity partial image 41, and can prevent erroneous recognition of the position of the subject 29. By properly calculating the position of the subject 29, the object processing device 1 can properly grasp the subject 29 and properly sort the subject 29.

[0043] As shown in FIG. 6, low-light partial image 51, which is cut out from a low-light image captured when multiple recyclable waste items are illuminated with low-light visible light, depicts image 52 of subject 29, for example. FIG. 6 is a diagram showing low-light partial image 51. Low-light partial image 51 is cut out from the low-light image based on a first elapsed time and a second elapsed time indicating the difference in imaging timing, so that the position of image 52 in low-light partial image 51 is equal to the position of image 42 in high-light partial image 41. Because the position of image 52 is equal to the position of image 42, control device 31 can easily calculate the position of subject 29 based on high-light partial image 41 using a calculation similar to that used to calculate the position of subject 29 based on high-light partial image 41.

[0044] The image 52 does not include any overexposed areas, i.e., the amount of light of the low-intensity visible light is set to be less than the amount of light of the high-intensity visible light so that overexposed areas do not occur in the image 52.

[0045] When multiple objects are shown in high-light partial image 41 and each of the multiple images showing the multiple objects contains a blown-out highlight area, the boundaries between the multiple images may not be shown in high-light partial image 41. In this case, control device 31 may not be able to properly distinguish between the multiple images shown in high-light partial image 41, and may not be able to properly calculate the position of the center of gravity of subject 29 included in the multiple objects based on high-light partial image 41 alone. When the position of the center of gravity of subject 29 cannot be properly calculated, object processing device 1 may not be able to properly grasp subject 29 and may not be able to properly classify subject 29.

[0046] The control device 31 can properly distinguish between multiple images of multiple objects in the low-light image by preventing blown-out highlight areas from being included in the low-light image captured when multiple recyclable waste items are illuminated with low-light visible light. Therefore, the control device 31 can properly extract the image 52 of the subject 29 from the low-light partial image 51 even when multiple objects are captured in the low-light partial image 51. By properly extracting the image 52 of the subject 29 from the low-light partial image 51, the control device 31 can prevent erroneous recognition of the position of the subject 29. By properly calculating the position of the subject 29, the object processing device 1 can properly grasp the subject 29 and properly sort the subject 29.

[0047] Fluorescent objects made of polyethylene terephthalate (PET) emit fluorescent light, which is visible light, when illuminated with ultraviolet light. The fluorescent image captured when multiple recyclable waste items are illuminated with ultraviolet light is captured using the fluorescent light emitted from the fluorescent objects when the multiple recyclable waste items contain fluorescent objects. The fluorescent partial image extracted from the fluorescent image also depicts an image of subject 29, similar to high-light-intensity partial image 41 and low-light-intensity partial image 51. The fluorescent partial image is extracted from the fluorescent image based on a first elapsed time indicating the difference in imaging timing so that the position of the image depicting subject 29 is equal to the position of image 42 and the position of image 52. That is, by processing the fluorescent image based on the first elapsed time, control device 31 can appropriately extract the fluorescent partial image from the fluorescent image so that the position of the image depicting subject 29 is equal to the position of image 42.

[0048] An image of an object made of glass and an image of an object made of polyethylene terephthalate PET may be displayed in the high-light partial image 41 or the low-light partial image 51. For this reason, the control device 31 may not be able to distinguish between an image of an object made of glass and an image of an object made of polyethylene terephthalate PET based on the high-light partial image 41 or the low-light partial image 51. The fluorescent image displays an image formed by fluorescence, and, for example, an image of an object made of polyethylene terephthalate PET may be displayed appropriately.

[0049] Therefore, the control device 31 can easily distinguish between images of objects other than fluorescent materials and images of fluorescent materials based on the fluorescent images captured when multiple recyclable waste items are illuminated with ultraviolet light. By distinguishing between images of fluorescent materials and other images, the control device 31 can determine whether the material from which the object 29 is made is polyethylene terephthalate PET. By determining whether the material from which the object 29 is made is polyethylene terephthalate PET, the control device 31 can appropriately determine the material from which the object 29 is made. Therefore, the object processing device 1 can cause a picking robot associated with polyethylene terephthalate PET, among the multiple picking robots 15, to appropriately grasp the polyethylene terephthalate PET object, thereby appropriately sorting the object 29.

[0050] As described above, by using the high-light-intensity partial image 41, the low-light-intensity partial image 51, and the fluorescent partial image, the control device 31 can appropriately extract an image of the subject 29 even when the subject 29 is made of various materials. By appropriately extracting an image of the subject 29, the control device 31 can appropriately calculate the position where the subject 29 is located and can appropriately calculate the position of the center of gravity of the subject 29. By appropriately calculating the position of the center of gravity of the subject 29, the control device 31 can appropriately grasp the subject 29 and appropriately classify the subject 29.

[0051] Although object recognition device 10 calculates the position of an object based on three images captured under three illumination conditions, the position of an object may also be calculated based on two images captured under two illumination conditions. Examples of pairs of two images include a pair of high-light-intensity partial image 41 and a low-light-intensity partial image 51, a pair of high-light-intensity partial image 41 and a fluorescent image, and a pair of low-light-intensity partial image 51 and a fluorescent image. Even when object recognition device 10 calculates the position of an object based on two images captured under two illumination conditions, it can properly extract an image of the object from the image and properly calculate the position of the object.

[0052] [Effects of the object recognition device 10 of the embodiment] The object recognition device 10 of the embodiment includes a lighting device 23, a camera 22, and a position calculation unit 36. The lighting device 23 illuminates the subject 29. The camera 22 captures multiple images of the subject 29 when the subject 29 is illuminated by the lighting device 23 under multiple lighting conditions. The position calculation unit 36 ​​calculates the position of the subject 29 by image processing the multiple images. The object recognition device 10 of the embodiment captures multiple images using the camera 22 when the subject 29 is illuminated under multiple lighting conditions, thereby easily capturing multiple images in which the subject 29 is captured in various ways without changing the settings of a single camera 22. Depending on the lighting conditions at the time of capture, the image of the subject 29 may or may not be captured properly in the image of the subject 29. The object recognition device 10 of the embodiment can appropriately extract the image of the subject 29 from the image by image processing the image in which the subject 29 is appropriately captured from the multiple images captured under multiple lighting conditions. The object recognition device 10 of the embodiment can appropriately calculate the position where the center of gravity of the subject 29 is located by appropriately extracting the image of the subject 29.

[0053] The object processing device 1 of the embodiment includes an object recognition device 10, a suction pad 16, an X-axis actuator 17, a Z-axis actuator 18, and a grip control unit 39. The X-axis actuator 17 and the Z-axis actuator 18 move the suction pad 16. The grip control unit 39 controls the X-axis actuator 17 and the Z-axis actuator 18 based on the position calculated by the position calculation unit 36 ​​so that the suction pad 16 grips the object 29. The object processing device 1 of the embodiment can properly grip the object 29 and properly separate multiple recyclable waste items because the object recognition device 10 properly calculates the position of the object 29.

[0054] In the object recognition device 10 of the above-described embodiment, the plurality of light sources 26 emit two types of visible light with different light intensities. However, the plurality of light sources 26 may emit multiple types of visible light with different wavelengths. Examples of the multiple types of visible light include red, green, and blue visible light. In this case, the control device 31 captures multiple images of the plurality of recyclable wastes using the camera 22 when the plurality of recyclable wastes are illuminated with multiple types of visible light. In a red light image captured when the plurality of recyclable wastes are illuminated with red visible light, the red portions of the plurality of recyclable wastes are not properly captured, but the non-red portions of the plurality of recyclable wastes are properly captured. In a green light image captured when the plurality of recyclable wastes are illuminated with green visible light, the green portions of the plurality of recyclable wastes are not properly captured, but the non-green portions of the plurality of recyclable wastes are properly captured. In a blue light image captured when multiple recyclable waste items are illuminated with blue visible light, the blue portions of the multiple recyclable waste items are not properly captured, but the non-blue portions of the multiple recyclable waste items are properly captured. In this case, the control device 31 can improve the probability of properly extracting images of multiple recyclable waste items from the multiple images, even when the surfaces of the multiple recyclable waste items have portions colored red, green, and blue.

[0055] Although the embodiments have been described above, the embodiments are not limited to the above content. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, at least one of various omissions, substitutions, and modifications of the components can be made without departing from the spirit of the embodiments. [Explanation of symbols]

[0056] 2: Automatic recyclable waste sorting device 3:Transportation device 1: Material processing equipment 10: Object recognition device 16: Suction pad 17: X-axis actuator 18: Z-axis actuator 22: Camera 23:Lighting equipment 31: Control device 34: Lighting control unit 35: Camera control unit 36: Position calculation section 37: Discrimination part 38: Grasp position and grasp timing calculation unit 39: Grasping control unit 41: High light intensity partial image 42: Statue 43: Blown-out highlight area 51: Low light partial image 52: Statue

Claims

1. A lighting device for illuminating an object; an imaging unit that captures a first image of the object when the object is illuminated by the lighting device under a first lighting condition, and captures a second image of the object when the object is illuminated by the lighting device under a second lighting condition different from the first lighting condition; a control device that calculates a position where the object is located by performing image processing on the first image and the second image; Equipped with An object recognition device in which the object is transported so that a first position at which the object is placed at a first timing when the first image is captured is different from a second position at which the object is placed at a second timing when the second image is captured.

2. The lighting device illuminates the object being transported by the transport device, the imaging unit captures the first image in parallel with the transport of the object, and after the first image is captured, captures the second image in parallel with the transport of the object; The control device cuts out a first partial image from the first image, and cuts out the second partial image from the second image so that a position where an image of the object is projected in the second partial image coincides with a position where an image of the object is projected in the first partial image. The object recognition device according to claim 1 .

3. The control device cuts out the second partial image from the second image based on an elapsed time that has passed from a first imaging time when the first image is captured to a second imaging time when the second image is captured. The object recognition device according to claim 2 .

4. The control device records the first image in association with the first image capturing time in a storage device, and records the second image in association with the second image capturing time in the storage device. The object recognition device according to claim 3 .

5. The control device determines the material from which the object is made by performing image processing on the first partial image and the second partial image. The object recognition device according to claim 2 .

6. The control device calculates a position where the object is located by performing image processing on the first partial image and the second partial image. The object recognition device according to claim 2 .

7. The first image is captured based on light reflected from the object when first light is projected onto the object under the first illumination conditions, The second image is captured based on fluorescence emitted from the object when second light is projected onto the object under the second illumination condition. The object recognition device according to claim 1 .

Images