Foreign object detection system

Abstract

The foreign matter detection system of the present application is provided with: an imaging device (9) that acquires an image of an object region (A); a transmission portion (7, 8) that can visually recognize the outside from the inside of the object region (A); and a polarizing filter (10a, 10b) on a path of light from the transmission portion (7, 8) to the imaging device (9). The polarizing filter (10a) on the front side of the imaging device (9) and the polarizing filter (10b) on the inside of the imaging device (9) are disposed so that the polarization directions intersect each other. A foreign matter (F) in the object region (A) is detected based on an image acquired by the imaging device (9).

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a system that detects a foreign object present in an object region such as a parking lot. BACKGROUND

[0002] In a space where a vehicle enters and stops, such as a loading / unloading space of a mechanical parking lot, when a door that opens and closes an entrance or an operation of carrying a pallet that carries a vehicle is performed, it is necessary to confirm that no driver or passenger, baggage, or the like is present as a foreign object other than the vehicle. Such a confirmation work of the absence of a foreign object can be performed by a driver or a supervisor of a facility or the like by visual inspection, but in the case of relying only on human power, it can not be possible to confirm thoroughly due to carelessness or the like. Therefore, in recent years, a technology is required in which a foreign object in an object region is automatically detected without relying only on human power, and confirmation of the absence of a foreign object is reliably performed.

[0003] In addition, if an automatic driving technology that is currently being developed in each country is officially put into practical use, it can be thought that not only the driving of a vehicle is automated, but also various operations performed in the periphery of the vehicle, such as the opening and closing of the door or the carrying of the pallet as described above, are also automated. For example, a case can be considered in which, after a vehicle transports a passenger to a destination by automatic driving, the vehicle automatically moves to a nearby mechanical parking lot and is stored in a storage space. In this case, the vehicle moves in a state in which no passenger is present, and stops in a loading / unloading space. At this time, the supervisor confirms the absence of a foreign object by visual inspection, and inputs an operation button to perform an operation such as the opening and closing of the door, the start of the movement of the pallet. However, if a configuration that automatically confirms the absence of a foreign object and starts the operation is provided, it is more smooth.

[0004] Such a configuration that performs foreign object detection can be widely required not only in a mechanical parking lot, but also in a place where some operation is performed in the periphery of a stopped vehicle, such as a power supply device for an electric vehicle. In addition, the same configuration is not limited to a vehicle, and can be widely applied to a situation in which it is desired to detect a foreign object in the periphery of some object.

[0005] As a technology for automatically detecting a foreign object in an object region, various methods have been proposed in which an image in an object region is acquired by a camera or the like, and a foreign object photographed in the image is detected. As a technical document that describes such a technology, for example, Patent Documents 1 to 7 can be cited.

[0006] Prior Art Documents

[0007] Patent Documents

[0008] Patent Document 1: Japanese Patent Application Publication No. 2016-3493;

[0009] Patent Document 2: Japanese Patent Application Publication No. 2015-55073;

[0010] Patent Literature 3: Japanese Patent Application Laid-Open (JP A) No. 2014-80735

[0011] Patent Literature 4: Japanese Patent Application Laid-Open (JP A) No. 08-42184

[0012] Patent Literature 5: Japanese Patent Application Laid-Open (JP A) No. 62-194366

[0013] Patent Literature 6: Japanese Patent Application Laid-Open (JP A) No. 2012-158968

[0014] Patent Literature 7: Japanese Patent Application Laid-Open (JP A) No. 2000-182195 SUMMARY

[0015] PROBLEMS TO BE SOLVED BY THE INVENTION

[0016] In the above-described technology for foreign matter detection, it goes without saying that the minimum level of accuracy is required in detection. In addition to this, from the viewpoint of costs related to construction or maintenance, it is important to simplify the device structure as much as possible. However, the technologies described in the above-described Patent Literatures 1 to 7 do not necessarily satisfy both the accuracy and the cost.

[0017] Therefore, in the present disclosure, a foreign matter detection system capable of detecting foreign matter with high accuracy with as simple a structure as possible is described.

[0018] SOLUTION TO THE PROBLEM

[0019] The present disclosure relates to a foreign matter detection system configured to include: an imaging device that acquires an image of an object region; a transmission portion that can visually recognize an outside from an inside of the object region; and a polarizing filter on a front side of the imaging device and a polarizing filter on a back side of the imaging device in a path of light from the transmission portion to the imaging device, the polarizing direction of the polarizing filter on the front side and the polarizing direction of the polarizing filter on the back side intersecting each other, the foreign matter detection system detecting foreign matter in the object region based on an image acquired by the imaging device.

[0020] In the above-described foreign matter detection system, the polarizing filter on the back side can be provided at the transmission portion located at a boundary of the object region and an outside space.

[0021] The above-described foreign matter detection system can detect foreign matter in an image acquired by the imaging device by a program using deep learning.

[0022] In the above-described foreign matter detection system, the object region can be a parking space in a mechanical parking lot.

[0023] Further, the present disclosure relates to a foreign matter detection system configured to include: an imaging device that acquires an image of an object region; a display unit that displays the image acquired by the imaging device; an operation unit that inputs an operation of each part configuring the system; and a foreign matter detection unit that detects a foreign matter in the object region, wherein in a case where it is determined by the foreign matter detection unit that a foreign matter exists, information of the detected foreign matter is displayed on the display unit together with the image acquired by the imaging device.

[0024] The above foreign matter detection system can be configured such that, in a case where it is determined by the foreign matter detection unit that a foreign matter exists, an operator confirms that no foreign matter exists in the object region, and then can input a gist that no foreign matter exists to the operation unit.

[0025] The above foreign matter detection system can be configured such that, in a case where it is determined by the foreign matter detection unit that a foreign matter exists, an operation of a device type configuring the system is prohibited, and thereafter, in a case where a gist that no foreign matter exists is input to the operation unit, the operation prohibition state of the device type configuring the system is released.

[0026] Effects of Invention

[0027] The foreign matter detection system according to the present disclosure can achieve an excellent effect of being able to detect a foreign matter with high precision with as simple a configuration as possible. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a schematic plan view that explains a device configuration in a foreign matter detection system according to an embodiment of the present disclosure.

[0029] Figure 2 is a front view that simply explains a form of an entrance and a window periphery in an embodiment of the present disclosure.

[0030] Figure 3 is a block diagram that explains a system configuration of a foreign matter detection system according to an embodiment of the present disclosure.

[0031] Figure 4 is a front view that conceptually shows one example of an image of a visual entrance and window periphery.

[0032] Figure 5 is a diagram that conceptually shows a reference example of an image to which a masking process is applied to an image of an entrance and window periphery.

[0033] Figure 6 is a diagram that conceptually shows one example of an image that can be acquired in a case where a polarizing filter is provided to an entrance, a window, and an imaging device.

[0034] Figure 7 is a flowchart that explains one example of a sequence when a foreign matter detection is performed in an embodiment of the present disclosure.

[0035] Figure 8 is a diagram illustrating a reference example of an image acquired by an imaging device.

[0036] Figure 9 is a diagram showing one example of an image acquired by an imaging device in an embodiment of the present disclosure.

[0037] Figure 10 is a diagram showing one example of an image displayed on a display section in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] Hereinafter, a mode of an embodiment in the present disclosure will be described with reference to the drawings. In the present disclosure, the same reference numerals are given to the same components, and repeated description will be omitted. Figure 1 In the foreign object detection system of the present embodiment shown in FIG. 1, a vehicle 1 is assumed as an object in an object region A. The object region A is a space in which the vehicle 1 is parked or taken out. The object region A is a space surrounded by a partition wall on four sides, and the vehicle 1 is parked or taken out from a parking / taking-out port 3 provided on the front side (lower side in the drawing). A space in which a carrying device 5 is provided is adjacent to the right side from the parking position of the vehicle 1. When the vehicle 1 is parked or taken out, the carrying device 5 carries a pallet 2 into or out of the object region A from a storage space (not shown) through a carrying port (not shown). The fixed position of the pallet 2 in the object region A is the central portion of the object region A.

[0039] As shown in FIG. 2, a guide mirror 6 is provided on the inner side of the partition wall, at a position opposite to the vehicle 1 parked on the pallet 2, from the parking / taking-out port 3. The guide mirror 6 assists the view of the driver of the vehicle 1 when the vehicle 1 is parked or taken out. An entrance / exit 7 for the driver, passenger, or a person in charge of the facility, or the like, is provided at a position on the side of the guide mirror 6 from the parking / taking-out port 3. Figure 2

[0040] The door provided at the entrance / exit 7 is made of a material such as glass that transmits light, so that the inside can be visually recognized from the outside of the entrance / exit 7. A window 8 is provided above the guide mirror 6.

[0041] ​Here, in the present specification, an optical opening portion capable of visually recognizing the outside from the inside of the subject region is referred to as a "transmissive portion". That is, in the present embodiment, the entrance / exit 7 and the window 8 correspond to the transmissive portion, but in addition, for example, a portion partitioning between an adjacent space with a fence or the like can also be assumed as the "transmissive portion". Further, the entrance / exit 3 can also be defined as the "transmissive portion" (however, in the foreign matter detection system described in the present disclosure, one of the main features is that the polarizing filter 10 is provided in the transmissive portion as described later. Although the entrance / exit 3 can be referred to as the "transmissive portion", it is an opening portion through which the vehicle 1 enters and exits, and thus it is not suitable for providing the polarizing filter 10 as in the entrance / exit 7 and the window 8).

[0042] Further, in the present specification, the "foreign matter" refers to an object other than the target object (in this case, the vehicle 1 being carried) present in the subject region, and is an object other than the object originally disposed in the subject region, and typically a person such as a driver, a passenger of the vehicle, a supervisor of the facility. Further, luggage or the like brought into the subject region A and placed outside the vehicle 1 is also cited as an example of the "foreign matter". In the present embodiment, the "foreign matter" is an object other than the target object present in the subject region, and is an object other than the object originally disposed in the subject region. Figure 1 In the present embodiment, as one example, one foreign matter F is illustrated inside the subject region A. Further, two objects O are illustrated outside the subject region A.

[0043] A camera or the like, that is, an imaging device 9 that takes an image inside the subject region A is disposed in the subject region A. In the case of the present embodiment, as illustrated in FIG. 1, a total of three imaging devices 9a to 9c are provided in the vicinity of the stop position of the vehicle 1. When looking into the subject region A from the entrance / exit 3, the imaging device 9a is disposed on one side (in the example illustrated in the present figure, the right side) of the left and right sides further inside than the stop position of the vehicle 1. The imaging devices 9b, 9c are disposed at positions inside the subject region A further forward than the vehicle 1 and on the left and right sides of the entrance / exit 3. The height at which each imaging device 9 is provided is in the vicinity of the ceiling of the subject region A. Figure 1

[0044] As illustrated by the broken lines in FIG. 1, each of the imaging devices 9a to 9c has a viewing angle of about 90° or so. The imaging device 9a provided on one side (the right side) of the left and right sides further inside than the stop position of the vehicle 1 when looking from the entrance / exit 4 is disposed so as to take in the space on the other side (the left side) of the left and right sides into the viewing angle. The imaging device 9b disposed on the left side when looking from the entrance / exit 4 is disposed so as to take in the space on the right side of the subject region A from the imaging device 9b into the viewing angle. The imaging device 9c disposed on the right side when looking from the entrance / exit 4 is disposed so as to take in the space on the left side of the subject region A from the imaging device 9c into the viewing angle. Figure 1

[0045] ​​Thus, three cameras 9 are disposed around the vehicle 1 as a shield so as to be able to acquire images of the subject area A without blind spots. Also, as will be described later, the angle of view of each camera 9 is made to be as little as possible to include the entrance / exit opening 3.

[0046] From the entrance / exit opening 3, the cameras 9b, 9c on the front side are disposed so as to face the entrance / exit opening 7 and the window 8 as the transmission portions, and the entrance / exit opening 7 and the window 8 are included in the angle of view. Also, a polarizing filter 10 is provided to each of the cameras 9b, 9c and the entrance / exit opening 7 and the window 8 so as not to include the outside of the entrance / exit opening 7 and the window 8 in the images acquired by the cameras 9b, 9c. The effect of the polarizing filter 10 will be described later in detail.

[0047] The partition surface near the entrance / exit opening 6 is provided with an operation portion 11 and a display portion 12. The operation portion 11 is an input device for inputting operations of each portion of the system, such as opening and closing of the entrance / exit opening 3 and transporting of the pallet 2. The display portion 12 is a display configured to display images of the subject area A acquired by the cameras 9 as needed. Also, in the case where something that should call the attention of a person occurs in the subject area A or the facility including the subject area A, the display portion 12 also issues an alarm. The something that should call the attention of a person is, for example, a case where a foreign object F other than the vehicle 1 is detected in the subject area A when the vehicle 1 is transported to a storage space outside the subject area A. The alarm issued by the display portion 12 is, for example, display of a warning message. In addition, a device having an alarm function capable of calling the attention of a person, such as an alarm sound and a warning light, can be separately provided.

[0048] Further, the positions where the operation portion 11 and the display portion 12 are provided are not limited to the positions shown here. For example, devices equivalent to these can be provided in a management room or the like not shown, and any one of these devices can be provided at a plurality of different places. In addition, as to the operation portion 11 or the display portion 12, appropriate changes can be made in accordance with the layout of the subject area A or other conditions.

[0049] Figure 3 An example of the system configuration related to the control of the cameras 9, the operation portion 11, and the display portion 12 is shown. The cameras 9, the operation portion 11, and the display portion 12 are informationally connected to a control device 13 via a communication bus such as Ethernet (registered trademark). The control device 13 is a device that monitors the entire mechanical parking lot including the subject area A as the entrance / exit space, and performs the operation of each portion, and controls opening and closing of the entrance / exit opening 3, transporting of the pallet 2, and operation of the cameras 9.

[0050] Further, the display section 12 is connected to the communication bus via the foreign matter detection section 14. The foreign matter detection section 14 is a section that performs foreign matter detection in the subject region, and in the case of the present embodiment, is configured as an information processing device that appropriately processes the image of the subject region A acquired in the imaging device 9. Further, the foreign matter detection section 14 also sends image data to the display section 12 and displays the image in the display section 12 as necessary.

[0051] With regard to detection of the foreign matter F (refer to Figure 1 ), image processing performed in the foreign matter detection section 14 is particularly assumed in the present embodiment. That is, the image of the subject region A acquired by the imaging device 9 is analyzed in the foreign matter detection section 14, and it is determined whether or not the foreign matter F such as a person is present in the image. When the foreign matter F is detected from the image that is the processing target in the foreign matter detection section 14, or the like, the control device 13 displays a message in the display section 12 as necessary.

[0052] A program that detects the foreign matter F photographed in the image is stored in the foreign matter detection section 14. The program is configured to detect a specific kind of object by a deep learning method using a multilayer neural network. As an algorithm of deep learning, various algorithms can be used, such as R-CNN (Regional-Convolutional Neural Network), Fast R-CNN, Faster R-CNN, YOLO (You Only Look Once), and SSD (Single Shot Multibox Detector), but other appropriate algorithms can also be used.

[0053] The program stored in the foreign matter detection section 14 determines the kind of the object photographed in the image using such an algorithm, and can output the probability that the object is the kind of object, the coordinates of the center of gravity of the object, the width and height of the object, and the like. The kind of the foreign matter F assumed in the subject region A, which is the entry / exit space of the mechanical parking lot, is first a person, so the program stored in the foreign matter detection section 14 should be configured to be able to at least distinguish a person from the image. Further, as objects that can enter the subject region A together with the vehicle 1, animals such as pets, handbags, strollers, and various objects are assumed, so the program can also detect these objects in addition to a person. Further, it can be considered that the kind of the object detected as a foreign matter differs depending on the kind of the subject region, which is the setting object of the foreign matter detection system. The program is not limited to detecting a person, and can be configured to be able to detect various objects assumed depending on the kind of the subject region.

[0054] The configuration and the effect of the polarizing filter 10 provided to the camera 9 and the entrance / exit 7 and the window 8 as the transmission portion will be described. In the present embodiment, the cameras 9b, 9c are provided at positions opposite to the entrance / exit 7 and the window 8, and the entrance / exit 7 and the window 8 enter the angle of view of the cameras 9b, 9c.

[0055] As shown in FIG. 9, from the inside of the target region A, there is a foreign object F in front of the entrance / exit 7, and in the case where there are two objects O outside the target region A and inside the entrance / exit 7 and the window 8, if the entrance / exit 7 and the window 8 are visually observed from the inside of the target region A with the naked eye, it is seen as shown in FIG. 10, for example. If such an image is taken as it is by the cameras 9b, 9c, and the foreign object detection is performed by the program in the foreign object detection section 14 as described above, it is needless to say that the foreign object F in front of the entrance / exit 7 is detected as a foreign object, but it is also possible that the object O inside is also detected as a foreign object. It is not a problem that the foreign object F existing in the target region A is detected as a foreign object, but if the object O existing outside the target region A is detected, it becomes over-detection. Figure 1 Figure 4 As shown in FIG. 9, from the inside of the target region A, there is a foreign object F in front of the entrance / exit 7, and in the case where there are two objects O outside the target region A and inside the entrance / exit 7 and the window 8, if the entrance / exit 7 and the window 8 are visually observed from the inside of the target region A with the naked eye, it is seen as shown in FIG. 10, for example. If such an image is taken as it is by the cameras 9b, 9c, and the foreign object detection is performed by the program in the foreign object detection section 14 as described above, it is needless to say that the foreign object F in front of the entrance / exit 7 is detected as a foreign object, but it is also possible that the object O inside is also detected as a foreign object. It is not a problem that the foreign object F existing in the target region A is detected as a foreign object, but if the object O existing outside the target region A is detected, it becomes over-detection.

[0056] In order to prevent over-detection, a method of performing the foreign object detection by masking the region corresponding to the transmission portion (the entrance / exit 7 and the window 8) in the image as shown in FIG. 11, for example, can be considered. However, in such a case, a part of the foreign object F in front of the entrance / exit 7 is also masked, and it is possible that it is not detected even if the foreign object F exists in the target region A. Figure 5 In addition, in addition to the masking process as shown in FIG. 11, various configurations such as determination of the inside and the outside of the target region A in the image (for example, a depth image sensor is separately provided, or another determination algorithm is introduced in the image processing process, etc.) can be adopted. However, even if such a process is assumed, it is not always possible to correctly determine the inside and the outside of the target region A, and in addition, there are disadvantages that other costs occur when the sensor or the like is provided, or the calculation load related to the foreign object detection section 14 (refer to FIG. 12) is increased.

[0057] Figure 5 Figure 3

[0058] ​​​​Alternatively, the illustration is omitted, but it is also considered that the camera 9 is disposed toward the lower side in the high place within the object region A, and the object region A is imaged from the upper side, thereby removing the outside of the object region A from the visual field, and covering the entire region within the object region A. However, the ceiling of the entrance / exit of the mechanical parking lot is generally low, and thus it is difficult for the camera 9 disposed toward the lower side to take an image of a sufficiently large range. In addition, even if it is assumed that the camera 9 can be disposed in the high place, there occurs a problem that the maintenance of the camera 9 must be performed in the high place. Furthermore, if some structures (for example, a mechanism for carrying the pallet 2) and the like are located between the floor of the object region A and the height at which the camera 9 is disposed, it is also possible that they become an obstruction and cannot image the entire region within the object region A. In addition, if the case where a person within the object region A is detected as a foreign object F is considered, the image of the person imaged from the upper side is insufficient as a feature amount of the person, and thus there is also a problem that the detection of the mechanical becomes difficult.

[0059] Therefore, in the present embodiment, as shown in Figure 1 the entrance / exit 7 and the window 8, and the cameras 9b, 9c opposed to them, respectively, to prevent light from reaching the cameras 9b, 9c on the inner side of the entrance / exit 7 and the window 8 from the outside space. That is, with respect to the light path from the entrance / exit 7 and the window 8 to the cameras 9b, 9c, the polarizing filters 10 are provided at positions on the front side and the inner side of the cameras 9b, 9c, respectively, and are arranged so that the polarization directions intersect each other, thereby masking a part of the light from the entrance / exit 7 and the window 8 to the cameras 9b, 9c. The polarizing filters 10 on the front side and the inner side can be arranged at appropriate positions of the light path from the transmission portion to the camera 9 in consideration of the actual spatial structure of the object region A and the like, but in this example, the case where the polarizing filters 10 on the front side and the inner side are provided in front of the cameras 9b, 9c and the entrance / exit 7 and the window 8 is illustrated. Further, hereinafter, for the purpose of explanation, the polarizing filter 10 illustrated in front of the camera 9b, 9c will be referred to as the polarizing filter 10a on the front side, and the polarizing filter 10 illustrated in the entrance / exit 7 and the window 8 will be referred to as the polarizing filter 10b on the inner side, as needed.

[0060] The polarizing filter 10 is a board-like or film-like article that allows only polarized light in a specific direction to pass. When the polarizing filter 10 is provided at the cameras 9b, 9c and the entrance / exit 7 and the window 8, for example, the polarizing filter 10 is arranged so as to block the front of the lens of the camera 9b, 9c, or a film-like polarizing filter 10 is attached so as to cover the entrance / exit 7 and the window 8. Further, it is also possible to constitute the following portions from a polarizing plate or the like: the components that constitute the camera 9b, 9c and the entrance / exit 7 and the window 8, or the raw material itself (for example, the lens or the lens cover provided in the camera 9b, 9c, the glass that constitutes the entrance / exit 7 and the window 8, and the like).

[0061] By so doing, when a person wants to visually recognize the inside and outside of the target region A through the entrance 7 and the window 8, the field of view is not blocked. That is, as shown in FIG. 6, the outside of the target region A can be visually recognized from the inside through the entrance 7 and the window 8, and the inside can be visually recognized from the outside as well. Figure 4

[0062] However, if an image inside the target region A is to be acquired by the imaging devices 9b, 9c, light from the outside space passing through the entrance 7 and the window 8 is blocked by the two polarizing filters 10 arranged on the optical path. Thus, as shown in FIG. 7, an image in which only the outside of the entrance 7 and the window 8 is entirely black is acquired. In this image, unlike the case where the masking process is performed as shown in FIG. 6, the object O on the inside of the entrance 7 and the window 8 is masked, but the foreign object F in front of the entrance 7 and the window 8 is not masked. Thus, if the foreign object detection section 14 performs foreign object detection based on the image acquired by the process, it is possible to accurately determine the presence or absence of the foreign object F without missing detection or false detection. Figure 6 Figure 5

[0063] Here, the respective polarization directions do not need to be strictly orthogonal. In a case where an image in which the entrance 7 and the window 8 are captured is acquired by the imaging devices 9b, 9c, and foreign object detection is to be performed by the foreign object detection section 14, it is sufficient that light from the entrance 7 and the window 8 to the imaging devices 9b, 9c is blocked to such an extent that the object O on the inside of the entrance 7 and the window 8 cannot be recognized.

[0064] Further, as described above, with respect to the entrance / exit 3 which can be defined as a "transmissive portion" identical to the entrance 7 or the window 8, the vehicle 1 is to enter and exit, and thus it is difficult to cover the entire surface with a polarizing filter. Thus, assuming a case where the entrance / exit 3 is included in the angle of view of any of the imaging devices 9, and a case where an object such as a person exists on the outside of the entrance / exit 3, it is possible that it is detected as a foreign object.

[0065] In the present embodiment, in response to this problem, the arrangement of the imaging devices 9 is made. That is, as described above, the imaging device 9a on the inside and the left side is arranged to include the space on the right side in the angle of view from the entrance / exit 3, and the imaging devices 9b, 9c on the front side are arranged to respectively include the space on the right side or the left side in the angle of view from the entrance / exit 3 (see FIG. 4). Figure 1

[0066] ​​​​Thus, if three cameras 9 are configured, the entire region within the target region A can be captured without blind spots, and the loading / unloading port 3 does not enter the angle of view of each camera 9, or even if it does, it is set to the edge within the angle of view. Thus, the loading / unloading port 3 is not captured in the image taken by each camera 9, or even if it is, it is the end of the image. If the position at which the loading / unloading port 3 is captured is the end, even if, for example, masking processing is performed on the region within the image corresponding to the loading / unloading port 3, the possibility of missing detection due to the foreign object F located within the target region A being masked is relatively low.

[0067] Thus, in the present embodiment, for the transmission portions (exit / entrance 7, window 8) opposite the cameras 9, light is physically masked by the polarizing filter 10, and for the transmission portion (loading / unloading port 3) for which a polarizing filter 10 cannot be provided, it is removed as much as possible from the central portion of the angle of view of the camera 9. By thus processing, foreign object detection can be performed with high accuracy, and the possibility of over-detection due to an object located outside the transmission portion (exit / entrance 7, window 8, loading / unloading port 3) being captured within the image, or missing detection due to masking processing, etc., is excluded.

[0068] Further, the guide mirror 6 reflects light from the outside that passes through the loading / unloading port 3, so the possibility that an object located outside the loading / unloading port 3 is reflected into the guide mirror 6 and the camera 9b, 9c takes the image as an image also needs to be considered. However, even if an object outside enters the angle of view of the camera 9b, 9c via the guide mirror 6 in that form, if the size or shape of the object that can be captured within the image is considered, it is actually considered that the possibility that the foreign object detection portion 14 detects the object within the image with high accuracy is relatively low.

[0069] However, when a guide mirror 6 of a large size is provided, etc., there is also a concern that reflection into the guide mirror 6 can involve over-detection. In this case, a polarizing filter can also be provided separately to the guide mirror 6, and light that reaches the camera 9b, 9c after being reflected by the guide mirror 6 can be masked. As explained above, the foreign object detection system disclosed in the present specification is characterized in that two polarizing filters 10 are provided on the optical path of light from the transmission portions 7, 8 to the camera 9, the image of an object located further forward than the polarizing filter 10b on the inner side is taken, and on the other hand, the inner side is masked. As long as such an effect is obtained, the configuration of the polarizing filter 10 can be appropriately changed. In Figure 1The polarizing filter 10 is provided at the entrance / exit 7 and the window 8, but it is also not a problem to provide the polarizing filter 10 at the guide mirror 6 which is in the light path. Further, the polarizing filter 10 can be provided at other positions, but in practice, it is considered to be convenient to provide it at the entrance / exit 7 or the window 8 which is at the boundary of the subject area A and the outside space, or at the guide mirror 6 which forms a refracting surface of light from the outside.

[0070] When the vehicle 1 is parked in the subject area A which is the parking / exit space of the mechanical parking lot, the driver, the passenger, the supervisor of the facility, or the like operates the operation section 11 to instruct the start of parking. The control device 13 opens the parking / exit opening 3 in accordance with the instruction. When the parking is started, the parking / exit opening 3 is opened in a state in which the empty pallet 2 is arranged at a fixed position, i.e., the center portion, in the subject area A.

[0071] If the parking / exit opening 3 is opened, the driver moves the vehicle 1 into the subject area A and stops it on the pallet 2. At this time, the driver can stop the vehicle 1 at an appropriate position on the pallet 2 with the aid of the field of view of the driver by the guide mirror 6. After the stop, the driver or another passenger gets off and exits from the subject area A through the entrance / exit 7. When the driver and the like have all exited, the driver and the like visually confirm the inside of the subject area A through the entrance / exit 7, and confirm whether or not there is a foreign object F such as a person or baggage in the subject area A, and then operates the operation section 11 to input the gist that the vehicle 1 has been properly stopped on the pallet 2. The control device 13 closes the parking / exit opening 3 and operates the carrying device 5 to carry the pallet 2 on which the vehicle 1 is carried to the storage space outside the subject area A.

[0072] When the vehicle 1 is parked in the subject area A which is the parking / exit space of the mechanical parking lot, the driver, the passenger, the supervisor of the facility, or the like operates the operation section 11 to instruct the start of parking. The control device 13 opens the parking / exit opening 3 in accordance with the instruction. When the parking is started, the parking / exit opening 3 is opened in a state in which the empty pallet 2 is arranged at a fixed position, i.e., the center portion, in the subject area A.

[0073] Further, the above series of operations are described taking the case in which the driver drives the vehicle 1 as an example, but there can be a case in which the vehicle 1 is an automated vehicle. In this case, some or all of the above operations of the vehicle 1 are performed without intervention of the driver.

[0074] Side reference Figure 7 The flowchart above illustrates an example of the sequence of foreign object detection performed in object region A.

[0075] First, in step S1, the closure sensor (not shown) installed at the entrance / exit 7 is used to confirm whether the door of the entrance / exit 7 is closed (step S1). When the door is open, the masking function implemented by the polarizing filter 10 of the entrance / exit 7 does not work well, so step S1 is repeated until it is confirmed that the entrance / exit 7 is closed, and the system remains in standby mode until the closure of the entrance / exit 7 is confirmed. At this time, a message prompting the closure of the entrance / exit 7 can also be displayed on the display unit 12.

[0076] If it is confirmed that entrance / exit 7 is closed, the control device 13 activates each camera device 9 to acquire an image of the target area A (step S2). If an image has been acquired, the foreign object detection unit 14 performs foreign object detection processing on the image acquired by each camera device 9 according to the aforementioned procedure (step S3).

[0077] In step S4, based on the results of the foreign object detection, it is determined whether there is a foreign object in the image. If no foreign object is found in any image acquired by each camera device 9, the confirmation that no foreign object is found is completed (step S5), and the foreign object detection process ends. The control device 13 performs operations such as closing the inbound / outbound port 3 or moving the pallet 2.

[0078] If, in step S4, it is determined that at least one of the images obtained in step S2 contains an object (e.g., a person) of the type assumed to be foreign object F, then proceed to step S11.

[0079] Here, if there is a foreign object in the target area A, assuming that the entrance / exit 7 and window 8, which serve as the transmission part, do not have a polarizing filter 10, then in step S2, for example, the image can be obtained by the camera device 9b. Figure 8 The image shown. In this case, in steps S3 and S4, in addition to detecting the person, i.e., the foreign object F, located in the object area A, there is also a concern that the object O located inside the entrance / exit 7 may also be detected as a foreign object.

[0080] However, in this embodiment, a polarizing filter 10 is provided in the camera device 9b and the entrance / exit 7 and window 8, so the actual light obtained by the camera device 9b in step S2 is, for example... Figure 9 The image shown is shown in the image. In this image, the inside of the entrance / exit 7 and window 8 is blocked by the polarizing filter 10, so only the foreign object F located in the object area A on the front side can be detected as a foreign object.

[0081] In step S11, operations of interlocking equipment are prohibited as needed (e.g., opening and closing of the inlet 4, handling of pallet 2, etc.). Then, step S6 is executed. In step S6, the message indicating that a foreign object has been detected is displayed on the display unit 12. At this time, a message using characters or sound is played, or the background color of the display unit 12 is changed, to draw the attention of the operator (driver of vehicle 1, or facility supervisor, etc.).

[0082] The display unit 12 displays images from those captured by each camera device 9 that are determined to contain foreign objects. Furthermore, if multiple images are found to contain foreign objects, displaying them all on the display unit 12 could be inconvenient. Therefore, the display unit 12 can only display images, for example, those where the detected foreign object is most likely to be of a specific type (in this case, a person).

[0083] like Figure 10 As shown, the results of foreign object detection can be further superimposed on the image displayed in step S6. In the image shown here, a box indicating the location of each object detected as a foreign object, the determined type of the object (here, "person"), and the probability that the object is of that type are all displayed around each object.

[0084] The operator refers to, for example Figure 10 The image shown indicates that the interior of object area A is visually identified from entrance / exit 7, or the presence of foreign objects is confirmed by entering object area A from entrance / exit 7 (step S7). If the confirmation result indicates that there is a foreign object F in object area A, appropriate measures are taken to remove foreign object F (steps S8-S9). If foreign object F is a person, the user is asked to exit; if it is an object, it is removed. If foreign object F is not present, it is confirmed that foreign object F is not present, and this information is input to operation unit 11 (step S10). If the input indicating that foreign object F is not present is made, the operation prohibition state of the device type is released (i.e., the interlocking performed in step S11 is released), and the device type is set to an operable state.

[0085] If the confirmation result in step S7 indicates that there are no foreign objects in object area A, the determination result in the previous step S4 is considered a false detection. For example, in the case of foreign object detection using image recognition, such as when drawing an illustration on the surface of vehicle 1, it may be falsely detected as a person. In such a case, the process moves from step S8 to step S10, and the input confirms that there are actually no foreign objects. If step S10 is entered, the confirmation that there are no foreign objects is completed (step S5).

[0086] Thus, in the process in which a person confirms that there is no foreign matter, if the image within the target region A is displayed at the display portion 12 in the case where it is determined that there is foreign matter, more careful attention can be drawn compared to the case where only a message or the like that prompts confirmation that there is no foreign matter is displayed. This is because the specific state within the target region A is displayed as an image, so the target region A will be confirmed in a form that matches the content. Furthermore, in the case where foreign matter is detected from the image by the program using deep learning, the position and kind of the detected foreign matter within the image are determined mechanically. Therefore, if the image within the target region A that is acquired by the imaging device 9 also displays information such as the position or kind of the determined foreign matter, the confirmer will confirm the target region A while considering the kind, position, and the like of the foreign matter, and more thorough confirmation and appropriate handling for removing the foreign matter F can be performed.

[0087] In addition, in the present embodiment, when it is determined that there is foreign matter F (step S4), the operator confirms that there is no foreign matter within the target region A, and then inputs the gist that there is no foreign matter to the operation portion 11 (step S10). In the case where foreign matter F is detected, the system side cannot confirm the presence or absence of the mechanically detected foreign matter F, or cannot perform such processing for removal. Therefore, after a person confirms that there is no foreign matter and performs appropriate handling according to the situation, the gist that there is no foreign matter is input to the mechanical side, so that it is possible to confirm that there is no foreign matter thoroughly.

[0088] In addition, in the present embodiment, when it is determined that there is foreign matter F (step S4), the operation of the device class that constitutes the system is prohibited (step S11), and thereafter, in the case where the gist that there is no foreign matter is input to the operation portion 11, the interlock is released accordingly (step S10). With this processing, a person confirms that there is no foreign matter, and the operation prohibition state of the device class is released by inputting the gist, so it is possible to ensure safety when the operation of the device class is restarted.

[0089] Further, in the case where it is determined that there is foreign matter within the plurality of images acquired by the imaging device 9, and in the case where it is determined that there are a plurality of foreign matters in the same image, the operator can be prompted to pay attention to each image or each object in order to perform the processes of steps S7 to S10. For example, an order can be adopted in which an [OK] button is displayed for each object, and after the confirmer confirms the target region A, if the [OK] button is input for one object, the [OK] button is displayed together with the next object.

[0090] The foreign matter detection process as above can be performed at an appropriate timing. For example, at the time of entry, the vehicle 1 stops on the pallet 2, and the driver inputs the gist to the operation section 11, and the foreign matter detection is performed at this time. At this time, the driver confirms the state in the object region A by visual confirmation, but if the foreign matter detection is mechanically performed by the foreign matter detection section 14 at the same time, it is possible to confirm thoroughly. Further, for example, the foreign matter detection can be performed at an arbitrary timing when it is considered that the state in the object region A needs to be confirmed, such as when a start of the delivery instruction is input, or when the entry / delivery port 3 is closed after the delivery is completed, or at a time of ordinary when no special operation is performed. In addition, regarding the acquisition of the image and the image processing (steps S2, S3), a use mode in which the continuous execution at some operation time or at a time of ordinary, and the issuance of the alarm once the foreign matter is detected can be considered.

[0091] Further, in the case where the foreign matter detection process as shown in Figure 7 is performed, an appropriate structure can be adopted as the foreign matter detection section 14. That is, in the present embodiment, the case where the foreign matter F is detected based on the image in the object region A acquired by the imaging device 9 is particularly described, but the structure of the foreign matter detection section 14 is not limited thereto. As long as it is a device such as a range sensor that can detect the foreign matter F in the object region A, it can be adopted as the foreign matter detection section 14. Further, it is sufficient that the information of the detected foreign matter F is displayed together with the image acquired by the imaging device 9 on the display section 12.

[0092] While comparing with the above patent documents 1 to 7, the advantageous aspects of the foreign matter detection system as in the present embodiment are further described. First, when the system based on the image detection of the foreign matter F is constructed, the following points are important.

[0093] A) Ensure sufficient detection performance. Reduce both false detection and missed detection. A stationary object can also be detected.

[0094] B) The system is composed of a small number of device types. As much as possible, no sensor or the like other than the imaging device is provided.

[0095] C) As much as possible, the appearance of the device is not affected.

[0096] In the patent document 1, a safety confirmation system is described, which is configured to display an image of a passenger compartment photographed by a camera on a touch panel type operation screen, and to input a safety confirmation button after a user confirms safety in the passenger compartment. In addition to the image information of the passenger compartment, the detection information of a sensor that detects the presence of a person or the like is also displayed on the operation screen. That is, in the case of the technology described in the patent document 1, when the presence of the foreign matter is mechanically detected, not only the image of the camera but also a plurality of sensors need to be provided separately, and the system structure becomes complicated. That is, the condition of the above B is not satisfied.

[0097] In the technology described in Patent Literature 2, a camera is provided at a parking / exit parking space in a mechanical parking lot, and pixel data of an image obtained is divided into a plurality of groups. Further, in a case where color change occurs between pixel data at a certain position at a certain time and pixel data at the same position at another time, it is determined that an object exists. In such a method, time variation is required when detecting an object, and there is a concern that a stationary foreign object cannot be detected. That is, the condition of the above A is not satisfied.

[0098] In the technology described in Patent Literature 3, an illuminating device is provided in a parking / exit parking space, and a person or an animal is detected by a shadow of movement or non-movement. Not only a stationary object cannot be detected, but also an illuminating device is required, and thus the conditions of the above A and B are not satisfied. In addition, if the influence of sunlight or a vehicle headlight is considered, there is a problem in that light of a considerable intensity needs to be irradiated.

[0099] In the technology described in Patent Literature 4, an initial parking room image of a vehicle entering and a subsequent image are compared, and in a case where the amount of change exceeds a set value, it is considered that a person or an animal exists. However, in such a method, it is easily affected by environmental light such as light irradiated from sunlight, a headlight of a subsequent vehicle, a surrounding building, and the like, and is easily affected by noise of a camera and the like, and it is conceivable that there is a tendency to have false detection. In addition, it is necessary to obtain an initial image at an appropriate timing (between when a vehicle stops and a headlight is turned off and when a door of the vehicle is opened), but it is difficult to determine such a timing from an image of a camera. Further, even if an image at an appropriate timing can be obtained as an initial image, since a passenger exists in the vehicle at the timing, it is considered that whether or not the passenger is obvious differs when compared with a subsequent image. That is, the condition of the above A is not satisfied.

[0100] In the technology described in Patent Literature 5, a floor is alternately painted in two different colors, and a color discontinuous portion in an image obtained by a camera from above is detected as a foreign object. Since a striped pattern is drawn on the floor, there is a problem that, in addition to the above C point being unsatisfactory, a cost of periodic maintenance occurs as the floor is contaminated.

[0101] In the technology described in Patent Literature 6, a person is detected from a video of a camera, and in a case where the number of people getting off and the number of people exiting do not agree, operation of an operation panel is disabled. However, since a person getting off usually pays attention to the floor, it is considered that it is difficult to obtain an image in which a face of the person getting off can be mechanically recognized by a camera, and the above A is not satisfied. In addition, since face recognition and matching are performed every time a person enters and exits, complicated calculation occurs every time.

[0102] In Patent Literature 7, a system of an unmanned parking lot and remote monitoring is described, but the user determines the foreign matter by visual observation, or an operator at a remote location determines the foreign matter by the image of a CCD camera, which does not satisfy the above-described A.

[0103] In contrast, in the foreign matter detection system of the present embodiment, the weaknesses of the technologies described in the above-described patent literatures can be solved by detecting a specific kind of foreign matter from the image. That is, even a stationary foreign matter can be detected, changes in light conditions due to ambient light and the like can be resisted, and there is no need to acquire an image to be a reference. By physically masking the transmission portion (entrance and exit 7, window 8) using the polarizing filter 10, false detection can be suppressed without complex calculations such as determination of the inside and outside. Furthermore, visual recognition by the naked eye through the transmission portion is not affected. In addition, as a mechanical minimum in the object area A, the camera 9 is sufficient, and the impact on design is slight.

[0104] Furthermore, in the present embodiment, the entrance / exit space of a mechanical parking lot is assumed as the object area A, but as long as it is a space having a transmission portion with the outside, the same foreign matter detection system can be applied regardless of the kind. For example, it can be assumed as a power supply device for an electric vehicle and the like, and in addition to the space for handling vehicles, it can be applied to safety assurance or security in some spaces.

[0105] For example, in a production device using a robot and the like, the operation area of the robot arm is sometimes surrounded by a transparent acrylic plate or the like. In such a place, in order to ensure safety in the operation area, a camera is provided in the operation area, and a polarizing filter is provided in the camera and the acrylic plate, so that visual recognition through the acrylic plate from the outside is possible, and a foreign matter detection based on the image of the camera is also possible. Or, when monitoring an elevator having a transparent partition, a polarizing filter is also considered to be provided in the camera and the partition to prevent false detection.

[0106] As above, the foreign matter detection system of the present embodiment is configured to include: the imaging device 9 that acquires an image of the subject region A; the transmission portion 7, 8 that can visually recognize the outside from the inside of the subject region A; and the front-side polarizing filter 10a that is in front of the imaging device 9 and the back-side polarizing filter 10b that is behind the imaging device 9 in a light path from the transmission portion 7, 8 to the imaging device 9, the polarization directions of the front-side polarizing filter 10a and the back-side polarizing filter 10b intersecting each other, the foreign matter F in the subject region A being detected on the basis of the image acquired by the imaging device 9. With this processing, light from the outside space is masked by the back-side polarizing filter 10b without affecting the visual recognition by the naked eye through the transmission portion 7, 8, and on the other hand, an image of an object in front can be acquired. Moreover, if the foreign matter detection is performed by the foreign matter detection portion 14 on the basis of the image thus acquired, it is possible to accurately determine the presence or absence of the foreign matter F without missing detection or over-detection.

[0107] In the foreign matter detection system described above, the back-side polarizing filter 10b is provided at the transmission portion 7, 8 at the boundary between the subject region A and the outside space. With this processing, the back-side polarizing filter 10b is easily arranged.

[0108] The foreign matter detection system described above is configured to detect the foreign matter F in the image acquired by the imaging device 9 by a program using deep learning. With this processing, in a system in which foreign matter detection is performed by deep learning, the foreign matter detection can be appropriately performed.

[0109] In the foreign matter detection system described above, the subject region A can be set as a parking-in / parking-out space of a mechanical parking lot, and with this processing, the foreign matter detection can be appropriately performed in the parking-in / parking-out space of the mechanical parking lot.

[0110] In addition, the foreign matter detection system of the present embodiment described above is configured to include: the imaging device 9 that acquires an image of the subject region A; the display portion 12 that displays the image acquired by the imaging device 9; the operation portion 11 that inputs operations of each portion constituting the system; and the foreign matter detection portion 14 that detects the foreign matter F in the subject region A, the information of the detected foreign matter F being displayed together with the image acquired by the imaging device 9 on the display portion 12 in a case where it is determined by the foreign matter detection portion 14 that the foreign matter F is present. With this processing, a person confirms the subject region A by considering the kind, position, and the like of the foreign matter F, and thus thorough confirmation and appropriate disposal for removing the foreign matter F can be performed.

[0111] The above foreign matter detection system is configured such that, in a case where it is determined by the foreign matter detection section 14 that a foreign matter F is present, the operator confirms that no foreign matter is present in the subject region A, and then can input a subject in which no foreign matter is present to the operation section 11. In this way, it is possible to confirm that no foreign matter is present.

[0112] The above foreign matter detection system is configured such that, in a case where it is determined by the foreign matter detection section 14 that a foreign matter F is present, operation of the device class that constitutes the system is prohibited, and thereafter, in a case where a subject in which no foreign matter is present is input to the operation section 11, the operation prohibition state of the device class that constitutes the system is released. In this way, it is possible to ensure safety with respect to the operation of the device class.

[0113] Therefore, according to the above embodiment, it is possible to detect a foreign matter with high precision by a structure that is as simple as possible.

[0114] Further, the foreign matter detection system described in the present disclosure is obviously not limited to the above embodiment, and various modifications can be added within the scope of the gist.

[0115] Explanation of Reference Signs

[0116] 1 vehicle; 2 pallet; 3 loading / unloading port; 5 carrying device; 6 guide mirror; 7 transmission section (entrance / exit); 8 transmission section (window); 9 camera device; 9a camera device; 9b camera device; 9c camera device; 10 polarizing filter; 10a polarizing filter (polarizing filter on the front side); 10b polarizing filter (polarizing filter on the back side); 11 operation section; 12 display section; 13 control device; 14 foreign matter detection section; A subject region; F object (foreign matter); O object.

Claims

1. A foreign object detection system configured to have: a plurality of imaging devices that acquire images of an object region and are arranged so that a warehouse / vehicle exit does not enter a visual angle of each imaging device or so that an edge of the warehouse / vehicle exit is within the visual angle even if the warehouse / vehicle exit enters the visual angle; a transmission portion through which an outside of the object region is visually recognized from an inside of the object region; and a polarizing filter that is on a front side of the imaging device in a path of light from the transmission portion to the imaging device and a polarizing filter that is on a back side of the imaging device, the polarizing filters having polarization directions that intersect each other, wherein the imaging device of the plurality of imaging devices that is on one of left and right sides on a back side of a stop position of a vehicle from the warehouse / vehicle exit is configured to include a space on the other of the left and right sides in the visual angle, the imaging device of the plurality of imaging devices that is arranged on a left side from the warehouse / vehicle exit is configured to include a space on a right side of the object region from the imaging device in the visual angle, the imaging device of the plurality of imaging devices that is arranged on a right side from the warehouse / vehicle exit is configured to include a space on a left side of the object region from the imaging device in the visual angle, and a foreign object in the object region is detected on the basis of the images acquired by the plurality of imaging devices. The polarizing filter on the back side is provided at the transmission portion that is a boundary between the object region and an outside space.

3. The foreign object detection system according to claim 1 or 2, configured to detect a foreign object in an image acquired by the imaging device by a program using deep learning. The object region is a warehouse / vehicle exit space of a mechanical parking lot. The object region is a warehouse / vehicle exit space of a mechanical parking lot. The object region is a warehouse / vehicle exit space of a mechanical parking lot.

7. A foreign object detection system configured to have: a plurality of imaging devices that acquire images of an object region and are arranged so that a warehouse / vehicle exit does not enter a visual angle of each imaging device or so that an edge of the warehouse / vehicle exit is within the visual angle even if the warehouse / vehicle exit enters the visual angle; a display portion that displays the images acquired by the plurality of imaging devices; an operation portion that inputs an operation of each portion constituting the system; and a foreign object detection portion that detects a foreign object in the object region by a program generated by a deep learning method on the basis of the images acquired by the plurality of imaging devices, wherein in a case where it is determined by the foreign object detection portion that a foreign object exists, information of the detected foreign object is displayed on the display portion in a vicinity of the foreign object in the image acquired by the plurality of imaging devices.

8. The foreign object detection system according to claim 7, configured so that, in a case where it is determined by the foreign object detection portion that a foreign object exists, an operator can input a theme that there is no foreign object in the object region to the operation portion in addition to confirmation that there is no foreign object in the object region.

2. The foreign object detection system of claim 1, wherein, 9. The foreign object detection system according to claim 8, configured so that: in a case where it is determined by the foreign object detection portion that a foreign object exists, an operation of a device type constituting the system is prohibited, ​ 4. The foreign object detection system of claim 1, wherein, ​ 5. The foreign object detection system of claim 2, wherein, ​ 6. The foreign object detection system of claim 3, wherein, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Thereafter, in a case where a subject in which no foreign matter is present is input to the operation section, the operation prohibition state of the device class constituting the system is released.

Images