AI-based judgment systems, and forklifts

Abstract

To achieve "downsizing" and "simplification" of a system determining the existence of a person based on the presence or absence of parts of the person in a prescribed area of an image.SOLUTION: A determination system 1 to determine a person in an image comprises a recognition unit 2 to recognize the person in the image, and a determination unit 3 to determine that a person is present in an image when there is the upper part of the person in an upper determination area of the image and when there is the lower part of the person in a lower determination area of the image. Each determination area includes a dangerous area and a detection area. The determination unit 3 determines there is the person closer when there is a part of the person in the dangerous area comparing with when there is a part of the person in the detection area. The determination unit may comprise a notification unit 4, a storage unit 5, and a communication unit 6. A forklift 10 equipped with the determination system 1 determines the person in the image by the determination unit 3 only when the forklift moves backwards. The determination unit 3 may determine the traveling direction and the turning direction of the forklift 10 based on the image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a determination system for determining the presence of a person in an image captured by a camera, and a forklift provided with this determination system.

Background Art

[0002] Conventionally, a human body detection device for detecting a human body based on visual data has been known (see Patent Document 1). This human body detection device includes an imaging unit that captures an image, a thermal information detection unit that detects information regarding a thermal distribution, detection range setting means that is arranged so as to overlap the imaging range in the imaging unit and the thermal detection range in the thermal information detection unit, and human body extraction means that extracts, as a human body extraction range, an imaging area corresponding to an area in the thermal detection range where the amount of heat possessed by a human body is detected, from the captured image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the human body detection device described in Patent Document 1 must include both a camera as an imaging unit and a thermopile array as a thermal information detection unit, so the system becomes large-sized. Further, since it is necessary to accurately overlap the imaging range and the thermal detection range, there is a problem that the system becomes complicated.

[0005] In view of such points, an object of the present invention is to provide a determination system that realizes "miniaturization" and "simplification" of the system by determining the presence of a person from the presence or absence of the upper part of a person in the upper determination area of an image and the presence or absence of the lower part of a person in the lower determination area of the image, and a forklift. [Means for solving the problem]

[0006] The determination system 1 according to the present invention is a determination system for determining the presence of a person in an image captured by a camera, wherein the image has at least an upper determination area and a lower determination area, and comprises a recognition unit that recognizes a person in the image, and a determination unit that determines that a person is present in the image if the upper part of the person recognized by the recognition unit is within the upper determination area and / or the lower part of the person recognized by the recognition unit is within the lower determination area. The upper determination area includes an upper danger determination area and an upper detection determination area, and the lower determination area includes a lower danger determination area and a lower detection determination area. The determination unit determines that a person is closer to the image when the upper part of a person recognized by the recognition unit is within the upper danger determination area and / or the lower part of a person recognized by the recognition unit is within the lower danger determination area, compared to when the upper part of a person recognized by the recognition unit is within the upper detection determination area and / or the lower part of a person recognized by the recognition unit is within the lower detection determination area. Its first characteristic is this.

[0008] The determination system 1 according to the present invention 2 In addition to the first feature described above, the system also includes a notification unit that notifies the system outside of itself that the determination unit has determined the presence of a person in the image.

[0009] The determination system 1 according to the present invention 3 The distinguishing feature of this system is that, in addition to the first feature described above, it also includes a storage unit for storing at least one of the image itself and a video containing the image, and / or a communication unit for communicating with the outside of the system.

[0010] These features allow the system to maintain accuracy in its determination. For example, if the upper part H1 of a person H is within the upper determination area A1 of image G, or if the lower part H2 of a person H is within the lower determination area A2 of image G, it can determine that a person H is present in image G. Unlike Patent Document 1, the system does not include a thermal information detection unit such as a thermopile array, making it smaller. Furthermore, it does not require the imaging range and thermal detection range to overlap precisely, thus simplifying the system (system "miniaturization" and "simplification"). Furthermore, as will be described later, the recognition unit 2 in the judgment system 1 may recognize a person H in the image G using artificial intelligence (AI), in which case the judgment system 1 can also be said to be an "AI-based judgment system."

[0011] Furthermore, if the upper part H1 of person H is within the upper danger detection area A11 of image G, or if the lower part H2 of person H is within the lower danger detection area A21 of image G, the system will determine that a person is closer to the subject compared to when the upper part H1 of person H is within the upper detection detection area A12 of image G, or when the lower part H2 of person H is within the lower detection detection area A22 of image G. This improves the accuracy of determining the presence of person H within image G.

[0012] Furthermore, if a notification unit 4 is also provided, it will be possible to inform the user of the judgment system 1 (for example, the driver of the forklift 10) of the presence of person H, thereby "reducing the degree of risk." If a storage unit 5 and a communication unit 6 are also provided, further "improvement of judgment accuracy" can be achieved based on the stored and communicated image G data.

[0013] The forklift 10 according to the present invention is a forklift equipped with the above-described determination system, wherein the forklift is equipped with a camera that captures the image, and the determination unit determines the presence of a person in the image only when the forklift is moving in the reverse direction.

[0014] A second feature of the forklift 10 according to the present invention is that, in addition to the first feature described above, the determination unit determines the direction of travel of the forklift based on the image. In this invention, "direction of travel" may refer to the forklift 10 being in the forward direction (forward state) or the backward direction (reverse state), and may also include a state in which the direction of travel is neither forward nor backward and the forklift is not moving (stopped state).

[0015] A third feature of the forklift 10 according to the present invention is that, in addition to the second feature described above, the determination unit also determines the turning direction of the forklift based on the image. In this invention, "turning direction" may refer to the turning direction of the forklift 10 being to the left (left-turning state) or to the right (right-turning state), and may also include a state in which the turning direction is neither left nor right, and the forklift is not turning (no-turning state).

[0016] These features make it possible to determine the presence of a person H in image G only when the forklift 10 is moving backward, thereby informing the driver that a person is behind the forklift 10 and prompting the driver to pay attention, thus reducing the "risk level" for the forklift 10.

[0017] Furthermore, by determining the direction of travel and turning of the forklift 10 based on image G, it becomes unnecessary to link the determination system 1 with the operation of the shift lever on the forklift 10 or when the gear shift is in rear. This allows for further miniaturization and simplification of the system, and also makes it easier to retrofit the determination system to existing forklifts 10. [Effects of the Invention]

[0018] According to the determination system and forklift of the present invention, the presence of a person can be determined from the presence or absence of the upper part of a person within the upper determination area of ​​the image and the presence or absence of the lower part of a person within the lower determination area of ​​the image, thereby achieving "reduction of risk" and "simplification of the system." [Brief explanation of the drawing]

[0019] [Figure 1] This is a schematic diagram illustrating the determination system according to the present invention. [Figure 2] (a) is a photographic representation of a diagram illustrating the rectangular information based on the recognition result of a person in the recognition unit of the judgment system, the upper judgment coordinates indicating the upper part of the person, and the lower judgment coordinates indicating the lower part of the person. (b) is a photographic representation of a diagram illustrating the image of the danger judgment area and the detection judgment area. In (b), the danger judgment area is labeled as the danger area and the detection judgment area is labeled as the detection area, and the dotted lines indicate the extent of each area. [Figure 3]It is a drawing substitute photograph exemplifying each area set in the image captured by the camera, where (a) shows the upper danger determination area, (b) shows the upper detection determination area, (c) shows the lower danger determination area, and (d) shows the lower danger determination area. [Figure 4] It is a wiring diagram exemplifying the determination system. [Figure 5] It is a schematic diagram exemplifying the forklift according to the present invention provided with the determination system. In the figure, the danger determination area is denoted as the danger area, and the detection determination area is denoted as the detection area. [Figure 6] It is a schematic diagram exemplifying the displacement vector used when determining the traveling direction and turning direction of the forklift. (a) shows the feature points in the image of the front frame (1 frame before), and (b) shows the feature points and the displacement vector in the image of the current frame. [Figure 7] It is a drawing substitute photograph exemplifying the image images of the displacement vector determination areas (left area and right area) in the forklift's determination system. [Figure 8] It is a schematic diagram exemplifying the traveling direction and turning direction of the displacement vector. [Figure 9] It is a schematic diagram exemplifying the image image and table of the displacement vector determination area when the forklift "moves forward without turning". [Figure 10] It is a schematic diagram exemplifying the image image and table of the displacement vector determination area when the forklift "moves backward without turning". [Figure 11] It is a schematic diagram exemplifying the image image and table of the displacement vector determination area when the forklift "moves forward while turning left". [Figure 12] It is a schematic diagram exemplifying the image image and table of the displacement vector determination area when the forklift "moves backward while turning left". [Figure 13] It is a schematic diagram exemplifying the image image and table of the displacement vector determination area when the forklift "moves forward while turning right". [Figure 14]This is a schematic diagram illustrating the image and table of the displacement vector determination area when a forklift is "reversing while turning right". [Figure 15] This is a state transition diagram for a forklift. [Figure 16] The flowcharts illustrate the judgment system, where (a) is a flowchart for determining the presence of a person based on the presence or absence of a part of a person within the hazard judgment area or detection judgment area of ​​an image, and (b) is a flowchart for determining the direction of travel and turning direction of a forklift. [Modes for carrying out the invention]

[0020] Embodiments of the present invention will be described in detail with reference to the drawings. <Overall configuration of judgment system 1> Figures 1 to 16 show the determination system 1 according to the present invention. This determination system 1 is a system that determines the presence of a person H in an image G captured by a camera.

[0021] In the judgment system 1, image G has at least an upper judgment area A1 and a lower judgment area A2. The determination system 1 includes a recognition unit 2 that recognizes a person H within an image G, and a determination unit 3 that determines that a person H exists in the image G if the upper part H1 of the person H recognized by the recognition unit 2 is within the upper determination area A1, and / or the lower part H2 of the person H recognized by the recognition unit 2 is within the lower determination area A2.

[0022] The determination system 1 may also include a notification unit 4 that notifies the system outside of the determination system 1 that the determination unit 3 has determined the presence of a person H in the image G, and may also include a storage unit 5 that stores the image G itself or a video containing the image G, and a communication unit 6, described later, that communicates with the system outside of the determination system 1. Furthermore, the judgment system 1 may include a casing 7 that houses the aforementioned recognition unit 2, judgment unit 3, communication unit 6, etc., a setting unit 8 for setting the ranges of each area A1, A2, etc., and may also include a camera for capturing image G (such as a camera 11 attached to the forklift 10), and the judgment system 1 may be composed of a computer.

[0023] <Recognition part 2> As shown in Figures 1 and 2(a), the recognition unit 2 is the part that recognizes that the person H in the image G described above is a real person. The recognition unit 2 may, for example, recognize that the person H in the image G described above is a real person based on the contour He of the person H, or it may recognize that it is a real person using an object recognition algorithm.

[0024] In addition, the recognition unit 2 may remove noise and distortion from the image G, emphasize the contours He of the person H in the image G, adjust the brightness and color (hue and saturation) of the image G, cut out the area of ​​the person H from the image G (area extraction), or display the extracted person H surrounded by a rectangle (rectangle or square) (display bounding box B). The following description will primarily focus on the assumption that the recognition unit 2 displays a bounding box B surrounding person H.

[0025] In the recognition unit 2, any image processing is acceptable as long as the contour He of person H can be extracted from image G. For example, the contour He of person H may be extracted (by gradient method) from the maximum and minimum values ​​calculated by the first derivative for changes in the pixel value (brightness, etc., at each pixel) of each pixel in image G, or from the inflection points calculated by the second derivative (by Laplacian method). In addition, the contour He of person H may be extracted from image G by smoothing, gradient method using weighted average, or Canny method. Furthermore, the extraction of the outline He of a person H from image G may be performed using artificial intelligence (AI), machine learning (ML), or deep learning (DL).

[0026] Such a recognition unit 2 can be said to be able to automatically learn the features (patterns) of person H and person H as a whole simply by receiving image G as input. Furthermore, the recognition unit 2 may be given multiple images G in advance (for example, tens of thousands of images) that have been labeled (with correct answers) indicating which lines are the contour He of person H, or which (which region, etc.) is person H (in other words, supervised learning), and the recognition unit 2 will automatically learn the features. For the image G input to the recognition unit 2, it may output a score (such as a percentage of 96%) indicating which lines are the contour He of person H, or which is person H, based on the previously learned features, or it may output an image G with an indication that person H has been recognized (for example, surrounded by a rectangle with the word "person" inside, see Figure 2(a)). In addition, the recognition unit 2 may perform unsupervised learning, reinforcement learning, or the like.

[0027] <h1> As shown in Figures 1 and 2(a), the recognition unit 2 extracts the outline He of person H from image G and displays the bounding box B surrounding person H. The center position of the upper edge of this bounding box B is defined as the upper determination coordinate B1, and the center position of the lower edge of the bounding box B is defined as the lower determination coordinate B2. In other words, whether or not the upper coordinate B1 is within the predetermined area described later can determine whether or not the upper part H1 of person H is within the predetermined area, and whether or not the lower coordinate B2 is within the predetermined area described later can determine whether or not the lower part H2 of person H is within the predetermined area.

[0028] In addition, the recognition unit 2 may recognize the head and feet (shoes worn, etc.) of the person H based on the extracted contour He of the person H, and use the coordinates of the head as the upper determination coordinate B1 (upper part H1 of person H) and the coordinates of the feet as the lower determination coordinate B2 (lower part H2 of person H). Furthermore, the recognition unit 2 determines the centroid of person H (the centroid of the area occupied by person H (a figure of a predetermined shape)) based on the contour He and region of person H within the image G, and sets the foot of the perpendicular from that centroid to the top edge of the bounding box B as the upper determination coordinate B1 (upper part H1 of person H), The foot of the perpendicular line from its center of gravity to the bottom edge of bounding box B can also be used as the lower determination coordinate B2 (the lower part of person H, H2).

[0029] <Upper judgment area A1, lower judgment area A2> As shown in Figure 3, image G has at least an upper judgment area A1 and a lower judgment area A2, as described above. Furthermore, the upper judgment area A1 may include the upper danger judgment area A11 and the upper detection judgment area A12, and the lower judgment area A2 may include the lower danger judgment area A21 and the lower detection judgment area A22.

[0030] The ranges of each of these areas A1, A2 (A11-A22), etc., are set within the size of the image G captured by the camera (for example, width: 720 pixels, height: 480 pixels). The height of the top edge of each area A1, A2 (A11-A22), as well as the height and shape of the top edge, etc., are determined according to the parameters of the camera that captures image G (such as the vertical direction of the camera's optical axis (pitch angle) and the field of view (angle of the horizontal range captured)), the height of the person being judged (for example, the average height of an adult), and the "actual" distance from the camera mounting position of the forklift 10, etc., which will be designated as danger judgment areas A11 and A21. To elaborate, in the case of the upper hazard determination area A11, the height of its top edge is determined by the vertical orientation of the camera's optical axis, the average height of an adult (for example, when determining a standing person), and the "actual" distance from the forklift 10, etc., which is designated as hazard determination areas A11 and A21. The shape of its top edge is determined by the upward curvature determined by the camera's field of view. Similarly, in the case of the upper detection area A12, the height of its top edge is determined by the vertical orientation of the camera's optical axis and the average height of an adult (for example, when detecting a person standing far away), and the shape of its top edge is determined by the camera's field of view, specifically by the degree of upward curvature. Furthermore, in the case of the lower hazard determination area A21, the height of its upper edge is determined according to the vertical orientation of the camera's optical axis and the "actual" distance from the forklift 10, etc., which is designated as hazard determination area A11 or A21, and the shape of its upper edge can be said to be determined by the downward curvature of the camera's field of view. Furthermore, in the case of the lower detection area A22, the height of its upper edge is determined according to the vertical orientation of the camera's optical axis, and the shape of its upper edge is determined by the downward curvature of the camera's field of view. Furthermore, the height of the bottom edge of each area A11 to A22 is the same as the bottom edge of image G, and its shape is also the same horizontal straight line as the bottom edge of image G. In addition, the left and right positions of each area A11 to A22 (the width of each area A11 to A22) may be the same as the left and right positions of the left and right edges of image G, respectively, or they may be shifted inward by a predetermined number of pixels from the left and right edges of image G (they may be narrower by a predetermined number of pixels than image G).

[0031] The range of each area A11 to A22 may be set by the setting unit 8 as described above, and this setting unit 8 may be adjustable in the range of each area A11 to A22. This adjustment may be made by adjusting the range of each area one pixel at a time, or by adjusting it every predetermined number of pixels (for example, every 10 pixels). The following description of setting section 8 mainly focuses on the fact that the range of each area A11 to A22 can be adjusted. The adjustments made by the setting unit 8 may be performed, for example, when installing the camera 11 on a forklift 10, along with adjusting the orientation of the camera 11, or they may be made while using the forklift 10 on which the camera 11 is installed.

[0032] <Judgment part 3> As shown in Figures 1-3 and 16(a), the determination unit 3 is the part that determines the presence of a person based on at least one of the presence or absence of the upper part H1 of person H within the upper determination area A1 (A11 or A12) of image G and the presence or absence of the lower part H2 of person H within the lower determination area A2 (A21 or A22) of image G, as described above. Furthermore, the determination unit 3 may determine that a person is closer to the image G when the upper part H1 of the person H recognized by the recognition unit 2 is within the upper danger determination area A11, and / or the lower part H2 of the person H recognized by the recognition unit 2 is within the lower danger determination area A21, compared to when the upper part H1 of the person H recognized by the recognition unit 2 is within the upper detection determination area A12, and / or the lower part H2 of the person H recognized by the recognition unit 2 is within the lower detection determination area A22.

[0033] Whether or not the upper part H1 or lower part H2 of person H is present in each area A11 to A22 in the determination unit 3 can be determined simply by whether the numerical values ​​(width and height, or X and Y values) of the upper determination coordinate B1 (upper part H1 of person H) are within the upper danger determination area A11 or upper detection determination area A12, or whether the numerical values ​​(width and height, or X and Y values) of the lower determination coordinate B2 (lower part H2 of person H) are within the lower danger determination area A21 or lower detection determination area A22, in which case it can be determined that a person is present. Alternatively, if the head of person H is defined as the upper part H1 of person H, or the feet of person H are defined as the lower part H2 of person H, the determination can be made using the artificial intelligence (AI) or the like as described above. The determination by the determination unit 3 (determination system 1) may be steps S1-1 and S1-2 in Figure 16(a).

[0034] In step S1-1, it is checked whether the upper part H1 of person H is in the upper judgment areas A11 and A12, and / or whether the lower part H2 of person H is in the lower judgment areas A21 and A22. If it is in the upper judgment areas A11 and A12, and / or the lower judgment areas A21 and A22, the process moves to step S1-2 described below; otherwise, it returns to step S1-1. In step S1-2, if the upper and lower determination areas A1 and A2 are divided into upper and lower danger determination areas A11 and A21 and upper and lower detection determination areas A12 and A22, the determination unit 3 determines that if the upper part H1 of person H is in the upper danger determination area A11 and / or the lower part H2 of person H is in the lower danger determination area A21, then the person is closer in image G. If the upper part H1 of person H is in the upper detection determination area A12 and / or the lower part H2 of person H is in the lower detection determination area A22, then the person is present (but not very close). If the upper and lower determination areas A1 and A2 are not divided into upper and lower danger determination areas A11 and A21 and upper and lower detection determination areas A12 and A22, the determination unit 3 simply determines that person H is present.

[0035] The processes described above (steps S1-1 and S1-2) are repeated at predetermined time intervals. This means that the determination unit 3 will be able to determine the presence of a person in real time (without predicting the future, but rather the presence of a person at that particular moment). Furthermore, if the determination system 1 is equipped with a notification unit 4, in step S1-2, the determination unit 3 may notify the driver of the forklift 10 or the like each time it determines the presence of a person in real time.

[0036] <Newspaper Section 4> As shown in Figures 1, 4, 5 and Figure 16(a), the notification unit 4 is the part that notifies the system (determination system 1) or outside of it that the determination unit 3 described above has determined the presence of a person H in the image G. The notification unit 4 may be, for example, a speaker capable of outputting predetermined audio data, and its size may be selectable from large, medium, and small. In addition, the notification unit 4 may be a rotating light (for example, a red rotating light to indicate the presence of person H in danger judgment areas A11 and A21, and a yellow rotating light to indicate the presence of person H in detection judgment areas A21 and A22), a buzzer or horn that emits a sound when the judgment unit 3 detects the presence of a person (the sound emitted may differ depending on which judgment area person H is in), an LED strobe light that lights up when the judgment unit 3 detects the presence of a person (the way it lights up may differ depending on which judgment area person H is in), or a monitor that displays predetermined characters or colors when the judgment unit 3 detects the presence of a person (the displayed content may differ depending on which judgment area person H is in), and these may coexist. The following discussion will primarily focus on the fact that Hochi-bu 4 is acting as the speaker.

[0037] The notification unit 4 may be able to switch between enabling and disabling notification of the detection of the presence of person H via the setting unit 8, etc., and the notification content may be selected from text (predetermined content), or it may be created by inputting text data (character data) into the setting unit 8, etc. by the user, etc., and converting this text data into audio data, etc. When the notification unit 4 is attached to a forklift 10 or the like, it may be linked to the state of the forklift 10 or the like (for example, it may only notify when it is reversing), or it may continue to notify a predetermined message as long as a certain state of the forklift 10 or the like persists.

[0038] <Storage section 5> As shown in Figure 1, the storage unit 5 is the part that stores the image G itself captured by the camera, and / or the video containing the image G. The storage unit 5 may be, for example, a USB (Universal Serial Bus) memory (a storage device used by connecting to a USB connector) for saving images G and videos. In addition, the storage unit 5 may be magnetic tape, DDS (Digital Data Storage), magnetic disks such as floppy disks (FD) and hard disk drives (HDD), optical disks such as compact discs (including CDs), laser discs (LD), digital versatile discs (DVDs), and Blu-ray discs (BDs), magneto-optical disks such as minidiscs (MDs), flash disks (Solid State Drives, SSDs), flash memory such as memory cards, or memory (RAM, etc.) built into PCs, mobile terminals, digital cameras, etc. The storage unit 5 will be described primarily as a USB memory stick.

[0039] When the storage unit 5 saves a video containing images G, it may set the frame rate of the video file (number of images G per second) to a predetermined value and reduce the number of image frames to be saved. The frame rate and the recording time of one video file may be changeable using the setting unit 8 or the like. The frame rate value may, for example, be set to 10 frames per second (FBS) by default and can be changed within a range of 1 FPS to 10 FPS using setting unit 8, etc. Similarly, the recording time may, for example, be set to 2 minutes by default and can be changed within a range of 1 minute to 10 minutes using setting unit 8, etc. If the judgment system 1 equipped with a storage unit 5 is attached to a forklift 10 or the like, after starting the judgment system 1, as long as the accessory power supply (ACC) of the forklift 10 or the like is ON (in other words, the engine of the forklift 10 or the like is ON), the judgment system 1 may continue to save new video files created by the judgment system 1 to the storage unit 5 at predetermined recording intervals. Furthermore, if the storage capacity of the USB memory or other storage device (5) becomes full, you may delete the oldest video file.

[0040] <Communications Section 6> As shown in Figure 1, the communication unit 6 is the part that communicates with the outside of the system (determination system 1). The communication unit 6 can also be described as the part that communicates wirelessly or via wired connections, either directly or via the internet or other communication networks, with equipment and devices such as PCs and computers located at a location separate from the judgment system 1. In particular, the communication unit 6, which communicates wirelessly, may be, for example, a USB-type data communication terminal that supports LTE (Long Term Evolution) Category 4 (Cat.4), 3G, GSM (Global System for Mobile Communications), or other mobile phone communication functions such as 4G and 5G, or a SIM (Subscriber Identity Module).

[0041] The communication unit 6 may send and receive images G or videos (such as multiple consecutive images G). Furthermore, the recognition unit 2 and the determination unit 3 may be configured as a server, or they may be configured to cooperate with or share the functions of the forklift 10's computer and the server. In such a communication unit 6, maintenance of the judgment system 1 may be performed, data under evaluation may be acquired, and the software of the judgment system 1 may be updated.

[0042] <Casing 7> As shown in Figures 1 and 4, the casing 7 is a housing that incorporates the recognition unit 2, determination unit 3, communication unit 6, etc. The casing 7 may have connection terminals for connecting cables from the notification unit 4, camera (camera 11), and the forklift 10 described later. In addition, if the storage unit 5 is a USB memory or the communication unit 6 is a USB type data communication terminal, the casing 7 may have a USB port to which these can be connected, or if the communication unit 6 is a SIM, it may have a tray or slot for connecting it. Furthermore, the setting unit 8 may be provided on the outside of the casing 7, or the setting unit 8 may be built into the casing 7 but a part of the setting unit 8 may be exposed. The notification unit 4 may or may not be built into the casing 7. The forklift 10 equipped with the judgment system 1 described above will be described below.

[0043] <Forklift 10> As shown in Figures 5 to 16, the forklift 10 according to the present invention is a vehicle equipped with the judgment system 1 described above. The forklift 10 can also be described as a powered (engine, motor, etc.) material handling vehicle equipped with a camera 11 that captures the aforementioned image G, forks 12 for loading (cargoing) luggage, cargo, and materials, a mast 13 for raising and lowering (lifting and lowering) the forks 12, and a head guard 14 to protect the driver.

[0044] Furthermore, in this invention, "a determination system 1 is provided" means not only that the determination system 1 is mounted on the forklift 10 (i.e., the forklift 10 has a determination system 1), but also that the determination system 1, in which the camera 11 and notification unit 4 etc. are mounted, but the recognition unit 2, determination unit 3 etc. are located in a different place from the forklift 10 (i.e., not mounted on the forklift 10), is connected via a communication unit 6 or the internet, etc., and determinations are made by the determination system 1. The determination system 1 (determination unit 3) in the forklift 10 determines the presence of a person H in the image G described above only when the forklift 10 is moving backward.

[0045] Whether or not the forklift is moving in the reverse direction can be determined by inputting a back signal (a signal output when the forklift 10 is moving backward) from the forklift 10 to the judgment system 1. Alternatively, whether or not the forklift 10 is turning can be determined by inputting a steering signal (a signal output when the steering wheel position (steering wheel operation) is turned left or right) from the forklift 10 to the judgment system 1. Furthermore, the determination system 1 (determination unit 3) in the forklift 10 may determine the direction of travel of the forklift 10 based on the image G described above, and may also determine the turning direction of the forklift 10. In particular, it may be possible to switch between determining the direction of travel based on input such as a reverse signal or based on the image G using a setting unit 8 or the like. In the following, the determination of the direction of travel will be mainly described based on image G, as this simplifies the installation of the determination system 1. Next, we will describe the camera 11 that captures image G.

[0046] <Camera 11> As shown in Figures 4-14, camera 11 is an imaging device that captures the image G described above. Camera 11 may be mounted in a position where the rear of the forklift 10 is within image G, and the forklift 10 itself does not need to be visible in image G.

[0047] Camera 11 may be mounted on more than one forklift 10 (i.e., it may be a stereo camera), and there are no particular limitations on the lens of camera 11, but for example, it may be a wide-angle lens. The camera 11 may be mounted, for example, on the rear of the head guard 14 of the forklift 10, on a balance weight, or on the rear edge of the upper surface of the rear bonnet, and the optical axis of its lens may be directed diagonally downward and rearward.

[0048] The range of the image G captured by such a camera 11 is not particularly limited, but if a wide-angle lens is used, for example, the sky may be visible in the upper part of the image G, or the left and right rear corners of the forklift 10 may be visible in the left and right parts of the image G, and a point at a predetermined distance behind the forklift 10 (approximately 10 to 30 m away) may also be visible in the image G. In addition, camera 11 may be used as a rearview camera for the driver of the forklift 10 to monitor the area behind the vehicle. Next, regarding the determination of the direction of travel and turning direction based on image G, we will first describe a first embodiment of the determination system 1 (determination unit 3, etc.) that determines only the direction of travel of the forklift 10.

[0049] <First Embodiment (Determination of Direction of Travel Only Based on Image G)> As shown in Figures 6, 7, and 16(b), the first embodiment of the determination system 1 (determination unit 3, etc.) in the forklift 10 determines the direction of travel based on the image G captured by the camera 11 described above. The determination of the direction of travel according to the first embodiment may also be performed in steps S2-1 to S2-3 in Figure 16(b). Alternatively, the determination system 1 (determination unit 3, etc.) may perform the process in steps S2-1 to S2-3 without dividing it into steps S2-1 to S2-3.

[0050] In step S2-1, in each of the left and right areas shown in Figure 7, corner points with edges (for example, corners of shelves in a warehouse, corners of arrows or path lines drawn on the floor, corners of buildings, etc.) are designated as feature points (one or more). A displacement vector is calculated from the coordinates of the feature points in the previous frame's image G and the coordinates of the feature points in the current frame's image G (see Figure 6), and the process proceeds to step S2-2 described below. In step S2-2, the number of displacement points (displacement vectors pointing upward or downward) in each of the left and right areas shown in Figure 7 is tallied, and the direction with the greater number of upward or downward movements is detected as the vertical displacement direction for each area, before proceeding to step S2-3 described below. In step S2-3, if the vertical displacement directions coincide in the left and right areas, the determination unit 3 determines that the forklift 10 is moving forward (forward determination) if the coincident displacement direction is upward, and determines that the forklift 10 is moving backward (backward determination) if the coincident displacement direction is downward. If the vertical displacement directions do not coincide in the left and right areas, the determination unit 3 determines that the forklift 10 is stopped (stopped determination), and then returns to step S2-1.

[0051] The processes described above (steps S2-1 to S2-3) are also repeated at predetermined time intervals. This means that the determination unit 3 will determine the direction of travel of the forklift 10 in real time (the direction of travel each time, without predicting the future). Furthermore, if the first embodiment of the determination system 1 includes a notification unit 4, in step S2-3, the determination unit 3 may notify the driver of the forklift 10 or the like each time it determines the direction of travel in real time.

[0052] Furthermore, as shown in Figure 7, in image G, a predetermined number of pixels (for example, 72 pixels) in the center of the width direction of the image G may be excluded from the left and right areas used for determining the direction of travel. This removal reduces the likelihood of misjudging the direction of travel of the forklift 10 when a person H, another forklift, car, bicycle, etc. crosses behind the forklift 10 while the forklift 10 is stationary. Furthermore, the reason for misjudgment in this case is that if the central vertical area is not removed, and the same vertical displacement vector is detected in the left and right areas near the center of image G (central vertical area), the judgment unit 3 will incorrectly determine that the forklift 10 is moving forward or backward, even though the forklift 10 is actually stopped. Furthermore, as shown in Figure 7, in image G, the lower horizontal area of ​​a predetermined number of pixels (for example, 240 pixels) may also be removed from the left and right areas used for determining the direction of travel. This removal reduces the likelihood of misjudging the direction of travel of the forklift 10, as the person will appear larger in image G when they are in close proximity to the forklift 10.

[0053] <Second Embodiment (Determination of Direction of Travel and Turning Direction Based on Image G)> As shown in Figures 7-14 and 16(b), the second embodiment of the determination system 1 (determination unit 3, etc.) in the forklift 10 differs from the first embodiment described above in that it determines not only the direction of travel but also the direction of turning based on the image G captured by the camera 11 described above. The determination of the direction of travel and the direction of turning according to the second embodiment may also be done in steps S2-1 to S2-3 in Figure 16(b). Furthermore, even in the second embodiment, the determination may not be divided into steps S2-1 to S2-3, but rather performed by the determination system 1 (determination unit 3, etc.) in steps S2-1 to S2-3.

[0054] In step S2-1, in each of the left and right areas shown in Figure 7, the corner points with edges are designated as feature points (one or more), and the displacement vector is calculated from the coordinates of the feature points in the previous frame's image G and the coordinates of the feature points in the current frame's image G (see Figure 8), and then the process moves to step S2-2 described below. In step S2-2, for each of the left and right areas shown in Figure 7, the number of displacement points in the upward or downward direction (displacement vectors pointing upward or downward) is tallied, as well as the number of displacement points in the leftward or rightward direction (displacement vectors pointing leftward or rightward). The direction with the most frequent upward or downward movement, and the direction with the most frequent leftward or rightward movement, are then identified as the vertical and horizontal displacement directions for each area. Furthermore, for each area, the magnitude of the displacement vector is determined (whether it is large or small, etc.) based on the average value of the scalars of the displacement vectors that have a certain or greater scalar value, and the process proceeds to step S2-3 described below.

[0055] In step S2-3, the direction of travel and turning of the forklift 10 is determined based on the lateral displacement direction, the magnitude of the displacement vector, and the vertical displacement direction. The lateral displacement direction in each left and right area determines whether the forklift 10 is turning or not. The magnitude of the displacement vector decreases on the tire side, which is the turning axis (the magnitude of the displacement vector indicates whether the turning axis of the forklift 10 is on the left or right side). The vertical displacement direction in each left and right area determines the direction of travel of the forklift 10 (forward or backward, etc.). The following provides a detailed explanation of each case. As shown in Figure 9, if the left-right displacement direction is different in each of the left and right areas, the determination unit 3 first determines that there is "no rotation," and the magnitude of the displacement vector is no longer used in the determination (indicated as "-" in the table in Figure 9). If the up-down displacement direction is the same in each of the left and right areas, the determination unit 3 determines that if the matching displacement direction is upward, the forklift 10 is moving forward (forward determination), and then returns to step S2-1. In other words, Figure 9 shows the case where the forklift 10 is "moving forward without rotation." As shown in Figure 10, if the left-right displacement direction is different in each of the left and right areas, the determination unit 3 first determines that there is "no rotation," and the magnitude of the displacement vector is no longer used in the determination. If the up-down displacement direction is the same in each of the left and right areas, the determination unit 3 determines that if the matching displacement direction is downward, the forklift 10 is moving backward (reverse), and then returns to step S2-1. In other words, Figure 10 shows the case where the forklift 10 is "reverse without rotation." As shown in Figure 11, if the left-right displacement directions coincide in the left and right areas, the determination unit 3 first determines that there is a turn, and the magnitude of the displacement vector is larger in the left area and smaller in the right area (if the right area is smaller in image G, the actual turning axis of the forklift 10 will be on the left side). At the same time, the vertical displacement directions will be different in the left and right areas, and if the vertical displacement direction is upward in the left area and downward in the right area, the determination unit 3 determines that the forklift 10 is moving forward (forward determination), and after determining that the tire (side) that becomes the turning axis is on the left, it returns to step S2-1. In other words, Figure 11 shows the case where the forklift 10 is "moving forward while turning left". As shown in Figure 12, if the left-right displacement direction is the same in each of the left and right areas, the determination unit 3 first determines that there is a turn. The magnitude of the displacement vector is larger in the left area and smaller in the right area, and the vertical displacement direction is different in each of the left and right areas. If the vertical displacement direction is downward in the left area and upward in the right area, the determination unit 3 determines that the forklift 10 is moving backward (reverse), and determines that the tire (side) that will be the axis of rotation will be on the left, before returning to step S2-1. In other words, Figure 12 shows the case where the forklift 10 is "reverse while turning left". As shown in Figure 13, if the left-right displacement direction is the same in each area, the determination unit 3 first determines that there is a turn, and the magnitude of the displacement vector is small in the left area and large in the right area (if the left area is small in image G, the actual turning axis of the forklift 10 will be on the right side instead). At the same time, the vertical displacement direction will be different in each area, and if the vertical displacement direction is down in the left area and up in the right area, the determination unit 3 determines that the forklift 10 is moving forward (forward determination), and after determining that the tire (side) that becomes the turning axis is on the right, it returns to step S2-1. In other words, Figure 13 shows the case where the forklift 10 is "moving forward while turning to the right". As shown in Figure 14, if the left-right displacement direction is the same in each area, the determination unit 3 first determines that there is a turn. The magnitude of the displacement vector is small in the left area and large in the right area, and the vertical displacement direction is different in each area. If the vertical displacement direction is upward in the left area and downward in the right area, the determination unit 3 determines that the forklift 10 is moving backward (reverse), and determines that the tire (side) that will be the axis of rotation will be to the right, before returning to step S2-1. In other words, Figure 14 shows the case where the forklift 10 is "reverse while turning to the right".

[0056] The processes described above (steps S2-1 to S2-3) are also repeated at predetermined time intervals. This means that the determination unit 3 will determine the direction of travel of the forklift 10 in real time (the direction of travel each time, without predicting the future). Furthermore, if the determination system 1 is equipped with a notification unit 4, in step S2-3, the determination unit 3 may notify the driver of the forklift 10 each time it determines the direction of travel in real time.

[0057] Furthermore, in the second embodiment as well, as shown in Figure 7, in image G, the central vertical area of ​​a predetermined number of pixels (for example, 72 pixels) in the center in the width direction may be removed from the left and right areas used for determining the direction of travel. This removal reduces the likelihood of misjudging the direction of travel of the forklift 10 when a person H, another forklift, car, bicycle, etc. crosses behind the forklift 10 while the forklift 10 is stationary. Furthermore, the reason for misjudgment in this case is that if the central vertical area is not removed, and the same vertical displacement vector is detected in the left and right areas near the center of image G (central vertical area), the judgment unit 3 will incorrectly determine that the forklift 10 is moving forward or backward, even though the forklift 10 is actually stopped. Furthermore, in the second embodiment as well, as shown in Figure 7, the lower horizontal area of ​​a predetermined number of pixels (for example, 240 pixels) at the bottom of image G may also be removed from the left and right areas used for determining the direction of travel. This removal reduces the likelihood of misjudging the direction of travel of the forklift 10, as the person will appear larger in image G when they are in close proximity to the forklift 10. Furthermore, in the second embodiment of the determination system 1 (determination unit 3, etc.), determination may be made without using the magnitude of the displacement vector.

[0058] <State transitions of forklift 10> As shown in Figure 15, the forklift 10 transitions between states based on the determination of the direction of travel, etc. (forward determination, reverse determination, stop determination) as described above: "stopped state", "starting to move forward state", "starting to move backward state", "forward state", and "reverse state". The state transition (1) in Figure 15 (the transition from "stopped state" to "moving forward state") occurs when the forward movement determination is made for a predetermined number of consecutive frames (for example, 2 frames). The state transition (2) in Figure 15 (the transition from "stopped state" to "reverse movement start state") occurs when the reverse movement determination is made for a predetermined number of consecutive frames (for example, 2 frames). The state transition (3) in Figure 15 (transition from "moving forward" to "stopped") occurs when a condition other than forward movement is not met for a predetermined number of frames (for example, 3 frames). The state transition (4) in Figure 15 (transition from "reverse movement start state" to "stopped state") occurs when a predetermined number of frames (for example, 3 frames) are consecutive except for the reverse movement detection. The state transition in (5) in Figure 15 (transition from "forward movement start state" to "forward state") occurs when, during a predetermined number of frames (e.g., 60 frames) in the "forward movement start state", nothing other than a forward movement judgment occurs for the predetermined number of frames (e.g., 3 frames) described in (3) above. To give a more specific example, if nothing other than a forward movement judgment occurs for 3 consecutive frames during the 60-frame count in the "forward movement start state", the state transition in (3) described above occurs immediately at that point. However, even if nothing other than a forward movement judgment occurs for only 1 frame or 2 consecutive frames during the 60-frame count, if a forward movement judgment occurs afterward, the count will continue, and if the count reaches 60 frames, the transition from "forward movement start state" to "forward state" will occur. The state transition (6) in Figure 15 (transition from "backward movement start state" to "backward state") occurs when, during a predetermined number of frames (e.g., 60 frames) in the "backward movement start state," nothing other than a backward detection occurs for the predetermined number of frames (e.g., 3 frames) described in (4) above. To give a more specific example, if nothing other than a backward detection occurs for 3 consecutive frames during the 60-frame count in the "backward movement start state," the state transition (4) described above occurs immediately at that point. However, even if nothing other than a backward detection occurs for only 1 frame or 2 consecutive frames during the 60-frame count, if a backward detection occurs afterward, the count will continue, and the transition from "backward movement start state" to "backward state" will occur when the count reaches 60 frames. The state transition (7) in Figure 15 (transition from "forward state" to "stopped state") occurs when a state other than forward movement is not detected for a predetermined number of frames (for example, 6 frames). The state transition (8) in Figure 15 (transition from "backward state" to "stopped state") occurs when a predetermined number of frames (for example, 6 frames) are consecutive except for the backward detection.

[0059] <Notification content based on state transitions, direction of movement, etc.> Depending on the state transitions and direction of movement determined so far, the determination system 1 may change the content of the notification unit 4 described above. For example, if the judgment system 1 determines that the forklift 10 is moving backward (the judgment unit 3 determines that it is moving backward, or it is in the "reverse state" described above) and that a person H is present in the danger judgment area A11, A21 or the detection judgment area A12, A22, it may output a predetermined notification content from the notification unit 4, such as a speaker. A concrete example of this notification content would be, if the notification unit 4 is a speaker, and it determines that the vehicle is in a "reverse state" and that person H is present within the danger zones A11 and A21. <1> If the system outputs the voice message "It is dangerous" and determines that person H is present in the "reverse state" and within the detection area A12 and A22, <2> In this case, audio data such as "Please be careful" may be output, and these audio data can be said to be warning messages. Also, in the case of "reverse state" <3> The system may output audio data saying "I'm going to back up," which can be considered a warning message. Furthermore, when determining the direction of travel based on the input of a back signal to the judgment system 1, the warning message may also be played when a back signal is input even while the vehicle is stopped (for example, when the transmission is put into reverse gear). The content of these audio data and other notifications will vary depending on the circumstances. <1> ~ <3> As long as it continues, it is acceptable for the output to be repeated, and also when multiple cases overlap (for example, case <1> and <3> For example, the notification content, such as audio data, may be output alternately from the notification unit 4.

[0060] <Other> The present invention is not limited to the embodiments described above. The judgment system 1, the individual components or the overall structure, design shape, dimensions, weight, etc. of the forklift 10 can be modified as appropriate in accordance with the spirit of the present invention. The judgment system 1 does not need to include other components such as the notification unit 4, storage unit 5, and communication unit 6, as long as it is equipped with the recognition unit 2 and the judgment unit 3. The range of each area, such as the upper judgment area A1 and the lower judgment area A2, the enable / disable status and content of the notification in the notification unit 4, the frame rate and recording time of the video file in the storage unit 5, the determination of the direction of travel of the forklift 10 (whether based on a reverse signal, etc., or on image G), and the number of frames that serve as conditions for state transitions may be set at a location other than where the judgment system 1 is installed (forklift 10, etc.) via the communication unit 6 or the internet, rather than in the setting unit 8 provided in the judgment system 1. The determination of the direction of travel based on image G may be performed by the determination unit 3, or by a separate unit (for example, a direction of travel determination unit, a turning direction determination unit, or a direction of travel / turning direction determination unit).

[0061] The judgment system 1 in the forklift 10 may be retrofitted to an existing forklift 10, or it may be installed from the time of manufacture or shipment. The forklift 10 may also be equipped with cameras 11 that capture images from its sides and front, and the images G from the cameras 11 that capture images from the sides, front, and rear may be combined and used by the judgment system 1 to determine the presence of a person. The state transition in (5) in Figure 15, as described above, <a5>It was stated that the transition from "forward movement start state" to "forward state" occurs only if, during a predetermined number of frames (e.g., 60 frames) in the "forward movement start state," nothing other than the forward movement determination occurs for a predetermined number of frames (e.g., 3 frames) as described in (3) above. <a5>In the case of, <b5>If a predetermined number of frames in the "forward movement initiation state" (for example, more than half (more than 30 frames out of 60), more than 1 / 3 (more than 20 frames out of 60), more than 30% (more than 18 frames out of 60), or more than 20% (more than 12 frames out of 60)) constitutes a forward movement determination (i.e., <a5>and <b5>In this case, it is acceptable to assume that a transition occurs from the "starting forward movement state" to the "forward movement state". Similarly, the state transition in (6) in Figure 15 is, as described above, <a6>It was stated that the transition from "reverse movement initiation state" to "reverse state" occurs only if, during a predetermined number of frames (e.g., 60 frames) in the "reverse movement initiation state," nothing other than the reverse movement determination occurs for a predetermined number of frames (e.g., 3 frames) as described in (4) above. <a6>In the case of, <b6>If a predetermined number of frames in the "reverse movement initiation state" (for example, more than half (more than 30 frames out of 60), more than 1 / 3 (more than 20 frames out of 60), more than 30% (more than 18 frames out of 60), or more than 20% (more than 12 frames out of 60)) constitutes a reverse movement (i.e., <a6>and <b6>In this case, it is acceptable to assume that a transition occurs from the "starting to move backward" state to the "reverse state". In the first embodiment described above, the determination system 1 (determination unit 3) determined that the system was stopped (stopped) when the vertical displacement direction did not coincide in the left and right areas. However, in the second embodiment, the system may also determine that the system is stopped (stopped) when the left and right displacement direction does not coincide in the left and right areas and the vertical displacement direction does not coincide in the left and right areas, or when the left and right displacement direction coincides in the left and right areas and the vertical displacement direction coincides in the left and right areas. In addition, the system may also determine that the system is stopped (stopped) when feature points cannot be extracted from the image G, or when the scalar of the calculated displacement vector is less than a certain value.

[0062] Below, we will provide a detailed explanation of the image G used in the judgment system 1 and forklift 10 described above, as well as the person H who is the target of the judgment and the various parts of the forklift 10. <Image G> As shown in Figures 1-3 and 6-14, Image G was captured by a designated camera, such as the camera 11 on the forklift 10. Image G, when captured by the camera 11 attached to the forklift 10 as described above, may be an image of the rear of the forklift 10. In this case, the image G may show the rear of the forklift 10 itself, or conversely, only the vicinity of the forklift 10 may be shown (i.e., the forklift 10 itself may not be visible in the image G).

[0063] Furthermore, image G may be captured by a camera attached to a vehicle other than the forklift 10, or to a device, product, facility, or equipment other than a vehicle. In this case, a portion of the other vehicle may be included in image G, or a predetermined area near a portion of the other vehicle may be included in image G. Alternatively, only the vicinity of the other vehicle may be included in image G (i.e., the other vehicle itself may not be included in image G). Image G may be captured not only by a camera 11 attached to the forklift 10, but also by an aerial vehicle such as a drone, or by a fixed camera (such as a security camera or surveillance camera) installed in a city, store, shopping street, large building, tenant, or apartment building.

[0064] Image G may be captured by a camera 11 or the like attached to the forklift 10, capturing the rear of the forklift 10 from approximately above, approximately from the side, or approximately from the rear. Furthermore, a single detection system 1 may detect a person H from two or more images G. In this case, for example, the images G may be two or more images captured by two or more cameras 11 (so-called stereo cameras) attached to a forklift 10, or two or more images G may be captured simultaneously by a camera 11 attached to a forklift 10 and two or more cameras such as cameras attached to an aircraft or the like. In addition, image G may be a color image (an image that has hue and saturation in addition to brightness), a grayscale image (an image that has only brightness and does not have hue or saturation), or a black and white two-color image such as a halftone (dot pattern). The person H who may exist within such an image G will be described below.

[0065] <Person H> As shown in Figures 1-3 and 5-14, Person H is the subject of judgment by the judgment system 1 described above, and is the person shown in the image G described above. Person H may include not only workers and others located behind the forklift 10, but also pedestrians, cyclists, and other drivers. Person H can be said to be captured in a single image G, along with other parts of the image (such as the forklift 10 itself or the area behind the forklift 10). The outline (edge) He of person H is the boundary between person H and the rest of the image G, and can be said to represent the shape of person H. Person H may carry luggage such as bread or other food items, a plastic bottle containing a drink, a bag, or other similar items.

[0066] <Parts of Forklift 10> As shown in Figure 5, the forklift 10 is equipped with forks 12, a mast 13, a head guard 14, and other components in addition to the camera 11. The forks 12 lift cargo, goods, and supplies via pallets or the like. The mast 13 can be described as a rail that moves the forks 12 up and down, and may have a chain or cylinder to move the forks 12 up and down. The head guard 14 is a guard that prevents falling cargo or other items from hitting the driver. In addition, the forklift 10 is equipped with balance weights, drive wheels, and steering wheels (wheels whose direction changes when the steering wheel is turned). Balance weights, also called counterweights, are weights that correspond to the load being handled or transported. The drive wheels are the front wheels and can be larger than the steering wheels, and they transmit power. The steering wheels are the rear wheels and are used for steering. [Industrial applicability]

[0067] The detection system according to the present invention can detect the presence of a person in an image, and can therefore be used to detect the presence of a person near a forklift or to notify of dangerous conditions. In addition to forklifts, it can be used to detect the presence of a person near any vehicle, device, product, facility, or equipment, such as dump trucks, water trucks, concrete pumps, mixer trucks, tailgate lifters, tank trucks, water tankers, powder and granular material transport vehicles, detachable body vehicles, vehicle transport vehicles, rail containers, fire-fighting labor-saving devices, forestry and biomass-related products, facilities, equipment and devices, multi-story parking systems, coin parking lots, waterproof panels, etc. The forklift according to the present invention can be used even in environments where people are present behind or around it while it is in operation. In particular, it can be used to determine the presence of people when they are standing behind the forklift or when people are crossing behind it. Furthermore, when lifting cargo, goods, or materials to a high position, it can be used to determine the presence of people to the left, right, or in front of the forklift, and can be used to respond to the presence of all people and dangerous situations around the forklift. [Explanation of Symbols]

[0068] 1. Judgment System 2 Recognition section 3 Judgment section 4. News Department 5 Storage section 6 Communications Department 10 Forklifts 11 Cameras G Image A1 Upper judgment area A11 Upper Hazard Judgment Area A12 Upper detection and determination area A2 Lower judgment area A21 Lower Hazard Assessment Area A22 Lower detection and judgment area H people H1 Person's H2 Person's lower

Claims

1. A detection system that determines the presence of a person in an image captured by a camera, The aforementioned image has at least an upper detection area and a lower detection area set, A recognition unit that recognizes a person in the aforementioned image, A determination unit determines that a person exists in the image if the upper part of the person recognized by the recognition unit is within the upper determination area, and / or the lower part of the person recognized by the recognition unit is within the lower determination area. Equipped with, The aforementioned upper determination area includes the upper danger determination area and the upper detection determination area. The lower determination area includes a lower danger determination area and a lower detection determination area. The determination unit, If the upper part of the person recognized by the recognition unit is within the upper danger determination area, and / or the lower part of the person recognized by the recognition unit is within the lower danger determination area, Compared to the case where the upper part of the person recognized by the recognition unit is within the upper detection determination area, and / or the lower part of the person recognized by the recognition unit is within the lower detection determination area, A determination system characterized by determining that a person is present closer to the image.

2. The determination system according to claim 1, further comprising a notification unit that notifies the system outside of the system that the determination unit has determined the presence of a person in the image.

3. The determination system according to claim 1, further comprising a storage unit for storing at least one of the aforementioned image itself and a video containing the aforementioned image, and / or a communication unit for communicating with the outside of the system.

4. A forklift equipped with a determination system according to any one of claims 1 to 3, The forklift is equipped with a camera that captures the aforementioned images. The determination unit is characterized in that it determines the presence of a person in the image only when the forklift is moving in the reverse direction.

5. The forklift according to claim 4, characterized in that the determination unit determines the direction of travel of the forklift based on the image.

6. The forklift according to claim 5, characterized in that the determination unit also determines the turning direction of the forklift based on the image.

Images