Device system that is activated at high speed in vehicle such as ai-based forklift

Abstract

To achieve "high-speed activation " or the like of a system by standby activation which becomes in an operation state in short time from main activation.SOLUTION: A device system 1 provided in a vehicle 10 such as a forklift is activated in response to an accessory interlocking signal from the vehicle 10 and becomes an operation state, and that activation includes a main activation K1 in which the device system becomes in the operation state after first time T1 has elapsed since the accessory interlocking signal is turned on, and a standby activation K2 in which the device system becomes in the operation state after shorter second time T2 has elapsed since the accessory interlocking signal is turned on after the device system becomes in a standby state from the operation state. In addition, after third time T3 has elapsed and the device system becomes in a pause state from the standby state since the device system becomes in the standby state, the device system may become in the operation state by the main activation K1, or may have a determination unit 2 or the like that determines presence of a person H in an image G for each detection area R1 farther from a set dangerous area R2. The traveling direction and the speed of the vehicles 10 in which the device system 1 is provided may be determined, and each area R1 and R2 may be changed in conjunction with the traveling direction and the speed.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a device system that starts up at high speed in a vehicle, and a vehicle such as an AI - equipped forklift provided with this device system.

Background Art

[0002] Conventionally, an object detection device provided in a vehicle such as a forklift is known (see Patent Document 1). This object detection device includes: an object detection means for detecting an object in the image data by image processing that compares a plurality of sequentially captured image data with reference image data; a counting means for counting the number of detected objects in the image data; and a determination means for comparing the number of detected objects between two pieces of image data at different shooting times, and determining that a moving object is included in the detected object when the number of detections is different.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Here, in a vehicle such as a forklift provided with the object detection device described in Patent Document 1, as shown in paragraph 0034, 0068, Figure 4, etc., there are cases where the vehicle is dismounted for use (for example, dismounted for work). When dismounting for work, in order to prevent accidents due to malfunction, etc., it is necessary to stop (turn off) the accessory switch of the forklift, the drive source such as the engine, etc. However, dismounting the forklift for work can occur dozens to hundreds of times a day. Therefore, every time the forklift or other accessory switches are turned off during the dozens to hundreds of disembarkation operations per day, and then every time the operator gets back on the forklift or other vehicle to resume work, the accessory switches and the like are turned on to activate the object detection device. However, since the accessory switches of the forklift or the like were turned off every time the disembarkation operation was performed, it took several seconds to dozens of seconds each time for the object detection device to become reusable again. During that time, the operator had to stop working and wait, resulting in a significant amount of work loss and a problem of reduced work efficiency. Such a problem is the same when taking a short break (for example, several minutes to more than ten minutes) after getting off the vehicle.

[0005] In view of such points, an object of the present invention is to provide a device system that realizes "high-speed startup" and the like by standby startup that becomes operational in a shorter time than the initial startup, and a vehicle such as a forklift.

Means for Solving the Problems

[0006] The device system 1 according to the present invention is a device system provided in a vehicle. The device system starts up in response to an accessory interlock signal that turns on and off in conjunction with the ON / OFF of the accessory switch in the vehicle and enters an operating state. For the startup in response to the accessory interlock signal, after a predetermined first time has elapsed since the accessory interlock signal became ON, the device system enters an operating state, which is the initial startup. In the operating state, when the device system enters a standby state due to the accessory interlock signal becoming OFF, after a predetermined second time shorter than the first time has elapsed since the accessory interlock signal became ON, the device system enters an operating state. The first feature is that there are at least two types of standby startups.

[0007] The second feature of the device system 1 according to the present invention is that, in addition to the first feature, after a predetermined third time has elapsed without the accessory interlock signal turning ON since the standby state, the standby state is released and the device system enters a sleep state. When the accessory interlock signal turns ON while in the sleep state, the device system is activated by the main startup.

[0008] The third feature of the device system 1 according to the present invention is that, in addition to the first or second feature, the device system has a determination unit that determines the presence of a person in an image captured by a camera. In the determination unit, at least a danger area closest to the device system and a detection area next closest to the device system after the danger area are set. The determination unit recognizes a person in the image, and the determination unit determines whether the recognized person is present in the danger area and whether the recognized person is present in the detection area, respectively.

[0009] The fourth feature of the device system 1 according to the present invention is that, in addition to the third feature, the device system has at least one of a notification unit that notifies the outside of the device system that the determination unit has determined that the recognized person is present in the danger area and / or the detection area, a storage unit that stores the image itself and / or a moving image including the image, and a communication unit that communicates with the outside of the device system.

[0010] Due to these features, for the activation according to the accessory interlock signal ACC, there are the main activation K1 that becomes the operating state after the first time T1 has elapsed since the accessory interlock signal ACC turned ON, and the standby activation K2 that becomes the operating state after the shorter second time T2 has elapsed since the accessory interlock signal ACC turned ON after changing from the operating state to the standby state. Different from Patent Document 1, every time a dismounting operation is performed or a short break is taken after dismounting, even if the accessory switch of the vehicle 10 such as the forklift 10A is turned OFF and the vehicle enters the standby state, the "high-speed activation" that takes only the shorter second time T2 until the equipment system 1 becomes the operating state is realized. Even if the dismounting operation is performed dozens to hundreds of times a day or a short break is taken, the time to stop and wait is shortened, the work loss is reduced, and the work efficiency can be improved. In addition, since the equipment system 1 can perform a standby activation that starts in a shorter time compared to the main activation, it can also be said to be an "equipment system that activates at high speed".

[0011] Also, after the third time T3 has elapsed since entering the standby state and the standby state changes to the rest state, it becomes the operating state by the main activation K1, so it does not always remain in the standby state. Therefore, it is possible to reduce the unnecessary power consumption in the standby state, and even if the standby state is forgotten, it will automatically enter the rest state ("power saving" and "prevention of forgetting to switch the standby state").

[0012] Furthermore, in the determination unit 2 that determines the presence of the person H, the closest danger area R1 and the next closest detection area R2 are set, and by determining whether the person H recognized in the image G exists within the danger area R1 or the detection area R2 respectively, the "improvement of the determination accuracy" of the presence of the person H in the image G can be achieved. In addition, as will be described later, the determination unit 2 in the equipment system 1 may recognize the person H in the image G or determine the presence of the person H for each area R1, R2 by means of artificial intelligence (AI), etc. In this case, the equipment system 1 can also be said to be an "equipment system that utilizes AI".

[0013] In addition, if the notification unit 3 is also provided, it becomes possible to notify the user of the device system 1 (for example, the driver of the vehicle 10 such as the forklift 10A, etc.) of the presence of person H, and "risk reduction" can be achieved. Also, if the storage unit 4 and the communication unit 5 are also provided, further "improvement of determination accuracy" can be achieved based on the data of the stored and communicated image G.

[0014] The vehicle 10 according to the present invention is a vehicle provided with a device system having the above-described first or second feature, and the vehicle is characterized in that it outputs the accessory interlock signal to the device system as a first feature.

[0015] The second feature of the vehicle 10 according to the present invention is a vehicle provided with a device system having the above-described third feature, and the vehicle outputs the accessory interlock signal to the device system and has a camera for imaging the image, and the determination unit determines at least the traveling direction of the vehicle based on the image.

[0016] The third feature of the vehicle 10 according to the present invention is that, in addition to the above-described second feature, the vehicle is a forklift, the traveling direction includes the turning direction of the vehicle, and the determination unit also determines the speed in the traveling direction.

[0017] The fourth feature of the vehicle 10 according to the present invention is that, in addition to the above-described second feature, the determination unit changes at least the danger area and / or the detection area in conjunction with the traveling direction.

[0018] Due to these features, by outputting the accessory interlock signal ACC from the vehicle 10 to the device system 1, different from Patent Document 1, in a vehicle 10 such as the forklift 10A, every time a dismounting operation or the like is performed, even if the accessory switch or the like is turned off and the vehicle is in a standby state, "high-speed startup" that can be completed in a shorter second time T2 is realized, and work loss can be reduced and work efficiency can be improved. In addition, since the vehicle 10 can perform a standby startup that starts in a shorter time than the present startup, it can also be said to be a "vehicle that starts up at high speed".

[0019] Further, an accessory interlock signal ACC is output from a vehicle 10 such as a forklift 10A to the equipment system 1, and a determination unit 2 of the provided equipment system 1 determines the traveling direction of the vehicle 10 based on the image G, etc. Thus, based on the situation of the vehicle 10, it is possible to notify (inform) the driver whether or not a person H exists in the vicinity thereof, prompting the driver to pay attention, and "reduce the risk" in the vehicle 10 can be achieved. In addition, in the equipment system 1 provided in the vehicle 10, its determination unit 2 may determine the presence of the person H in the image G or the traveling direction of the vehicle 10, etc. by means of artificial intelligence (AI), etc. as described later. In this case, the vehicle 10 can also be said to be "a vehicle such as an AI - enabled forklift that starts up quickly".

[0020] In addition, by the determination unit 2 determining the turning direction and speed of the forklift 10A which is the vehicle 10, further "risk reduction" can be achieved in the forklift 10A. Also, by the determination unit 2 changing the danger area R1 and the detection area R2 in conjunction with the traveling direction, etc. in the vehicle 10, further "improvement of determination accuracy" can be achieved.

Advantages of the Invention

[0021] According to the equipment system and the vehicle according to the present invention, "quick start" etc. can be realized by standby start - up that becomes an operating state in a shorter time than the normal start.

Brief Description of the Drawings

[0022]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0023] The embodiments of the present invention will be described in detail with reference to the drawings. <Overall Configuration of Device System 1> In FIGS. 1 to 11, a device system 1 according to the present invention is shown. This device system 1 is a system provided in a vehicle 10 described later. Here, the "device system 1 provided in the vehicle 10" in the present invention can be paraphrased as mounted, installed, or attached, and it may be retrofitted to an existing vehicle 10, or may be attached since the vehicle 10 is manufactured, and currently includes the device system 1 that is not attached to the vehicle 10. The device system 1 is activated according to the accessory interlock signal ACC from the vehicle 10, and may enter the operating state S1 described later, the standby state S2 described later, or other states such as the rest state S0. In the device system 1, there are at least two types of activations according to the accessory interlock signal ACC, such as the main activation K1 and the standby activation K2 described later.

[0024] The specific configuration of the device system 1 is not particularly limited. For example, it may have a determination unit 2 described later, a notification unit 3, a storage unit 4, a communication unit 5 described later, or other parts having functions other than the determination by the determination unit 2. Incidentally, the device system 1 having the determination unit 2 can also be said to be a determination system. Furthermore, the device system 1 may include a casing 6 that houses the above-described determination unit 2, communication unit 5, etc., or may include other components such as a camera (such as camera 20 attached to vehicle 10) that captures image G. The device system 1 may be configured by a computer. First, the accessory interlock signal ACC input from the vehicle 10 to the device system 1 will be described below.

[0025] <Accessory interlock signal ACC, etc.> As shown in FIGS. 1 to 3, the accessory interlock signal ACC is a signal that turns ON / OFF in conjunction with the ON / OFF of an accessory switch in the vehicle 10 described later. Here, the accessory switch is a switch in the vehicle 10 such as a forklift 10A that is interlocked with the start switch of a drive source such as an engine, is provided beside the start switch, or is turned on when the key of the vehicle 10 is inserted and turned. By turning on this accessory switch, power supply from the accessory power supply (constant power supply) to the device system 1 and other electrical components retrofitted to the vehicle 10 becomes ON. It can also be said that when the accessory switch is turned OFF, the power supply from the accessory power supply to the device system 1 and the like also becomes OFF. In addition, the "accessory interlock signal ACC that turns ON / OFF in conjunction with the ON / OFF of the accessory switch" in the present invention means that when the accessory switch is turned ON, the ON accessory interlock signal ACC is output from the vehicle, and when the accessory switch is turned OFF, the OFF accessory interlock signal ACC is output from the vehicle. Furthermore, the device system 1 may have a terminal (accessory input terminal) 1a for inputting an accessory interlock signal ACC from the vehicle 10 and a terminal (constant power input terminal) 1b for inputting power from a power source (constant power source) from the vehicle 10. These may be respectively connected to output terminals (accessory output terminal 10a and constant power output terminal 10b, to be described later) from the vehicle 10 via wiring such as a cable. In particular, the constant power input terminal 1b may be provided with a plus (+) terminal and a minus (-) terminal respectively. In addition, the device system 1 may have a terminal (backup signal input terminal) 1c for inputting a backup signal from the vehicle 10, and the terminal may be connected to an output terminal (backup signal output terminal 10c, to be described later) for outputting a backup signal from the vehicle 10 via wiring such as a cable. Note that the term "terminal" includes plugs, jacks, connectors (such as waterproof connectors), giboshi terminals, etc., and the term "wiring" includes NTSC (National Television Standards Committee) cables, etc.

[0026] <Operating state S1, standby state S2, rest state S0> As shown in FIGS. 2 and 3, the operating state S1 is a state in which the above-described device system 1 is started in response to the ON accessory interlock signal ACC output from the vehicle 10 when the above-described accessory switch is turned ON. It can be said that when the device system 1 is in such an operating state S1, the determination unit 2 described later operates, and determination by the determination unit 2 can be performed, or in addition, the functions of the device system 1 operate normally.

[0027] As shown in FIGS. 2 and 3, the standby state S2 is a state in which the above-described device system 1 stops in response to the OFF accessory interlock signal ACC output from the vehicle 10 when the accessory switch is turned OFF in the above-described operating state S1. When the accessory interlock signal ACC becomes ON in this standby state, the device system 1 becomes the operating state S1 by a standby start K2 described later. When the device system 1 enters such a standby state S2, although the functions such as the determination by the determination unit 2 described later and other functions of the device system 1 do not operate, it returns to the operating state S1 again in a shorter second time T2 by the standby activation K2 described later.

[0028] As shown in FIG. 3, the rest state S0 is a state in which, after the accessory interlock signal ACC does not turn on after entering the above-described standby state S2 and a predetermined third time T3 has elapsed (in other words, after the time (standby time) t after entering the standby state S2 exceeds the third time T3), the standby state S2 is released. When the accessory interlock signal ACC turns on when the rest state S0 is entered, the device system 1 enters the operating state S1 by the main activation K1 described later. It can also be said that the third time T3 is the standby upper limit time. Here, the value of the third time T3 is not particularly limited. For example, it may be longer than the first time T1 described later. In addition, regardless of the first time T1, it may be 0 minutes (0 seconds) or more and less than 1 minute (60 seconds), or 1 minute (60 seconds) or more and 60 minutes (3600 seconds) or less. When the device system 1 enters such a rest state S0, the functions such as the determination by the determination unit 2 described later and other functions of the device system 1 do not operate, and it returns to the operating state S1 again in the first time T1 by the main activation K1 described later. In addition, in FIGS. 2 and 3, the “operating state S1” - “stop ACC / OFF” → “standby state S2” means that during the operating state S1, when the accessory interlock signal ACC becomes OFF, it stops and enters the next “standby state S2”. The “standby state S2” - “standby start K2 ACC / ON T2” → “operating state S1” in FIG. 2 means that during the standby state S2, when the accessory interlock signal ACC becomes ON, it starts at the second time T2 and enters the next “operating state S1”. The “standby state S2” - “t>T3” No - “standby start K2 ACC / ON T2” → “operating state S1” in FIG. 3 means that during the standby state S2, when the standby time t is less than or equal to the third time T3 and the accessory interlock signal ACC becomes ON, it starts at the second time T2 and enters the next “operating state S1”. The “standby state S2” - “t>T3” Yes → “rest state S0” in FIG. 3 means that during the standby state S2, when the standby time t exceeds the third time T3, it enters the next “rest state S0”. The “rest state S0” - “main start K1 ACC / ON T1” → “operating state S1” in FIG. 3 means that during the rest state S0, when the accessory interlock signal ACC becomes ON, it starts at the first time T1 and enters the next “operating state S1”.

[0029] <main start K1, standby start K2> As shown in FIGS. 2 and 3, the main start K1 can be said to be a so-called full-scale start in the device system 1. It is a start that makes the device system 1 enter the operating state after a predetermined first time T1 has elapsed since the accessory interlock signal ACC became ON. In addition, the first time T1 can also be said to be the main start time. Here, the value of the first time T1 is not particularly limited. For example, it may be 12 seconds or less, or 2 seconds or more and 20 seconds or less, preferably 3 seconds or more and 12 seconds or less, and more preferably 4 seconds or more and 10 seconds or less. When the present activation K1 activates the device system 1, as described above, in addition to the case where the accessory interlock signal ACC turns ON when the device system 1 changes from the standby state S2 to the sleep state S0, the device system 1 may also be activated by the present activation K1 when the device system 1 is retrofitted to an existing vehicle 10 for the first time and activated, or when the power switch of the device system 1 provided in the vehicle 10 is turned ON while it is OFF.

[0030] As shown in FIGS. 2 and 3, the standby activation K2 can be said to be an activation from the standby state S2. When the device system 1 enters the standby state S2, after the accessory interlock signal ACC turns ON and a predetermined second time shorter than the above-described first time T1 has elapsed, the device system 1 is activated to enter the operating state. The second time T2 can also be said to be the standby activation time. Here, the value of the second time T2 is not particularly limited as long as it is shorter than the first time T1. For example, it may be 1 second or less, or may be 0.1 second or more and 2.0 seconds or less, preferably 0.2 second or more and 1.0 second or less, and more preferably 0.3 second or more and 0.7 second or less.

[0031] <Determination unit 2> As shown in FIGS. 1, 4 to 10, the determination unit 2 is a part that determines the presence of a person H in the image G captured by the camera. It recognizes the person H in the image G and determines whether the person H is present in the later-described danger area R1 or detection area R2 that is set. The specific configuration of the determination unit 2 is not particularly limited. For example, it may include a recognition part 2a and a determination part 2' described later, or may include a detection part 2b and a setting part 2c described later. In addition, in the determination unit 2, a settable area R0 described later may be set in addition to the danger area R1 and the detection area R2.

[0032] <Recognition part 2a> As shown in FIGS. 1 and 5, the recognition part 2a is a part that recognizes the person H in the above-described image G and can also be said to be a part of the determination unit 2. The specific configuration of the recognition part 2a is not particularly limited. For example, based on the outline of person H, etc., person H in the image G may be recognized as an actual person, or may be recognized as an actual person by an object recognition algorithm. In addition, the recognition part 2a may remove noise, distortion, etc. of the image G, emphasize the outline, etc. of the person H in the image G, adjust the brightness and color tone (color tone, hue, and saturation) of the image G, cut out the area of the person H, etc. from the image G (region extraction), or display the extracted person H circumscribed by a rectangle (rectangle or square) (display the bounding box B). Hereinafter, the recognition part 2a will be mainly described on the assumption that it displays the bounding box B surrounding the person H.

[0033] In the recognition part 2a, any image processing may be performed as long as the outline of the person H can be extracted from the image G, etc. For example, for the change in the pixel value (brightness, etc. of each pixel) in the image G, it may be extracted from the maximum and minimum values calculated by the first derivative (by the gradient method), or may be extracted from the inflection points calculated by the second derivative (by the Laplacian method). In addition, the outline of the person H, etc. may be extracted from the image G by using a gradient method using smoothing or weighted average, the Canny method, or the like. Also, the extraction of the outline of the person H, etc. from the image G itself may be performed by artificial intelligence (AI), machine learning (ML), deep learning (DL), etc., or may be performed by an inference process that applies the result to an inference model generated by machine learning, deep learning, etc.

[0034] Such a recognition part 2a can be said to be able to automatically learn the feature amounts (patterns) of the person H or the entire person H just by inputting the image G. Note that, for the recognition part 2a, a plurality (for example, tens of thousands) of images G labeled (with correct answers) regarding which lines in the image G are the contours of the person H and which part (which area, etc.) is the person H are given in advance (so to speak, performing supervised (with correct answers) learning). The recognition part 2a automatically learns the feature amounts, and for the image G input to the recognition part 2a, from the previously learned feature amounts, it outputs, as scores (such as a ratio of 96%, etc.), which lines are the contours of the person H and which part is the person H, or outputs an image G with a display indicating that the person H has been recognized for the input image G. In addition, the recognition part 2a may perform unsupervised learning, reinforcement learning, or the like.

[0035] <Danger area R1, detection area R2, settable area R0, etc.> As shown in FIGS. 4, 5, 7 to 10, in the device system 1 having the determination part 2, at least a danger area R1 and a detection area R2 are set, and in addition, a settable area R0 may also be set. The danger area R1 is the area closest to the device system 1, the detection area R2 is the area next closest to the device system 1 relative to the danger area R1 (farther than the danger area R1), and the settable area R0 is the outer peripheral area farther from the device system 1 than the detection area R2. Note that the settable area R0 may be the settable area in FIGS. 8 to 10. This settable area indicates the range in which the danger area R1 and the detection area R2 can be set wider, and can also be said to be the outer peripheral area. The specific configurations of these areas R1, R2, and R0 are not particularly limited. For example, each of the areas R1, R2, and R0 may be set within the image G captured by the camera. In this case, it can be said that each of the areas R1, R2, and R0 is set within the size of the image G (for example, width: 720 pixels, height: 480 pixels). The position, shape, size, etc. of each of the areas R1, R2, and R0 are determined according to parameters of the camera that captures the image G (the vertical direction (pitch angle) of the optical axis of the camera, the angle of view (the angle of the captured range, such as 138° in the horizontal direction)), the height of the person H to be determined (for example, the average height of an adult), the "actual distance" from the camera mounting position in the vehicle (such as the forklift 10A, etc.) 10 described later, etc.

[0036] Specifically, in the case of the danger area R1, the position, shape, size, etc. within the image G are, for example, such that the center is approximately at the center of each of the device system 1 and the vehicle 10, and the shape is approximately circular or elliptical at the camera mounting position, or it may be a shape with a part missing from these shapes (such as a substantially sector shape with a predetermined central angle) (see FIGS. 4 and 5). The values of their radii are not particularly limited. For example, they may be the "distance within the image G" corresponding to an "actual distance" greater than 0.0 m and less than or equal to 5.0 m, preferably the "distance within the image G" corresponding to an "actual distance" of 0.1 m or more and 4.0 m or less, and more preferably the "distance within the image G" corresponding to an "actual distance" of 0.5 m or more and 3.0 m or less. In addition, the position, shape, and size of the danger area R1 may be the "position, shape, and size within the image G" corresponding to the "actual position, shape, and size" in the plan view shown in FIGS. 8 to 10. Also, in the case of the detection area R2, the position, shape, size, etc. within the image G are as follows. For example, the center is at approximately the center of each of the device system 1 and the vehicle 10, which is the same as the above-described danger area R1 and is the camera mounting position. Moreover, it may be in a substantially circular or substantially elliptical annular shape (doughnut shape), or a shape with a part missing from these shapes (such as a substantially annular sector shape with a predetermined central angle) (see FIGS. 4 and 5). The value of its width is not particularly limited. For example, it may be the "distance within the image G" corresponding to an "actual distance" greater than 5.0 m and less than or equal to 8.0 m from the same center, preferably the "distance within the image G" corresponding to an "actual distance" greater than 4.0 m and less than or equal to 6.5 m, or more preferably the "distance within the image G" corresponding to an "actual distance" greater than 3.0 m and less than or equal to 5.0 m. Additionally, the position, shape, and size of the detection area R2 may also be the "position, shape, and size within the image G" corresponding to the "actual position, shape, and size" in the plan view shown in FIGS. 8 to 10. Furthermore, in the case of the settable area R0, the position, shape, size, etc. within the image G are as follows. For example, the center is at approximately the center of each of the device system 1 and the vehicle 10, which is the same as the above-described danger area R1 and detection area R2 and is the camera mounting position. Moreover, it may be in a substantially circular or substantially elliptical annular shape (doughnut shape), or a shape with a part missing from these shapes (such as a substantially annular sector shape with a predetermined central angle) (see FIGS. 4 and 5). The value of its width is not particularly limited. For example, it may be the "distance within the image G" corresponding to an "actual distance" greater than 8.0 m and less than or equal to 20.0 m from the same center, preferably the "distance within the image G" corresponding to an "actual distance" greater than 6.5 m and less than or equal to 15.0 m, or more preferably the "distance within the image G" corresponding to an "actual distance" greater than 5.0 m and less than or equal to 12.0 m. Note that the width of the settable area R0 may not have an upper limit. In that case, for example, it may be the "distance within the image G" corresponding to an "actual distance" greater than 8.0 m, preferably greater than 6.5 m, and more preferably greater than 5.0 m from the same center. Additionally, the position, shape, and size of the settable area R0 may also be the "position, shape, and size within the image G" corresponding to the "actual position, shape, and size" in the plan view shown in FIGS. 8 to 10.

[0037] In addition, instead of being provided within the image G, each of the areas R1, R2, and R0 may be set by detecting the "actual distance". In this case, the setting of each of the areas R1, R2, and R0 means that a range where the detected "actual distance" is, for example, within a predetermined first distance D1 (such as 0.0 m or more and 5.0 m or less) is set as the dangerous area R1, a range of a predetermined second distance D2 longer than the first distance D1 (such as greater than 5.0 m and 8.0 m or less) is set as the detection area R2, and a range of a predetermined settable distance D0 longer than the second distance D2 (such as greater than 8.0 m and 20.0 m or less, or greater than 8.0 m) may be set as the settable area R0. Also, the timing for detecting the "actual distance" may be when the above-described recognition part 2a recognizes the person H within the image G, and the distance to the recognized person H becomes the "actual distance". The detection of the "actual distance (distance to the person H)" may be performed by the detection part 2b, and the detection part 2b will be described below.

[0038] <Detection part 2b> As shown in FIGS. 1 and 5, the detection part 2b is a part that detects the distance to the person H within the image G recognized by the above-described recognition part 2a (that is, the "actual distance"), and can also be said to be a part of the determination part 2. In addition, the "distance to the person H" in the present invention means the distance from the device system 1 to the person H (the distance between the device system 1 and the person H), and may also mean the distance from the camera, or the distance from a vehicle such as the forklift 10A provided with the device system 1 (the front end or the rear end of the vehicle, and more precisely, the front left corner, the front right corner, the rear left corner, or the rear right corner) 10. The specific configuration of the detection part 2b is not particularly limited. For example, the distance to the lower side of the bounding box B (so to speak, the feet of the person H or the position where the person H stands) displayed via the recognition part 2a, or the intersection of the diagonal lines in the bounding box B, etc. may be detected as the distance to the person H. Hereinafter, the distance to the person H will be mainly described as the distance to the lower side of the bounding box B (so to speak, the feet of the person H).

[0039] The distance to such a person H, for example, when the camera for imaging the image G is a compound eye type (a stereo camera with two cameras or two cameras), in the case of two cameras installed separately in the left - right direction or the like, for each image G obtained by imaging the same object (person H, bounding box B, etc.) with the two cameras, there are differences. The greater the parallax, which is a parameter obtained by quantifying this difference, the greater the distance to the object. Using the principle that the distance to the object increases, the distance to person H is calculated using the formula "distance to person H=(distance between the two cameras×focal length of each camera)÷parallax". In addition, within each image G captured by the two cameras, whether it is the same object can be determined, for example, by a process of considering that if the same pattern appears in the two cameras, it is regarded as the same object.

[0040] When the camera for imaging the image G is a monocular type (one camera), for example, it may be a Tof (Time of Flight) camera that irradiates light (such as infrared light) from the camera and measures the time until the light is reflected by the object and returns to the camera to obtain the distance. Alternatively, the distance to the object may be calculated by a process of calculating from the shape and color tone of the blur due to aberration (lens aberration) according to the pixel position in the captured image G. In addition, in the process of calculating from the shape and color tone of the blur (pinhole blur, out - of - focus blur), for the association between the distance to the object and the shape and color tone of the blur, it may be performed by artificial intelligence (AI), machine learning (ML), deep learning (DL), deep neural network (DNN, Deep Neural Network), etc., or by an inference process of applying to an inference model generated by these learnings and obtaining a result.

[0041] <Setting part 2c> As shown in FIGS. 1, 4, 5, 7 to 10, the positions, shapes, sizes, etc. of each area R1, R2, R0 such as the above - mentioned dangerous area R1 and detection area R2 may be set by the setting part 2c, and this setting part 2c may be able to adjust (set) the dangerous area R1, detection area R2, etc. Therefore, the setting part 2c is a part for setting the above-described danger area R1, detection area R2, etc., and can also be said to be a part of the determination unit 2. When the areas R1, R2, and R0 are set in the image G, the adjustment by the setting part 2c can be performed for each pixel of the areas R1, R2, and R0, or can also be performed for each predetermined pixel (for example, for each 10 pixels). When the "actual distance" is detected and set, the distance to the person H can be adjusted for each 0.1 m or for each 1.0 m.

[0042] The adjustment by the setting part 2c may be performed together with the adjustment of the orientation of the camera 20 when installing the camera 20 on a vehicle 10 such as a forklift 10A, or may also be adjusted while using the vehicle 10 on which the camera 20 is installed. Among the areas R1, R2, and R0, only the danger area R1 and the detection area R2 may have their positions, shapes, sizes, etc. adjusted by the setting part 2c. Also, among the areas R1, R2, and R0, the positions, shapes, sizes, etc. of the danger area R1, the detection area R2, etc. may be changed according to the traveling direction of the vehicle 10 and its speed, which will be described later. Furthermore, the setting part 2c may be configured to perform not only the adjustment of the areas R1, R2, and R0 but also various other settings. The setting by the setting part 2c described so far may be performed via the communication part 5, which will be described later. In this case, it can also be said that the communication part 5 is included in the setting part 2c.

[0043] <Determination unit 2'> As shown in FIGS. 1, 4 to 10, the determination part 2' is a part for determining whether or not the person H recognized by the above-described recognition part 2a exists in the above-described danger area R1 and whether or not the recognized person H exists in the above-described detection area R2, and can also be said to be a part of the determination unit 2. The determination part 2' may also determine whether or not the recognized person H exists in the above-described settable area R0.

[0044] Whether a person H exists within each area R1, R2, etc. in the determination part 2' can be determined by whether the numerical values (X value and Y value, or height position value and horizontal length value) of the lower side (the feet of the person H) of the bounding box B described above are within the numerical values (X value and Y value) indicating the dangerous area R1 set within the image G, whether they are within the numerical values indicating the detection area R2, or whether they are within the settable area R0, when each area R1, R2, etc. is provided within the image G. In addition, when the determination part 2' uses the "actual distance" detected by the detection part 2b described above, the determination part 2' may determine whether the value of the "actual distance" is within the range of the first distance D1 indicating the dangerous area R1, whether it is within the range of the second distance D2 indicating the detection area R2, or whether it is within the range of the settable distance D0 indicating the settable area R0. Incidentally, if the positions, shapes, and sizes of the areas R1, R2, and R0 are the "actual positions, shapes, and sizes" in the plan view shown in FIGS. 8 to 10, the determination part 2' may determine whether they are within the ranges of the first distance D1, the second distance, and the settable distance D0 indicating the areas R1, R2, and R0, according to the direction (orientation) from the camera to the position where the person H exists, so as to be the "actual positions, shapes, and sizes" in the plan view shown in FIGS. 8 to 10. The determination part 2' may determine whether a person H exists within each area R1, R2, etc. by artificial intelligence (AI), machine learning, deep learning, etc., or by inference processing after these learnings. The determination by the determination part 2' may be repeated at regular intervals, and thus it can be said that the determination part 2' determines the presence of a person in real time (without predicting the future, but the presence of a person each time). Incidentally, if the device system 1 is provided with the notification part 3, each time the determination part 2' determines the presence of a person in real time, it may notify a user such as a driver of the vehicle 10 like the forklift 10A.

[0045] <Notification part 3> As shown in FIGS. 1 and 11, the notification unit 3 is a part that notifies the outside of the system (equipment system 1) that the determination unit 2 has determined the presence of the person H in the image G. The notification unit 3 may be, for example, a speaker 3a capable of outputting predetermined voice data (including those externally attached (retrofit) to the equipment system 1 and the vehicle 10), and the volume of the voice data (voice) can also be selected from large, medium, small, etc., or it may be the speaker of the monitor 3b described later. In addition, the notification unit 3 may be a rotating light (for example, a red rotating light notifies the presence of the person H in the danger area R1, and a yellow rotating light notifies the presence of the person H in the detection area R2, etc.), or a buzzer or horn that makes a sound when the determination unit 2 determines the presence of a person (the way of making a sound is different depending on whether the person H exists in any of the areas R1, R2, etc.), or an LED strobe light that lights up when the determination unit 2 determines the presence of the person H (the way of lighting is different depending on whether the person H exists in any of the areas R1, R2, etc.), or a monitor 3b on which predetermined characters, colors, etc. are displayed when the determination unit 2 determines the presence of the person H (the display content is different depending on whether the person H exists in any of the areas R1, R2, etc., such as an NTSC monitor), etc., or these may coexist. These notification units 3 may be connected via wiring such as cables and terminals to the casing 6 (so to speak, the main body of the equipment system 1) described later. Furthermore, the notification unit 3 may output a digital (or analog) warning signal. In this case, for example, the output of this warning signal may be configured to be set to any of no output, output only when the presence of the person H is determined in the danger area R1, and output when the presence of the person H is determined in both the danger area R1 and the detection area R2. Note that the output destination of this warning signal is not particularly limited, and may be, for example, a patrol light (registered trademark) outside the equipment system 1, a warning light, a rotating light, a stacked signal light, etc. Note that the notification unit 3 (equipment system 1) may have a terminal (warning signal output terminal) 1d for outputting the above-described warning signal to the vehicle 10 side, etc., and may be connected via wiring such as a cable to the input terminal (warning signal input terminal 10d) of the vehicle 10 described later. Hereinafter, the notification unit 3 will be mainly described assuming that it is the speaker 3a.

[0046] The notification unit 3 may be switchable between valid / invalid in the above-described setting portion 2c or the like to notify that the presence of the person H has been determined, and the notification content may be selected from text (predetermined content), or text data (character data) may be used by the user or the like using the above-described setting portion 2c, or may be set (input / changed, etc.) via the communication unit 5 described later, or may be created by converting this text data into voice data or the like via the communication unit 5 described later. When the notification unit 3 is attached to a vehicle 10 such as a forklift 10A, etc., it may be linked to the state of the vehicle 10 or the like (for example, notify only when moving forward or backward (conversely, do not notify when stopped)), and may continue to notify a predetermined content while a certain state of the vehicle 10 or the like continues. In addition, the notification unit 3 may switch and notify the content with a higher priority (importance) even during notification. For example, when it is determined that the person H is present in the detection area R2 and it is determined that the person H is present in the danger area R1 during the notification of the content corresponding thereto, the content being notified may be interrupted (stopped), and the content when it is determined that the person H is present in the danger area R1 with a higher priority may be notified. Also, the notification unit 3 may be able to switch the output destination of the content to be notified. For example, as the output destination, it may be possible to switch between an externally attached speaker 3a or the speaker of the monitor 3b.

[0047] <Storage unit 4> As shown in FIG. 1, the storage unit 4 is a part that stores (or records, memorizes) the image G itself captured by the camera and / or the moving image including the image G. The storage unit 4 may be, for example, a USB (Universal Serial Bus) memory (a storage device used by connecting to a USB connector) that stores an image G or a video. Alternatively, the storage unit 4 may be a magnetic tape, DDS (Digital Data Storage), a magnetic disk such as a floppy (registered trademark) disk (FD) or a hard disk drive (HDD), an optical disk such as a compact disk (including CD), a laser disk (registered trademark) (LD), a digital versatile disk (DVD), a Blu-ray disk (BD), a magneto-optical disk such as a mini disk (MD), a flash memory such as a flash disk (Solid State Drive, SSD) or a memory card, or a memory (such as RAM) built in a PC, a mobile terminal, a digital camera, etc. Hereinafter, the storage unit 4 will be mainly described as a USB memory.

[0048] When the storage unit 4 stores a video including the image G, the frame rate of the video file (the number of image Gs per second) may be set to a predetermined value, and the stored image frames may be thinned out. The frame rate and the recording time of one video file may be changeable at the setting portion 2c or the like. The value of this frame rate is, for example, 10 frames / second (FBS) in the standard setting, and may be changeable at the setting portion 2c or the like within the range of 1 FPS or more and 10 FPS or less. The recording time is, for example, 2 minutes in the standard setting, and may be changeable at the setting portion 2c or the like within the range of 1 minute or more and 10 minutes or less. When the equipment system 1 equipped with the storage unit 4 is attached to a vehicle 10 such as a forklift 10A, after starting the equipment system 1, while the accessory power supply (accessory interlock signal ACC) of the vehicle 10 etc. is ON (in other words, the engine of the vehicle 10 etc. is ON), for each set recording time set, the equipment system 1 may continue to save the newly created video file to the storage unit 4. Additionally, when a plurality of cameras are attached to the vehicle 10 etc. both front and rear together with the storage unit 4, for example, when the vehicle 10 is moving forward, an image or video obtained by combining images G captured by the plurality of front cameras may be saved to the storage unit 4, or when the vehicle 10 is moving backward, an image etc. obtained by combining images G captured by the plurality of rear cameras may be saved to the storage unit 4, or when the vehicle 10 is stopped, an image G or video captured by any one of the plurality of cameras both front and rear may be saved to the storage unit 4. In addition, when the storage capacity of the storage unit 4 such as a USB memory becomes full, the oldest video file may be deleted.

[0049] <Communication unit 5> As shown in FIGS. 1 and 11, the communication unit 5 is a part that communicates with the outside of the system (equipment system 1). The communication unit 5 is also a part that communicates wirelessly or by wire with a vehicle 10 such as a forklift 10A to which the equipment system 1 is attached, a computer such as a PC located at a place separated from the equipment system 1, and a communication device terminal 5a such as a tablet, smartphone, or mobile phone, via the Internet or other communication networks or directly. In particular, the communication unit 5 that communicates wirelessly may, for example, support the function of Wi-fi (registered trademark, including its access point), and in this case, the communication unit 5 When the communication unit 5 (machine system 1) is used as an access point (see Fig. 11(a)), and It may be configured to be able to select any of the cases when used via an access point such as another router 5b different from the communication unit 5 (see Fig. 11(b)). Note that when used via an access point such as another router 5b different from the communication unit 5, one communication device terminal 5a may communicate with a plurality of device systems 1 or vehicles 10. Also, the same may apply when the communication unit 5 supports the function of Bluetooth (registered trademark, including its access point).

[0050] In addition, the communication unit 5 may be a USB - type data communication terminal corresponding to the functions of Category 4 (Cat.4) in LTE (Long Term Evolution), 3G, GSM (Global System for Mobile Communications), or other mobile phone communication functions such as 4G and 5G, or it may be a SIM (Subscriber Identity Module). It is possible to download the image G and video (such as a plurality of consecutive images G) stored in the above - mentioned storage unit 4 via the communication unit 5, and conversely, it is also possible to upload a predetermined image or video. Also, the determination unit 2, etc., may be configured by a server, or may be configured in cooperation or sharing with a computer such as the vehicle 10 and a server. In addition, it is possible to acquire data being evaluated or perform updates of the device system 1 (software, learning data such as AI, etc.). With such a communication unit 5, maintenance of the device system 1 and the above - mentioned settings may be performed. In this case, operations for maintenance and settings (adjustment of predetermined values, requests, commands, etc.) may be performed using a web browser on the communication device terminal 5a located at a place separate from the device system 1. In addition, when the device system 1 including the communication unit 5 is attached to a vehicle 10 or the like, after the device system 1 is activated, after the accessory power supply (accessory interlock signal ACC) of the vehicle 10 or the like is turned ON (in other words, the engine of the vehicle 10 or the like is turned ON), the communication unit 5 may be in a state where maintenance and settings via the communication unit 5 are valid (possible) for a predetermined time (for example, 10 minutes). Also, when there is an operation of maintenance or setting via the communication unit 5 during the valid state of maintenance or setting, the valid state may be extended.

[0051] <Casing 6> As shown in FIGS. 1 and 11, the casing 6 is a housing that incorporates the determination unit 2 and the like described above. In addition to the determination unit 2, the casing 6 may also incorporate the storage unit 4 and the communication unit 5 described above. The casing 6 may have connection terminals (such as the accessory input terminal 1a, the constant power input terminal 1b, the back signal input terminal 1c, and the warning signal output terminal 1d described above, or terminals of the wiring connecting the notification unit 3 and the casing 6) for connecting to cables from vehicles 10 such as the notification unit 3, the camera (camera 20), and the forklift 10A described later. In addition, when the storage unit 4 is a USB memory or the communication unit 5 is a USB type data communication terminal, the casing 6 may have a USB port capable of connecting these. When the communication unit 5 is a SIM, the casing 6 may have a tray or a slot for connecting this. Also, the casing 6 may have the setting portion 2c provided on its outer surface, or even if the setting portion 2c is incorporated in the casing 6, a part of the setting portion 2c may be exposed. The casing 6 may or may not incorporate the notification unit 3. Regarding the vehicle 10 provided with the device system 1 described so far, the following will be described.

[0052] <Vehicle 10> As shown in FIGS. 1, 4 to 10, the vehicle 10 according to the present invention is a vehicle provided with the device system 1 described above. Here, the "vehicle 10 provided with the device system 1" in the present invention not only means a vehicle in which the device system 1 (determination unit 2) is provided in the vehicle 10 (so to speak, the vehicle 10 has the device system 1), but also includes cases where, among the camera 20 and the device system 1, the notification unit 3 etc. are mounted, but the determination unit 2 etc. are located at a place different from the vehicle 10 (that is, not provided in the vehicle 10), and are connected to the device system 1 via the communication unit 5, the Internet, etc., and determinations by the device system 1 are performed.

[0053] The vehicle 10 outputs the above-described accessory interlock signal ACC to the above-described device system 1. The vehicle 10 may have a terminal (accessory output terminal) 10a for outputting the accessory interlock signal ACC to the device system 1, or may also have other terminals such as a terminal (constant power supply output terminal 10b) for outputting power to the device system 1, a terminal (back signal output terminal) 10c for outputting a back signal to the device system 1, and a terminal (warning signal input terminal) 10d for inputting a warning signal from the device system 1. These terminals may be respectively connected to the terminals on the device system 1 side (accessory input terminal 1a, constant power supply input terminal 1b, back signal input terminal 1c, warning signal output terminal 1d) via wiring such as a cable. In particular, the constant power supply output terminal 10b may be provided with a plus (+) terminal and a minus (-) terminal, similar to the constant power supply input terminal 1b. In addition, the drive source of the vehicle 10 may be a motor etc. in addition to the engine, and the constant power supply for outputting (supplying) power to the device system 1 may be a power source from an auxiliary battery (battery for accessory power) instead of a battery for motor drive.

[0054] The vehicle 10 may have a camera 20 for imaging the above-described image G. The vehicle 10 may have a provided device system 1 having a determination unit 2, and may also have parts having functions other than the determination by the determination unit 2. Vehicle 10 is such that the determination unit 2 of the equipment system 1 determines at least the traveling direction of the vehicle 10 based on at least the image G. Here, the traveling direction of the vehicle 10 may also include the turning direction of the vehicle 10. Also, the equipment system 1 (determination unit 2) provided in the vehicle 10 may also determine the speed in the traveling direction of the vehicle 10. Furthermore, the determination unit 2 provided in the vehicle 10 may change at least the above-described danger area R1 and / or detection area R2 in conjunction with at least the traveling direction of the vehicle 10. Such a vehicle 10 is not particularly limited. For example, it may be a forklift 10A, or other industrial vehicles other than the forklift 10A, or general vehicles such as trucks and buses may also be acceptable. Hereinafter, the vehicle 10 will be mainly described as being a forklift 10A.

[0055] <Forklift 10A> As shown in FIGS. 1, 4 to 10, the forklift 10A is a type of the above-described vehicle 10. The forklift 10A has forks 11A for loading (stacking) luggage, goods, and materials, a mast 12A for raising and lowering the forks 11A, a head guard 13A for protecting the driver, etc., and can also be said to be a powered (also called a drive source, such as an engine or a motor) cargo handling and transporting vehicle. The forks 11A lift luggage, goods, and materials via a pallet or the like. The mast 12A can also be said to be a rail for raising and lowering the above-described forks 11A, and may have a chain, a cylinder, or the like for raising and lowering the forks 11A. The head guard 13A is a guard for preventing a dropped luggage or the like from hitting the driver. In addition, the forklift 10A is provided with a balance weight, drive wheels, and steering wheels (wheels whose direction changes by turning the steering wheel). The balance weight is also called a counterweight and has weights corresponding to the cargo handling and the load to be transported. The drive wheels are the front wheels, may be larger than the steering wheels, and transmit power. The steering wheels may be the rear wheels and perform steering. In the forklift 10, a camera 20 may be provided on the head guard 13A or the like described above. Next, the camera 20 that captures the image G will be described.

[0056] <Camera 20> As shown in FIGS. 1, 4 to 11, the camera 20 is an apparatus (imaging device) that captures the above-described image G. The camera 20 may be attached to a position where the front, rear, or inside of the vehicle 10 such as the forklift 10A is within the image G, and the forklift 10A itself does not have to be reflected in the image G. For one forklift 10A or the like, only one or two or more cameras 20 may be attached, regardless of whether it is a compound-eye type or a single-eye type. Also, the lens of the camera 20 is not particularly limited, and for example, it may be a wide-angle lens. For example, in the forklift 10A, the camera 20 may be attached to the above-described head guard 13A, or may be attached to the upper part of the mast 12A, the upper rear end of the rear bonnet of the forklift 10A, etc., and the optical axis of the lens may be directed obliquely downward. Hereinafter, the camera 20 will be mainly described on the assumption that it is attached to the head guard 13A of the forklift 10A.

[0057] Specific attachment positions of the camera 20 on the head guard 13A are, for example, on the left and right front end sides (front left corner, front right corner) and the left and right rear end sides (rear left corner, rear right corner) of the upper surface of the head guard 13A in a plan view. The camera 20 may be attached to each of them, and a total of four cameras 20 may be provided for one forklift 10A. Also, for one forklift 10A, cameras 20 may be installed on the front end side and the rear end side of approximately the center of the left and right of the upper surface of the head guard 13A, respectively, and a total of two cameras 20 may be provided. In addition, only one camera 20 may be provided on the rear end side or the front end side of approximately the center of the left and right of the upper surface of the head guard 13A, or five or more cameras 20 may be provided for one forklift 10A. Hereinafter, mainly described is a case where cameras 20 are respectively attached to the front left corner, front right corner, rear left corner, and rear right corner of the upper surface of the head guard 13A of a single forklift 10A (that is, a total of four cameras 20 including the front left camera 20a1, front right camera 20a2, rear left camera 20b1, and rear right camera 20b2 are attached).

[0058] The range of the image G captured by such a camera 20 is not particularly limited. However, when a wide-angle lens is used, for example, the sky may be reflected in a part of the upper side within the image G, or a part of the forklift 10A may be reflected in a part of the image G. Also, a point at a predetermined distance (such as about 10 to 30 m behind) from the forklift 10A may be reflected in the image G. Next, regarding the determination of the traveling direction and its speed based on the image G, first, a device system 1 (such as a determination unit 2) for determining the traveling direction in the forklift 10A will be described. Note that the traveling direction includes not only forward (forward state) and backward (backward state), but also a stopped state (stopped state), and may also include a turning direction.

[0059] <Determination of the traveling direction in the forklift 10A> The determination of the traveling direction in the forklift 10A may be determined by the device system 1 (determination unit 2) in the forklift 10A based on at least the above-described image G. In addition, the presence or absence of an input of a back signal from the forklift 10A to the device system 1 (a signal that outputs ON when the forklift 10A is moving backward (or when the reverse gear is engaged) and outputs OFF when the forklift 10A is not moving backward (or when the reverse gear is not engaged)) may also be considered for the determination. In addition, the determination of the turning direction in the forklift 10A may be determined by the determination unit 2 in the forklift 10A based on at least the above-described image G. Alternatively, it may be determined by inputting a steering signal (a signal output when the position of the steering wheel (steering operation) is turned left or right) from the forklift 10A to the equipment system 1.

[0060] <Determination of the traveling direction (including the turning direction and speed) based on the image G> The equipment system 1 (determination unit 2) in the forklift 10A performs at least the determination of the traveling direction based on the image G captured by the above-described camera 20. Regarding the specific determination of the traveling direction based on the image G, there is no particular limitation. For example, in the image G, corner points with edges (for example, corners of shelves in a warehouse, corners of arrows or lines indicating paths drawn on the floor, corners of buildings, etc.) are used as feature points (one or more), and a process of calculating a displacement vector is performed from the coordinates of the feature points in the image G of the previous frame and the coordinates of the feature points in the image G of the current frame. After that process, in the image G, the number of displacement locations (displacement vectors directed upward or downward) in the upward or downward direction is counted, and a process of detecting the direction with the larger number of upward and downward directions as the vertical displacement direction of the image G is performed. After that process, in the image G, if the vertical displacement direction is upward, it is determined that the forklift 10A is traveling forward (forward movement determination), and if the displacement direction is downward, it is determined that the forklift 10A is traveling backward (backward movement determination). If the detected numbers of upward and downward directions are substantially the same, etc., the determination unit 2 may perform a process of determining that the forklift 10A is stopped (stop determination). Such a process may be repeated at regular intervals, and thus it can be said that the determination unit 2 determines the traveling direction of the forklift 10A in real time (without predicting the future, but the traveling direction at each moment).

[0061] In addition, the specific determination of the traveling direction based on the image G may include the determination of the turning direction. In this case, within the image G, corner points with edges are used as feature points (one or more), and a process of calculating a displacement vector is performed based on the coordinates of the feature points in the image G of the previous frame and the coordinates of the feature points in the image G of the current frame. After that process, within the image G, not only the number of displacement locations in the upward or downward direction (displacement vectors directed upward or downward) but also the number of displacement locations in the leftward or rightward direction (displacement vectors directed leftward or rightward) are counted, and a process of detecting the direction with the larger number between the upward and downward directions and the direction with the larger number between the leftward and rightward directions as the vertical displacement direction and the horizontal displacement direction of the image G is performed. After that process, within the image G, based on the average value of the scalars of the displacement vectors having a scalar of a certain level or more, a process of determining the speed in each direction according to the magnitude of the displacement vector (such as large or small) is performed. After that process, within the image G, based on the horizontal displacement direction, the magnitude of the displacement vector, and the vertical displacement direction, the traveling direction and the turning direction of the forklift 10A are determined. Whether the forklift 10A turns is determined by the horizontal displacement direction, and the side of the tire serving as the turning axis has a smaller magnitude of the displacement vector (it can be known whether the turning axis of the forklift 10A is on the left side or the right side based on the magnitude of the displacement vector), and a process of determining the traveling direction (such as forward or backward) of the forklift 10A by the vertical displacement direction may be performed. Such a process may also be repeated at regular time intervals, and thus it can be said that the determination unit 2 determines the traveling direction of the forklift 10A in real time (without predicting the future, but the traveling direction at each moment).

[0062] <Determination Transition of Forklift 10A> As shown in FIG. 6, among the determinations of the traveling direction of the forklift 10A, the forward determination, the backward determination, and the stop determination undergo the following determination transitions (state transitions). The determination transition of (1) in FIG. 6 (transition from "stop determination" to "forward determination") may occur when the determination based on image G is a forward determination and the back signal is OFF. Note that this determination transition may occur only when the determination based on image G is a forward determination. Here, the forward determination includes not only the case where it is determined as a forward determination even in one frame in image G, but also the case where it is continuous for a predetermined number of frames (for example, 2 frames, 3 frames, etc.). The determination transition of (2) in FIG. 6 (transition from "forward determination" to "stop determination") may occur when the determination based on image G is other than a forward determination and the back signal is OFF. Note that this determination transition may occur only when the determination based on image G is other than a forward determination. Here, other than the forward determination includes not only the case where it is determined as other than a forward determination even in one frame in image G, but also the case where it is continuous for a predetermined number of frames (for example, 6 frames, etc.). The determination transition of (3) in FIG. 6 (transition from "stop determination" to "backward determination") may occur only when the back signal is ON. Note that this determination transition may occur when the back signal is ON and the determination based on image G is a backward determination, or may occur only when the determination based on image G is a backward determination. Here, the backward determination includes not only the case where it is determined as a backward determination even in one frame in image G, but also the case where it is continuous for a predetermined number of frames (for example, 2 frames, 3 frames, etc.). The determination transition of (4) in FIG. 6 (transition from "backward determination" to "stop determination") may occur when the determination based on image G is other than a backward determination and the back signal is OFF. Note that this determination transition may occur only when the determination based on image G is other than a backward determination. Here, other than the backward determination includes not only the case where it is determined as other than a backward determination even in one frame in image G, but also the case where it is continuous for a predetermined number of frames (for example, 6 frames, etc.). The determination transition of (5) in FIG. 6 (transition from "forward determination" to "backward determination") may occur only when the back signal is ON. In addition, this determination transition may occur when the back signal is ON and the determination based on the image G is a backward determination, or may occur only when the determination based on the image G is a backward determination. Here, the backward determination includes not only the case where it is determined as a backward determination even for one frame in the image G, but also the case where it continues for a predetermined number of frames (for example, 2 frames, 3 frames, etc.). In addition, in the determination transition of the forklift 10A, there may be a transition from "backward determination" to "forward determination" (not shown). In this determination transition, it may occur when the determination based on the image G is a forward determination and the back signal is OFF. In addition, this determination transition may occur only when the determination based on the image G is a forward determination. Here, the forward determination includes not only the case where it is determined as a forward determination even for one frame in the image G, but also the case where it continues for a predetermined number of frames (for example, 2 frames, 3 frames, etc.).

[0063] <Notification content according to determination transitions, the presence area of person H, etc.> According to the determination of the traveling direction and the like described so far, in the device system 1, the content notified from the above-described notification unit 3 may be changed. For example, when the forklift 10A is traveling backward (determined as a backward determination by the determination unit 2 or in the above "backward state") and it is determined that a person H exists in the danger area R1 or the detection area R2, the device system 1 may output predetermined notification content from a notification unit 3 such as a speaker. As a specific example of this notification content, if the notification unit 3 is a speaker, in the case of <1> of "backward determination", voice data of "I will back up" may be output. This voice data can be said to be a kind of attention-grabbing message. In the case of <2> where it is determined that a person H exists in the detection area R2 in "forward determination" or "backward determination", voice data of "Please be careful" may be output. This voice data can be said to be a kind of warning message. When it is determined that a person H exists within the danger area R1 during "forward movement determination" or "backward movement determination" <3>, voice data saying "Danger" may be output, and this voice data can also be regarded as a warning message so to speak. In addition, when determining the traveling direction in consideration of the input of a back signal to the equipment system 1, when a back signal is input even while stopped (for example, when the transmission is put into reverse gear), a caution message may be played. The notification contents such as these voice data may be repeatedly output as long as each case <1> to <3> continues. Also, when multiple cases overlap (for example, case <1> and <3>, etc.), during the notification, it may be switched to and notified of content with a higher priority. For example, when the content of case <1> is being notified and case <3> occurs, the content being notified may be interrupted (stopped), and the content of case <3> may be notified. Alternatively, when multiple cases overlap, the notification contents such as voice data may be alternately output from the notification unit 3, etc.

[0064] <Switching of the camera 20 to be displayed to the user according to the traveling direction, etc., display of the result of the presence determination> According to the traveling direction, etc. described above, in the equipment system 1, the camera 20 to be displayed to the user of the equipment system 1 (for example, the driver of the vehicle 10 such as the forklift 10A, etc.) may be switched, or the result of the presence determination of the person H in each area may be shown (displayed) to the user. The specific configuration of the switching of the camera 20 to be displayed to the user and the display of the result of the presence determination is not particularly limited. For example, it may be displayed by the monitor 3b. Alternatively, the switching of the camera 20 and the determination results of each area may be shown (notified) by the speaker of the notification unit 3, or may be shown by a rotating red or yellow light, etc. Hereinafter, assuming that the switching of the camera 20 according to the traveling direction and the display of the result of the presence determination are performed by the monitor 3b, it will be mainly described.

[0065] As shown in FIG. 7, the display on the monitor 3b may be capable of displaying the image G captured by each provided camera 20 (each camera 20 of the front left camera 20a1, front right camera 20a2, rear left camera 20b1, and rear right camera 20b2 on the upper surface of the above-described head guard 13A). The specific configuration of switching the image G of the camera 20 to be displayed to the user according to the traveling direction of the forklift 10A and the like described above on the monitor 3b is not particularly limited. For example, when the above-described equipment system 1 (determination unit 2) determines that the forklift 10A is moving forward, only the images G captured by the cameras 20 on the front side (two cameras 20 of the front left camera 20a1 and front right camera 20a2) among the four cameras 20 are displayed on the monitor 3b. When the forklift 10A is determined to be moving backward, only the images G captured by the cameras 20 on the rear side (two cameras 20 of the rear left camera 20b1 and rear right camera 20b2) may be displayed on the monitor 3b. In addition, when the equipment system 1 (determination unit 2) determines that the forklift 10A has stopped, only the image G captured by the camera 20 on the front side among the four cameras 20 may be displayed, or only the image G captured by the camera 20 on the rear side may be displayed, or the image G captured by any of the cameras 20 may not be displayed. At the time of the stop determination, whether to display any of the images G or not can be changed at the setting part 2c or the like. Further, the display on the monitor 3b may indicate the presence or absence of the person H in the danger area R1 by lighting and extinguishing a red display lamp for each imaging direction (direction of the optical axis of the camera) of each camera 20 (each camera 20 of the front left camera 20a1, front right camera 20a2, rear left camera 20b1, and rear right camera 20b2), and may indicate the presence or absence of the person H in the detection area R2 by lighting and extinguishing a red display lamp. The forklift 10A may also be shown surrounded by the display lamps indicating the determination results of the presence in each area R1, R2. In addition, according to the area in which the person H in the image G displayed on the monitor 3b is located, the color of the bounding box B surrounding the person H may be changed and displayed. For example, the color of the bounding box B when the person H is in the danger area R1 may be red, or the color of the bounding box B when the person H is in the detection area R2 may be yellow, or the color of the bounding box B when the person H is in the settable area (displayed as "outside the area" in FIG. 7) R0 may be blue.

[0066] <Change of each area R1, R2 linked to the traveling direction of the forklift 10A, etc.> As shown in FIGS. 8 to 10, the equipment system 1 (determination unit 2) provided in the forklift 10A may change the danger area R1 and / or the detection area R2 in conjunction with at least the traveling direction of the forklift 10A (the turning direction may or may not be included). Furthermore, not only the traveling direction but also the speed of the forklift 10A may be linked to change the danger area R1 and / or the detection area R2. Incidentally, in this way, each area R1, R2 changed in conjunction with the traveling direction of the forklift 10A, etc. can also be said to be a variable area. Each of the areas R1, R2 may be changed in terms of its position, shape, and size within the settable area R0, and may correspond to the "position, shape, and size within the image G" corresponding to the "actual position, shape, and size" in the plan view shown in FIGS. 8 to 10, or whether it is within the ranges of the first and second distances D1, D2 indicating the areas R1, R2 may be determined by the determination unit 2 (determination part 2') to be changed to the "actual position, shape, and size" in the plan view shown in FIGS. 8 to 10 in conjunction with the traveling direction of the forklift 10A, etc.

[0067] The shape of the settable area is not particularly limited. For example, from the viewing angles of each camera 20 (each camera 20 of the front left camera 20a1, the front right camera 20a2, the rear left camera 20b1, and the rear right camera 20b2) (such as 138°, 120°, 110°, etc. in the horizontal direction) and the number of cameras 20 (4 cameras), in a plan view, the viewing angles of the two front cameras 20 overlap at approximately the center in front of the forklift 10A (in FIGS. 5(a) and 5(b), the same person H exists in the danger area R1 where the viewing angles of the two front cameras 20 overlap at approximately the center in front). At the same time, the viewing angles of each front camera 20 also extend rearward, and the viewing angles of the two rear cameras 20 overlap at approximately the center in the rear of the forklift 10A, and the viewing angles of each rear camera 20 may also extend forward. Also, the shape of the outer edge of the settable area may be, for example, a substantially circular shape centered on approximately the center of the forklift 10A as shown in FIGS. 4 and 7. However, since the plan view shape of the forklift 10A including the fork 11A is substantially rectangular, the shape of the outer edge of the settable area may have substantially right-angled corners on the left and right in the front and rear (see FIGS. 8 to 10). First, for the case where each area R1, R2 is interlocked (the interlock is effective) in the traveling direction of the forklift 10A, FIG. 8 details the case where the traveling direction is forward, and FIG. 9 details the case where the traveling direction is backward. Note that the change of each area R1, R2 interlocked with the traveling direction, etc. may be performed within a predetermined time (such as 100 mSec (milliseconds)) after detecting a change in the traveling direction, turning direction, speed, etc. of the forklift 10A. Also, the shape of the settable area and each area R1, R2 in the case where the traveling direction in FIG. 8 is forward and the case where the traveling direction in FIG. 9 is backward is substantially front-back symmetric, or the shape of the danger area R1 in the case where the traveling direction in FIG. 9 is backward may be slightly longer in the front-back direction than the shape of the danger area R1 in the case where the traveling direction in FIG. 8 is forward. Further, whether to change each area R1, R2 in conjunction with the traveling direction, etc. of the forklift 10A may be set for each of the front side (the two front cameras 20, the change of each area R1, R2 shown in FIG. 8) and the rear side (the two rear cameras 20, the change of each area R1, R2 shown in FIG. 9). After that, in FIG. 10, a case where each area R1 and R2 is not interlocked (the interlock is invalid) with respect to the speed and turning direction of the forklift 10A will be described in detail.

[0068] As shown in FIG. 8, when the traveling direction of the forklift 10A is forward (forward determination), when the forklift 10A in FIG. 8(a) is stopped (at the stop), the front-rear length of the danger area R1 is from the front end of the fork 11A of the forklift 10A to a point a predetermined distance (such as 2 or 3 m) forward, and the left-right width of the danger area R1 may be substantially the same as the left-right width of the above-described settable area. On the other hand, the front-rear length of the detection area R2 is from a point a predetermined distance (such as 2 or 3 m) forward from the front end of the above-described danger area R1, and the left-right width of the detection area R2 may be from a point a predetermined distance (such as 2 or 3 m) to the left and right from the left and right ends of the forklift 10A. In addition, at the stop in FIG. 8(a), it may include when starting to move forward from the stop (for example, when an image G of a predetermined number of frames (such as 1 frame) is determined to move forward after the image G of the stop determination). When the forklift 10A in FIG. 8(b) is moving forward at a low speed (during low-speed forward movement), although the front-rear length of the danger area R1 is from the front end of the fork 11A of the forklift 10A to a point a predetermined distance (such as 2 or 3 m) forward, the same as at the stop in FIG. 8(a), the left-right width of the danger area R1 may be from a point a shorter predetermined distance (such as 1 m) to the left and right from the left and right ends of the forklift 10A and may be narrower than at the stop in FIG. 8(a). On the other hand, the front-rear length of the detection area R2 is from a point a predetermined distance (such as 2 or 3 m) forward from the front end of the above-described danger area R1, the same as at the stop in FIG. 8(a), and the left-right width of the detection area R2 may also be from a point a predetermined distance (such as 2 or 3 m) to the left and right from the left and right ends of the forklift 10A, the same as at the stop in FIG. 8(a). In addition, during the low-speed forward movement in FIG. 8(b), it can be said that it is moving straight forward (not turning). When the forklift 10A in Fig. 8(c) is moving forward at high speed (during high-speed forward movement), the front-back length of the danger area R1 is longer in the front than when moving forward at low speed in Fig. 8(b), and it extends to a point at a predetermined distance (such as 4 or 5 m) further forward from the front end of the fork 11A of the forklift 10A. The left-right width of the danger area R1 may be the same as when moving forward at low speed in Fig. 8(b), extending to points at a predetermined distance (such as 1 m) shorter from the left and right ends of the forklift 10A to the left and right. On the other hand, the front-back length of the detection area R2 extends to a point at a predetermined distance (such as 2 or 3 m) forward from the front end of the above-mentioned danger area R1, and the left-right width of the detection area R2 may also be the same as when moving forward at low speed in Fig. 8(b), extending to points at a predetermined distance (such as 2 or 3 m) from the left and right ends of the forklift 10A to the left and right. It should be noted that during high-speed forward movement in Fig. 8(c), it can be said that it is moving straight forward (not turning). When the forklift 10A in Fig. 8(d) is moving forward while turning to the left (during left-turn forward movement), although the front-back length of the danger area R1 extends to a point at a predetermined distance (such as 2 or 3 m) forward from the front end of the fork 11A of the forklift 10A, the same as when stopped in Fig. 8(a) or moving forward at low speed in Fig. 8(b), the left-right width of the danger area R1 may extend from the left end of the forklift 10A to the left end of the settable area and to a point at a predetermined distance (such as 1 m) shorter from the right end of the forklift 10A to the right. On the other hand, the front-back length of the detection area R2 extends to a point at a predetermined distance (such as 2 or 3 m) forward from the front end of the above-mentioned danger area R1, the same as when stopped in Fig. 8(a) or moving forward at low speed in Fig. 8(b), and the left-right width of the detection area R2 may also extend from the left end of the forklift 10A to the left end of the settable area and to a point at a predetermined distance (such as 2 or 3 m) from the right end of the forklift 10A to the right. When the forklift 10A in Fig. 8(e) is moving forward while turning to the right (when turning right and moving forward), the front-to-back length of the dangerous area R1 is, as in the case of stopping in Fig. 8(a), moving forward at low speed in Fig. 8(b), and turning left and moving forward in Fig. 8(d), from the front end of the fork 11A of the forklift 10A to a point a predetermined distance (such as 2 or 3 m) forward. However, the left-to-right width of the dangerous area R1 can be from the right end of the forklift 10A to the right end of the settable area to the right, and also from the left end of the forklift 10A to a point a shorter predetermined distance (such as 1 m) to the left. On the other hand, the front-to-back length of the detection area R2 is, as in the case of stopping in Fig. 8(a), moving forward at low speed in Fig. 8(b), and turning left and moving forward in Fig. 8(d), from the front end of the above-mentioned dangerous area R1 to a point a predetermined distance (such as 2 or 3 m) forward. The left-to-right width of the detection area R2 can also be from the right end of the forklift 10A to the right end of the settable area to the right, and also from the left end of the forklift 10A to a point a predetermined distance (such as 2 or 3 m) to the left. When the forklift 10A in Fig. 8(f) is moving forward at high speed while turning to the left (when turning left and moving forward at high speed), the front-to-back length of the dangerous area R1 is longer forward than when turning left and moving forward in Fig. 8(d) (and is the same as when moving forward at high speed in Fig. 8(c)), from the front end of the fork 11A of the forklift 10A to a point a longer predetermined distance (such as 4 or 5 m) forward. However, the left-to-right width of the dangerous area R1 is, as in the case of turning left and moving forward in Fig. 8(d), from the left end of the forklift 10A to the left end of the settable area to the left, and also from the right end of the forklift 10A to a point a shorter predetermined distance (such as 1 m) to the right. On the other hand, the front-to-back length of the detection area R2 is, as in the case of moving forward at high speed in Fig. 8(c) and turning left and moving forward in Fig. 8(d), from the front end of the above-mentioned dangerous area R1 to a point a predetermined distance (such as 2 or 3 m) forward. The left-to-right width of the detection area R2 is also, as in the case of turning left and moving forward in Fig. 8(d), from the left end of the forklift 10A to the left end of the settable area to the left, and also from the right end of the forklift 10A to a point a predetermined distance (such as 2 or 3 m) to the right. When the forklift 10A in Fig. 8(g) is moving forward at high speed while turning right (during right turn and high-speed forward movement), the front-to-back length of the danger area R1 is longer in the front than when moving forward during a right turn in Fig. 8(e) (and is the same as when moving forward at high speed in Fig. 8(c)). It extends to a point at a predetermined distance (such as 4 or 5 m) further forward from the front end of the fork 11A of the forklift 10A. However, the left-to-right width of the danger area R1 can be the same as when moving forward during a right turn in Fig. 8(e), from the right end of the forklift 10A to the right end of the settable area and from the left end of the forklift 10A to a point at a shorter predetermined distance (such as 1 m) to the left. On the other hand, the front-to-back length of the detection area R2 is the same as when moving forward at high speed in Fig. 8(c) or when moving forward during a right turn in Fig. 8(e), extending to a point at a predetermined distance (such as 2 or 3 m) forward from the front end of the above-mentioned danger area R1. The left-to-right width of the detection area R2 can also be the same as when moving forward during a right turn in Fig. 8(e), from the right end of the forklift 10A to the right end of the settable area and from the left end of the forklift 10A to a point at a predetermined distance (such as 2 or 3 m) to the left.

[0069] As shown in Fig. 9, when the traveling direction of the forklift 10A is backward (backward determination), when the forklift 10A in Fig. 9(a) is stopped (at the stop time), the front-to-back length of the danger area R1 extends to a point at a predetermined distance (such as 4 or 5 m) further backward from the rear end of the forklift 10A. The left-to-right width of the danger area R1 can be approximately the same as the left-to-right width of the above-mentioned settable area. On the other hand, the front-to-back length of the detection area R2 extends to a point at a predetermined distance (such as 2 or 3 m) further backward from the rear end of the above-mentioned danger area R1. The left-to-right width of the detection area R2 can be from the left and right ends of the forklift 10A to points at a predetermined distance (such as 2 or 3 m) to the left and right. Note that at the stop time in Fig. 9(a), it may include the time when starting to move backward from the stop (for example, when, after the image G of the stop determination, the image G of a predetermined number of frames (such as 1 frame) is determined to be moving backward). When the forklift 10A in Fig. 9(b) is moving backward at a low speed (during low-speed backward movement), although the front-to-back length of the danger area R1 is the same as that at the stop in Fig. 9(a), reaching a point a longer predetermined distance (such as 4 or 5 m) behind the rear end of the forklift 10A, the left-to-right width of the danger area R1 may be narrower than that at the stop in Fig. 9(a), reaching a point a shorter predetermined distance (such as 1 m) from the left and right ends of the forklift 10A to the left and right. On the other hand, the front-to-back length of the detection area R2 is the same as that at the stop in Fig. 9(a), reaching a point a predetermined distance (such as 2 or 3 m) behind the rear end of the above-mentioned danger area R1, and the left-to-right width of the detection area R2 may also be the same as that at the stop in Fig. 9(a), reaching a point a predetermined distance (such as 2 or 3 m) from the left and right ends of the forklift 10A to the left and right. Note that during low-speed backward movement in Fig. 9(b), it can also be said that it is moving straight backward (not turning). When the forklift 10A in Fig. 9(c) is moving backward at a high speed (during high-speed backward movement), the front-to-back length of the danger area R1 is longer behind than during low-speed backward movement in Fig. 9(b), reaching a point an even longer predetermined distance (such as 7 or 8 m) behind the rear end of the forklift 10A, and the left-to-right width of the danger area R1 may be substantially the same as that during low-speed backward movement in Fig. 8(b), reaching a point a shorter predetermined distance (such as 1 m) from the left and right ends of the forklift 10A to the left and right. On the other hand, the front-to-back length of the detection area R2 is a point a longer predetermined distance (such as 4 or 5 m) behind the rear end of the above-mentioned danger area R1, and the left-to-right width of the detection area R2 may be the same as that during low-speed backward movement in Fig. 8(b), reaching a point a predetermined distance (such as 2 or 3 m) from the left and right ends of the forklift 10A to the left and right. Note that during high-speed backward movement in Fig. 8(c), it can also be said that it is moving straight backward (not turning). When the forklift 10A in Fig. 9(d) is reversing while turning left (left turn reverse), the front-to-back length of the danger area R1 is, as in the case of stopping in Fig. 9(a) or low-speed reverse in Fig. 9(b), from the rear end of the forklift 10A to a point at a longer predetermined distance (such as 4 or 5 m) backward. However, the left-to-right width of the danger area R1 may be from the left end of the forklift 10A to the left end of the settable area to the left, and also to a point at a shorter predetermined distance (such as 1 m) to the right from the right end of the forklift 10A. On the other hand, the front-to-back length of the detection area R2 is, as in the case of stopping in Fig. 9(a) or low-speed reverse in Fig. 9(b), from the rear end of the above-mentioned danger area R1 to a point at a predetermined distance (such as 2 or 3 m) backward, and the left-to-right width of the detection area R2 may also be from the left end of the forklift 10A to the left end of the settable area to the left, and also to a point at a predetermined distance (such as 2 or 3 m) to the right from the right end of the forklift 10A. When the forklift 10A in Fig. 9(e) is reversing while turning right (right turn reverse), the front-to-back length of the danger area R1 is, as in the case of stopping in Fig. 9(a), low-speed reverse in Fig. 9(b), and left turn reverse in Fig. 9(d), from the rear end of the forklift 10A to a point at a longer predetermined distance (such as 4 or 5 m) backward. However, the left-to-right width of the danger area R1 may be from the right end of the forklift 10A to the right end of the settable area to the right, and also to a point at a shorter predetermined distance (such as 1 m) to the left from the left end of the forklift 10A. On the other hand, the front-to-back length of the detection area R2 is, as in the case of stopping in Fig. 9(a), low-speed reverse in Fig. 9(b), and left turn reverse in Fig. 9(d), from the rear end of the above-mentioned danger area R1 to a point at a predetermined distance (such as 2 or 3 m) backward, and the left-to-right width of the detection area R2 may also be from the right end of the forklift 10A to the right end of the settable area to the right, and also to a point at a predetermined distance (such as 2 or 3 m) to the left from the left end of the forklift 10A. When the forklift 10A in Fig. 9(f) is making a high-speed reverse while turning left (left-turn high-speed reverse), the front-to-back length of the danger area R1 is longer behind than when making a left-turn reverse in Fig. 9(d) (and similar to when making a high-speed reverse in Fig. 9(c)), reaching a point at a further predetermined distance (such as 7 or 8 m) behind from the rear end of the forklift 10A. However, the left-to-right width of the danger area R1 is the same as when making a left-turn reverse in Fig. 9(d), from the left end of the forklift 10A to the left end of the settable area to the left, and may also be from the right end of the forklift 10A to a point at a shorter predetermined distance (such as 1 m) to the right. On the other hand, the front-to-back length of the detection area R2 is the same as when making a high-speed reverse in Fig. 9(c) or a left-turn reverse in Fig. 9(d), reaching a point at a predetermined distance (such as 2 or 3 m) behind from the rear end of the above-mentioned danger area R1. The left-to-right width of the detection area R2 is also the same as when making a left-turn reverse in Fig. 9(d), from the left end of the forklift 10A to the left end of the settable area to the left, and may also be from the right end of the forklift 10A to a point at a predetermined distance (such as 2 or 3 m) to the right. When the forklift 10A in Fig. 9(g) is making a high-speed reverse while turning right (right-turn high-speed reverse), the front-to-back length of the danger area R1 is longer behind than when making a right-turn reverse in Fig. 9(e) (and similar to when making a high-speed reverse in Fig. 9(c)), reaching a point at a further predetermined distance (such as 7 or 8 m) behind from the rear end of the forklift 10A. However, the left-to-right width of the danger area R1 is the same as when making a right-turn reverse in Fig. 9(e), from the right end of the forklift 10A to the right end of the settable area to the right, and may also be from the left end of the forklift 10A to a point at a shorter predetermined distance (such as 1 m) to the left. On the other hand, the front-to-back length of the detection area R2 is the same as when making a high-speed reverse in Fig. 9(c) or a right-turn reverse in Fig. 9(e), reaching a point at a predetermined distance (such as 2 or 3 m) behind from the rear end of the above-mentioned danger area R1. The left-to-right width of the detection area R2 is also the same as when making a right-turn reverse in Fig. 9(e), from the right end of the forklift 10A to the right end of the settable area to the right, and may also be from the left end of the forklift 10A to a point at a predetermined distance (such as 2 or 3 m) to the left.

[0070] As shown in Fig. 10, in the case where each area R1, R2 is not interlocked with the speed or turning direction of the forklift 10A, when the forklift 10A in Fig. 10(a) is moving forward (during forward movement), the front - to - back length of the danger area R1 is from the front end of the fork 11A of the forklift 10A to a point a predetermined distance (such as 2 or 3 m) forward. The left - to - right width of the danger area R1 may be substantially the same as the left - to - right width of the above - mentioned settable area. On the other hand, the front - to - back length of the detection area R2 is from the front end of the above - mentioned danger area R1 to a point a predetermined distance (such as 2 or 3 m) forward. The left - to - right width of the detection area R2 may be substantially the same as the left - to - right width of the above - mentioned settable area. Note that during the forward movement in Fig. 10(a), it may include the time when starting to move forward from a stop (for example, when, after the image G for stop determination, an image G for a predetermined number of frames (such as 1 frame) is determined to be a forward movement). When the forklift 10A in Fig. 10(b) is moving backward (during backward movement), the front - to - back length of the danger area R1 is from the rear end of the forklift 10A to a point a longer predetermined distance (such as 4 or 5 m) backward. The left - to - right width of the danger area R1 may be substantially the same as the left - to - right width of the above - mentioned settable area. On the other hand, the front - to - back length of the detection area R2 is from the rear end of the above - mentioned danger area R1 to a point a predetermined distance (such as 2 or 3 m) backward. The left - to - right width of the detection area R2 may be substantially the same as the left - to - right width of the above - mentioned settable area. Note that during the backward movement in Fig. 10(b), it may include the time when starting to move backward from a stop (for example, when, after the image G for stop determination, an image G for a predetermined number of frames (such as 1 frame) is determined to be a backward movement). For the cases of Figs. 10(a) and (b) described so far, although each area R1, R2 is not interlocked with the speed or turning direction of the forklift 10A, it can be said that they are interlocked with the traveling direction (such as forward determination or backward determination). In addition, in the cases of Figs. 10(a) and (b), even if each area R1, R2 is not interlocked with the traveling direction of the forklift 10A (not shown), regardless of whether the forklift 10A is moving forward or backward, each area R1, R2 shown in Figs. 10(a) and (b) may always exist simultaneously.

[0071] <Others> The present invention is not limited to the above-described embodiments. Each component or the overall structure, design shape, dimensions, weight, etc. of the device system 1 or the vehicle 10 can be appropriately changed in accordance with the gist of the present invention. In the device system 1, there may be types of activation other than the above-described main activation K1 and standby activation K2. For example, after connecting the accessory input terminal 1a on the device system 1 side and the accessory output terminal 10a on the vehicle 10 side, there may be an initial activation that starts for the first time. Incidentally, the initial activation may be such that the device system enters an operating state after a predetermined fourth time has elapsed since the accessory interlock signal ACC became ON. This fourth time may be approximately the same as the first time T1 of the above-described main activation K1, or may be longer or shorter. Incidentally, the fourth time can also be said to be the initial activation time. Between the device system 1 and the vehicle 10, it is sufficient that the accessory interlock signal ACC from the vehicle 10 is input to the device system 1, and signals other than the accessory interlock signal ACC do not necessarily need to be input and output between the device system 1 and the vehicle 10. Between the device system 1 and the vehicle 10, it is not necessarily the case that signals are input and output or power is output (supplied) via the connected input and output terminals. In that case, the device system 1 and the vehicle 10 may simply be connected by wiring such as a cable. The device system 1 does not necessarily need to be connected to the vehicle 10's constant power supply. In that case, the device system 1 may separately have a power supply within it. The device system 1 does not necessarily need to have the above-described determination unit 2. In this case, the device system 1 may have, as a part having a function other than the determination by the determination unit 2, for example, a travel instruction (navigation) unit (not shown), or an automatic driving unit (not shown). Additionally, it may have a storage unit 4 and a camera 20 may be provided on the vehicle 10 (so to speak, it may have a drive recorder unit (not shown)). Incidentally, the device system 1 may have the determination unit 2 and may also have at least one of the above-described navigation unit, automatic driving unit, and drive recorder unit. The machine system 1 does not necessarily have a notification unit 3, a storage unit 4, a communication unit 5, etc.

[0072] The determination unit 2 does not necessarily have a detection part 2b and a setting part 2c. For one machine system 1 (or vehicle 10), one or more notification units 3 may be provided. For example, both a speaker 3a and a monitor 3b may be provided, or either one may be provided. The machine system 1 does not necessarily have a configurable area R0. In vehicles 10 other than the forklift 10A, the machine system 1 (determination unit 2) may also perform the above-described determination of the traveling direction, determination of the traveling direction (including the turning direction and speed) based on the image G, determination transition, notification content according to the determination transition and the presence area of the person H, etc., and variable of the area linked to the traveling direction, etc. The ranges of the respective areas R1, R2, etc., the validity / invalidity of notification and the notification content in the notification unit 3, the frame rate value and recording time of the video file in the storage unit 4, the determination of the traveling direction of the vehicle 10 such as the forklift 10A (whether it is a reverse signal or based on the image G), the number of frames that are the conditions for determination transition, etc. are not set in the setting part 2c provided in the machine system 1, but may be set at a location different from the location where the machine system 1 is provided (such as the vehicle 10 like the forklift 10A) via the communication unit 5, the Internet, etc. In vehicles 10 other than the forklift 10A, the turning direction in the vehicle 10 may be included in its traveling direction, and the machine system 1 (determination unit 2) provided in the vehicle 10 may also determine the speed of the traveling direction of the vehicle 10. Also, the machine system 1 (determination unit 2) may change at least the danger area R1 and / or the detection area R2 in conjunction with the traveling direction of the vehicle 10. Hereinafter, the image G used in the vehicle 10 such as the machine system 1 and the forklift 10A described so far, and the person H to be determined will be explained in detail.

[0073] <Image G> As shown in FIGS. 1 to 3 and 6 to 14, the image G is captured by a predetermined camera such as a camera 20 of a vehicle 10 such as a forklift 10A. When the image G is captured by the camera 20 attached to the forklift 10A among the above-described vehicles 10, it may be an image that captures the front or rear of the forklift 10A. In this case, the front or rear part of the forklift 10A itself may be reflected in the image G, or conversely, only the vicinity of the forklift 10A may be reflected (that is, the forklift 10A itself is not reflected in the image G), etc.

[0074] Alternatively, the image G may be captured by a camera attached to another vehicle 10 other than the forklift 10A, or a device, product, facility, equipment, etc. other than the vehicle 10. In this case, a part of another vehicle or the like may be reflected in the image G, or a predetermined area near a part of another vehicle or the like may be reflected in the image G. Alternatively, only the vicinity of another vehicle or the like may be reflected in the image G (that is, the other vehicle or the like itself is not reflected in the image G), etc. In addition to being captured by a camera 20 or the like attached to the vehicle 10, the image G may be captured by a flying object such as a drone, or a (fixed) camera (such as a security camera or a surveillance camera) installed in a street, a store, a shopping street, a large building facility, a tenant, an apartment, etc.

[0075] The camera 20 or the like attached to the vehicle 10 such as the forklift 10A may capture the front or rear of the vehicle 10 from substantially above, from substantially the side, or from substantially the front or the rear. Note that one device system 1 may detect a person H from two or more images G. In this case, for example, it may be two or more images G captured by two or more cameras 20 (so-called stereo cameras) attached to a vehicle 10 such as a forklift 10A, or two or more images G captured simultaneously by two or more cameras including the camera 20 attached to the vehicle 10 and a camera attached to a flying object or the like. In addition, the image G may be a color image (an image having hue and saturation in addition to brightness), a grayscale image (an image having only brightness and no hue or saturation), or a two-color black-and-white image such as a halftone (screen dots). Regarding the person H who may exist in such an image G, the following will be described.

[0076] <Person H> As shown in FIGS. 1, 4 to 10, the person H is the object to be determined by the above-described equipment system 1 and is the person shown in the above-described image G. In addition to workers present in front of or behind the vehicle 10 such as the forklift 10A, the person H may include pedestrians, drivers of bicycles, etc. It can be said that in one image G, the person H and the parts other than the person H (the vehicle 10 itself, the conditions in front of or behind the vehicle 10, etc.) are shown. The contour (edge) of the person H is the boundary between the person H and the parts other than the person H in one image G and represents the shape of the person H. Incidentally, the person H may be holding food such as bread, a plastic bottle containing a drink, or luggage such as a bag.

Industrial Applicability

[0077] Since the equipment system according to the present invention is a system that starts up at high speed, in a vehicle, when it has a determination unit, it can be used when frequently interrupting the determination of the presence of a person in the captured image, the notification of the presence of a person in the vicinity of the vehicle, or a dangerous state, or the detection by a sensor not based on an image. In addition, even when the vehicle does not have a determination unit, as a function other than the determination of the presence of a person in the image, for example, driving instructions, automatic driving, a drive record, etc. can also be used when frequently interrupted. The vehicle according to the present invention is a vehicle that starts up at high speed. When it mainly has a determination unit in a forklift or the like, during getting off work or taking a break, etc., it can be used to determine the presence of people in the image captured by a camera, to notify the presence of people in the vicinity of the forklift or a dangerous state, or to detect by a sensor not based on an image, etc., when frequently interrupting. In addition, during driving, it can be used in an environment where people are present around it. In particular, in addition to determining the presence of people when people stay around the forklift or when people cross around it, when lifting goods, cargo, or materials in the forklift to a high position, it can also be used to determine the presence of people on the left, right, front, rear, etc. of the forklift, and can be used to handle the presence and dangerous state of all people around industrial vehicles. In addition, even when a forklift or the like does not have a determination unit, as a function other than determining the presence of people in the image, for example, it can also be used when frequently interrupting a drive record or automatic driving, etc. Here, the vehicle in the present invention may be other than a forklift, and may be any industrial vehicle such as a dump truck, a water sprinkler, a concrete pump, a mixer truck, a tailgate lifter, a tank truck, a water supply vehicle, a powder and granular material transport vehicle, a detachable body vehicle, a stacked vehicle transport vehicle, etc. In addition, it may be a general vehicle such as a truck, a bus, a passenger car, a motorcycle, etc.

Explanation of Signs

[0078] 1 Equipment system 2 Determination unit 3 Notification unit 4 Storage unit 5 Communication unit 10 Vehicle 10A Forklift 20 Camera ACC Accessory interlock signal S1 Operating state S2 Standby state S0 Rest state T1 First time T2 Second time T3 Third time K1 This start K2 Standby Startup G Image H Person R1 Danger Area R2 Detection Area

Claims

1. An equipment system provided in a vehicle, the equipment system is activated according to an accessory interlock signal that turns on and off in conjunction with the ON / OFF of an accessory switch in the vehicle, and enters an operating state, For activation according to the accessory interlock signal, after a predetermined first time has elapsed since the accessory interlock signal turned ON, the equipment system enters an operating state, which is the main activation, and in the operating state, when the equipment system enters a standby state due to the accessory interlock signal turning OFF, after a predetermined second time shorter than the first time has elapsed since the accessory interlock signal turned ON, the equipment system enters an operating state. The equipment system is characterized by having at least two types of standby activation.

2. After a predetermined third time has elapsed since the equipment system entered the standby state without the accessory interlock signal turning ON, the standby state is released and the equipment system enters a dormant state, When the accessory interlock signal turns ON while the equipment system is in the dormant state, the equipment system enters an operating state by the main activation. The equipment system according to claim 1 is characterized by this.

3. The equipment system has a determination unit that determines the presence of a person in an image captured by a camera, in the determination unit, at least a danger area closest to the equipment system and a detection area next closest to the equipment system after the danger area are set, the determination unit recognizes a person in the image, the determination unit determines whether the recognized person is present in the danger area and whether the recognized person is present in the detection area, respectively. The equipment system according to claim 1 or 2 is characterized by this.

4. A notification unit that notifies outside the device system that the determination unit has determined that the recognized person is present within the dangerous area and / or within the detection area; a storage unit that stores the image itself and / or a moving image including the image; and a communication unit that communicates with outside the device system. The device system according to claim 3, characterized by having at least one of them.

5. A vehicle provided with the device system according to claim 1 or 2, The vehicle is characterized by outputting the accessory interlock signal to the device system.

6. A vehicle provided with the device system according to claim 3, The vehicle outputs the accessory interlock signal to the device system and has a camera that captures the image, The determination unit determines the traveling direction of the vehicle based at least on the image. The vehicle is characterized by this.

7. The vehicle is a forklift, The traveling direction also includes the turning direction of the vehicle, The determination unit also determines the speed in the traveling direction. The vehicle according to claim 6 is characterized by this.

8. The determination unit changes the dangerous area and / or the detection area in conjunction with at least the traveling direction. The vehicle according to claim 6 is characterized by this.

Images