Location Monitoring System

Abstract

To provide a position monitoring system capable of determining the degree of risk for an object person from a position and a visual line direction of the object person.SOLUTION: A position monitoring system includes: a marker 1 worn by an object person; an image acquisition section 2 for capturing the marker 1; an ID information storage section 311 for storing the object person, the marker 1, and ID information associating them; a marker recognition section 321 for recognizing the information on the marker 1 captured by the image acquisition section 2; a determination area storage section 314 for storing a determination area; and a state determination section 323 for determining a state of the object person having an ID corresponding to the marker 1 on the basis of the information on the determination area and the marker 1.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present embodiment relates to a location monitoring system that utilizes markers that can identify subjects. [Background technology]

[0002] Technologies for accurately measuring the current position and angle of an object are used in a variety of fields. For example, in game machines and vehicles, devices are used that track head movement by wearing a helmet or the like equipped with optical markers such as reflectors or light-emitting bodies, and measuring the positions of the optical markers with a camera device capable of stereoscopic vision (Patent Document 1).

[0003] Furthermore, a position monitoring system for determining the position of a subject includes a system in which the subject wears a headset equipped with optical markers and the position of the subject is calculated or predicted wirelessly (Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-289645 [Patent Document 2] Special Publication No. 2021-505898 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned conventional technologies have low accuracy in recognizing the target person, and for example, the safety of the target person cannot be sufficiently ensured at construction sites where heavy machinery, fuel trucks, and other large vehicles are used. In particular, wireless location monitoring systems have the problem that radio waves become unstable depending on the status of surrounding equipment and obstacles, making it impossible to constantly and accurately monitor the target person's location.

[0006] Furthermore, conventional technology requires the subject to wear a headset equipped with numerous accessories, such as sensors and cameras, which increases the number and weight of the devices worn, reducing the subject's work efficiency. Furthermore, in recent years, there have been cases where subjects have collapsed while working due to heatstroke or sudden illness, and if the condition is not discovered in time, the situation can become serious. However, while conventional technology can detect the distance to dangerous objects or obstacles, it cannot detect signs of the subject's poor health if there are no such dangerous objects or obstacles around the subject.

[0007] This embodiment has been proposed to solve the above-mentioned problems. That is, the object of this embodiment is to provide a position monitoring system that can determine the degree of danger and other conditions of a target person based on the target person's position and body orientation. [Means for solving the problem]

[0008] The location monitoring system according to the embodiment has the following configuration. (1) A marker attached to the subject. (2) An image acquisition unit that captures the marker. (3) An ID information storage unit that stores the subject, the marker, and ID information linking the two. (4) A marker recognition unit that recognizes information about the marker captured by the image acquisition unit. (5) A judgment region storage unit that stores the judgment region. (6) A state determination unit that determines the state of the subject having the ID corresponding to the marker based on information about the determination area and the marker.

[0009] The destination point guidance system according to the embodiment may further include the following configuration. (1) The marker information is the position and / or angle of the marker, the judgment area is the distance between the marker and the object to be judged and / or the angle range of the marker, and the status judgment unit judges the status of the subject and the object based on the position and / or angle of the marker and the distance from the object and / or the angle range of the marker. (2) A marker tracking unit that, when the position and / or angle of the marker cannot be recognized by the marker recognition unit, estimates the current position and / or angle of the marker from the past position and / or angle of the marker. (3) A control device that controls the object placed in the determination area based on the determination result determined by the state determination unit. (4) The marker information is the angle of the marker, the judgment area is the angle range of the marker, and the status judgment unit judges the status of the subject based on the angle of the marker recognized by the marker recognition unit and the angle range stored in the judgment area. (5) A warning device that notifies the subject of the judgment result in stages according to the judgment result determined by the state judgment unit. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a functional block diagram of a location monitoring system according to a first embodiment. [Figure 2] FIG. 2 is an image diagram showing the shape and coordinates of an AR marker in the first embodiment. [Figure 3] FIG. 4 is a diagram showing the positional relationship between the marker size and the image acquisition unit in the first embodiment. [Figure 4] 3A to 3C are diagrams for explaining setting of a relative coordinate system and tracking of markers in the first embodiment. [Figure 5] 1A and 1B are conceptual diagrams showing the overall configuration of the location monitoring system in the first embodiment, in which (a) the image acquisition unit is installed above and in front of the driver's seat of the heavy equipment, and (b) the image acquisition unit is installed in a location where the work site can be monitored. [Figure 6] 4 is a flowchart showing the operation of the first embodiment. [Figure 7] 10A and 10B are conceptual diagrams showing the overall configuration of the location monitoring system in the second embodiment, in which (a) the image acquisition unit is installed above and in front of the driver's seat of the heavy equipment, and (b) the image acquisition unit is installed in a location where the work site can be monitored. [Figure 8]10A and 10B are conceptual diagrams showing the overall configuration of the location monitoring system in the third embodiment, in which (a) the image acquisition unit is installed above and in front of the driver's seat of the heavy equipment, and (b) the image acquisition unit is installed in a location where the work site can be monitored. [Figure 9] FIG. 13 is an image diagram showing marker coordinates and subject state determination in the fourth embodiment. [Figure 10] 1A to 1C are conceptual diagrams showing examples of how markers are attached in the first embodiment and other embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0011] [1. First embodiment] [1-1.Configuration] The following describes a location monitoring system according to the first embodiment. The location monitoring system according to the first embodiment calculates the position and angle of a marker attached to a helmet worn by a worker at a construction site, and determines the worker's level of danger.

[0012] The position monitoring system of the first embodiment uses an AR marker as the marker 1. This position monitoring system includes the marker 1 attached to the helmet of a worker who is a subject of assessment, an image acquisition unit 2 that captures an image of the marker 1, and a monitoring device 3.

[0013] The monitoring device 3 is configured by a computer including a CPU, memory, etc., and performs various calculation processes and controls. As an example, the monitoring device 3 is installed in an appropriate location such as a monitoring room, a driver's cab of heavy machinery, or a construction site.

[0014] The monitoring device 3 includes a storage unit 31 that stores various data, a determination processing unit 32 that determines the degree of danger to the worker, a warning device 33, and a control device .

[0015] The storage unit 31 has an ID information storage unit 311 that stores the worker, the marker, and ID information linking the two, a relative coordinate storage unit 312 used to calculate the position and angle of the marker 1, and a position information storage unit 313 that stores the position and angle of the photographed marker 1. The position information storage unit 313 also stores information on the predicted position and angle of the marker 1, if any. The storage unit 31 also has a determination area storage unit 314 that stores areas that are determined to be dangerous if a worker wearing a marker enters.

[0016] The determination processing unit 32 includes a marker recognition unit 321 and a marker tracking unit 322. The marker recognition unit 321 recognizes the position and angle of the marker 1 based on the image of the marker 1 captured by the image acquisition unit 2. The marker tracking unit 322 predicts and calculates the position and angle of the marker 1 based on the speed, acceleration, and other information of the marker 1 captured in previous images, even when the image acquisition unit 2 cannot capture the marker 1. The determination processing unit 32 includes a state determination unit 323. The state determination unit 323 determines the state (e.g., the level of danger) of the worker having the ID corresponding to the marker 1 based on area information, such as the determination area, stored in the determination area storage unit 314 and the position and angle of the marker 1 recognized by the marker recognition unit 321. When the image acquisition unit 2 cannot capture the image of the marker 1, the state determination unit 323 determines the state of the worker based on area information, such as the determination area, stored in the determination area storage unit 314 and the predicted position and angle of the marker 1 acquired by the marker tracking unit 322.

[0017] The configuration of each part will be described in detail below. (1) Marker 1 The marker 1 in the first embodiment is, for example, an AR marker as shown in Fig. 2. The AR marker is a sign for specifying the content of various content information such as AR (Augumented Reality) content, the position where the content information is to be displayed, etc., and is an image or the like in which a predetermined pattern or character pattern is formed within a predetermined area, such as a two-dimensional code. The marker 1 is linked in advance to the identification information (ID) of each worker to be judged according to the pattern. The worker, the marker 1, and the ID information linking them are stored in the ID information storage unit 311.

[0018] 2, the position and angle information of the marker 1 is defined using a relative coordinate system (XYZ coordinate system) based on the image acquisition unit 2. This relative coordinate system is stored in the relative coordinate storage unit 312. The position and angle of the marker acquired by the image acquisition unit 2, i.e., the coordinate values ​​of the marker 1 in the relative coordinate system, are stored in the position information storage unit 313.

[0019] Marker 1 is attached, for example, by attaching a flat fixing base to the top of the worker's helmet and attaching a sticker with marker 1 printed on it to the top surface of the fixing base. Any material can be used for the fixing base, such as a wooden board, plastic board, polystyrene foam board, or metal board, as long as marker 1 can be read when viewed from directly above.

[0020] Next, the size of marker 1 will be explained using Figure 3. First, when using an AR marker as marker 1, the maximum area size and minimum marker size when the widest area is set will be calculated. The following conditions are assumed. (1) Height (distance from marker 1 to the camera): 10 [m] (2) Camera resolution: (W) 3,840 x (H) 2,160 [pixels] (3) Lens distortion: None (4) Minimum number of marker pixels (per side): 30 [pixels] (5) Maximum marker tilt: 45° In addition, since it is difficult to recognize a marker tilt of 45° or more, (6) The camera angle of view is up to 90°.

[0021] Under the above conditions, the horizontal angle of view of the camera becomes wider, so if the horizontal angle of view is based on 90°, the maximum area size of the recognition range is 20 m (horizontal) x 11.2 m (vertical), as shown in the upper part of Figure 3. The minimum size of marker 1 in this case is 20,000 [mm] / 3,840 [pixels] ≒ 5.2 [mm] Size per pixel The minimum marker pixel size is 30 pixels per side, so 5.2 [mm] x 30 [pixels] ≒ 156.3 [mm].

[0022] On the other hand, if the marker size is reduced, the judgment area must also be reduced. For example, as shown in the bottom of Figure 3, if the judgment area is set to 5 m (width) x 2.8 m (height), the minimum size of Marker 1 is: 5,000 [mm] / 3,840 [pixels] x 30 [pixels] ≒ 39.1 [mm]. If you want to make the marker size even smaller, it is possible to use a simpler marker that does not require higher resolution, rather than an AR marker.

[0023] (2) Video acquisition unit 2 The image acquisition unit 2 is an interface with an imaging function, such as a camera, and in this embodiment, is composed of one or more cameras. The camera is installed in a direction and at a height that allows the camera to capture images in a way that the marker 1 fixed to the worker's helmet can be recognized. For example, as shown in FIG. 5, the image acquisition unit 2 is installed in a location above and in front of the driver's seat of a heavy machine or in a location that allows the work site to be monitored. In this case, in order to be able to express the position of the marker 1 in spatial coordinates, a relative coordinate system, which has the center of the camera as its origin and moves along with the camera, is called from the relative coordinate storage unit 312 and used.

[0024] The relative coordinate system can arbitrarily define the origin and the direction of each coordinate axis. In this embodiment, as shown in Fig. 4, the X-axis direction is parallel to the ground and points in the direction in which a dangerous object exists in the determination area, the Z-axis direction is perpendicular to the X-axis direction and perpendicular to the ground, pointing upward, and the Y-axis direction is perpendicular to the X-axis direction, horizontal to the ground, and points away from the dangerous object. The position and angle of the marker 1 in the relative coordinate system are defined with the origin as the midpoint of the camera, and are set in the marker recognition unit 321.

[0025] (3) Marker recognition unit 321 The marker recognition unit 321 performs image processing using binarization, pattern matching, Hough transform, AI recognition processing, etc. to extract features of the photographed marker 1. For the marker 1 whose features have been extracted by image processing, the worker wearing the photographed marker 1 is identified based on information linking the marker 1 to the worker stored in the ID information storage unit 311.

[0026] The marker recognition unit 321 performs processing such as extracting the distance and angle of marker 1 from the camera and feature points of marker 1 based on the image of marker 1 acquired by the image acquisition unit 2, and converts the position and angle of marker 1 in the image into coordinate values ​​of a relative coordinate system. As a result, it calculates the position (P0) and angle (V0) of marker 1 based on the relative coordinate system whose origin is the center of the camera, and recognizes the position of the worker in the relative coordinate system and the angle of marker 1 attached to the helmet. In addition, it determines the worker's line of sight based on the angle of marker 1.

[0027] The marker tracking unit 322 calculates and stores the movement vector and orientation vector of the marker 1, thereby estimating the current position and orientation of the worker when the marker 1 becomes unrecognizable due to the worker bending over or being hidden by luggage, etc. The marker tracking unit 322 reads the past position information of the worker's marker 1 stored in the position information storage unit 313, and predicts current or future data using linear prediction, non-linear prediction, AI prediction, etc. For example, in FIG. 4, the recognition result of the marker 1 at time t is (ID,P t (x,y,z),Vt (x,y,z)), P indicates position and V indicates direction.

[0028] When the worker is moving at a constant speed, the predicted result F^ of marker 1 is calculated using Equation 1. t Even when the object to be judged for risk level is a moving object such as a vehicle and a marker 1 is attached to the moving object to calculate its position and angle, Equation 1 can be used as long as the moving object moves at a constant speed in a straight line.

[0029]

number

[0030] When the worker or moving object is moving at a constant speed, the predicted result F^ of marker 1 is calculated using the following formula 2. t (i) may be calculated. Formula 2 allows for more detailed calculation of the prediction result of Marker 1.

[0031]

number

[0032] On the other hand, if the object to be judged, such as a worker or vehicle, is subject to accelerated motion, such as a falling object or a ball, the predicted result F^ of Marker 1 can be calculated using Equation 3. t Calculate (i).

[0033]

number

[0034] The marker tracking unit 322 calculates whether the current or future data calculated using the above formulas 1 to 3 is within the range of prediction validity. For example, the marker tracking unit 322 stores information such as information on the movement speed of the worker or vehicle, the most recent image capture time of marker 1, and the predicted time. When calculating validity, this information is referenced to calculate the maximum value of the worker or vehicle movement between the image capture time and the predicted time, and this maximum value is compared with the calculated value of the predicted data. If the predicted data is within the range of the maximum value, the predicted data is determined to be valid.

[0035] The state determination unit 323 determines the state of the subject having the ID corresponding to marker 1 based on the information on the determination area and marker 1. Specifically, the state determination unit 323 determines the danger level of the subject based on the positional relationship between the determination area stored in the determination area storage unit 314 and marker 1 recognized by the marker recognition unit 321, and the line of sight direction of the worker obtained from the angle of marker 1. For example, as shown in FIGS. 5(a) and 5(b), if the area within the turning radius of the heavy equipment is set as a warning area and a 3-meter perimeter surrounding that is set as a caution area in advance, the danger level determination criteria are set as follows, for example: (1) Level 0: The worker is in the safety area (2) Level 1: When a worker enters the caution area and faces the heavy machinery. (3) Level 2: When a worker enters the caution area and does not face the direction of the heavy equipment. (4) Level 3: When a worker enters the warning area and faces the direction of heavy machinery. (5) Level 4: When a worker enters the warning area and is not facing the direction of the heavy machinery. In this way, the state determination unit 323 performs a comparison operation to determine whether the position of the marker 1 recognized by the marker recognition unit 321 is within the range of the determination area and whether the worker is facing the direction of the dangerous object, and determines that there is a danger if the worker has entered the determination area and the error exceeds a preset allowable error. The danger level may also be appropriately set or changed taking into account the time the worker has been in the determination area.

[0036] The ID information storage unit 311 stores information about electronic data identified by marker 1, including at least the pattern of marker 1 and the worker's identification information, such as the worker's name, gender, age, experience, and medical history, linked to the pattern.

[0037] The relative coordinate storage unit 312 stores a relative coordinate system that serves as a reference for calculating the position (P0) and angle (V0) of the marker 1. In this embodiment, the relative coordinate system is set with the origin set to the midpoint of the camera.

[0038] The position information storage unit 313 stores the position information of the marker 1 in the relative coordinate system. By accumulating and storing the position information of the marker 1, it becomes possible to perform more accurate current or future data prediction.

[0039] The determination area storage unit 314 stores the determination area based on the distance from the object. The distance between the marker 1 and the object to be determined and / or the angle range of the marker 1 is stored as coordinate values ​​in a relative coordinate system based on the image acquisition unit 2. As shown in FIG. 5, if the dangerous object is heavy machinery, the warning area is set within the turning radius of the heavy machinery, and a 3m perimeter around that is set as a caution area, and the determination area is saved. When using heavy machinery, it is recommended to adjust the determination area depending on the arm length of the heavy machinery.

[0040] (4) Warning device 33 The warning device 33 is a device that notifies the worker of danger in stages according to the determination result made by the state determination unit 323. The warning device 33 can be a lighting warning device 331 and / or an audio warning device 332. The lighting warning device 331 notifies the worker of danger by flashing, lighting up, or displaying an LED warning light or a warning message. The audio warning device 332 notifies the worker of danger by sounding a warning buzzer or sound. The warning device 33 is installed in an appropriate location such as on heavy machinery or at a construction site.

[0041] As shown in FIG. 5, when the area within the turning radius of the heavy equipment is defined as a warning area and the area 3 meters around it as a caution area, the criteria for the warning device 33 to notify the worker of danger in stages will be described. For example, if a worker enters the caution area and is not facing the direction of the heavy equipment, the audio warning device 332 emits an intermittent "beep, beep, beep" sound to alert the driver and worker of the heavy equipment. In contrast, when a worker enters the warning area, even if the worker is facing the direction of the heavy equipment, the audio warning device 332 emits a continuous "beep" sound, a stronger warning. In this way, when the worker is facing the heavy equipment, the worker is considered to be in a monitoring state, and a warning is issued in the warning area and a caution is issued in the caution area. On the other hand, when the worker is not facing the heavy equipment, the worker is not aware of the movement of the heavy equipment, so a warning is issued in the warning area and a caution is issued in the caution area. In addition, a warning is also issued if the worker remains in the caution area for 10 seconds or more. In this way, the warning device 33 notifies the worker of danger in stages according to the judgment result determined by the state judgment unit 323 based on the positional relationship between the judgment area stored in the judgment area storage unit 314 and the marker 1 recognized by the marker recognition unit 321, and the direction of the worker's line of sight.

[0042] (5) Control device 34 The control device 34 controls the object placed in the determination area based on the determination result determined by the state determination unit 323. By providing the control device 34 in the heavy equipment and disabling and stopping the driver's operation, collisions can be prevented.

[0043] [1-2. Effect] The operation of the location monitoring system of the first embodiment having the above-described configuration will be described with reference to the flowchart in Fig. 6. In the flowchart in Fig. 6, the main processes are explained with step codes S01 to S07. The flowchart also indicates which part of the location monitoring system performs each process, and what method and data are used.

[0044] The worker wears a helmet with a marker 1 fixed to the top of his / her head. When the worker is captured within the image capture range of the image capture unit 2, the marker 1 is captured and sent to the monitoring device 3 together with other image data.

[0045] The marker recognition unit 321 in the monitoring device 3 reads the marker 1 and extracts the characteristics of the marker 1 (step S01). If the characteristics of the marker 1 are present (Yes in step S01), it is determined whether the size of the marker 1 is within the recognition range (step S02). If the size of the marker 1 is within the recognition range (Yes in step S02), the ID of the marker 1 is read. From the ID, position, and angle of the read marker 1, the position (P0) and angle (V0) of the worker relative to the relative coordinate system, and the line of sight direction of the worker are calculated.

[0046] If there are no features of the marker 1 (No in step S01) or if the size of the marker 1 is outside the recognition range (No in step S02), the marker tracking unit 322 reads the worker's past data stored in the position information storage unit 313. Based on the past data, current or future data is predicted using linear prediction or nonlinear prediction. Then, a calculation is made as to whether the current or future data is within the range of prediction validity. If the current or future data is within the range of prediction validity, the process proceeds to step S03.

[0047] The position (P0) and angle (V0) of the worker calculated by the marker recognition unit 321 are sent to the state determination unit 323. The state determination unit 323 determines whether or not the position (P0) and angle (V0) of the worker overlap with the warning area (step S03). If the position (P0) and angle (V0) of the worker overlap with the warning area (Yes in step S03), the warning device 33 issues a warning to the worker.

[0048] If the worker's position (P0) and angle (V0) do not overlap with the warning area (No in step S03), it is determined whether or not there is overlap with the caution area (Step S04). If the worker's position (P0) and angle (V0) overlap with the caution area (Yes in step S04), it is determined whether or not the worker's line of sight is facing a dangerous direction (Step S05). If the worker's line of sight is facing a dangerous direction (Yes in step S05), the number of times the worker faces the dangerous direction is counted using an increment counter (Step S06). If the increment counter has counted N times or more (Yes in step S06), the warning device 33 issues a warning to the worker.

[0049] If the worker's position (P0) and angle (V0) do not overlap with the warning area (step S03: No), if the worker's line of sight is not directed in a dangerous direction (step S05: No), or if the increment counter has counted less than N (step S06: No), the warning device 33 will issue a warning to the worker.

[0050] [1-3.Effects] The effects of this embodiment are as follows. (1) The degree of danger to the subject is determined based on the positional relationship between the determination area and the marker 1, and the direction of the subject's body. This makes it possible to determine the degree of danger to the subject based on the subject's position and body direction, achieving highly accurate position monitoring and reducing unnecessary monitoring due to overdetection.

[0051] (2) The only equipment worn by the worker is the marker 1 attached to the helmet. This eliminates the need to charge the marker, reducing the burden on the worker and making the equipment lighter, thereby improving work efficiency.

[0052] (3) The system of this embodiment can be constructed with only the marker 1 attached to the worker's helmet, the image acquisition unit 2 that captures the image of the marker 1, and the monitoring device 3. Therefore, compared to systems that use wireless communication, it is possible to reduce the costs of constructing and managing the system.

[0053] (4) The marker tracking unit 322 calculates and stores the movement vector and orientation vector of the marker 1. Therefore, even if the marker 1 becomes unrecognizable due to bending over or being hidden by luggage, it is possible to estimate the current position and orientation of the worker.

[0054] (5) By using the marker 1 as a marker that can be identified by the marker recognition unit 321, a system can be constructed more easily than with human recognition technology. In addition, since the ID information storage unit 311 stores the pattern of the marker 1 and the identification information of the worker associated with the pattern, it becomes possible to more accurately determine the risk level based on the worker information.

[0055] [2. Second Embodiment] The second embodiment will be described with reference to Fig. 7. The second embodiment assumes the use of a mobile crane C as heavy equipment at a construction site. In addition to the configuration of the first embodiment, the second embodiment is equipped with an angular position sensor such as a rotary encoder R, and a marker 1 is also installed on the mobile crane C itself. Fig. 7(a) shows a camera fixed to the upper front of the driver's seat of the heavy equipment, while Fig. 7(b) shows a fixed camera installed on the building side.

[0056] In the second embodiment, by providing an angular position sensor and installing a marker 1 on the mobile crane C itself, it becomes possible to recognize not only the position of the crane C but also the distance between the target person's marker position and the position of the crane C. Furthermore, by providing an angular position sensor on the mobile crane C, the status determination unit 323 acquires information on whether the mobile crane C is moving or stopped.

[0057] When the mobile crane C is stopped, it is likely that work is being done to place or unload cargo, so the state determination unit 323 determines that the risk to the target person is low, and the warning device 33 does not issue a warning or caution. Also, a worker must run alongside the crane C behind the direction of movement of the mobile crane C while visually checking the status of the cargo. Therefore, if there is no worker behind the crane C, the state determination unit 323 determines that there is a danger at the work site, and the warning device 33 issues a warning.

[0058] In the second embodiment, by installing a marker 1 on the mobile crane C itself, it is possible to prevent accidents in which a worker gets under a load and collides with the load due to the load moving or falling.

[0059] 3. Third Embodiment The third embodiment will be described with reference to FIG. 8. The third embodiment assumes a case where refueling is performed from a refueling truck to a refueling device at a gas station or the like. As shown in FIG. 8(a), the configuration is the same as the first embodiment, with the image acquisition unit 2 installed in the refueling truck, the determination processing unit 32 installed in the driver's cab of the refueling truck, and the warning device 33 installed in the refueling device. In addition to the configuration similar to the second embodiment, FIG. 8(b) also has a marker 1 installed on the refueling truck itself.

[0060] In the third embodiment, by monitoring the position and orientation of the worker, if the worker is not facing the fuel filler opening, the worker will not be able to recognize oil leaks or overflows of gasoline or the like due to overfilling, so a warning is issued by the warning device 33 and refueling is stopped by the control device 34.

[0061] According to the third embodiment, the risk level of a subject at the site of refueling work can be determined from the positional relationship between the determination area and the marker 1 and the direction of the subject's body. Therefore, with the spread of self-service refueling, it is possible to prevent accidents during refueling work even when refueling work is performed by people who are not professional workers.

[0062] Even for self-service fueling, the refueling operation requires permission from a specialized worker, so a camera is installed near the fueling device to confirm that the operation is correct. According to the third embodiment, by using a camera already installed near the fueling device and placing a marker 1 at the fuel filler opening, a system can be constructed without installing a new camera.

[0063] [4. Fourth Embodiment] A fourth embodiment will be described with reference to Fig. 9. In recent years, there have been cases where workers have collapsed during work due to heat stroke or sudden illness, and if the situation is not discovered in time, the situation can become serious. The fourth embodiment is intended to avoid dangerous situations by discovering a collapsed worker early.

[0064] In the fourth embodiment, the information about the marker 1 is the angle of the marker, and the determination area is the angle range of the marker 1. The state determination unit 323 determines the state of the subject based on the angle of the marker 1 recognized by the marker recognition unit 321 and the angle range stored in the determination area.

[0065] As shown in Fig. 9, the range of angle changes that occur with normal work for the angle of the worker's marker (within the range of an inverted cone calculated from the XYZ coordinates) and the angle range that is expected if the worker falls (outside the range of the inverted cone calculated from the XYZ coordinates; see the upper right and lower right diagrams in Fig. 9) are stored in advance as judgment regions in the judgment region storage unit 314. If the worker remains outside the range of the judgment region for a certain period of time or longer, the state judgment unit 323 judges that there is a high possibility that the worker has fallen, and notifies the surrounding area of ​​the danger using the warning device 33.

[0066] The angle range of the marker 1 in the judgment area may be determined by learning in advance, through AI processing, movements that may indicate a risk of falling. By storing movements that indicate a risk of falling as a judgment area in advance in the judgment area storage unit 314, danger can be recognized in advance and a warning can be issued.

[0067] According to the fourth embodiment, even if there are no dangerous objects around the subject, it is possible to discover a worker who has collapsed while working due to heatstroke or sudden illness, thereby ensuring the safety of the worker.

[0068] 3. Other Embodiments Although multiple embodiments according to the present invention have been described herein, these embodiments are presented as examples and are not intended to limit the scope of the invention. The above-described embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the scope of the invention. These embodiments and their modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as set forth in the claims.

[0069] (1) As shown in Figure 10, instead of AR markers, motion capture points or other appropriate markers can be used. In this case, it is recommended that the markers have distinctive shapes and colors that are easy for the camera to recognize. In addition, markers that reflect infrared or ultraviolet light that can be captured by a camera or that emit light themselves can also be used, regardless of the color of the visible light. Figure 10 shows side views, top views, top views (facing downward), top views (facing upward), and top views (facing sideways) of cases where motion capture points, AR markers, or other unique markers are used. Regardless of which marker is used, by being able to detect not only the position of the subject but also the body orientation, it is possible to more accurately determine the subject's condition and achieve highly accurate position monitoring.

[0070] (2) Instead of fixing the marker to the top of the helmet, it may be fixed to the shoulder or back of the work clothes, printed as a logo mark on the chest, or used as a name tag, etc. Also, if the object is equipment, a product, or a box, a marker sticker may be attached.

[0071] (3) To make the marker easier to recognize and improve the accuracy of marker recognition, this system may be combined with a location recognition device using Bluetooth or a radio beacon. In other words, it is also possible to incorporate a wireless transmitter / receiver into the marker to perform location recognition.

[0072] (4) By placing markers on other heavy machinery and cargo in addition to workers, unintended collisions between heavy machinery or cargo can be avoided, just as with workers. In addition, by placing markers on the ground or walls, it can be used to prevent the heavy machinery itself from tipping over or falling.

[0073] (5) The information recognized by the marker recognition unit may be only the position of the marker or only the angle of the marker. In other words, when only the angle of the marker is recognized, the angle range of the marker is stored as a judgment area in the judgment area storage unit 314, and the state determination unit 323 determines the state of the subject based on the angle of the marker recognized by the marker recognition unit 321 and the angle range stored in the judgment area storage unit 314. For example, when the angle of the marker becomes parallel to the ground, it may be determined that the subject has fallen and a warning may be issued. In this case, combining the change in the angle of the marker and the position information enables more accurate determination. [Explanation of symbols]

[0074] 1...Marker 2...Video acquisition unit 3…Monitoring device 31...Storage section 311…ID information storage section 312...Relative coordinate storage section 313...Location information storage section 314...Judgment area storage section 32...Determination processing unit 321...Marker recognition unit 322...Marker tracking section 323...Status determination unit 33...Warning device 331...Lighting warning device 332…Acoustic warning device 34...Control device C...Crane R...Rotary encoder

Claims

1. A marker attached to the subject; an image acquisition unit that captures an image of the marker; an ID information storage unit that stores the subject, the marker, and ID information linking the subject and the marker; a marker recognition unit that recognizes information about the marker captured by the image acquisition unit; a determination region storage unit for storing the determination region; a state determination unit that determines a state of the subject having an ID corresponding to the marker based on information of the determination area and the marker; Equipped with The information about the marker is an angle of the marker, The determination region is an angle range of the marker, The state determination unit determines the state of the subject based on the angle of the marker recognized by the marker recognition unit and the angle range stored in the determination region.

2. The information on the marker is the position and angle of the marker, The determination area is a distance between the marker and an object to be determined and an angle range of the marker, The position monitoring system according to claim 1 , wherein the state determination unit determines the state of the subject and the object based on the position and angle of the marker, the distance to the object, and the angle range of the marker.

3. When the position or angle of the marker cannot be recognized by the marker recognition unit, a marker tracking unit that estimates a current position or angle of the marker from a past position or angle of the marker; 3. The location monitoring system of claim 2, comprising:

4. Based on the determination result determined by the state determination unit, a control device that controls the object placed in the determination area; 4. The location monitoring system according to claim 2 or 3, comprising:

5. Depending on the determination result determined by the state determination unit, a warning device that notifies the subject of the determination result in stages; 5. A location monitoring system according to claim 1, further comprising:

Images