surveillance device

Abstract

To provide a highly accurate and convenient monitoring device with a simple and inexpensive configuration.SOLUTION: One or more markers 200 are attached to a monitored object. A monitoring device 100 includes: a camera 110; a trained marker recognition unit 120 that detects the coordinates in the captured image of the marker 200 included in the captured image from the captured image by the camera 110 and the distance between the marker 200 and the camera 110 in the real space; a marker position calculation unit 130 that calculates the position of the marker 200 in the physical space based on the coordinates and the distance detected by the marker recognition unit 120; and a warning processing unit 140 that outputs a warning based on the position of the marker in the physical space calculated by the marker position calculation unit 130 and area information of a monitoring area 10.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a monitoring device that monitors the position or behavior of a monitoring target within a monitoring area. [Background technology]

[0002] Conventionally, known monitoring devices of this type are described in Patent Document 1 and Non-Patent Document 1. The device described in Patent Document 1 is equipped with two cameras to recognize a monitored object and detects the position of the monitored object using a stereo method, and issues a warning when the monitored object enters a monitoring map. The device described in Non-Patent Document 1 detects the position of the monitored object in a planar space using laser scanning, and issues a warning when the monitored object enters a specified monitoring area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 10-79934 [Non-patent literature]

[0004] [Non-Patent Document 1] "Laser Barrier System LMS Series," [online], [searched September 2, 2021], Internet<https: / / www.netis.mlit.go.jp / netis / pubsearch / details?regNo=KT-130018%20> Summary of the Invention [Problem to be solved by the invention]

[0005] However, the system described in Patent Document 1 requires the installation of two cameras, which is inconvenient. In particular, in order to improve the detection accuracy of monitored objects and issue more accurate warnings, the distance between the cameras needs to be increased. However, increasing the distance between the cameras increases the possibility that the monitored object will be located in a blind spot of other obstacles, which limits the installation locations of the cameras. On the other hand, reducing the distance between the cameras to reduce the possibility that the monitored object will be located in a blind spot of other obstacles reduces the detection accuracy of the monitored object.

[0006] Furthermore, the systems described in Patent Document 1 and Non-Patent Document 1 do not distinguish between monitored objects and other objects within a monitored area. This makes it necessary to prevent objects or people other than monitored objects from entering the monitored area, and in this respect, they are also inconvenient.

[0007] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a monitoring device that is simple in construction, inexpensive, highly accurate, and convenient. [Means for solving the problem]

[0008] In order to achieve the above object, the present invention provides a monitoring device for monitoring the position or behavior of a monitored object within a monitored area, the monitored object having one or more markers attached thereto, the monitoring device comprising: a camera; a marker recognition unit that has been trained by machine learning to identify the marker and recognize the distance between the captured position of the marker image in real space and the marker itself, using a plurality of marker images containing the markers as learning data; the marker recognition unit that detects, from an image captured by the camera, the coordinates of the marker contained in the captured image and the distance between the marker and the camera in real space; a marker position calculation unit that calculates the position of the marker in real space based on the coordinates and distance detected by the marker recognition unit and the imaging direction of the camera; and a monitoring processing unit that monitors the position or behavior of the monitored object based on the position of the marker in real space calculated by the marker position calculation unit and area information of the monitored area.The monitoring processing unit estimates a future position of the marker based on a change over time in the position of the marker in real space calculated by the marker position calculation unit, and outputs a warning based on the estimated position and area information of the monitoring area. It is characterized by: [Effects of the Invention]

[0009] According to the present invention, monitoring of a monitored object is possible by attaching a marker to the monitored object and installing a single camera in a position where it can capture an image of the monitored area, resulting in a highly convenient and inexpensive system. Furthermore, by performing sufficient training on the marker recognition unit, high detection accuracy can be achieved even with a single camera. Furthermore, since the target of detection is not the monitored object itself but the marker attached to the monitored object, it is possible to easily improve the learning accuracy and recognition accuracy compared to detecting the monitored object itself. Furthermore, since objects and people other than the marker are not detected, management of the monitored area is easy, resulting in a highly convenient system. [Brief explanation of the drawings]

[0010] [Figure 1] A diagram illustrating how a monitoring device is used. [Figure 2] 1 is a functional block diagram of a monitoring device according to a first embodiment; [Figure 3] FIG. 10 is a diagram showing an example of image data. [Figure 4] Diagram explaining the marker position calculation algorithm [Figure 5] Perspective view of the marker [Figure 6] A diagram explaining the relationship between the camera position and the image relative to the marker. [Figure 7] A diagram explaining the relationship between the camera position and the image relative to the marker. [Figure 8] Functional block diagram of the learning device [Figure 9] 10 is a functional block diagram of a monitoring device according to a second embodiment. [Figure 10] 10 is a functional block diagram of a monitoring device according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] (First embodiment) A monitoring device according to a first embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a diagram illustrating a usage pattern of the monitoring device, Fig. 2 is a functional block diagram of the monitoring device according to the first embodiment, Fig. 3 is a diagram showing an example of image data, and Fig. 4 is a diagram illustrating a calculation algorithm for marker positions.

[0012] 1, the monitoring device 100 according to this embodiment has a load 2 suspended from a crane 1 as a monitoring object, a part or all of the work area of the crane 1 as a monitoring area 10, and monitors the position or behavior of the load 2 as a monitoring object within the monitoring area 10 and outputs a predetermined warning. Here, one or more markers 200 are attached to the load 2. In other words, the monitoring device 100 according to the present invention monitors the position or behavior of the markers 200, thereby monitoring the position or behavior of the load 2 as a monitoring object.

[0013] As shown in FIG. 2, the monitoring device 100 includes a camera 110, a marker recognition unit 120, a marker position calculation unit 130, a monitoring processing unit 140, and a monitoring area information storage unit 141.

[0014] The monitoring device 100 can be configured by a conventionally known computer equipped with a main processing unit, a main memory unit, an auxiliary memory unit, a display device, an input device, etc. The monitoring device 100 can be implemented by installing a program that causes each of the above-mentioned units to function on a computer. The monitoring device 100 can also be implemented as dedicated hardware. The monitoring device 100 can also be implemented in a distributed manner across multiple devices. For example, in this embodiment, the camera 110 is implemented as a part of the monitoring device 100, but the camera 110 may also be implemented separately from the other components of the monitoring device 100. In this embodiment, a highly functional mobile communication terminal called a "smartphone" with a built-in camera 110 is used as the monitoring device 100.

[0015] The camera 110 is a well-known imaging device that captures an image of a predetermined area in real space and outputs two-dimensional image data. The camera 110 outputs a plurality of digital image data over time as a moving image. The format, resolution, frame rate, angle of view, focal length, etc. of the image data output by the camera 110 are arbitrary. Each pixel of the two-dimensional image data captured by the camera 110 is specified by a two-dimensional Cartesian coordinate system. In this embodiment, the center point of the image data captured by the camera 110 is set as the origin of the Cartesian coordinate system. The center point of the image data captured by the camera 110 coincides with the direction in which the camera 110 is facing, i.e., the optical axis.

[0016] The camera 110 is installed in a position and orientation such that all or part of the monitored area 10 is within its imaging range. The camera 110 is installed in a position and orientation such that an area of the monitored area 10 that is the target of an alert is within its imaging range. From the viewpoint of detection accuracy, the camera 110 is preferably installed in a position and orientation such that a warning area of the monitored area 10 that is the target of an alert is located directly in front of the camera 110. In this embodiment, to simplify calculations, it is assumed that the camera 110 is installed so that its imaging direction, i.e., its optical axis, is horizontal.

[0017] The monitored area 10 will now be described. The monitored area 10 is an area of any shape defined in three-dimensional real space. Here, the position in the real space can be defined in a three-dimensional Cartesian coordinate system. The position in the real space can be a relative value based on the installation position and imaging direction, i.e., the optical axis direction, of the camera 110. The position in the real space may also be an absolute value using, for example, latitude, longitude, and altitude. In this embodiment, the position in the real space is a relative value based on the installation position and imaging direction, i.e., the optical axis direction, of the camera 110, with the vertical direction being the Z axis, the direction in which the camera 110 is facing, i.e., the optical axis direction, being the X axis, and the direction perpendicular to the direction in which the camera 110 is facing, i.e., the optical axis direction, being parallel to the horizontal plane, being the Y axis.

[0018] The monitoring area 10 defines the spatial position reference for the warning output by the monitoring device 100, and is defined as a three-dimensional space in real space. Multiple monitoring areas 10 can be set. Area information of the monitoring areas 10 is stored in the monitoring area information storage unit 141.

[0019] 1, in this embodiment, the entire area in front of and to the left of the camera 110 is set as the monitored area 10, based on the installation position of the camera 110. The monitored area 10 is defined as a relative value based on the installation position and imaging direction of the camera 110, i.e., the optical axis direction.

[0020] The marker recognition unit 120 receives as input image data captured by the camera 110, in which the marker 200 is captured, and is configured with a learning device that has undergone deep learning using a teacher signal to detect the presence or absence of the marker 200 captured in the image data, its identifier, its two-dimensional coordinates M (Mx, My) in the image data, and the distance D between the marker 200 and the camera 110 in real space. Here, in the image data, an image of the marker 200 is formed in an area whose size corresponds to the distance from the camera 110. The marker recognition unit 120 regards the representative coordinates, such as the center coordinates or center of gravity coordinates, of the area in which the image of the marker 200 is formed as the coordinates M of the marker 200. An example of the image data is shown in FIG. 3.

[0021] The marker recognition unit 120 is a well-known learning device capable of deep learning, configured by a neural network having multiple intermediate layers. The marker recognition unit 120 includes an inference engine and parameters for putting the inference engine into a trained state. The marker recognition unit 120 may be configured such that the inference engine and the parameters are inseparable, or may be configured such that the two are separable.

[0022] When the marker recognition unit 120 detects the presence of the marker 200 captured in image data captured by the camera 110, the marker position calculation unit 130 calculates the position P (Px, Py, Pz) of the marker 200 in a three-dimensional Cartesian coordinate system in real space based on the two-dimensional coordinate M in the image data of the marker 200 detected by the marker recognition unit 120, the distance D between the marker 200 and the camera 110 in real space, and the imaging direction of the camera 110, i.e., the optical axis direction. The position P of the marker 200 calculated by the marker position calculation unit 130 may be a relative value based on the known installation position and imaging direction of the camera 110, i.e., the optical axis direction, or may be an absolute value. In this embodiment, the position P of the marker 200 calculated by the marker position calculation unit 130 is a relative value based on the known installation position and imaging direction of the camera 110, i.e., the optical axis direction.

[0023] The calculation algorithm of the marker position calculation unit 130 will be described with reference to FIG. 4. First, consider a virtual reality space based on image data. That is, consider a virtual three-dimensional Cartesian coordinate system corresponding to the imaging direction (optical axis direction) of the camera 110 and the tilt of the camera 110 with the imaging direction as its central axis. In this virtual three-dimensional Cartesian coordinate system, the depth direction of the image data, i.e., the imaging direction (optical axis direction) of the camera 110 in real space, is the X-axis direction; the left-right direction of the image data, i.e., the left-right direction relative to the camera 110, is the Y-axis direction; and the up-down direction of the image data, i.e., the up-down direction relative to the camera 110, is the Z-axis direction. The installation position of the camera 110 is the origin. The marker position calculation unit 130 calculates the angle θ (θxy, θz) of the line connecting the camera 110 to the marker 200 with respect to the central axis (optical axis) extending in the X-axis direction from the front of the camera 110 in the virtual reality space, from the two-dimensional coordinate M in the image data of the marker 200. Here, θxy is the horizontal component of angle θ, i.e., the azimuth angle in the virtual reality space, and θz is the vertical component, i.e., the elevation angle in the virtual reality space. The calculation process for converting coordinate M to angle θ can be performed based on known specification information such as the focal length of camera 110 and the resolution (number of pixels) of image data output by camera 110. The specification information for camera 110 may be stored in advance in a predetermined storage device (not shown) of monitoring device 100. Through this process, if the distance in real space between marker 200 detected by marker recognition unit 120 and camera 110 is D, the position of marker 200 in virtual reality space is identified by polar coordinates (D, θ) with the installation position of camera 110 as the origin. Then, marker position calculation unit 130 converts the polar coordinates (D, θ) into position P' in a virtual three-dimensional Cartesian coordinate system using a known coordinate conversion process, and further converts it into position P in the three-dimensional Cartesian coordinate system in real space based on the installation position of camera 110 in real space and the imaging direction of the camera, i.e., the optical axis direction. Information about the installation position of the camera 110 and the imaging direction of the camera may be stored in advance in a predetermined storage device (not shown) of the monitoring device 100.

[0024] The monitoring processing unit 140 performs an arbitrary warning output process based on the position P of the marker 200 in real space calculated by the marker position calculation unit 130 and area information of the monitoring area 10. When multiple monitoring areas 10 are set, the monitoring processing unit 140 performs a warning output process for each monitoring area 10. Furthermore, when there are multiple markers 200, the monitoring processing unit 140 performs a warning output process for each marker 200.

[0025] In some embodiments, the monitoring processing unit 140 outputs a warning based on the positional relationship between the monitoring area 10 and the boundary surface 11 outside the monitoring area 10. For example, the monitoring processing unit 140 outputs a warning when a position P that was within the monitoring area 10 reaches the boundary surface 11. In another example, the monitoring processing unit 140 outputs a warning when a position P that was within the monitoring area 10 passes the boundary surface 11 and enters outside the monitoring area 10. In another example, the monitoring processing unit 140 outputs a warning when the distance between the position P and the boundary surface 11 within the monitoring area 10 becomes equal to or less than a predetermined value. In some embodiments, a predicted position of the future marker 200 is calculated from changes over time in the position P (e.g., speed or acceleration), and a warning is output based on this predicted position and area information of the monitoring area 10. For example, the monitoring processing unit 140 calculates the speed of the position P and outputs a warning when the speed at which the position P approaches the boundary surface 11 is equal to or greater than a predetermined value. The warning output processes according to the various embodiments can be combined in any manner.

[0026] The form of the warning output by the monitoring processing unit 140 is not important. In some embodiments, the monitoring device 100 outputs a predetermined alarm sound from a speaker or the like, displays a predetermined warning display, or activates a vibration generator. In some embodiments, the monitoring device 100 transmits a warning signal to another warning device, and the other warning device outputs a predetermined alarm sound from a speaker or the like, displays a predetermined warning display, activates a vibration generator, or controls equipment. In this case, it is preferable that the other warning device be located on or near the person or device operating the monitored object. The other warning device may also be located anywhere within the monitored area 10 or anywhere near the monitored area 10. The other warning device may also be located on the monitored object or the marker 200. The transmission path of the warning signal to the other warning device may be wireless or wired.

[0027] Next, the marker 200 will be described with reference to FIG. 4. As shown in FIG. 4, the marker 200 includes a cylindrical marker body 210. An identification pattern 220 is formed on the peripheral surface located at one end of the marker body 210. The identification pattern 220 is formed by axially arranging multiple band-shaped identification bands 221 extending around the entire circumference. In this embodiment, the identification bands 221 are adjacent to each other without any gaps, and adjacent identification bands 221 have different colors. Here, different colors means that at least one of hue, brightness, and saturation is different from each other. An end face of the marker body 210 is colored differently from the identification bands 221 adjacent to that end face. An end face at one end of the marker body 210 may be colored differently from the colors of all the identification bands 221. An end face of the marker body 210 may be colored the same as any of the identification bands 221 that are not adjacent to that end face. The identification pattern 220 may be disposed at a position where a gap is formed between the identification pattern 220 and the end face of the marker body 210. The identification pattern 220 may also be formed on the entire outer periphery of the marker body 210.

[0028] The identification pattern 220 is used to identify the marker 200 by the combination, permutation, or sequence of the colors of the multiple identification bands 221. The colors used for the identification bands 221 are selected from a group of colors that differ from each other in at least one of hue, brightness, and saturation. However, except when the identification pattern 220 is formed on the entire outer periphery of the marker body 210, the color used for the identification band 221 differs from the color of the portion of the outer periphery of the marker body 210 other than the identification pattern 220 in at least one of hue, brightness, and saturation. The multiple identification bands 221 may be assigned the same color. The identification pattern 220 may encode the identification information of the marker 200. For example, a single-digit number may be assigned to each color, such as black = 0, red = 1, ... green = 9, and the identifier of the marker 200 may be expressed as a three-digit number depending on the order of the three identification bands 221.

[0029] The marker 200 may be installed on a monitored object in real space in any manner. Typically, the marker 200 is installed so that the longitudinal direction of the cylindrical marker body 210 is aligned with the vertical direction and the identification pattern 220 is at the top. In the example of Fig. 1, the markers are installed at the four corners of a roughly rectangular parallelepiped suspended load 2 so that the longitudinal direction of the cylindrical marker body 210 is aligned with the vertical direction of the suspended load 2.

[0030] In such a marker 200, if the angle of the camera 110 with respect to the axial direction of the marker body 210 is the same, the shape of the map of the identification pattern 220 in the image data will be the same regardless of the circumferential direction of the marker body 210 from which the image is taken, as shown in Fig. 5. Also, in such a marker 200, if the angle of the camera 110 with respect to the axial direction of the marker body 210 is right angle (see Fig. 6(a)), as shown in Fig. 6, each identification band 221 in the map 231 in the image data 230 of the identification pattern 220 will be rectangular (see Fig. 6(b)). Also, in such a marker 200, even if the angle of the camera 110 with respect to the axial direction of the marker body 210 is deviated up or down from a right angle (see Fig. 7(a)), as shown in Fig. 7, each identification band 221 in the map 233 in the image data 232 of the identification pattern 220 will be a substantially rectangular shape, specifically, the upper and lower sides of the rectangle will be slightly curved in the vertical direction.

[0031] Therefore, with such a marker 200, the image of the marker 200 to be recognized by the marker recognition unit 120 has a stable shape without being significantly affected by the imaging position or angle of the camera 110, thereby improving recognition accuracy. In particular, when the distance between the camera 110 and the marker 200 is large, the number of pixels occupied by the image of the marker 200 in the image data decreases, but even in this case, a decrease in recognition accuracy can be suppressed. Similarly, when the image of the marker 200 is significantly deviated from the center of the image, distortion occurs in the image due to lens distortion, but even in this case, a decrease in recognition accuracy due to distortion can be suppressed. Note that if the marker recognition unit 120 is provided with a distortion correction unit that performs correction processing for lens distortion using the eigenvalues of the lens, a decrease in recognition accuracy due to distortion can be further suppressed.

[0032] Furthermore, in the learning process of the marker recognition unit 120, the number of samples of learning data input to the learning device can be small, which reduces the effort required to acquire learning data and also reduces the effort required for the learning process.

[0033] Next, the learning device 300 for generating the marker recognition unit 120 will be described with reference to Fig. 8. The learning device 300 receives as input a still image containing a marker 200 as learning data, and trains a learning module through manual annotation processing.

[0034] The input learning data is a plurality of still images of a plurality of markers 200 with different identification patterns 220, captured under different imaging conditions such as angle, distance, angle of view, and lighting. In this embodiment, the learning data is a still image, but it may also be a video. In this case, a plurality of still images can be obtained by dividing the video into frames.

[0035] As shown in FIG. 8, the learning device 300 includes an image preprocessing unit 310, an annotation processing unit 320, and a learning device 330 that is the learning target.

[0036] The image preprocessing unit 310 performs image processing such as brightness adjustment on the learning data as preprocessing for learning.

[0037] The annotation processing unit 320 performs annotation processing on the learning data. The annotation processing is performed by displaying an image on a display device (not shown) and having the user input the area where the recognition pattern 220 exists using an input device (not shown) such as a mouse. In other words, this input annotation information becomes a teacher signal. Note that an identifier of the recognition pattern 220 may be added as the teacher signal. The annotation processing unit 320 may also augment the learning data by performing black and white conversion, inversion, rotation, etc. on the learning data.

[0038] The learning device 330, which is the learning target, has the same structure as the marker recognition unit 120 except for its learning state. The learning device 330 learned by the learning device 300 is installed in the monitoring device 100 as the marker recognition unit 120. The learning device 330 may be installed from the learning device 300 to the monitoring device 100 via a network or via a predetermined recording medium. When installing the learning device 330 from the learning device 300 to the monitoring device 100, the learning device 330 itself may be installed, or only various parameters corresponding to the learning state of the learning device 330 may be installed in the marker recognition unit 120 of the monitoring device 100.

[0039] Such a monitoring device 100 allows monitoring of a monitored object by attaching a marker 200 to the monitored object and installing a single camera 110 at a position in the monitored area 10 where the marker can be captured, resulting in a highly convenient and inexpensive device. Furthermore, by adequately training the marker recognition unit 120, high detection accuracy can be achieved even with a single camera 110. Furthermore, since the target of detection is the marker 200 attached to the monitored object, rather than the monitored object itself, learning accuracy and recognition accuracy can be easily improved compared to detecting the monitored object itself. Furthermore, since objects and people other than the marker 200 are not detected, management of the monitored area 10 is simplified, resulting in a highly convenient device.

[0040] (Second embodiment) A monitoring device according to a second embodiment of the present invention will be described with reference to Fig. 9. Fig. 9 is a functional block diagram of a monitoring device according to the second embodiment. This embodiment differs from the first embodiment in the method of setting the monitoring area. Since the rest of the embodiment is the same as the first embodiment, only the differences will be described here.

[0041] 9, the monitoring device 100a includes a camera 110, a marker recognition unit 120, a marker position calculation unit 130, a monitoring processing unit 140, a monitoring area information storage unit 141, and a monitoring area setting unit 150. Prior to the start of warning processing by the monitoring processing unit 140, the monitoring area setting unit 150 sets a monitoring area 10 based on the position in real space of an area setting marker 200a placed within the image capture range of the camera 110. The position in real space of the area setting marker 200a is detected by the marker recognition unit 120 and the marker position calculation unit 130. Here, the area setting marker 200a can be identified by an identification pattern 220 that is different from the marker 200 attached to the monitored object.

[0042] According to such a monitoring device 100a, the monitoring area 10 can be flexibly set at the monitoring site, thereby improving convenience. Other functions and effects are the same as those of the first embodiment.

[0043] (Third embodiment) A monitoring device according to a second embodiment of the present invention will be described with reference to Fig. 10. Fig. 10 is a functional block diagram of a monitoring device according to a third embodiment. This embodiment differs from the first and second embodiments in that a calibration processing unit 131 is provided in a marker position calculation unit 130. Since the rest of the configuration is the same as the first and second embodiments, only the differences from the first embodiment will be described here.

[0044] 10, the monitoring device 100b includes a camera 110, a marker recognition unit 120, a marker position calculation unit 130, a monitoring processing unit 140, a monitoring area information storage unit 141, and a monitoring area setting unit 150. In this embodiment, the marker position calculation unit 130 includes a calibration processing unit 131.

[0045] Prior to the start of warning processing by the monitoring processing unit 140, the calibration processing unit 131 performs position-related calibration processing based on the position and distance of a calibration marker 200b that is placed at a known position in real space within the image capture range of the camera 110. The position of the calibration marker 200b in real space is detected by the marker recognition unit 120 and the marker position calculation unit 130. The calibration marker 200b can be identified by an identification pattern 220 that is different from the marker 200 attached to the monitored object. The calibration processing unit 131 performs calibration processing based on the difference between the position of the marker 200b detected by the marker recognition unit 120 and the marker position calculation unit 130 and the known position of the marker 200b. The known position of the calibration marker 200b can be input from outside.

[0046] Such a monitoring device 100b can perform monitoring with high accuracy because it improves the positional accuracy of the marker 200. Other functions and effects are the same as those of the first embodiment.

[0047] The first to third embodiments of the present invention have been described in detail above, but the present invention is not limited to the above embodiments, and various improvements and modifications may be made within the scope that does not deviate from the gist of the present invention.

[0048] For example, in the above embodiment, relative values from the camera 110 are used as the positions of the monitored area 10 and the marker 200, but absolute values may also be used. In this case, a position detection device such as a GNSS (Global Navigation Satellite System) receiver may be provided in the monitoring device to obtain the absolute position of the camera 110. Also, the position of the camera 110 may be measured using a known surveying device, and the measured position may be input to the monitoring device.

[0049] In the above embodiment, the load 2 suspended from the crane 1 is exemplified as the monitored object, but other objects may be monitored. The monitored area 10 can also be set arbitrarily depending on the usage of the monitoring device 100, etc.

[0050] Furthermore, in the above embodiment, an example has been described in which the imaging direction of camera 110, that is, the optical axis, is set horizontally to facilitate calculation, but the imaging direction of camera 110 is arbitrary. [Explanation of symbols]

[0051] 1. Crane 2…Hanging load 10…Monitoring area 11...Boundary 100,100a,100b…Monitoring device 110...Camera 120...Marker recognition unit 130...Marker position calculation unit 131...Calibration processing unit 140...Monitoring processing unit 141...Monitoring area information storage unit 150...Monitoring area setting section

Claims

1. A monitoring device that monitors the position or behavior of a monitoring target within a monitoring area, One or more markers are attached to the monitored object; The monitoring device A camera and a marker recognition unit that has been trained by machine learning to use a plurality of marker images containing the marker as learning data to identify the marker and recognize the distance between the marker and the image capture position of the marker image in real space, and that detects, from the image captured by the camera, the coordinates of the marker in the captured image and the distance between the marker and the camera in real space; a marker position calculation unit that calculates the position of the marker in real space based on the coordinates and distance detected by the marker recognition unit and the imaging direction of the camera; a monitoring processing unit that monitors the position or behavior of a monitoring target based on the position of the marker in real space calculated by the marker position calculation unit and area information of the monitoring area, The monitoring processing unit estimates a future position of the marker based on a change over time in the position of the marker in real space calculated by the marker position calculation unit, and outputs a warning based on the estimated position and area information of the monitoring area. A monitoring device characterized by:

2. The monitoring processing unit outputs a warning based on the positional relationship between the position of the marker in real space calculated by the marker position calculation unit and the boundary surface of the monitoring area.

2. The monitoring device according to claim 1.

3. The monitoring processing unit outputs a warning when the position of the marker in real space calculated by the marker position calculation unit goes beyond the boundary surface from the monitoring area.

3. The monitoring device according to claim 2.

4. The monitoring area is determined relative to the installation position and imaging direction of the camera.

4. The monitoring device according to claim 1, wherein the monitoring device comprises: a first detecting means for detecting a first signal;

5. The monitoring processing unit includes a monitoring area setting unit that sets the monitoring area based on the position in real space of a marker for area definition that is installed within the imaging range of the camera.

4. The monitoring device according to claim 1, wherein the monitoring device comprises: a first detecting means for detecting a first signal;

6. The marker position calculation unit includes a calibration processing unit that performs calibration processing based on the position and distance of a calibration marker that is set at a known position in real space within the image capture range of the camera.

6. The monitoring device according to claim 1, wherein the monitoring device is a monitoring device for monitoring a plurality of objects.

7. The marker has a cylindrical marker body, and a plurality of band-shaped identification portions extending around the entire circumference are formed on the outer surface of the marker body and arranged in the axial direction.

7. The monitoring device according to claim 1, wherein the monitoring device comprises: a first detecting means for detecting a first error;

Images