Work site monitoring system

Abstract

To provide a work site monitoring system capable of suppressing a load on a monitoring person even when a plurality of imaging units are set.SOLUTION: A work site monitoring system 100 comprises: a plurality of cameras 10 disposed at a work site; a monitor 30 capable of displaying video imaged by the cameras 10; a monitoring target position identification unit 41 for identifying a position of a monitoring target; a candidate selection unit 42 that on the basis of a predetermined candidate condition, selects, from the plurality of cameras 10, candidate imaging units to be candidates for displaying video on the monitor 30; and a determination unit 43 that out of the candidate imaging units selected by the candidate selection unit 42, determines a display target imaging unit for displaying video on the monitor 30 to make video imaged by the display target imaging unit be displayed on the monitor 30. The candidate condition includes a first candidate condition that no shielding objects are disposed between a position of the monitoring target identified by the monitoring target position identification unit 41 and a position of the camera 10.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a work site monitoring system that monitors a work site. [Background technology]

[0002] Patent Document 1 discloses a safety management system that ensures safety during construction work. This safety management system uses a photographing device such as a CCD camera to photograph the entire construction site, captures the outlines of targets such as construction machinery and workers from the photographed image, sets target areas, and issues an alarm or other warning when the set target areas come into contact with each other or when the target area deviates from the work area. The photographing device is made up of a CCD camera, and is mounted at an elevated position that allows it to photograph the entire construction site. A monitoring monitor is also installed in the site office or elsewhere. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-203677 Summary of the Invention [Problem to be solved by the invention]

[0004] Meanwhile, construction work sites can be vast. When a work site is vast, construction work is carried out in various locations, making it difficult to monitor the work site with a single camera (photographing unit) like the safety management system described in Patent Document 1. For this reason, when the work site is vast, multiple cameras are installed in various locations on the work site, and the work site is sometimes monitored using multiple monitoring monitor screens corresponding to the cameras, or using a single monitoring monitor screen divided into multiple screens.

[0005] However, the images displayed on the screens of multiple surveillance monitors or on multiple split screens of a single surveillance monitor may include images that are unrelated to the desired surveillance target. This requires the surveillance person checking the images on the surveillance monitor screen to find the desired surveillance target from the multiple images, which increases the surveillance person's burden.

[0006] Furthermore, installing multiple monitoring monitors in a site office or the like increases the burden on the monitor, and can also lead to problems such as increased costs and the need for space to install the monitors. On the other hand, if one monitoring monitor screen is split into multiple screens, each split screen becomes smaller, which can further increase the burden on the monitor.

[0007] Therefore, an object of the present disclosure is to provide a work site monitoring system that can reduce the burden on the monitor even when multiple imaging units are provided. [Means for solving the problem]

[0008] In order to solve the above problem, a work site monitoring system of a first aspect of the present invention comprises a plurality of image capturing units arranged at a work site, a display unit capable of displaying images captured by the image capturing units, a monitoring target position identification unit that identifies the position of a monitoring target, a candidate selection unit that selects a candidate image capturing unit from the plurality of image capturing units to be a candidate for displaying images on the display unit based on predetermined candidate conditions, and a determination unit that determines a display target image capturing unit from the candidate image capturing units selected by the candidate selection unit to display images on the display unit, and displays the images captured by the display target image capturing unit on the display unit, wherein the candidate conditions include a first candidate condition that no obstructions are located between the position of the monitoring target identified by the monitoring target position identification unit and the position of the image capturing unit.

[0009] A second aspect of the present invention is a work site monitoring system of the first aspect, wherein the candidate conditions include a second candidate condition that the distance between the position of the monitoring target identified by the monitoring target position identification unit and the position of the photographing unit is shorter than a predetermined distance, and the candidate selection unit selects the photographing unit that satisfies both the first candidate condition and the second candidate condition as the candidate photographing unit.

[0010] A third aspect of the present invention is a work site monitoring system of the first aspect or the second aspect, further comprising a virtual space information storage unit that stores virtual space information for recreating the work site in a virtual space, and the candidate selection unit determines whether the first candidate condition is satisfied based on the image acquired by the shooting unit or the virtual space information.

[0011] A fourth aspect of the present invention is a work site monitoring system of the first aspect or the second aspect, wherein the determination unit determines the target shooting unit to be displayed from among the candidate shooting units based on predetermined determination conditions, and the determination conditions include a determination condition that the image captured by the candidate shooting unit contains a predetermined area of ​​interest.

[0012] A fifth aspect of the present invention is a work site monitoring system of the first aspect or the second aspect, comprising an environment detection unit disposed at the work site and detecting environmental values ​​of the work site, and the monitored object position identification unit identifies the position of the monitored object based on the environmental values ​​detected by the environment detection unit and the image captured by the imaging unit. [Effects of the Invention]

[0013] According to the present disclosure, it is possible to provide a work site monitoring system that can reduce the burden on the monitor even when multiple imaging units are provided. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram showing the configuration of a work site monitoring system according to a first embodiment of the present invention. [Figure 2]FIG. 2 is an explanatory diagram illustrating an example of a hardware configuration. [Figure 3] FIG. 1 is an explanatory diagram illustrating an example of a virtual space. [Figure 4] FIG. 2 is an explanatory diagram showing an example of an image captured by a camera. [Figure 5] 10 is a flowchart of a work site monitoring process. [Figure 6] FIG. 10 is a block diagram showing the configuration of a work site monitoring system according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0016] The work site monitoring system according to the present disclosure is a system for monitoring the safety of a work site. The work site is not particularly limited as long as it is a site where various types of work are performed, and examples thereof include work sites for construction work, building construction, etc. Furthermore, the work site is not limited to a work site on land, and may be, for example, an underwater work site.

[0017] First Embodiment Fig. 1 is a block diagram showing the configuration of a work site monitoring system according to a first embodiment of the present invention, Fig. 2 is an explanatory diagram showing an example of the hardware configuration.

[0018] (Workplace monitoring system) 1, a work site monitoring system 100 according to a first embodiment of the present invention includes a plurality of cameras (imaging units) 10, a monitor (display unit) 30 capable of displaying images captured by the cameras 10, and a server 40. In addition to the above, the work site monitoring system 100 of this embodiment also includes a virtual space information storage unit 20 and a GNSS 50 provided in a monitoring target 1, which will be described later.

[0019] (Monitored) The monitored object 1 (see FIGS. 3 and 4) in the work site monitoring system 100 is not particularly limited, but examples include moving objects at the work site such as heavy machinery, transport vehicles, workers, and loads being lifted during lifting work, or parts of these moving objects. The monitored object 1 is not limited to one, and various types of moving objects that can be monitored may be stored (registered) in advance in the server 40 as the monitored object 1. Note that, although the present embodiment will be described using a hydraulic excavator (heavy machinery) as the monitored object 1, the monitored object is not limited to this.

[0020] (GNSS50) A GNSS (Global Navigation Satellite System) 50 is provided in the monitoring target 1 and detects the position of the monitoring target 1. Specifically, the GNSS 50 acquires three-dimensional position information (latitude, longitude, altitude, etc.) of the monitoring target 1. The three-dimensional position information of the monitoring target 1 acquired by the GNSS 50 is transmitted to the server 40. Note that in this embodiment, the position of the monitoring target 1 is detected using the GNSS 50, but this is not limiting, and for example, the position of the monitoring target 1 may be detected using a GPS (Global Positioning System).

[0021] Furthermore, in this embodiment, a plurality of GNSSs 50 (for example, a pair on the left and right) are provided at mutually different positions with respect to the monitoring target 1. This makes it possible to detect the orientation of the monitoring target 1 based on three-dimensional position information acquired by the plurality of GNSSs 50. Note that, in this embodiment, the orientation of the monitoring target 1 is detected using the plurality of GNSSs 50, but this is not limited thereto, and for example, the orientation of the monitoring target 1 may be detected using a plurality of GPS. Also, in this embodiment, a plurality of GNSSs 50 are provided on the monitoring target 1 to detect the orientation of the monitoring target 1, but this is not limited thereto, and a direction sensor or the like capable of detecting the orientation of the monitoring target 1 may be provided on the monitoring target 1.

[0022] (camera) The cameras 10 are cameras (e.g., wide-angle cameras) arranged at multiple locations on the work site and capable of capturing images of a predetermined area of ​​the work site. The locations at which the multiple cameras 10 are arranged are not particularly limited, and the cameras may be arranged at relatively high or low positions. The multiple cameras 10 may also include a camera 10 fixed to the tip of a crane jib, for example. Three-dimensional position information representing the positions (arrangement positions) of the cameras 10 may be included in virtual space information (described later) stored in the virtual space information storage unit 20. The imaging ranges (angles of view) of the cameras 10 may also be included in the virtual space information. The cameras 10 may be PTZ (Pan-Tilt-Zoom) cameras whose orientation can be controlled, or may be fixed cameras whose orientation cannot be controlled. Images captured by the cameras 10 are transmitted to the server 40 in real time. While the present embodiment describes the cameras 10 as PTZ cameras, the present invention is not limited to this.

[0023] (Virtual space information storage unit) The virtual space information storage unit 20 stores virtual space information, which is information (three-dimensional information) for reproducing a work site in a virtual space VS (see FIG. 3). The virtual space information is three-dimensional information for reproducing a three-dimensional model of a structure such as a building at the work site in the virtual space VS. The virtual space information may be, for example, point cloud data of the work site, a digital twin generated based on the point cloud data of the work site, or a BIM (Building Information Modeling) model or a CIM (Construction Information Modeling) model (hereinafter referred to as a "BIM / CIM model") in which attribute information is added to the digital twin of the work site. Point cloud data is, for example, data indicating a set of points associated with each of three-dimensional coordinates. A digital twin is, for example, data in which a copy of a physical space is reproduced in a digital space (virtual space) based on information acquired from the physical space. A BIM / CIM model is a three-dimensional model of a building or construction site.

[0024] The virtual space information storage unit 20 may be included in a storage unit H4 of the server 40, which will be described later, or may be a database server separate from the server 40. If it is a database server, the database server may be located on a local network, or may be a cloud server that stores data via the Internet.

[0025] (monitor) The monitor 30 is a display device capable of displaying images captured by the camera 10, and is installed, for example, in a site office or the like that manages the safety of the work site. A supervisor or the like who monitors (manages) the safety of the work site can monitor the safety of the work site by checking the images displayed on the monitor 30 in the site office or the like. The number of monitors 30 to be installed is not particularly limited, but the fewer the number, the better, from the viewpoints of reducing the burden on the supervisor, reducing costs, and saving space. Note that, although one monitor 30 is illustrated in FIG. 1, the present invention is not limited to this, and multiple monitors 30 may be provided.

[0026] (server) The server 40 determines which image to display on the monitor 30 from among images captured by the multiple cameras 10. The server 40 includes a monitoring target position identification unit 41, a candidate selection unit 42, and a determination unit 43. In addition to the above, the server 40 of this embodiment also includes a camera control unit 44. The server 40 may be placed in a field office together with the monitor 30, or may be placed in a location different from the monitor 30. Although the camera control unit 44 is provided in this embodiment, the camera control unit 44 need not be provided.

[0027] (Hardware configuration) 2, the server 40 is a device that performs calculations such as a computer, and includes a communication unit H1, an input unit H2, a display unit H3, a storage unit H4, and a processor H5, all of which are connected so as to be accessible via a bus H6. Note that this hardware configuration is an example, and it can also be realized by other hardware.

[0028] The communication unit H1 is a communication interface that establishes a communication route with other devices and transmits and receives data, and is, for example, a network interface or a wireless interface. The communication unit H1 of the server 40 in this embodiment is connected to a communication device (not shown) on the GNSS 50 side of the monitored object 1 and a communication device (not shown) on the camera 10 side via a network so that they can communicate with each other. The network may be a closed network or an open network.

[0029] The input unit H2 is a device that accepts input of various information, such as a mouse, a keyboard, etc. The input unit H2 also accepts input of location information acquired by the GNSS 50, images captured by the camera 10, etc. via the communication unit H1.

[0030] The display unit H3 is a display unit such as a monitor (display) that displays various information. When the server 40 is placed in the same position as the monitor 30, the display unit H3 of the server 40 may be the monitor 30, and when the server 40 is placed in a different position from the monitor 30, the display unit H3 of the server 40 may be a device different from the monitor 30. Note that when the display unit such as a display is a touch panel display unit that can handle input, the display unit may have a function as the input unit H2 in addition to the function as the display unit H3.

[0031] The storage unit H4 is a storage device that stores data and various programs for executing various functions of the server 40. Examples of the storage unit H4 include a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk.

[0032] The processor H5 controls each process in the server 40 using programs and data stored in the storage unit H4. Examples of the processor H5 include a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The processor H5 loads programs stored in a ROM or the like into a RAM and executes various processes for each process. Specifically, the processor H5 functions as a monitoring target position identification unit 41, a candidate selection unit 42, a determination unit 43, and a camera control unit 44 by executing programs stored in the storage unit H4. Note that some of the functions of the processor H5 may be extracted and provided in another information processing device, and each process may be executed using multiple information processing devices.

[0033] (Monitoring target location identification unit) The monitoring target position specifying unit 41 executes processing to specify the position (hereinafter referred to as "monitoring target position") of the monitoring target 1. The monitoring target position is three-dimensional position information that can specify the location where the monitoring target 1 exists.

[0034] In this embodiment, the monitoring target position identifying unit 41 identifies the monitoring target position based on the position of the monitoring target 1 detected by the GNSS 50. For example, the monitoring target position identifying unit 41 may directly identify three-dimensional position information detected by a predetermined GNSS 50 as the monitoring target position. Alternatively, the monitoring target position identifying unit 41 may store in advance the positional relationship between the arrangement position of the GNSS 50 on the monitoring target 1 and a predetermined position of the monitoring target 1 (such as the center position of the heavy equipment), calculate three-dimensional position information of the predetermined position of the monitoring target 1 based on the three-dimensional position information detected by the GNSS 50, and identify the calculated three-dimensional position information as the monitoring target position.

[0035] (Candidate selection section) The candidate selection unit 42 executes a process of selecting, from the plurality of cameras 10, a candidate image capture unit that will be a candidate for displaying video on the monitor 30, based on predetermined candidate conditions. That is, the candidate image capture unit is one or more cameras 10 selected from the plurality of cameras 10 by the candidate selection unit 42. The candidate conditions used by the candidate selection unit 42 to select a candidate image capture unit include at least a first candidate condition, and may additionally include a second candidate condition or other candidate conditions.

[0036] The candidate selection unit 42 of this embodiment executes a process of selecting a candidate shooting unit in a virtual space VS that recreates a work site based on the virtual space information stored in the virtual space information storage unit 20. That is, the candidate selection unit 42 of this embodiment selects a candidate shooting unit by using the virtual space VS without performing image analysis or the like of the image from the camera 10. Note that the candidate selection unit 42 may select a candidate shooting unit by performing image analysis or the like of the image from the camera 10 without using the virtual space VS.

[0037] (First candidate condition) The first candidate condition is a condition that no obstruction is placed between the monitoring target position identified by the monitoring target position identifying unit 41 and the position of the camera 10. The candidate selecting unit 42 selects, from among the multiple cameras 10, a camera 10 that satisfies at least the first candidate condition as a candidate imaging unit. An obstruction is something that blocks the monitoring target 1 so that at least a portion of the monitoring target 1 is not captured in the image captured by the camera 10. Examples of obstructions include structures such as buildings at the work site, heavy machinery other than the monitoring target 1, and materials and equipment temporarily placed at the work site. The determination of the first candidate condition may be performed using a virtual space VS, or may be performed using image analysis of the image captured by the camera 10. In this embodiment, a case where the virtual space VS is used will be described.

[0038] FIG. 3 is an explanatory diagram showing an example of a virtual space.

[0039] For example, in an example of a three-dimensional model of a work site (hereinafter referred to as the "work site model") reproduced in a virtual space VS shown in FIG. 3, multiple (four in the figure) structures St1 to St4 are arranged. The work site model also reflects the positions of multiple cameras 10 (six cameras 10a to 10f in the figure), making it possible to identify the position of each camera 10. The work site model may also reflect the shooting range (angle of view) of each camera 10. Note that in FIG. 3, for ease of understanding, multiple cameras 10 are displayed (illustrated) as three-dimensional models in the virtual space VS, but this is not limiting. As long as information regarding the three-dimensional position information and shooting range of the cameras 10 is reflected, the cameras 10 do not necessarily have to be displayed as three-dimensional models.

[0040] In the case of the work site model of the virtual space VS shown in Fig. 3, the candidate selection unit 42 selects, as a candidate image capture unit, a camera 10 for which no three-dimensional models of structures St1 to St4, such as buildings, that serve as obstructions are located between the monitoring target position P identified by the monitoring target position identification unit 41 and the positions of the cameras 10a to 10f that are to be placed at the work site. For example, as shown in Fig. 3, the candidate selection unit 42 may select, as a candidate image capture unit, a camera 10 for which, when a straight line is drawn between the monitoring target position P and the positions of the cameras 10a to 10f, the line does not pass through (is not obstructed by) the structures St1 to St4. In Fig. 3, of the six cameras 10a to 10f, two cameras 10c and 10f have structures St3 and St4 located between them and the monitoring target position P, while four cameras 10a, 10b, 10d, and 10e do not have structures St1 to St4 located between them and the monitoring target position P. Therefore, in this case, the candidate selection unit 42 selects four cameras 10a, 10b, 10d, and 10e out of the six cameras 10a to 10f as candidate imaging units. Note that in Fig. 3, of the straight lines connecting the monitoring target position P and the locations of the cameras 10a to 10f, the portions that pass through the structures St1 to St4 are indicated by dashed lines.

[0041] (Second candidate condition) The second candidate condition is a condition that the distance between the monitoring target position identified by the monitoring target position identifying unit 41 and the position of the camera 10 is shorter than a predetermined distance. When the candidate conditions include the first candidate condition and the second candidate condition, the candidate selecting unit 42 selects, from among the multiple cameras 10, a camera 10 that satisfies both the first candidate condition and the second candidate condition as a candidate imaging unit. The determination of the second candidate condition may be performed using a virtual space VS, or may be performed by calculation using pre-stored position information of the camera 10. In this embodiment, a case where the virtual space VS is used will be described. Note that the second candidate condition may include a condition that the distance between the monitoring target position and the position of the camera 10 is longer than a second distance, in addition to the condition that the distance is shorter than a first distance (a predetermined distance).

[0042] For example, the candidate selection unit 42 may select, as candidate image capture units, from among the multiple cameras 10 in the virtual space VS shown in Fig. 3, cameras 10 that satisfy the first candidate condition and are located inside a spherical surface S that has the monitoring target position P as its center and the predetermined distance as its radius. In Fig. 3, of the six cameras 10a to 10f, five cameras 10a, 10b, 10c, 10e, and 10f, excluding one camera 10d, are located inside the spherical surface S that has the predetermined distance as its radius. Therefore, in this case, the candidate selection unit 42 of this embodiment selects, as candidate image capture units, three cameras 10a, 10b, and 10e that satisfy both the first and second candidate conditions from among the six cameras 10a to 10f.

[0043] The candidate conditions used by the candidate selection unit 42 to select a candidate image capture unit from among the multiple cameras 10 are not limited to the first and second candidate conditions, and may include other candidate conditions. For example, the candidate condition may include a condition that the imaging range (angle of view) of the camera 10 in the virtual space VS includes the monitored position P. In this case, in FIG. 3, the two cameras 10a and 10c are facing away from the monitored position P, and the monitored position P is outside the imaging range of the camera 10, so they will not be selected as candidate image capture units.

[0044] (Decision section) The determination unit 43 determines a display target imaging unit, which is a camera 10 whose image will be displayed on the monitor 30, from among the candidate imaging units selected by the candidate selection unit 42, and executes a process of displaying the image captured by the display target imaging unit on the monitor 30. Specifically, the determination unit 43 executes a process of outputting the image captured by the determined display target imaging unit to the monitor 30. The display target imaging unit is one or more cameras 10 determined from among the candidate imaging units by the determination unit 43. The display target imaging unit determined by the determination unit 43 is preferably one camera 10, but may be multiple cameras 10. Note that the determination unit 43 may determine all cameras 10 included in the candidate imaging units selected by the candidate selection unit 42 as display target imaging units. In other words, the determination unit 43 may cause images of all candidate imaging units selected by the candidate selection unit 42 to be displayed on the monitor 30.

[0045] The determination unit 43 preferably determines the imaging units to be displayed based on predetermined determination conditions from among the candidate imaging units selected by the candidate selection unit 42. This makes it possible to further narrow down the number of candidate imaging units selected by the candidate selection unit 42 and determine the imaging units to be displayed. Below, three determination conditions (first determination condition, second determination condition, and third determination condition) will be described as the predetermined determination conditions, but other determination conditions different from these may also be included.

[0046] (First decision condition) The first determination condition is that the video captured by the candidate image capture unit includes the surveillance target 1. For example, the determination unit 43 can use a known image analysis technique to determine whether the video captured by the candidate image capture unit includes the surveillance target 1. For example, the determination unit 43 may make the determination by comparing frames of the video data captured by the candidate image capture unit (hereinafter referred to as "video data") with pre-stored images of the surveillance target 1, or may use a learning model that outputs information on whether the surveillance target 1 is included in response to input video data. If it is determined that the surveillance target 1 is included in the video data, the first determination condition is met. Note that "image analysis" includes "image recognition," which extracts features of objects captured in the video, and "image analysis," which analyzes the extracted features to determine the identity of the object.

[0047] FIG. 4 is an explanatory diagram showing an example of an image captured by a camera.

[0048] 4 includes a heavy machine, which is the monitored object 1. Therefore, the camera 10 capturing the image X as shown in FIG. 4 is included in the display target capturing section determined by the determining unit 43.

[0049] 3 selected as candidate image capturing units by the candidate selection unit 42 described above, the image captured by the camera 10a does not capture the monitoring target 1, but the images captured by the other two cameras 10b and 10e do capture the monitoring target 1. In this case, the cameras 10 that satisfy the first determination condition are the two cameras 10b and 10e among the candidate image capturing units, and therefore the determination unit 43 determines the two cameras 10b and 10e as the image capturing units to be displayed.

[0050] Since it is preferable that the number of cameras 10 that can be the display target imaging units be small, when there are multiple cameras 10 that satisfy the first determination condition, the determination unit 43 may determine, as the display target imaging unit, a candidate imaging unit that satisfies both the first determination condition and other determination conditions. Examples of other determination conditions include a condition regarding the size of the monitoring target 1 displayed in the video (the size of the frame indicated by the two-dot chain line 1a in FIG. 4) and a condition regarding the position of the monitoring target 1 in the video. When determining these conditions, the zoom performance and swivel performance of the camera 10 may be taken into consideration. In addition, the other determination conditions may be the second determination condition and the third determination condition described below.

[0051] (Second decision condition) The second determination condition is a condition that the orientation of the monitoring target 1 satisfies a preset orientation condition. When the determination unit 43 includes the first determination condition and the second determination condition as determination conditions when determining the display target imaging unit, the determination unit 43 determines the camera 10 that satisfies both the first determination condition and the second determination condition from among the candidate imaging units as the display target imaging unit.

[0052] The preset orientation condition (hereinafter simply referred to as "orientation condition") in the second determination condition may be an orientation condition of the monitored object 1 relative to the camera 10, and may be, for example, a condition that a point of interest of the monitored object 1 (for example, the front or rear of the monitored object 1) faces the camera 10. If the monitored object 1 is heavy machinery, the point of interest may be a direction that is a blind spot for an operator (operator) operating the heavy machinery.

[0053] The determination unit 43 of this embodiment determines whether or not the orientation of the monitoring target 1 satisfies a preset orientation condition in the virtual space VS, based on the orientation of the monitoring target 1 detected by the multiple GNSSs 50. Note that the determination of whether or not the orientation condition is satisfied is not limited to a determination using information from the multiple GNSSs 50, and may be a determination using, for example, a known image analysis technique or a learning model.

[0054] For example, if the orientation condition in the second determination condition is that the front of the monitoring target 1 faces the camera 10, then of the two cameras 10b and 10e that satisfy the first determination condition shown in Fig. 3, the camera 10 that satisfies the second determination condition is the camera 10e that is capturing the video X as shown in Fig. 4. Therefore, the determination unit 43 determines the camera 10e that is capturing such video X as the display target capturing unit.

[0055] (Third decision condition) The third determination condition (determination condition) is a condition that a predetermined portion of interest is included in the video captured by the candidate imaging unit. When the determination conditions include the first determination condition and the third determination condition, the determination unit 43 determines, from among the candidate imaging units, the camera 10 that satisfies both the first determination condition and the third determination condition as the imaging unit to be displayed.

[0056] The predetermined attention portion in the third determination condition (hereinafter simply referred to as "attention portion") may be a part of the monitored object 1, or may be an object different from the monitored object 1. The attention portion is not particularly limited, but examples include a bucket when the monitored object 1 is a hydraulic excavator (heavy machinery), a hook when the monitored object 1 is a crane truck (heavy machinery), or a worker working around the monitored object 1. The method for detecting the attention portion in the video data is not particularly limited, but can be performed using known image analysis techniques or learning models.

[0057] For example, if the part of interest in the third determination condition is the bucket 1b of the hydraulic excavator, which is the monitoring target 1, then the bucket 1b of the hydraulic excavator appears in the image X of the camera 10e as shown in Fig. 4, and therefore the camera 10e capturing such image X will be included in the captured part to be displayed determined by the determination unit 43. Alternatively, if the part of interest in the third determination condition is workers around the monitoring target 1, then the image X of the camera 10e will show workers m1 and m2 around the hydraulic excavator, which is the monitoring target 1, as shown in Fig. 4, and therefore the camera 10e capturing such image X will be included in the captured part to be displayed determined by the determination unit 43.

[0058] (Camera control unit) The camera control unit 44 may execute a process to control the zoom and panning of the camera 10 determined as the display target imaging unit by the determination unit 43. For example, when the monitoring target 1 is moving, the camera control unit 44 may control the panning of the camera 10 so that the monitoring target 1 appears approximately in the center of the image output to the monitor 30. The camera control unit 44 may also control the zoom of the camera 10 so that the size of the monitoring target 1 in the image output to the monitor 30 becomes a predetermined size.

[0059] (Worksite monitoring processing) FIG. 5 is a flowchart of the work site monitoring process.

[0060] Next, an example of the work site monitoring process executed by the monitoring target position specifying unit 41, the candidate selecting unit 42, and the deciding unit 43 will be described with reference to Fig. 5. The work site monitoring process is repeatedly executed from when the work site monitoring system 100 is started until it is stopped.

[0061] 5, in the work site monitoring process, first, the monitoring target position specifying unit 41 specifies the position of the monitoring target 1 (step S1). In this embodiment, the monitoring target position specifying unit 41 specifies the position of the monitoring target 1 based on information from the GNSS 50.

[0062] Next, the candidate selection unit 42 selects a candidate image capturing unit from among the multiple cameras 10 (step S2). In this embodiment, the candidate selection unit 42 selects (extracts) as a candidate image capturing unit a camera 10 that satisfies a first candidate condition that a 3D model of a structure such as a building at the work site that acts as an obstruction is not located between the monitoring target position P identified by the monitoring target position identification unit 41 and the position of the camera 10 in the virtual space VS. Note that the candidate selection unit 42 may determine other candidate conditions (including a second candidate condition) in addition to the first candidate condition.

[0063] Next, the determination unit 43 determines the imaging units to be displayed from among the candidate imaging units selected by the candidate selection unit 42 (step S3). Note that the determination unit 43 may determine all the candidate imaging units selected by the candidate selection unit 42 as imaging units to be displayed, or may determine the imaging units to be displayed based on the above-mentioned predetermined determination condition (first determination condition, etc.).

[0064] Next, the determination unit 43 outputs the video captured by the display target capturing unit to the monitor 30 for display (step S4).

[0065] Finally, the camera control unit 44 controls the zoom and pan of the camera 10 determined by the determination unit 43 as the display target photographing unit so as to track the monitoring target 1 (step S5). Note that the control of the camera 10 by the camera control unit 44 (step S5) does not necessarily have to be executed.

[0066] In the worksite monitoring system 100 configured as described above, the candidate selection unit 42 selects, from the multiple cameras 10, a candidate image capture unit that will be a candidate for displaying video on the monitor 30, based on predetermined candidate conditions. The candidate conditions used by the candidate selection unit 42 to select a candidate image capture unit include a first candidate condition that no obstructions are located between the position of the monitoring target 1 and the position of the camera 10. Therefore, the video displayed on the monitor 30 does not include at least video that does not show the monitoring target 1 because it is obstructed by obstructions such as structures on the worksite. This means that when checking the video displayed on the monitor 30, the monitor does not have to check video that does not show the monitoring target 1 because it is obstructed by obstructions, thereby reducing the burden on the monitor.

[0067] Furthermore, the candidate selection unit 42 selects candidate photographing units so as to eliminate images that do not show the monitored object 1 due to obstructions, so that the number of images displayed on the monitor 30 can be reduced, thereby reducing the burden on the monitor.

[0068] Furthermore, since the number of images (number of cameras 10) displayed on the monitor 30 can be reduced, even when one monitor 30 screen is divided into multiple screens, the size of each divided screen can be ensured to be large, thereby reducing the burden on the observer.

[0069] In addition, since the number of images (number of cameras 10) displayed on the monitors 30 can be reduced, the number of monitors 30 installed in the field office, etc. can be reduced, thereby reducing costs and the installation space for the monitors 30.

[0070] As described above, according to this embodiment, it is possible to provide a work site monitoring system 100 that can reduce the burden on the monitor even when multiple cameras (photographing units) 10 are provided.

[0071] Furthermore, the candidate conditions used by the candidate selection unit 42 to select a candidate imaging unit include a second candidate condition that the distance between the position of the monitoring target 1 and the position of the camera 10 is shorter than a predetermined distance. This makes it possible to eliminate in advance images in which the monitoring target 1 is captured small because the camera 10 is too far from the monitoring target 1, thereby increasing the proportion of images in which the monitoring target 1 is captured relatively large on the monitor 30 and further reducing the burden on the monitor.

[0072] Furthermore, since the monitoring target position specifying unit 41 specifies the position of the monitoring target based on the position of the monitoring target 1 detected by the GNSS 50, the accurate position of the monitoring target 1 can be easily specified.

[0073] Furthermore, since the determination unit 43 determines the image capture unit to be displayed from among the candidate image capture units based on predetermined determination conditions, when multiple cameras 10 are selected as candidate image capture units, the number of cameras 10 can be further reduced based on the above-mentioned predetermined determination conditions. This makes it possible to reduce the number of images (the number of cameras 10) displayed on the monitor 30, further reducing the burden on the observer.

[0074] Furthermore, if the first determination condition is included in the determination conditions when the determination unit 43 determines the shooting area to be displayed, the image displayed on the monitor 30 will always include the monitored object 1, thereby reducing the burden on the monitor.

[0075] Furthermore, when the candidate selection unit 42 selects candidate image capture units using the virtual space VS, the processing load of the image analysis process on the server 40 can be reduced.

[0076] Furthermore, if the second determination condition is included in the determination conditions when the determination unit 43 determines the image capturing unit to be displayed, the image displayed on the monitor 30 shows the monitored object 1 from the desired viewing direction, further reducing the burden on the monitor. Also, by appropriately setting the orientation condition, the safety of the work can be ensured.

[0077] Furthermore, since the GNSS 50 that detects the orientation of the monitoring target 1 is provided, the determining unit 43 can easily determine the second determination condition based on the orientation of the monitoring target 1 without performing image analysis processing.

[0078] Furthermore, if the determination condition when the determination unit 43 determines the photographed area to be displayed includes the third determination condition, the image displayed on the monitor 30 will show the area of ​​interest within the monitored object 1 or the area of ​​interest around the monitored object 1, further reducing the burden on the monitor. Also, by appropriately setting the area of ​​interest, the safety of the work can be ensured.

[0079] Second Embodiment Next, a second embodiment of the present invention will be described with reference to the drawings. The work site monitoring system 200 of this embodiment differs from the first embodiment in the processing of a monitoring target position specifying unit 41A, a candidate selecting unit 42A, and a determining unit 43A. Note that the same components as those of the first embodiment are denoted by the same reference numerals, and their description will be omitted.

[0080] FIG. 6 is a block diagram showing the configuration of a work site monitoring system according to the second embodiment of the present invention.

[0081] (Workplace monitoring system) As shown in Fig. 6, a work site monitoring system 200 according to a second embodiment of the present invention is a system for monitoring the safety of a work site, and includes an environment sensor (environment detection unit) 60, a plurality of cameras (imaging units) 10, a monitor (display unit) 30 capable of displaying images captured by the cameras 10, and a server 40A. In addition to the above, the work site monitoring system 200 of this embodiment also includes a virtual space information storage unit 20 and a direction sensor 70 provided on the monitoring target 1. Note that the direction sensor 70 does not necessarily have to be provided.

[0082] (environmental sensor) The environmental sensor 60 is placed at the work site and detects the environmental values ​​of the work site. Multiple environmental sensors 60 may be placed at the work site. The environmental values ​​of the work site are not particularly limited, but examples include noise values, vibration values, and temperature at the work site. That is, the environmental sensor 60 may be a noise sensor, a vibration sensor, a temperature sensor, etc. The environmental sensor 60 detects environmental values ​​in the vicinity of the location where the work site is installed. The environmental values ​​of the work site detected by the environmental sensor 60 are transmitted to the server 40A.

[0083] (orientation sensor) The direction sensor 70 is provided on the monitoring target 1 and detects the direction, thereby detecting the orientation of the monitoring target 1. The orientation of the monitoring target 1 detected by the direction sensor 70 is transmitted to the server 40.

[0084] (server) The server 40A determines an image to be displayed on the monitor 30 from among images captured by the multiple cameras 10. The server 40A includes a monitoring target position identification unit 41A, a candidate selection unit 42A, and a determination unit 43A. In addition to the above, the server 40A of this embodiment also includes a camera control unit 44. Although the camera control unit 44 is provided in this embodiment, the camera control unit 44 does not necessarily have to be provided. The hardware configuration of the server 40A of this embodiment may be the same as the hardware configuration of the server 40 of the first embodiment.

[0085] (Monitoring target location identification unit) The monitoring target position specifying unit 41A executes a process for specifying the position of the monitoring target 1 (monitoring target position).

[0086] In this embodiment, the monitoring target position specifying unit 41A specifies the position of the monitoring target based on the environmental value detected by the environmental sensor 60 and the video captured by the camera 10. When the environmental value detected by the environmental sensor 60 falls outside a predetermined range (a range in which at least one of an upper limit value and a lower limit value is specified), the monitoring target position specifying unit 41A starts processing to specify the position of the monitoring target.

[0087] When the environmental value detected by the environmental sensor 60 falls outside the predetermined range (abnormal value), the monitoring target position identifying unit 41A first identifies the area where the environmental value is detected by the environmental sensor 60. The method for identifying the area where the environmental value is detected by the environmental sensor 60 (hereinafter referred to as the "detection area") is not particularly limited, but may be stored in advance in the server 40A, for example.

[0088] Next, the monitoring target position identification unit 41A extracts cameras 10 whose shooting range includes the detection area from among the multiple cameras 10. The number of cameras 10 to be extracted is not limited to one, and multiple cameras 10 may be extracted. The method for extracting cameras 10 whose shooting range includes the detection area is not particularly limited, but may be, for example, extraction using a table or the like in which the positions of cameras 10 and detection areas are previously associated and stored, or extraction using a virtual space VS.

[0089] Next, the monitoring target position identifying unit 41A determines whether or not the monitoring target 1 is included in the video captured by the extracted camera 10. This determination may be made, for example, using a known image analysis technique or a learning model. Then, if the monitoring target 1 is included in the video captured by the camera 10, the monitoring target position identifying unit 41A identifies the position of the monitoring target 1 included in the video. The method for identifying the position of the monitoring target 1 included in the video is not particularly limited, but may be, for example, identified using a virtual space VS based on the orientation and shooting range of the camera 10, and the size and position of the monitoring target 1 in the video.

[0090] (Candidate selection section) The candidate selection unit 42A executes a process of selecting, based on predetermined candidate conditions, a candidate imaging unit from among the multiple cameras 10 that will be a candidate for displaying images on the monitor 30. The candidate conditions include at least the first candidate condition, and may also include the second candidate condition and other candidate conditions.

[0091] The candidate selection unit 42A of this embodiment executes a process of selecting a candidate shooting unit in a virtual space VS that recreates a work site based on the virtual space information stored in the virtual space information storage unit 20. That is, the candidate selection unit 42 of this embodiment selects a candidate shooting unit by using the virtual space VS without performing image analysis or the like of the image from the camera 10. Note that the candidate selection unit 42 may select a candidate shooting unit by performing image analysis or the like of the image from the camera 10 without using the virtual space VS.

[0092] The candidate selection unit 42A of this embodiment may select a candidate imaging unit that will be a candidate for displaying images on the monitor 30 from among the cameras 10 extracted in the processing executed by the monitoring target position identification unit 41A (cameras 10 whose imaging range includes a detection area where the environmental value is outside the above-mentioned specified range) based on the above-mentioned specified candidate conditions.

[0093] (Decision section) The determination unit 43A determines a display target imaging unit, which is a camera 10 whose image will be displayed on the monitor 30, from among the candidate imaging units selected by the candidate selection unit 42A, and executes a process of displaying the image captured by the display target imaging unit on the monitor 30. Specifically, the determination unit 43A executes a process of outputting the image captured by the determined display target imaging unit to the monitor 30. The display target imaging unit determined by the determination unit 43A is preferably one camera 10, but may be multiple cameras 10. Note that the determination unit 43A may determine all cameras 10 included in the candidate imaging units selected by the candidate selection unit 42A as display target imaging units. In other words, the determination unit 43A may cause the monitor 30 to display images of all the candidate imaging units selected by the candidate selection unit 42A.

[0094] It is preferable that the determination unit 43A determines the image capture unit to be displayed from among the candidate image capture units selected by the candidate selection unit 42A based on predetermined determination conditions (the above-mentioned first determination condition, the above-mentioned second determination condition, the above-mentioned third determination condition, etc.). As described above, when the candidate selection unit 42A selects the candidate image capture units from among the cameras 10 whose image capture range includes the detection area of ​​the environmental value, the selected candidate image capture unit is a camera 10 whose image capture range includes the detection area where the monitoring target 1 is located and no obstruction is located between the camera 10 and the monitoring target position. Therefore, since the image of the camera 10 selected as the candidate image capture unit is likely to include the monitoring target 1, the determination conditions used by the determination unit 43A to determine the image capture unit to be displayed do not need to include the above-mentioned first determination condition.

[0095] According to the work site monitoring system 200 configured as above, like the work site monitoring system 100 of the first embodiment, even if a plurality of cameras (photographing units) 10 are provided, the burden on the monitor can be reduced.

[0096] Furthermore, when the environmental value detected by the environmental sensor 60 indicates an abnormal value, the monitoring target 1 in the detection area of ​​the environmental sensor 60 can be monitored intensively, thereby ensuring safety.

[0097] Although the present invention has been described above based on the above embodiment, the present invention is not limited to the content of the above embodiment, and can be modified as appropriate without departing from the scope of the present invention. In other words, all other embodiments, examples, operational techniques, etc. made by those skilled in the art based on this embodiment are naturally included in the scope of the present invention. [Explanation of symbols]

[0098] 10: Camera (photography section) 20: Virtual space information storage unit 30: Monitor (display) 40, 40A: Server 41, 41A: Monitoring target position identification unit 42, 42A: Candidate selection section 43,43A:Decision section 60: Environmental sensor (environment detection unit) 100,200: Workplace monitoring system

Claims

1. A plurality of imaging units disposed at the work site; a display unit capable of displaying an image captured by the imaging unit; a monitoring target position specifying unit that specifies the position of a monitoring target; a candidate selection unit that selects a candidate image capture unit that will display an image on the display unit from among the plurality of image capture units based on predetermined candidate conditions; a determination unit that determines a display target imaging unit that will display an image on the display unit from among the candidate imaging units selected by the candidate selection unit, and causes the image captured by the display target imaging unit to be displayed on the display unit; The candidate conditions include a first candidate condition that no obstruction is disposed between the position of the monitoring target identified by the monitoring target position identifying unit and the position of the imaging unit. A work site monitoring system characterized by:

2. the candidate conditions include a second candidate condition that the distance between the position of the monitoring target identified by the monitoring target position identifying unit and the position of the imaging unit is shorter than a predetermined distance; The candidate selection unit selects the imaging unit that satisfies both the first candidate condition and the second candidate condition as the candidate imaging unit.

2. The work site monitoring system according to claim 1.

3. a virtual space information storage unit that stores virtual space information for reproducing the work site in a virtual space; The candidate selection unit determines whether the first candidate condition is satisfied based on the image acquired by the image capturing unit or the virtual space information.

3. The work site monitoring system according to claim 1 or 2.

4. the determination unit determines the display target imaging unit from among the candidate imaging units based on a predetermined determination condition; The determination condition includes a determination condition that a predetermined part of interest is included in the image captured by the candidate image capturing unit.

3. The work site monitoring system according to claim 1 or 2.

5. an environment detection unit disposed at the work site to detect an environmental value of the work site; The monitoring target position specifying unit specifies the position of the monitoring target based on the environmental value detected by the environmental detection unit and the video captured by the imaging unit.

3. The work site monitoring system according to claim 1 or 2.

Images