Safety management system, safety management method, and safety management program

Abstract

To avoid the danger of an accident such as a collision by confirming the safety of a dead angle in a working site or the like by using the image of a virtual space.SOLUTION: Using environmental information acquired by an environmental sensor installed at a predetermined position in an area to be monitored, an object in the area is recognized, a position of the object in the area is acquired, a three dimensional model of the object is generated in a virtual space corresponding to the area, and an image of the virtual space is generated based on a viewpoint of the object.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a safety management system, a safety management method, and a safety management program. [Background technology]

[0002] Digital twin technology is attracting attention as a method for virtualizing physical objects in a digital space and monitoring and managing them in real time. Traditional safety management systems at construction sites have relied primarily on physical monitoring using cameras and sensors. These systems are capable of continuously monitoring specific areas and have been used to prevent accidents from occurring. In particular, systems using video cameras and motion detection sensors have increased safety at construction sites by instantly detecting abnormalities and issuing alarms.

[0003] In recent years, with advances in technology, these monitoring systems have become even more sophisticated. For example, image recognition systems using AI technology and sensor systems utilizing IoT (Internet of Things) technology are able to detect and respond to abnormalities more accurately and quickly. Furthermore, by linking these systems with cloud technology, it is possible to monitor from remote locations and analyze data in real time, improving the safety of construction sites. Furthermore, the introduction of digital twin technology has made it possible to recreate physical objects in virtual space and perform simulations, making it possible to detect and address potential risks in advance.

[0004] The monitoring system 1 described in Patent Document 1 is a system that monitors an object 2 located in a monitoring area R using a monitoring terminal 10. This system has an acquisition unit 31 that acquires virtualization information related to the object 2 from at least one of an environmental sensor 3 and a communication terminal 4. It also has a generation unit 32 that generates a virtual space of the monitoring area R including static objects that are virtualized static objects and dynamic objects that are virtualized dynamic objects based on the acquired information. The system also has a reception unit 33 that receives input of an arbitrary viewpoint position in the virtual space V from the monitoring terminal 10, and an output unit 34 that outputs an image of the virtual space V seen from the viewpoint position to the monitoring terminal 10, thereby providing a system that can easily and appropriately monitor objects in the monitoring area. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2024-025909 Summary of the Invention [Problem to be solved by the invention]

[0006] At work sites such as factories and warehouses, there are many blind spots and poor visibility, which poses a risk of collisions between work vehicles and workers. In particular, when transporting goods, blind spots are created ahead due to the load, so a system to support safety management at work sites is required.

[0007] In order to solve the above-mentioned problems, an object of the present invention is to provide a technology for improving safety at work sites and the like by using an image of a virtual space corresponding to a monitored area. [Means for solving the problem]

[0008] [1] A safety management system comprising: an object recognition unit that recognizes objects within a monitored area using environmental information acquired by an environmental sensor installed at a predetermined position in the area; a model generation unit that acquires the position of the object within the area and generates a three-dimensional model of the object in a virtual space corresponding to the area; and an image generation unit that generates an image of the virtual space based on the viewpoint of the object. [2] The safety management system described in [1], further comprising a viewpoint determination unit that acquires the position and orientation of the object and determines a viewpoint of a three-dimensional model corresponding to the object. [3] A safety management system described in [1] or [2], wherein the environmental sensor is an imaging device, and the object recognition unit recognizes objects within the area using an image captured by the imaging device. [4] The safety management system described in [3], wherein the object recognition unit recognizes feature points of the object in the captured image, and the model generation unit determines the orientation of the three-dimensional model based on the feature points and generates a three-dimensional model of the object in the virtual space. [5] A safety management system described in any of [1] to [4], wherein the object recognition unit distinguishes and recognizes moving objects within the area, and the image generation unit generates an image of the virtual space based on the viewpoint of a three-dimensional model of the moving object. [6] A safety management system as described in [5], further comprising a prediction unit that predicts an intersection area based on the respective movement paths of a first moving object and a second moving object, wherein the image generation unit generates an image including a notification of approach to the intersection area. [7] A safety management system described in any of [1] to [6], wherein the object recognition unit distinguishes and recognizes stationary objects within the area, and the image generation unit generates an image of a three-dimensional model of the stationary object with a predetermined transmittance.

[0009] The invention of [1] provides an image of a virtual space from a viewpoint corresponding to an object in the area, and enables safety confirmation of blind spots.

[0010] The invention according to [2] makes it possible to determine the viewpoint in the virtual space with high accuracy.

[0011] [3] The invention of the present invention makes it possible to introduce a safety management system using existing surveillance cameras, etc. as imaging devices.

[0012] According to the invention described in [4], the direction of an object can be determined with high accuracy using feature points obtained by image recognition.

[0013] The invention according to [5] makes it possible to recognize a moving object and generate an image of the moving object.

[0014] The invention according to [6] makes it possible to predict an intersection area based on the movement paths of multiple moving objects, generate an approach notification, and improve safety.

[0015] The invention according to [7] can improve safety by making objects in blind spots transparent. [Effects of the Invention]

[0016] According to the present invention, it is possible to provide a technique for improving safety at a work site by using an image of a virtual space corresponding to a monitored area. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a block diagram of a system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating a hardware configuration of the present embodiment. [Figure 3] FIG. 3 is a schematic diagram illustrating information acquisition in a monitoring area according to the present embodiment. [Figure 4] FIG. 2 is a schematic diagram of a virtual space according to the present embodiment. [Figure 5] FIG. 2 is a schematic diagram of a viewpoint in a virtual space according to the present embodiment. [Figure 6] 10 shows a display example of a viewpoint image according to the present embodiment. [Figure 7] FIG. 10 is a schematic diagram of safety management in an intersection area according to the present embodiment. [Figure 8] 3 is a flowchart of the overall processing of the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, a safety management system and a safety management method according to an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment shown below is an example of the present invention, and the present invention is not limited to the embodiment below, and various configurations can be adopted.

[0019] In this embodiment, the configuration, operation, etc. of a safety management system and a safety management device are described, but a safety management method, a computer program, and a program recording medium on which the program is recorded, each having a similar configuration, also achieve the same effects. For example, by using a program recording medium, the program can be installed on a computer. The series of processes according to this embodiment described below are provided as a computer-executable program, and can be provided via a non-transitory computer-readable recording medium such as a CD-ROM or a flexible disk, or even via a communication line.

[0020] The safety management system is composed of multiple computer devices. The first computer device has an arithmetic device such as a CPU (Central Processing Unit) and a storage device. The computer device can function as a safety management device by executing a first program stored in the storage device using the arithmetic device. The second computer device has an arithmetic device such as a CPU and a storage device. The computer device can function as a user terminal by executing a second program stored in the storage device using the arithmetic device. The safety management method is realized by processing of the computer devices including the safety management device and the user terminal.

[0021] In this description, the area to be monitored is a work site such as a factory or warehouse where work vehicles and workers work. The monitored area may be any area where there is a risk of collision between moving objects, and is not limited to a work site.

[0022] FIG. 1 shows a block diagram of a safety management system 1. The safety management system 1 includes a safety management device 2, a user terminal 3, and an environmental sensor 4. The safety management device 2 is communicatively connected to the user terminal 3 and the environmental sensor 4. The communication connection is, for example, wireless communication via a communication network, but wired communication may also be employed. Although only one user terminal 3 and one environmental sensor 4 are shown in FIG. 1, there may be multiple of these.

[0023] The environmental sensor 4 is installed at a predetermined position in the area to be monitored, and acquires environmental information within the area. The environmental sensor 4 can transmit the acquired environmental information to the safety management device 2.

[0024] The safety management device 2 has, as functional components, an object recognition unit 21 that recognizes objects within the area using environmental information acquired by the environmental sensor 4, a model generation unit 22 that generates a three-dimensional model of the object, a viewpoint determination unit 23 that determines the viewpoint of the three-dimensional model, an image generation unit 24 that generates an image of the virtual space based on the viewpoint of the object, a prediction unit 25 that predicts the intersection area of ​​multiple moving objects, and an output unit 26 that outputs images of the virtual space, etc. to the user terminal 3.

[0025] The user terminal 3 is a terminal device operated by a user. The user terminal 3 functions as an input / output interface for the safety management system 1. The user terminal 3 can transmit various input information to the safety management device 2. The user terminal 3 can also receive various outputs from the safety management device 2. In this embodiment, the user terminal 3 can acquire an image of the virtual space generated by the safety management device 2 and display it on a display.

[0026] FIG. 2(a) shows a hardware configuration diagram of the safety management device 2. The safety management device 2 includes a control device 201, a storage device 202, and a communication device 203 as its hardware configuration, and each component device is connected via a bus interface. In this embodiment, the safety management device 2 can be a computer device such as a server device or a personal computer. The safety management device 2 is not limited to the configuration shown in the example, as long as it is made up of multiple computer devices and can realize the above-mentioned functional components (21-26) as a whole.

[0027] The control device 201 is configured with one or more processors such as a CPU, and executes a safety management program, an OS (Operating System), and other applications to control the overall processing in the safety management device 2. The storage device 202 is a hard disk drive (HDD), solid state drive (SSD), flash memory, random access memory (RAM), etc., and stores the safety management program and various data. The communication device 203 is a communication interface for wired communication, wireless communication, etc., and controls data communication with external devices.

[0028] 2(b) shows a hardware configuration diagram of the user terminal 3. The user terminal 3 includes, as its hardware configuration, a control device 301, a storage device 302, a communication device 303, an input device 304, and an output device 305. The user terminal 3 may also include a GPS communication device. In this embodiment, the user terminal 3 is a tablet terminal, but is not limited to this, and may also be, for example, a smartphone or a personal computer.

[0029] The control device 301 is composed of one or more processors such as a CPU, and controls the overall processing of the user terminal 3 by executing terminal programs, an OS, other applications, etc. The storage device 302 is an HDD, SSD, flash memory, RAM, etc., and stores a browser application and various data. The communication device 303 controls communication with the communication network NW and realizes data communication with at least the safety management device 2. The input device 304 is an input interface that accepts input operations by the user, and is composed of a microphone, touch panel, mouse, keyboard, etc. The output device 305 is composed of a display that displays and outputs, etc. The GPS communication device can acquire the position coordinates of the user terminal 3 via GPS communication.

[0030] The environmental sensor 4 acquires environmental information in the monitored area. The environmental sensor 4 can be an imaging device, a LiDAR (Light Detection and Ranging), a radar, an ultrasonic sensor, etc. The environmental information is used to identify the type and positional relationship of objects present in the monitored area.

[0031] In this embodiment, an example will be described in which an imaging device is used as the environmental sensor 4. The imaging device captures images of the monitored area and transmits the captured images to the safety management device 2 in real time. The imaging device is installed in a position where it can capture images of the monitored area from above, and multiple imaging devices may be installed. The imaging device can be a surveillance camera or the like that is installed in the monitored area in advance, and can be introduced as the environmental sensor 4 without the need to install new equipment.

[0032] FIG. 3(a) is a schematic explanatory diagram of acquiring environmental information of a monitoring area R by an environmental sensor 4. In FIG. 3, objects O1 to O4, including moving objects and stationary objects, exist in the monitoring area R. The object O1 is a work vehicle and is classified as a moving object. The object O2 is a worker and is classified as a moving object. The object O3 is luggage and is classified as a stationary object. The object O4 is a storage shelf and is classified as a stationary object. Note that the objects O1 to O4 are merely examples and may include other objects.

[0033] The moving objects include a target moving object to which the image generated by the image generation unit 24 is output. In this embodiment, the target moving object is a work vehicle, and an image is output to a user terminal 3 on the work vehicle. There may be multiple target moving objects in the monitoring area R. When viewed from a certain target moving object, other target moving objects are distinguished as moving objects.

[0034] 3(b) shows the plane coordinates of the monitoring area R. The object recognition unit 21 recognizes each object present in the monitoring area R based on the environmental information acquired by the environmental sensor 4. The object recognition unit 21 maps the position of each of the recognized objects O1 to O4 onto the plane coordinates of the monitoring area R.

[0035] The object recognition unit 21 recognizes the type of object based on the environmental information. The types of objects include vehicles (work vehicles), people (workers), luggage, storage shelves, etc. Objects of the same type are assigned individual identification information (ID) and are distinguished from one another.

[0036] The object recognition unit 21 recognizes feature points of an object. Feature points indicate parts that characterize the object, such as its shape, color, pattern, or a combination of these. Feature points differ depending on the type of object. By recognizing feature points, the object recognition unit 21 can recognize the direction (orientation) of the object.

[0037] The object recognition unit 21 recognizes planar objects having a predetermined plane. A planar object is, for example, a stationary object having a plane area equal to or larger than a predetermined plane. The object recognition unit 21 acquires the texture of the plane of an object recognized as a planar object. An example of a planar object is a storage shelf. Items such as luggage are stored on a storage shelf, and the object recognition unit 21 recognizes the stored items and the storage shelf as a whole as the storage shelf. The object recognition unit 21 recognizes luggage taken out of the storage shelf as an individual object. It is assumed that textures are also acquired for floor surfaces, wall surfaces, etc. in the same way.

[0038] The object recognition unit 21 can recognize objects that move together with the target moving object as a single unit. The target moving object includes, for example, a work vehicle. In this case, the workers on board the work vehicle and the luggage being transported are recognized as a single work vehicle. In addition to this, luggage, shelves, and the like that are continuously arranged may also be recognized as a single unit.

[0039] In one embodiment, the object recognition unit 21 recognizes objects within an area using a captured image. The object recognition unit 21 has an image recognition model, and by inputting the captured image into the image recognition model, it is possible to recognize the position, type, and direction (feature points) of each object in the image. The image recognition model may be a neural network model or the like.

[0040] The process flow for recognizing an object by the object recognition unit 21 is as follows: Step S12 and step S13 are performed in no particular order. Step S11: Detect the object. Step S12: Recognize the position of the object. Step S13: Recognize the type of object. Step S13-1: Recognize feature points of the object. Step S13-2: The texture of the planar object is obtained.

[0041] The object recognition unit 21 generates object information as a result of object recognition. The object information includes an object ID, a type, a position, and a direction. The object information is classified into a stationary object or a moving object depending on the type. Stationary objects include luggage, storage shelves, etc. Moving objects include work vehicles, workers, etc. The object information further includes texture for planar objects. Furthermore, the object information may further include a moving speed for moving objects.

[0042] The position, direction, and movement speed included in the object information may be acquired by various sensors mounted on the user terminal 3 or the work vehicle. The various sensors include a position sensor such as a GPS module, a direction sensor such as a gyro sensor or a magnetic sensor, and a speed sensor such as an acceleration sensor. These sensors may be used in conjunction with the environmental sensor 4.

[0043] The model generation unit 22 generates a 3D model of an object in a virtual space corresponding to the monitored area. The virtual space is a space in which the positional relationship of objects existing in the monitored area, which is real space, is digitized. The model generation unit 22 references the object information, acquires the position of each object, and generates a 3D model of the object at a corresponding position in the planar coordinates of the monitored area. The model generation unit 22 also references the object information, acquires the orientation of each object, and determines the orientation of the 3D model of the object, thereby generating the 3D model.

[0044] FIG. 4(a) is an explanatory diagram illustrating an overview of a virtual space V. In FIG. 4(a), simplified three-dimensional models M1 to M6 are shown in the virtual space V. The three-dimensional model M1 represents a three-dimensional model corresponding to luggage. The three-dimensional models M2 and M3 represent three-dimensional models corresponding to a storage shelf. The three-dimensional model M4 represents a three-dimensional model corresponding to a work vehicle. The three-dimensional models M5 and M6 represent three-dimensional models corresponding to workers. The three-dimensional models M2 and M3 of the storage shelf, which is a planar object, represent a recognized plane as a model.

[0045] 4(a) shows the rough positions, types, and directions of the three-dimensional models M1 to M6. The model generation unit 22 acquires model information and texture information and applies them to the three-dimensional models M1 to M6 to generate a detailed virtual space V2 shown in FIG. 4(b).

[0046] The model information includes a model ID, an object type, model data, texture data, etc. The model information is pre-stored in the storage device 202. The model generation unit 22 acquires model information of an object type corresponding to the type of object information, and applies the model data and texture data based on the position and orientation of the object information, thereby generating the three-dimensional models M1 to M6 shown in FIG. 4(b).

[0047] The model generation unit 22 can generate a three-dimensional model of a planar object using texture information included in the object information. In FIG. 4(b), the texture of the object information is applied to the planar areas of the three-dimensional models M2 and M3, which are shelves. Note that the model generation unit 22 may also generate a three-dimensional model of a planar object by applying texture data stored as model information. By applying the texture of the object information, a virtual space that is closer to the real space can be reproduced.

[0048] The viewpoint determination unit 23 determines the viewpoint of an object in virtual space. In this description, the viewpoint indicates where the view is from and in which direction. In other words, the viewpoint includes a position and a direction. The viewpoint in this embodiment reproduces the viewpoint from a predetermined object present in the monitoring area, and is the viewpoint of a three-dimensional model corresponding to the object in virtual space.

[0049] The viewpoint determination unit 23 acquires the position and direction of the object information and determines the viewpoint of the object in the virtual space. Note that the viewpoint determination unit 23 may also acquire the position and direction of a three-dimensional model in the virtual space and determine the viewpoint of the three-dimensional model in the virtual space. The viewpoint determination unit 23 can determine the viewpoint for each target moving object identified by the object ID.

[0050] The viewpoint determination unit 23 sets a height position for the acquired plane coordinate position and determines the viewpoint position in the virtual space. The height may be set according to the type of object or may be set as a default value. In this embodiment, the viewpoint particularly includes the viewpoint of a 3D model corresponding to the target moving object.

[0051] The viewpoint determination unit 23 determines the viewpoint direction based on the direction of the object or three-dimensional model. The viewpoint direction is usually determined as the direction forward of the object or three-dimensional model. Note that the viewpoint direction may also be determined according to an instruction or the traveling direction of the object, and includes, for example, the rearward direction, leftward or rightward directions. For example, when a work vehicle is backing up, the rearward direction of the work vehicle is set as the viewpoint direction.

[0052] FIG. 5 is an explanatory diagram outlining the viewpoint in the virtual space V2. In FIG. 5, the viewpoint VP of the three-dimensional model M4 of the work vehicle is shown. The viewpoint VP includes a viewpoint position and a viewpoint direction indicated by an arrow. The shaded area in FIG. 5 indicates the field of view VF. The field of view VF can be adjusted by instructions from the user terminal 3.

[0053] 5, the work vehicle is carrying a load in the front. In this case, in real space, the worker operating the work vehicle has his or her front blocked by the load, creating a risk of colliding with another object when moving forward. To avoid this, the viewpoint determination unit 23 may set the viewpoint position to be in front of the loaded load.

[0054] The image generation unit 24 generates a viewpoint image of the virtual space V based on the viewpoint of the object. The image generation unit 24 generates the viewpoint image based on the viewpoint determined by the viewpoint determination unit 23. The viewpoint image is usually a front image of the object, but is not limited to this and includes a rear image and an image in another direction. The direction of the viewpoint image is determined by the viewpoint determination unit 23 described above. The image generation unit 24 can generate a viewpoint image for each target moving object identified by an object ID.

[0055] The image generation unit 24 can change the transmittance of a predetermined three-dimensional model to generate a viewpoint image. The transmittance is set, for example, in the range of 0 to 1. At a transmittance of "0", the three-dimensional model is not transparent, and at a transmittance of "1", the three-dimensional model is completely transparent and not visible.

[0056] The image generation unit 24 can make the 3D model of the target moving object transparent or invisible. Furthermore, the image generation unit 24 can make the 3D model of an object recognized as moving together with the target moving object transparent or invisible. Furthermore, the image generation unit 24 can make the 3D model that exists within a predetermined distance from the viewpoint of the target moving object transparent or invisible. As a result, the luggage carried by the work vehicle, which is the target moving object, becomes transparent or invisible. Note that the viewpoint determination unit 23 may be configured to hide the luggage by setting the viewpoint position further forward of luggage, etc., loaded in front of the target moving object.

[0057] The image generating unit 24 may set a transmittance (excluding 0) for the three-dimensional model under any of the following conditions: The transmittance may be any value that allows other three-dimensional models that are located beyond the target three-dimensional model to be recognized. Condition A: Transmittance is set for a three-dimensional model of a stationary object. Condition B: Set transmittance to a 3D model that is close to a specified distance. Condition C: If there is a possibility of intersection, transmittance is set for the 3D model of an object that is between at least the 3D model of the moving object that may intersect.

[0058] Fig. 6(a) shows an example of a viewpoint image based on the viewpoint VP in Fig. 5. This allows the driver to avoid blind spots caused by luggage blocking the road ahead in real space, check for safety ahead, and operate the work vehicle.

[0059] Fig. 6(b) shows an example of a perspective image based on the viewpoint VP in Fig. 5, in which a 3D model M2 of the storage shelf is seen through. This shows a 3D model M6 of a worker standing at the back of the storage shelf, allowing the operator to avoid blind spots caused by the storage shelf in real space, check for safety in the direction of travel, and operate the work vehicle.

[0060] The output unit 26 outputs the generated viewpoint image to the user terminal 3. The user terminal 3 may be mounted on the target moving object or carried by the user. The user terminal 3 displays the acquired viewpoint image on a display.

[0061] An example will be described in which a plurality of target moving objects exist in a monitoring area. The object recognition unit 21 recognizes a first target moving object and a second target moving object, respectively, and generates two pieces of object information. The image generation unit 24 generates a first viewpoint image and a second viewpoint image corresponding to these two target moving objects. The output unit 26 outputs the generated viewpoint images to the first user terminal 3A and the second user terminal 3B. At this time, the output unit 26 needs to determine, for example, whether to output the first viewpoint image to the first user terminal 3A or the second user terminal 3B.

[0062] The output unit 26 acquires a link signal indicating the correspondence between the user terminal 3 and the target moving object, and thereby determines the output destination.

[0063] In one aspect, the output unit 26 receives an input selecting a first target moving object or a second target moving object as a link signal via the user terminal 3. This links the target moving object and the user terminal in a correspondence relationship. The output unit 26 determines an output destination in accordance with the correspondence relationship and outputs a viewpoint image. The selection input is transmitted from the user terminal 3 to the safety management device 2, for example, by a selection operation by the user on a display screen (not shown) of a virtual space or a plan view on which a three-dimensional model of each target moving object is displayed.

[0064] In one aspect, the target moving object or the user terminal 3 is equipped with a position sensor. The position sensor can collect sensor values ​​indicating the position of the sensor. The position sensor or the user terminal 3 transmits the sensor values ​​to the safety management device 2 as link information. The output unit 26 refers to the sensor value and the position included in the object information of the target moving object, and if the respective positions are close to each other, it can link the target moving object and the user terminal. Note that the position sensor can be replaced with an acceleration sensor, a gyro sensor, driving information (including wheel rotation amount, steering angle, etc.), etc. In this case, the sensor value indicating a predetermined movement is transmitted to the safety management device 2 as link information. The output unit 26 refers to the sensor value and the movement of the target moving object recognized by the object recognition unit 21, and if the movements match, it can link the target moving object and the user terminal 3.

[0065] Alternatively, the link information may be a predetermined sign presented to the environment sensor 4 from near the target moving object. The sign may include a predetermined gesture, presentation of a predetermined image, emission of light, emission of sound, etc.

[0066] The prediction unit 25 predicts an intersection region of moving objects. The intersection region indicates an area where multiple moving objects intersect on their movement paths, posing a risk of collision.

[0067] The prediction unit 25 predicts an intersection area using object information. The prediction unit 25 predicts an intersection area based on the positions, directions, and velocities of multiple moving objects. The prediction unit 25 predicts the intersection area and can also predict an intersection prediction time, which is the time required until the intersection occurs.

[0068] The prediction unit 25 generates an approach notification to the intersection area. In this embodiment, the approach notification is a notification that notifies the target moving object that there is a risk of collision with another moving object in the intersection area. The approach notification can be in an output form that can be perceived by the user, such as an image, sound, vibration, or light.

[0069] The image generation unit 24 generates a warning display based on the approach notification by superimposing it on the viewpoint image. The output unit 26 outputs the viewpoint image including the warning display to the user terminal 3. The output unit 26 may also output a warning to the user terminal 3 by sound, vibration, or light based on the approach notification.

[0070] Fig. 7(a) shows a plan view V3 of the virtual space. Fig. 7(a) shows a first moving object (work vehicle) M11, a second moving object (worker) M12, and a stationary object (shelf) M13. The first moving object M11 and the second moving object M12 are each moving forward in the direction of the arrows and are predicted to intersect in an intersection region CR.

[0071] FIG. 7(b) shows an example of a screen of a viewpoint image W10 including warning displays W11 to W13 displayed on the user terminal 3. The viewpoint image W10 is an image based on the viewpoint of the first moving object M11 in FIG. 7(a). The warning display W11 is a display indicating an intersection area. The warning display W12 is a display warning of the risk of collision with another object in the intersection area. The warning display W13 is a display indicating that a second moving object M12 is approaching from the left. The warning displays W11 to W13 may alert the user by at least one of these. In addition to the warning displays, the viewpoint image W10 can more effectively alert the user by displaying a three-dimensional model of an object in the blind spot as shown in FIG. 6(b).

[0072] FIG. 8 shows a flowchart of the overall processing of the safety management system 1. The object recognition unit 21 recognizes objects in the area using environmental information acquired by the environmental sensor 4 (S101). The model generation unit 22 generates a 3D model corresponding to the object in virtual space based on the object information of the recognized object (S102). The viewpoint determination unit 23 determines the viewpoint of the target object based on the object information or the 3D model (S103). The image generation unit 24 generates a viewpoint image of the virtual space based on the determined viewpoint of the object (S104). The output unit 26 outputs the generated viewpoint image to the user terminal 3 (S105). [Explanation of symbols]

[0073] 1 Safety Management System 2 Safety management device 21 Object recognition section 22 Model Generation Unit 23 Viewpoint determination unit 24 Image generation unit 25 Prediction Department 26 Output section 3. User terminal 4 Environmental Sensors

Claims

1. an object recognition unit that recognizes objects within a monitoring target area using environmental information acquired by an environmental sensor installed at a predetermined position in the monitoring target area; a model generation unit that acquires a position of an object within the region and generates a three-dimensional model of the object in a virtual space corresponding to the region; and an image generation unit that generates an image of the virtual space based on the viewpoint of the object.

2. The safety management system according to claim 1 , further comprising a viewpoint determination unit that acquires the position and orientation of the object and determines a viewpoint of a three-dimensional model corresponding to the object.

3. the environmental sensor is an imaging device; The safety management system according to claim 1 , wherein the object recognition unit recognizes an object within the area using an image captured by the imaging device.

4. the object recognition unit recognizes feature points of the object in the captured image; The safety management system according to claim 3 , wherein the model generation unit determines an orientation of the three-dimensional model based on the feature points and generates a three-dimensional model of the object in the virtual space.

5. the object recognition unit distinguishes and recognizes moving objects within the area; The safety management system according to claim 1 or 2, wherein the image generation unit generates an image of the virtual space based on a viewpoint of a three-dimensional model of the moving object.

6. a prediction unit that predicts an intersection area based on the movement paths of the first moving object and the second moving object, The safety management system according to claim 5 , wherein the image generation unit generates an image including a notification of approach to the intersection area.

7. the object recognition unit distinguishes and recognizes stationary objects within the area; The safety management system according to claim 1 or 2, wherein the image generating unit generates an image of the three-dimensional model of the stationary object with a predetermined transmittance.

8. Recognizing objects within a monitored area using environmental information acquired by environmental sensors installed at predetermined positions within the monitored area; acquiring a position of an object within the region, and generating a three-dimensional model of the object in a virtual space corresponding to the region; A safety management method in which a computer executes each process to generate an image of the virtual space based on the viewpoint of the object.

9. an object recognition unit that recognizes objects within a monitoring target area using environmental information acquired by an environmental sensor installed at a predetermined position in the monitoring target area; a model generation unit that acquires a position of an object within the region and generates a three-dimensional model of the object in a virtual space corresponding to the region; an image generation unit that generates an image of the virtual space based on the viewpoint of the object.

Images