Safety equipment monitoring method and system

Abstract

The present invention relates to a method performed by a safety equipment monitoring system, and provides a safety equipment monitoring method comprising the steps of: receiving an image stream, which is collected by a camera provided in at least one construction site, and in which a dangerous area having safety equipment for fall prevention to be provided in a building under construction is captured; extracting data from the image stream and processing the data; and using the data to detect the safety equipment for fall prevention provided in the dangerous area. According to the present invention, safety equipment for fall prevention provided on respective floors can be monitored according to changes in the position of a camera and the number of floors of a building under construction.

Description

Safety facility monitoring method and system

[0001] The present invention relates to a safety facility monitoring method and system, and more particularly, to a method and system for monitoring the situation in which fall prevention safety facilities are installed at a construction site.

[0002] In architecture, the core refers to the vertical space for circulation and services. Circulation and services refer to the movement of people and materials, and include things necessary for building use, such as stairs, elevators, wiring, and plumbing. If a building were compared to the human body, the core would correspond to the circulatory system—the blood vessels and heart—that supply oxygen and nutrients to all tissues and organs. The core occupies a pivotal position in architecture.

[0003] The construction of apartments and office buildings can involve concrete pouring, depending on the number of floors. As concrete pouring begins on a floor, the next floor begins, and until the building reaches its highest point, the cores formed at each floor remain empty. This means that until the cores are filled, the cores containing stairs, elevators, electrical wires, and water and sewer pipes remain empty, posing a risk of falls for construction workers. Opening covers, such as barbed wire, are required to cover these openings. As the number of floors increases, these covers must be installed in the empty cores.

[0004] The present invention relates to a safety facility monitoring method and system capable of detecting the presence or absence of safety facilities, such as opening covers, installed during construction.

[0005] As a technology related to the present invention, the Korean Patent Publication discloses a system for monitoring the stability of temporary construction facilities and structures using CCTV image recognition technology of three-dimensional markers installed on the surface of the temporary construction facilities and structures, which calculates displacement information and sends an alarm. This related technology monitors the stability of temporary construction facilities and structures using three-dimensional markers, and is distinct from the present invention, which monitors the installation status of temporary construction facilities, in terms of its composition and effects.

[0006] The problem to be solved by the present invention is to provide a method and system for monitoring the installation of fall prevention safety facilities on each floor as the number of floors of a building being constructed changes.

[0007] The problem to be solved by the present invention is to provide a method and system that can automatically recognize fall prevention safety facilities installed in similar locations on each floor as the location of the camera and the number of floors of the building change.

[0008] The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0009] In order to achieve the above object, according to one embodiment of the technical idea of ​​the present invention, a safety facility monitoring method is disclosed, which comprises a step of receiving a video stream collected by at least one camera installed at a construction site, the video stream capturing a dangerous area where a fall prevention safety facility is to be installed in a building under construction; a step of extracting data from the video stream and processing the data; and a step of detecting the fall prevention safety facility installed in the dangerous area using the data, the method being performed by a safety facility monitoring system.

[0010] In addition, the safety facility monitoring method is characterized in that the camera is installed at a position higher than the highest floor on the crane to photograph the core exposed on the highest floor where the skeleton is completed.

[0011] In addition, the safety facility monitoring method may further include a step of constructing a simulation model in which a virtual building corresponding to the building under construction and a virtual camera corresponding to the camera are installed, and the step of receiving a video stream may be configured to include a step of tracking the location of a core on a third floor or higher of the building under construction using the virtual building and the virtual camera based on the location of the core indicated on the second floor of the virtual building; and a step of controlling the operation of the camera based on the location of the core.

[0012] In addition, the safety facility monitoring method is characterized in that the step of tracking the location of a core on the third floor or higher is based on a simulation model, and the location information of the camera and building information are used to track the location of the core.

[0013] In addition, the safety facility monitoring method may be configured such that the step of controlling the operation of the camera includes a step of controlling the camera to a preset position of at least one core through control of the pan, tilt, and zoom of the camera; and a step of collecting an image of the core captured using the camera.

[0014] In addition, the safety facility monitoring method may be configured such that the step of detecting a fall prevention safety facility installed in a hazardous area includes a step of recognizing the opening cover by comparing a core without an opening cover and a core with an opening cover installed using an object detection algorithm; and a step of generating an alarm if the opening cover is not recognized.

[0015] In addition, the safety facility monitoring method further includes a step of training an artificial intelligence recognition model to recognize an opening cover, and the step of recognizing the opening cover is characterized in that the opening cover is recognized using the artificial intelligence recognition model.

[0016] In addition, the safety facility monitoring method may be configured such that the step of training the artificial intelligence recognition model includes a step of generating a dataset necessary for training through data synthesis by modeling a virtual building included in a simulation model; and a step of training the recognition model to be able to recognize the presence or absence of installed safety facilities using the dataset.

[0017] In order to achieve the above object, according to one embodiment of the technical idea of ​​the present invention, a safety facility monitoring system is disclosed, which comprises: a plurality of cameras installed at construction sites; an image receiving unit that receives an image stream collected by the cameras, wherein the image stream captures a dangerous area where a fall prevention safety facility is to be installed in a building under construction; an image processing unit that extracts data from the image stream and processes the data; and an object recognition unit that detects the fall prevention safety facility installed in the dangerous area using the data.

[0018] In addition, the safety facility monitoring system may be configured to further include a simulation unit that generates a virtual building model including a virtual building corresponding to a building under construction and a virtual camera corresponding to the camera.

[0019] In addition, the safety facility monitoring system may be configured to further include a dataset generation unit that synthesizes images of safety facilities installed in a hazardous area using a simulation unit.

[0020] Additionally, the safety facility monitoring system may be configured to further include a camera control unit that controls the operation of the camera.

[0021] In addition, the safety facility monitoring system may be configured to further include a storage device that stores camera location information and construction status information regarding the location where the camera is installed for a building under construction.

[0022] Specific details of other embodiments are included in the “Specific Details for Carrying Out the Invention” and the attached “Drawings.”

[0023] The advantages and / or features of the present invention and the methods for achieving them will become clear with reference to the various embodiments described in detail below together with the accompanying drawings.

[0024] However, the present invention is not limited to the configuration of each embodiment disclosed below, but may be implemented in various different forms, and each embodiment disclosed in this specification is provided only to ensure that the disclosure of the present invention is complete and to fully inform a person having ordinary skill in the art to which the present invention pertains of the scope of the present invention, and it should be understood that the present invention is defined only by the scope of each claim of the claims.

[0025] According to the present invention, monitoring is possible to recognize whether safety facilities are installed by identifying the location of safety facilities installed on each floor of a building.

[0026] Additionally, if safety facilities are not installed or a dangerous situation is detected, an alarm can be generated to alert users to the danger.

[0027] The effects that can be obtained by the safety facility monitoring method and system according to the technical idea of ​​the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0028] Figure 1 is a network relationship diagram of a safety facility monitoring system according to one embodiment of the present invention.

[0029] FIG. 2 is an exemplary diagram of a building, a core, and a camera according to one embodiment of the present invention.

[0030] Figure 3 is a block diagram of a safety facility monitoring device according to another embodiment of the present invention.

[0031] Figure 4 is a flowchart of a safety facility monitoring method according to one embodiment of the present invention.

[0032] Figure 5 is a flowchart of a safety facility monitoring method according to one embodiment of the present invention.

[0033] Figure 6 is a flowchart of a safety facility monitoring method according to one embodiment of the present invention.

[0034] Figure 7 is a flowchart of a safety facility monitoring method according to one embodiment of the present invention.

[0035] Figure 8 is a flowchart of a safety facility monitoring method according to one embodiment of the present invention.

[0036] Figure 9 is a flowchart of camera control and event transmission according to one embodiment of the present invention.

[0037] Figure 10 is a flowchart of event detection according to one embodiment of the present invention.

[0038] Figure 11 is an exemplary diagram of a process for training an artificial intelligence recognition model according to one embodiment of the present invention.

[0039] Figure 12 is an example diagram of an artificial intelligence recognition model according to one embodiment of the present invention.

[0040] Before describing the present invention in detail, it should be understood that the terms or words used in this specification should not be interpreted as being unconditionally limited to their usual or dictionary meanings, and that the inventor of the present invention may appropriately define and use the concepts of various terms in order to explain his or her invention in the best possible manner, and further, that these terms or words should be interpreted as meanings and concepts that are consistent with the technical idea of ​​the present invention.

[0041] That is, it should be noted that the terms used in this specification are only used to describe preferred embodiments of the present invention, and are not intended to specifically limit the contents of the present invention, and that these terms are defined in consideration of various possibilities of the present invention.

[0042] Additionally, it should be noted that in this specification, singular expressions may include plural expressions unless the context clearly indicates a different meaning, and similarly, even if expressed in plural, may include a singular meaning.

[0043] Throughout this specification, whenever a component is described as "including" another component, it may mean that the component may further include any other component, rather than excluding any other component, unless specifically stated otherwise.

[0044] Furthermore, when a component is described as being "inside or connected to" another component, it should be understood that the component may be installed in direct connection with or in contact with the other component, may be installed spaced apart from the other component by a certain distance, and if installed spaced apart from the other component by a certain distance, there may be a third component or means for fixing or connecting the component to the other component, and the description of this third component or means may be omitted.

[0045] On the other hand, if a component is described as being "directly connected" or "directly connected" to another component, it should be understood that no third component or means exists.

[0046] Likewise, other expressions that describe the relationship between components, such as "between" and "directly between", or "adjacent to" and "directly adjacent to", should be interpreted as having the same meaning.

[0047] Additionally, it should be noted that the terms “one side,” “the other side,” “one side,” “the other side,” “first,” “second,” etc. in this specification, if used, are used to clearly distinguish one component from another component, and that the meaning of the component is not limited by such terms.

[0048] In addition, terms related to position, such as “upper,” “lower,” “left,” and “right,” etc., in this specification, if used, should be understood to indicate relative positions of the corresponding components in the corresponding drawings, and unless absolute positions are specified for these positions, these position-related terms should not be understood to refer to absolute positions.

[0049] In addition, in this specification, when specifying the drawing numbers for each component of each drawing, the same component has the same drawing number even if the component is shown in a different drawing, that is, the same reference number indicates the same component throughout the specification.

[0050] In the drawings attached to this specification, the size, position, connection relationship, etc. of each component constituting the present invention may be described with some exaggeration, reduction, or omission in order to sufficiently clearly convey the idea of ​​the present invention or for convenience of explanation, and therefore the proportions or scales may not be strict.

[0051] In addition, in the following description of the present invention, a detailed description of a configuration that is judged to unnecessarily obscure the gist of the present invention, for example, a known technology including a prior art, may be omitted.

[0052] Hereinafter, embodiments of the present invention will be described in detail with reference to the relevant drawings.

[0053] Figure 1 is a network relationship diagram of a safety facility monitoring system according to one embodiment of the present invention.

[0054] Referring to FIG. 1, a safety facility monitoring system (10) according to one embodiment of the present invention may be configured to include a camera (100), a safety facility monitoring device (200), a server (300), and a terminal (400) that are connected to each other so as to be able to communicate with each other through a network (500).

[0055] The camera (100) has a function of capturing a subject, such as a building under construction, and providing a video stream to a safety facility monitoring device (200). The safety facility monitoring device (200) has a function of monitoring dangerous situations captured in the video stream. The camera (100) can be installed at a construction site. Dangerous situations that can be captured at a construction site include situations where a worker is at risk of falling due to a lack of safety facilities, such as opening covers, installed in openings such as cores formed on each floor of a building.

[0056] FIG. 2 is an exemplary diagram of a building, a core, and a camera according to one embodiment of the present invention.

[0057] Referring to FIG. 2, a camera (100) included in a safety facility monitoring system (10) according to one embodiment of the present invention, a building (A1) at a construction site that is the subject of the camera (100), a crane (A2) at the construction site where the camera (100) is installed, a danger zone (A3) included in the building, and a safety facility (A4) for preventing falls installed in the danger zone (A3) are depicted.

[0058] The camera (100) may be installed at a construction site, for example, at a location where it can view the upper part of a building (A1) under construction, such as a crane (A2). The camera (100) may be installed at at least one construction site, and multiple cameras may be installed at one construction site. The camera (100) for viewing the building under construction may be installed at a location where it can view the planned highest floor, or as the number of floors of the building increases during the construction process, the location of the camera (100) may be adjusted upward to a low floor, a middle floor, and a high floor as one or more floors are added. Therefore, the camera (100) is preferably installed at a location higher than the currently constructed highest floor from the crane in order to photograph openings such as cores exposed on the highest floor where the frame has been completed.

[0059] The camera (100) may be a camera used for CCTV purposes, may be an IP camera with an IP assigned, and may be a PTZ camera capable of pan, tilt, and zoom operations.

[0060] The safety facility monitoring device (200) basically includes components of a computer, such as a main processing unit (CPU), memory, input / output devices, network equipment, and a power supply, and has a function of recognizing a danger zone (A3) and a fall prevention safety facility (A4) using a video stream received from a camera (100), and a function of sending an alarm to a terminal (400) when a danger is detected.

[0061] The server (300) may include one or more servers performing different functions, and has a function of providing various information to the safety facility monitoring device (200) or training the learning of an artificial intelligence recognition model.

[0062] Various information may include camera information. Camera information may include camera location information, including camera hardware information and camera installation location and height. Camera location information may include camera horizontal information and camera vertical information. Camera horizontal information may include the xy coordinates of the camera installation on the ground (xy plane) occupied by the building under construction. Camera horizontal information may include the height value (z coordinate) of the camera installation location.

[0063] Various types of information may include building information. Building information may include pre-determined building plan information and information on the progress of construction based on the building plan information, such as information on how many floors have been poured and how many floors contain hazardous areas.

[0064] The terminal (400) has a function of connecting to the safety facility monitoring device (200) to monitor the risk area and fall prevention safety facilities through the camera (100), or to receive an alarm transmitted by the safety facility monitoring device (200) when a risk is detected at a construction site. The terminal (400) may be configured to include a personal terminal (410), such as a smart phone, carried by a staff member participating in the construction, a computer (420) of the control center, and a speaker (430) installed at the construction site. Referring again to FIG. 2, a building (A1) in which a risk area (A3) is indicated is depicted. The risk area (A3) defined in the present invention refers to an area where there is a risk of falling, such as an opening penetrating each floor of the building, for example, a core. The core exists as an opening during the concrete pouring construction of the building, and after the pouring is completed, facility work is performed to fill the opening, for example, with an elevator, stairs, water pipes, or wires.

[0065] In the risk area (A3), fall prevention safety devices (A4), such as opening covers, may be installed. Since openings vary depending on the core type, opening covers may be installed in various sizes.

[0066] Figure 3 is a block diagram of a safety facility monitoring device according to another embodiment of the present invention.

[0067] Referring to FIG. 3, the safety facility monitoring device (200) may be configured to include an image receiving unit (210), an image processing unit (220), a simulation unit (230), a data set generation unit (240), a camera control unit (250), an object tracking unit (260), an event detection unit (270), an environment setting unit (281), an event transmission unit (282), a display (283), a storage device (284), an external interface (285), and a control unit (290).

[0068] The video receiving unit (210) has a function of receiving, in real time, the video collected by the camera (100), for example, video streaming. The purpose of video collection is to check whether a safety facility for preventing falls, for example, an opening cover, is installed in the core provided on each floor of a building, for example, a danger area indicated in the video. Accordingly, the video receiving unit (210) can continuously receive a video streaming from the camera (100), or, when the number of floors increases due to concrete pouring, can receive a video streaming that captures the danger area indicated on the corresponding floor. Accordingly, when determining the timing of video reception, building plan information or building situation information can be utilized.

[0069] The image processing unit (220) extracts image data to be used for image analysis from the received image stream, and has a preprocessing function for processing the extracted image data, such as noise removal, brightness adjustment, and binarization.

[0070] The simulation unit (230) has a function of constructing a simulation model (231) including a virtual building of the same specifications as the building, a virtual danger zone at the same location as the danger zone formed in the building under construction, and a virtual camera corresponding to the camera (100) that photographs the building. The constructed simulation model (231) can be used for generating a learning dataset and object tracking.

[0071] The dataset generation unit (240) has a function of generating a learning dataset used for training an artificial intelligence recognition model. An artificial intelligence recognition model can be used to perform object recognition to recognize fall prevention safety facilities in a dangerous area captured within the angle of a camera (100). The artificial intelligence recognition model receives learning training to be able to recognize fall prevention safety facilities, and the dataset generation unit (240) has a function of generating a learning dataset used for learning training. The dataset generation unit (240) can generate image data depicting a dangerous area and a fall prevention safety facility installed in the dangerous area, for example, an opening cover installed in a core, using a simulation model (231).

[0072] Training AI cognitive models requires a large and diverse amount of data. However, due to practical limitations in data collection, simulation models can be used to obtain a large and diverse dataset through various settings. This data serves as the basis for the training dataset.

[0073] The camera control unit (250) has a function to automatically control the camera (100) remotely. The camera (100) can be installed in a position where it can photograph the entire upper floor of a building under construction, for example, on a tower crane located higher than the upper floor. The height of the camera (100) can be adjusted upward as the number of floors being constructed increases. That is, both the position of the camera (100) and the height of the upper floor of the building change, and the camera (100) must track a dangerous area formed in the upper floor of the building, for example, an opening such as a core. The camera control unit (250) has a function to control the direction and zoom magnification of the camera (100) to photograph a desired subject, i.e., a dangerous area and a fall prevention safety facility, located at the center of the screen based on the results of object tracking by the object tracking unit (260) and the object recognition unit (270).

[0074] The object tracking unit (260) has a function of tracking the location of the risk area formed on each floor and the safety facilities for fall prevention installed in the risk area as the number of floors of the building increases.

[0075] The object tracking unit (260) can basically use a method of tracking an object by comparing previous image data with changed image data based on object recognition and recognizing an object to be tracked in the changed image data.

[0076] The object tracking unit (260) has a function of tracking the location of an actual danger zone based on the location of a virtual danger zone displayed on a virtual building using an additionally constructed simulation model. When the location of the danger zone changes in the angle of the camera (100) as the number of floors of the building increases, the object tracking unit (260) can control the location of the virtual camera through the simulation model so that the location of the virtual danger zone comes within the angle of the virtual camera, and the camera position control value at this time can be used to control the actual camera (100).

[0077] According to one embodiment of the present invention, a method may be used to track the location of a risk area on the third floor or higher, based on a risk area formed on the second floor, using a virtual building corresponding to a building under construction and a virtual camera corresponding to an installed camera included in a simulation model (231). That is, when a virtual building of the same specifications as a building under construction is modeled by the simulation unit (230) and a virtual camera is installed in the same virtual location as an actual camera, the camera control unit (250) can control the virtual camera to track the location of a risk area on the third floor or higher.

[0078] If there are multiple risk areas on each floor, if a number is assigned to each risk area, the location of each risk area is set as a preset, and the control value of the preset is saved, the camera (100) can capture the risk area using the control value of the saved preset. The changed control value is found through object tracking based on the simulation model, and the camera control unit (250) can control the pan operation, tilt operation, and zoom operation of the camera (100) using the changed control value.

[0079] The event detection unit (270) has a function of recognizing a risk area corresponding to an object in the present invention, for example, various cores and openings formed on the floor of a building after construction, and safety facilities for preventing falls installed in the risk area, for example, an opening cover, using an object recognition algorithm. An artificial intelligence recognition model (271) can be used as the object recognition algorithm. An artificial neural network, for example, CNN, YOLO, etc. can be used as an example of an artificial intelligence recognition model (271).

[0080] The environment setting unit (281) has a function of setting the environment required for object tracking by inputting the preset, ROI, status information, and dwell time related setting values ​​of the camera (100) required for tracking the risk area by the object tracking unit (260).

[0081] The event transmission unit (282) has a function of transmitting the result of event detection by the event detection unit (270) to a display (283), a storage device (284), and an external interface (285).

[0082] The display (283) has the function of receiving the results of event detection and displaying them on the screen. The storage device (284) has the function of storing various information and data, such as camera information and building information, as described above. The storage device (284) has the function of receiving and storing the results of event detection.

[0083] The external interface (285) has a function of transmitting the result of event detection to an external device, such as a task server. The control unit (290) has a function of detecting dangers at a construction site by utilizing the object tracking unit (260) and the event detection unit (270) and generating an alarm to notify of the detected danger.

[0084] On construction sites, workers can be exposed to openings, such as cores, that pose a risk of falling while working. These openings are classified as hazardous areas because they remain empty after the entire building is completed, until work begins to fill them, such as installing elevators, stairwells, or utilities like water, sewer, gas, and electricity.

[0085] The control unit (290) has a function of recognizing the location of each floor when a floor is poured and an opening such as a core is formed based on building information of a building under construction, for example, building plan information for the future and building situation information for the past, recognizing whether a safety facility, for example, an opening cover, is installed at the recognized location, and notifying the terminal (400) of the danger through an alarm if the opening cover is not installed.

[0086] Figure 4 is a flowchart of a safety facility monitoring method according to one embodiment of the present invention.

[0087] Referring to Fig. 4, the safety facility monitoring method (S100) can be performed by the safety facility monitoring system (10). The safety facility monitoring method (S100) can be configured to include building a simulation model (231) (S110), learning and training an artificial intelligence recognition model (S120), receiving a video stream in which a dangerous area is captured from a camera (100) (S130), extracting video data from the video stream and processing the video data (S140), and detecting a safety facility using the video data (S150).

[0088] The safety facility monitoring system (10) can perform the construction of a simulation model (S110) and the learning and training (S120) of an artificial intelligence recognition model (271) prior to the reception of a video stream (S130), regardless of the order of the two. If the artificial intelligence recognition model (271) is not utilized in the detection (S150) of a safety facility for fall prevention, the learning and training (S120) of the artificial intelligence recognition model (271) may be omitted.

[0089] The safety facility monitoring system (10) can construct a simulation model (231) in which a virtual building corresponding to a building under construction (A1) and a virtual camera corresponding to a camera (100) are installed using a simulation unit (230). The simulation unit (230) can include a 3D modeling program capable of modeling a virtual building, and a program capable of controlling the walk and zoom of the virtual camera as the position of the subject (virtual danger zone) changes. The safety facility monitoring system (10) can output a camera position control value of an actual camera (100) through the walk and zoom of the virtual camera.

[0090] The safety facility monitoring system (10) can train an artificial intelligence recognition model (271) to recognize an opening cover corresponding to a safety facility prior to detecting the safety facility. The artificial intelligence recognition model (271) can include an artificial neural network such as CNN, YOLO, YOLO TINY, YOLACT, and U-NET that can learn through deep learning. The artificial intelligence recognition model (271) is equipped with an artificial intelligence capable of recognizing a hazardous area and a safety facility for fall prevention through learning training using a learning dataset.

[0091] Next, the safety facility monitoring system (10) can receive a video stream collected by at least one camera installed at a construction site, wherein the video stream captures a risk area where a fall prevention safety facility is to be installed in a building under construction (S130). Next, the safety facility monitoring system (10) can extract video data from the video stream and process the video data (S140). Here, the processing of the video data includes various preprocessing steps used prior to video analysis, such as reducing noise contained in the video, correcting brightness, etc., or binary processing.

[0092] Next, the safety facility monitoring system (10) can detect a fall prevention safety facility (A4) installed in a risk area (A3) using video data (S150). Detection of the safety facility (A4) can be comprised of tracking the risk area (A3) based on recognition of the risk area (A3) and recognition of the fall prevention safety facility (A4) installed in the risk area (A3).

[0093] The danger zone (A3) and the safety facility (A4) can be distinguished from each other, from the perspective of the pixels that compose them, through differences in the intensity of light and shade, differences in texture, and differences in color, compared to the danger zone (A3). For example, if an opening cover corresponding to a fall prevention safety facility is installed in an opening corresponding to the danger zone (A3), the object recognition unit (270) can recognize that the intensity of light and shade inside the opening is weaker, recognize a mesh-like material, or recognize a color that stands out, such as yellow.

[0094] In particular, the safety facility monitoring system (10) can recognize the opening cover corresponding to the safety facility for fall prevention by using the recognition model (271) of artificial intelligence that has been trained in advance at the stage of recognizing the opening cover.

[0095] Figure 5 is a flowchart of a safety facility monitoring method according to one embodiment of the present invention.

[0096] Referring to FIG. 5, in the step of training the learning of the artificial intelligence recognition model (S120), the safety facility monitoring system (10) can generate a learning dataset through data synthesis regarding the core of a virtual building (S121). In addition, the safety facility monitoring system (10) can train the learning of the recognition model to recognize the presence or absence of installed safety facilities using the learning dataset (S122).

[0097] Deep learning, a subset of machine learning, refers to the training of artificial neural networks using training datasets for supervised or unsupervised learning. The training datasets required for training typically require massive amounts, and the intelligence of the artificial neural network increases proportionally to the amount of data. Conventional techniques typically rely on publicly available datasets for various data types, relying on widely known public data rather than generating the data needed for training. However, public data often suffers from limitations and lack of customization, making it unsuitable for the intended training tasks.

[0098] According to one embodiment of the present invention, by constructing a simulation model, a camera position control value of an actual camera can be calculated through a virtual camera work based on the constructed simulation, and based on the simulation, a dangerous area and a safety facility for preventing falls can be directly modeled and created, and a photographed image of the created model can be used to construct a learning data set necessary for recognizing the dangerous area and the safety facility for preventing falls. By designing the size and shape of an opening corresponding to a dangerous area in various ways, designing the shape, size, material, texture, and color of an opening cover corresponding to a safety facility for preventing falls in various ways, and adjusting the intensity and direction of lighting for a virtual environment in various ways, an image of a dangerous area and an image of a safety facility for preventing falls having various characteristics can be generated.

[0099] Figure 6 is a flowchart of a safety facility monitoring method according to one embodiment of the present invention.

[0100] Referring to FIG. 6, in the step of receiving a video stream (S130), the safety facility monitoring system (10) can track the location of a core on the third floor or higher of a building under construction using a virtual building and a virtual camera based on the location of the core indicated on the second floor of the virtual building (S131). In addition, the safety facility monitoring system (10) can control the operation of a camera (100) at the location of the core (S132) and receive a video stream in a controlled state.

[0101] In a building under construction, a core is an opening formed through the interior of the building. While the opening may extend from the ground floor to the basement, assuming the core connects the first and top floors, the core formed on the second floor would penetrate the cores on the third and fourth floors, and so on, until it reaches the core on the top floor, for example, the 15th floor.

[0102] As the skeleton for each floor is completed, cores are formed for each floor, and cores on the third floor and higher can be captured with angles and zooms that are different from those of the cores on the second floor at the location where the camera (100) is installed. As the location of the cores changes, the location of the camera (100) may also change as the building under construction becomes taller. Therefore, it is necessary to change the walk and zoom of the camera (100) to match the location of the new core, and prior to controlling the location value of the camera (100), the object tracking unit (260) has a function of tracking the location of the core.

[0103] In the step (S131) ​​of tracking the location of the core on the third floor or higher, the safety facility monitoring system (10) can track the location of the core based on the simulation model (231) and using the location information of the camera (100) and the building information. The location information and the building information of the camera (100) determine the constructed floor of the virtual building and the location of the house camera in the simulation model. When the location of the virtual camera, i.e., the XY coordinate value, is already determined, the changing Z coordinate value is determined, and the number of floors of the virtual building is determined according to the pouring of the actual building, the object tracking unit (260) can identify the location change for the virtual risk area formed on the changed floor by comparing it with the previous floor, and output a camera location control value that controls the location of the camera according to the location change.

[0104] Figure 7 is a flowchart of a safety facility monitoring method according to one embodiment of the present invention.

[0105] Referring to FIG. 7, in the step of controlling the operation of the camera (S132), the safety facility monitoring system (10) can control the camera to a preset position of at least one core by controlling the pan, tilt, and zoom of the camera (S133). In addition, the safety facility monitoring system (10) can collect images of the cores captured using the camera (100) (S134).

[0106] The safety facility monitoring system (10) can sequentially capture multiple risk areas formed in multiple locations using a single camera (100) through preset control. The camera (100) includes the function of a PTZ camera, so the camera control unit (250) can control the position of the camera (100) to multiple preset positions on the next floor using the camera position control value.

[0107] Figure 8 is a flowchart of a safety facility monitoring method according to one embodiment of the present invention.

[0108] Referring to FIG. 8, in the step (S150) of detecting a fall prevention safety facility installed in a core, the safety facility monitoring system (10) can recognize the opening cover by comparing the core without the opening cover and the core with the opening cover installed using an object detection algorithm (S151). In addition, if the opening cover is not recognized, the safety facility monitoring system (10) can generate an alarm in various ways on the terminal (400) (S152).

[0109] The alarm can be transmitted to a personal terminal (410) with an alarm application installed and a computer (420) of the control center through a signal, or can be output through a speaker (430) installed at a construction site through sound.

[0110] FIG. 9 is a flowchart of camera control and event transmission according to an embodiment of the present invention. Referring to FIG. 9, the safety facility monitoring system may be configured with a step (S211) related to a camera video interface related to image analysis that serves as the basis for object recognition, and a step (212) related to an interface that controls the camera according to the result of object tracking. Various information and data used for object detection (S220) may be subject to search (S232) according to user settings (S231). The camera (100) may be controlled to point toward a dangerous area, which is a subject, through preset control and correction (S240). From the video stream collected by the state-controlled camera (100), state change detection (S251) is possible, and event data is generated (S252) according to the result of the state detection, and event transmission (S253), which is an alarm output, occurs.

[0111] Referring to FIG. 9, an image (B1) of an opening without an opening cover installed, an image (B2) of an opening with an opening cover installed, an image (B3) processed by an image processing unit (220), and an image (B4) in which a bounding box is displayed according to opening recognition are depicted.

[0112] The camera (100) can output an image of an opening with an opening cover installed (PRESET 1), an image of an opening with a safety guard installed (PRESET 2), and an image of an opening without a safety facility installed (PRESET 3) according to the preset settings.

[0113] Figure 10 is a flowchart of event detection according to one embodiment of the present invention.

[0114] Referring to FIG. 10, the occurrence of an event (S300) is depicted, which detects a dangerous situation performed by the safety facility monitoring device (200), for example, a dangerous area (A3), particularly a dangerous area (A3) where a safety facility (A4) is not installed, and an approach of a field worker to the dangerous area (A3) where a safety facility (A4) is not installed. In general, the location of the dangerous area can be tracked by the object tracking unit (260), the ROI of the camera (100) can be set to the dangerous area by the camera control unit (250), and the occurrence of an event can be detected by the event detection unit (270). Finally, when the event is detected, the event transmission unit (282) can display the result of the event through the display (283), store the result of the event in the storage device (284), and transmit the result of the event through the external interface (285).

[0115] The camera (100) can be moved in a direction according to a plurality of presets, for example, preset1, preset2, preset3, and preset4 (S311, S321, S331, S341). When there is a preset delay (S312, S322), when the preset delay is greater than the dwell time (S313, S323), AI image analysis is performed (S314, S324), when an event is detected (S315, S325), and when there is a change in the position of the danger area (A3) (S316, S326), an update control value is estimated based on the simulation and the PTZ of the camera is controlled (S317, S327). The direction of the camera (100) can be moved by the camera control unit (250) to the next preset, the preset time can be set (S332, S342), an object can be detected (S333, S343), an event can be detected (S334, S344), and preset correction (S335, S345) can be performed.

[0116] Figure 11 is an exemplary diagram of a process for training an artificial intelligence recognition model according to one embodiment of the present invention.

[0117] Referring to Fig. 11, images D1 to D8 are depicted. The depicted images are part of a learning dataset generated by the dataset generation unit (240). The artificial intelligence recognition model (271) can learn the image (D1) labeled as a risk area to recognize the risk area. Next, the artificial intelligence recognition model (271) recognizes that a fall prevention safety facility is installed using the image (D2) in which a worker approaches the risk area, and recognizes the fall prevention safety facility using the image (D3) of the risk area in which the fall prevention safety facility is installed. The image (D7) represents an image processed by the image processing unit (220). The risk area is not marked in the image (D4). As construction progresses and a new floor is added, a risk area may not yet be formed on the new floor, or even if a risk area is formed, it may be out of the angle of the camera (100) as the floor increases. Image (D4) corresponds to an image in which the risk area is not indicated for this reason.

[0118] The image (D5) corresponds to an image showing a danger zone formed on a new layer. If the danger zone is not displayed in the image because it is outside the angle range of the camera (100) or is displayed outside the center, the object tracking unit (260) has a function of recognizing the location of the current danger zone. The object tracking unit (260) can track the location of the danger zone through two different methods. One of the object tracking methods is to track the location of the virtual danger zone using a virtual building and a virtual camera based on a simulation model (231), and the other is to track the location of the current danger zone by comparing the image (D1) showing the danger zone formed on the previous lower layer with the image (D5) showing the danger zone formed on the current upper layer.

[0119] Image (D6) corresponds to an image in which the location of the danger zone is displayed in the center of the screen through object tracking and camera control. Image (D8) corresponds to an image included in the learning dataset that displays the danger zone.

[0120] The object tracking unit (260) can track an object, i.e., a dangerous area, and output camera control values ​​necessary for camera operation due to a change in the position of the dangerous area. The camera control values ​​can include information regarding the direction of the camera's pan and tilt and the scale of the zoom. The camera control unit (250) can perform PTZ control using the camera control values.

[0121] Figure 12 is an example diagram of an artificial intelligence recognition model according to one embodiment of the present invention.

[0122] Referring to Fig. 12, an object recognition process utilizing an object recognition unit (270) and an artificial intelligence recognition model (271) that have undergone the learning training process depicted in Fig. 11 is depicted. Images C1 to C3 correspond to images input to the artificial intelligence recognition model (271) for recognition testing. Image C1 is an image depicting only a dangerous area, for example, an opening, and image C2 is an image indicating a dangerous area where fall prevention safety facilities are installed. Image C3 corresponds to an image processed by the image processing unit (220).

[0123] Images of C4 and C5 are preset images. When multiple risk areas exist on a single layer, the camera position control values ​​for the cameras that capture the locations of each risk area are stored in advance. By immediately controlling the cameras using the stored camera position control values, the corresponding risk areas can be immediately displayed on the screen. Therefore, through the camera preset, the cameras can immediately identify the locations of risk areas in different locations and capture the corresponding risk areas.

[0124] C6 represents a bounding box displayed in a danger zone as a result of recognition of the image of C4 by the object recognition unit (270). C7 represents a bounding box displayed in a danger zone where a fall prevention safety facility is installed as a result of recognition of the image of C5 by the object recognition unit (270). The image of C8 represents a case where there is no change in status in the danger zone, and the image of C9 represents a case where a change in status occurs in the danger zone. The object recognition unit (270) can detect a change in status through an image depicting only the danger zone and an image depicting the danger zone where fall prevention safety facility is installed, and recognizes that the danger zone has not been equipped with a safety facility. As a subsequent measure, the control unit (290) can output an alarm notifying that a danger zone where fall prevention safety facility is not installed has been recognized to a personal terminal (410), a computer (420) of a control center supervising the construction site, and a speaker (430) installed in the construction site.

[0125] According to one embodiment of the present invention, monitoring is possible to recognize whether a fall prevention safety facility is installed by identifying the location of the safety facility installed on each floor of a building.

[0126] Additionally, if safety facilities for fall prevention are not installed or a dangerous situation is detected, an alarm can be generated to alert the user to the danger.

[0127] Additionally, it will be possible to quickly respond to risk areas that continue to form during the building construction process.

[0128] Above, although some examples have been given and various preferred embodiments of the present invention have been described, the description of the various embodiments described in the “Specific Details for Carrying Out the Invention” section is merely exemplary, and those skilled in the art to which the present invention pertains will readily understand that they can carry out various modifications of the present invention or carry out equivalent implementations of the present invention based on the above description.

[0129] In addition, since the present invention can be implemented in various other forms, the present invention is not limited by the above description, and the above description is provided only to make the disclosure of the present invention complete and to fully inform a person having ordinary skill in the art to which the present invention belongs of the scope of the present invention, and it should be understood that the present invention is defined only by each claim of the claims.

[0130] The present invention can be used in the fields of monitoring and safety of industrial facilities.

Claims

1. A method performed by a safety facility monitoring system, A step of receiving a video stream collected by at least one camera installed at a construction site, wherein the video stream captures a risk area where a fall prevention safety facility is to be installed in a building under construction; A step of extracting video data from the video stream and processing the video data; and A safety facility monitoring method, comprising a step of detecting the fall prevention safety facility installed in the risk area using the above image data.

2. In claim 1, the camera, A safety facility monitoring method characterized in that a crane is installed at a position higher than the highest floor to photograph the core exposed on the highest floor where the skeleton has been completed.

3. In claim 1, It further includes a step of constructing a simulation model in which a virtual building corresponding to the building under construction and a virtual camera corresponding to the camera are installed, The step of receiving the above video stream is: A step of tracking the location of a core on the third floor or higher of a building under construction using the virtual building and the virtual camera based on the location of the core indicated on the second floor of the virtual building; and A safety facility monitoring method, comprising a step of controlling the operation of the camera according to the position of the core.

4. In claim 3, the step of tracking the position of the core of the third or more layers comprises: Based on the above simulation model, characterized in that the location of the core is tracked using the location information and building information of the camera. Safety facility monitoring method.

5. In claim 3, the step of controlling the operation of the camera comprises: A step of controlling the camera to a preset position of at least one core by controlling the pan, tilt, and zoom of the camera; and A safety facility monitoring method comprising a step of collecting images of a core captured using the above camera.

6. In claim 3, the step of detecting a fall prevention safety facility installed in the risk area is: A step of recognizing the aperture cover by comparing the core without the aperture cover and the core with the aperture cover installed using an object detection algorithm; and A safety facility monitoring method, comprising a step of generating an alarm when the above opening cover is not recognized.

7. In claim 6, It further includes a step of training the learning of an artificial intelligence recognition model for recognition of the above opening cover, A safety facility monitoring method, characterized in that the step of recognizing the above opening cover recognizes the opening cover using the recognition model of the artificial intelligence.

8. In claim 7, the step of training the learning of the recognition model of the artificial intelligence comprises: A step of creating a learning dataset through data synthesis by modeling a virtual building included in the above simulation model; and It is configured to include a step of training the learning of the recognition model so that it can recognize the presence or absence of installed safety facilities using the above learning data set. Safety facility monitoring method.

9. Cameras installed at multiple construction sites; A video receiving unit that receives the video stream collected by the above camera, wherein the video stream captures a risk area where a fall prevention safety facility is to be installed in a building under construction; An image processing unit that extracts data from the above image stream and processes the data; and A safety facility monitoring system configured to include an object recognition unit that detects the fall prevention safety facility installed in the risk area using the above data.

10. In claim 9, A safety facility monitoring system further comprising a simulation unit that generates a virtual building model including a virtual building corresponding to a building under construction and a virtual camera corresponding to the camera.

11. In claim 10, A safety facility monitoring system further comprising a dataset generation unit that synthesizes images of safety facilities installed in a hazardous area using the above simulation unit.

12. In claim 9, It is configured to further include a camera control unit that controls the operation of the above camera. Safety facility monitoring system.

13. In claim 9, It is configured to further include a storage device that stores camera location information and construction status information regarding the location where the camera is installed for a building under construction. Safety facility monitoring system.

Images