A systems for responding to event with real-time visualization map based on image data

Abstract

The present invention relates to an event response system with a real-time visualization map based on image data, comprising: a database storing rescue unit data that can be deployed to an event site; a 3D event environment data generation unit that generates virtual 3D event environment data corresponding to the event site based on event site data obtained by photographing the event site and rescue target data for the rescue target; a rescue simulation execution unit that performs a plurality of rescue simulations by reflecting rescue unit data and rescue target data on the generated 3D event environment data; and a rescue scenario data provision unit that generates rescue scenario data based on the results of the rescue simulation execution and provides it to a rescue management terminal.

Description

Technology Field

[0001] The present invention relates to an event response system with a video data-based real-time visualization map applied thereto. More specifically, it relates to an event response system with a video data-based real-time visualization map applied thereto that implements a virtual three-dimensional event environment based on video data acquired by shooting various event sites in real time, and conducts rescue simulations based on changes in the physical environment through a simulation engine, thereby enabling rapid and accurate identification of real-time event situations and rapid and accurate response within a few hours. Background Technology

[0003] In general, various events—including natural occurrences such as typhoons, heavy rain, heatwaves, droughts, strong winds, and heavy snowfall, as well as fires, wildfires, traffic accidents, explosions, and hazardous material spills—cause significant property damage along with human casualties. For such situations, not only is prediction crucial, but a swift and accurate response upon occurrence is also vital.

[0004] However, central event headquarters or local governments responsible for managing event situations or sites generally lack proper devices or systems to broadcast real-time event situations. Furthermore, even with CCTVs installed at event sites, the limited field of view and difficulty in accurately assessing the scene from various angles make it difficult to provide appropriate response guidelines for event situations. Prior art literature

[0006] Korean Registered Patent No. 10-2328272 The problem to be solved

[0007] The present invention aims to solve the aforementioned problems by providing an event response system with a video data-based real-time visualization map that implements a virtual 3D event environment based on video data acquired by shooting various event sites in real time, and conducts rescue simulations based on changes in the physical environment through a simulation engine, thereby enabling rapid and accurate identification of real-time event situations and rapid and accurate response within a few hours. means of solving the problem

[0009] An event response system (100) with a real-time visualization map based on image data according to one embodiment of the present invention may include a database (110) that stores rescue unit data that can be deployed to an event site, a 3D event environment data generation unit (120) that generates virtual 3D event environment data corresponding to the event site based on event site data obtained by photographing the event site and rescue target data for the rescue target, a rescue simulation execution unit (130) that performs a plurality of rescue simulations by reflecting the rescue unit data and the rescue target data on the generated 3D event environment data, and a rescue scenario data provision unit (140) that generates rescue scenario data based on the results of the rescue simulation execution and provides it to a rescue management terminal.

[0010] According to one embodiment of the present invention, the database (110) can acquire the rescue unit data through external linkage or user input.

[0011] According to one embodiment of the present invention, the three-dimensional event environment data generation unit (120) determines the location of the event site and the location of the rescue target based on the event site data and the rescue target data, and then transmits it to a mobile body capable of moving to the location of the event site and the location of the rescue target through one or more paths among a ground path, a sea path, or an air path, so that the event site is photographed through a camera device placed on the mobile body, and the mobile body is an unmanned aerial vehicle (UAV), and the camera device is a 360-degree camera capable of omnidirectional shooting, or is arranged in multiple units to enable multi-angle shooting facing different directions, and the camera device may be configured to photograph the event site to generate video data or multiple image data and transmit it in real time to the three-dimensional event environment data generation unit (120).

[0012] According to one embodiment of the present invention, the three-dimensional event environment data generation unit (120) can collect position values ​​in real time according to the position movement of the moving body from a GPS device placed on the moving body, and also collect distance values ​​for the vertical downward height of the moving body in real time from a LiDAR sensor placed on the moving body.

[0013] According to one embodiment of the present invention, the 3D event environment data generation unit (120) receives shooting data in a real-time streaming manner from the camera device placed on the moving body, a position value according to the position movement of the moving body during the process of transmitting the shooting data, and a distance value for the vertical downward height of the moving body, and then generates the 3D event environment data corresponding to the shooting data based on a 3D Gaussian Splatting method, analyzes the event site within the shooting data based on the 3D Gaussian Splatting method, implements a virtual space for the analyzed event site, and implements the rescue target placed at the event site in 3D and places it in the virtual space.

[0014] According to one embodiment of the present invention, the three-dimensional event environment data generation unit (120) implements a virtual space for the event site and, in the process of implementing the rescue target placed at the event site in three dimensions and placing it in the virtual space, obtains one or more of the topographic data, plant distribution data, past flood damage record data, and past flood record data for the event site from the database (110) and reflects them when generating the three-dimensional event environment data.

[0015] According to one embodiment of the present invention, the 3D event environment data generation unit (120), in the process of implementing a virtual space for the event site, divides the frame-by-frame image of the captured data into a plurality of regions, implements a virtual space cell for each divided region, and then merges the virtual space cells for each region into a single virtual space. When the similarity of boundary vector values ​​between adjacent virtual space cells exceeds a preset error range, it is determined that there is an unimplemented region between the adjacent virtual space cells, and based on the results of learning the unimplemented region based on deep learning, a virtual space cell corresponding to the unimplemented region is generated and then additionally reflected between the adjacent virtual space cells.

[0016] According to one embodiment of the present invention, the three-dimensional event environment data generation unit (120) can correct the boundary vector value of each virtual space cell corresponding to the additionally reflected unimplemented area between the adjacent virtual space cell and the adjacent virtual space cell so that the similarity is within a preset error range, identify the dimming value of the external geometry for the virtual space cell corresponding to the additionally reflected unimplemented area between the adjacent virtual space cell and the adjacent virtual space cell, and correct the dimming value of each virtual space cell so that if the dimming values ​​for each virtual space cell do not match, the similarity of the dimming value for each virtual space cell is within a preset error range.

[0017] According to one embodiment of the present invention, a preprocessing step is performed to contrast the color of a rescue target identified within the shooting data, and numerical data is generated based on the rescue target data for the preprocessed rescue target, wherein the rescue target data includes coordinate values ​​on a satellite map for the rescue target, and if the coordinate values ​​on the satellite map are not identified, the 3D event environment data generation unit (120) can infer and input the coordinate values ​​on the satellite map for the rescue target based on the coordinate values ​​of the moving object on the satellite map.

[0018] According to one embodiment of the present invention, the three-dimensional event environment data generation unit (120) is connected to a correction position value transmission module that is placed on the ground and has an absolute position value, and the correction position value transmission module receives a position value from a satellite, outputs a GPS correction value of the moving body based on the position value, and then wirelessly transmits the outputted GPS correction value to the three-dimensional event environment data generation unit (120). When the GPS correction value is received, the three-dimensional event environment data generation unit (120) compares the position value received in real time from the GPS device with the GPS correction value wirelessly transmitted through the correction position value transmission module, and if the difference in position values ​​exceeds a preset error range, it maintains the state of receiving the GPS correction value from the correction position value transmission module and then resets the GPS device provided in the moving body.

[0019] According to one embodiment of the present invention, the rescue simulation progress unit (130), in the process of conducting the rescue simulation, obtains weather information, ocean information, and sea surface image information regarding the event site from a connected external weather information server as input data, and then inputs the weather information, ocean information, and sea surface image information regarding the event site into a machine learning-based weather change estimation model trained to output weather conditions, ocean conditions, and weather changes according to the location of the moving object based on the obtained input data, thereby obtaining image pattern information regarding the event site and additionally reflecting the image pattern information in the rescue simulation.

[0020] According to one embodiment of the present invention, the rescue simulation progress unit (130), in the process of conducting the rescue simulation, may acquire one or more of wildfire simulation data, earthquake simulation data, flood simulation data, landslide simulation data, river or stream flooding simulation data, typhoon simulation data, storm simulation data, tsunami simulation data, heavy snowfall simulation data, heat wave simulation data, cold wave simulation data, collapse simulation data, fire simulation data, infectious disease simulation data, and chemical spill simulation data corresponding to the event site from a connected external event information database, and then add to the rescue simulation along with the image pattern information. Effects of the invention

[0022] According to one embodiment of the present invention, a virtual three-dimensional event environment is implemented based on video data obtained by shooting various event sites in real time, and a rescue simulation is performed according to changes in the physical environment through a simulation engine, thereby having the advantage of being able to quickly and accurately identify real-time event situations.

[0023] In addition, according to one embodiment of the present invention, since the event situation can be identified quickly and accurately in real time, it has the advantage of enabling a rapid and accurate response within a few hours.

[0024] In addition, according to one embodiment of the present invention, by applying a 3D Gaussian splatting method to field image data obtained through real-time shooting to visualize the map as a three-dimensional map, and by predicting and visualizing areas that were not captured based on deep learning, it has the advantage of being able to identify the realistic field conditions regarding specific terrain and features in real time. Brief explanation of the drawing

[0026] FIG. 1 is a schematic diagram showing the configuration of an event response system (100) to which a real-time visualization map based on image data is applied according to one embodiment of the present invention. FIG. 2 is a flowchart showing the process of generating virtual 3D event environment data corresponding to the event site in sequence in the 3D event environment data generation unit (120). Specific details for implementing the invention

[0027] Hereinafter, specific details for implementing the present invention will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk that the gist of the present invention may be unnecessarily obscured.

[0028] In the attached drawings, identical or corresponding components are given the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0029] The advantages and features of the invented embodiments and the methods for achieving them will become clear by referring to the embodiments described below together with the accompanying drawings. However, the present invention is not limited to the embodiments described below but can be implemented in various different forms, and these embodiments are provided merely to make the present invention complete and to fully inform a person skilled in the art of the scope of the invention.

[0030] The terms used in this specification will be briefly explained, and the invented embodiments will be described in detail. The terms used in this specification have been selected to be as generally used as possible, taking into account their functions in the present invention; however, these terms may vary depending on the intent of those skilled in the relevant field, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined not merely by their names, but based on the meanings they possess and the content of the invention as a whole.

[0031] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0032] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within the 'module' or 'part' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0034] FIG. 1 is a schematic diagram showing the configuration of an event response system (100) to which a real-time visualization map based on image data is applied according to one embodiment of the present invention.

[0035] Referring to FIG. 1, an event response system (100) with a real-time visualization map based on image data according to one embodiment of the present invention may largely include a database (110), a three-dimensional event environment data generation unit (120), a rescue simulation progress unit (130), and a rescue scenario data provision unit (140).

[0036] First, the database (110) can store data on rescue units that can be deployed to the event site. Here, the term "event site" may refer to a place (location) where an event occurs, such as an earthquake, typhoon, storm, flood, landslide, tsunami, heavy snowfall, drought, heatwave, cold wave, industrial accident, traffic accident, building collapse, fire, outbreak of infectious disease, or chemical spill. An earthquake refers to a disaster caused by vibration of the surface of the earth due to crustal deformation, and typhoons and storms are disasters caused by strong winds and rainfall, which may include floods, strong winds, and tsunamis. Floods are regional or global water disasters caused by rainfall or coastal tides. Landslides are disasters caused by the movement of soil in mountainous terrain due to rain, etc., and tsunamis are a type of wave that occurs in the sea due to earthquakes, submarine earthquakes, or ocean volcanic eruptions, which can cause significant damage to coastal areas. Heavy snowfall can cause road closures, power outages, and traffic congestion. Droughts can cause damage to crops and livestock due to insufficient precipitation, while heatwaves and cold waves can lead to human casualties and health problems, respectively.

[0037] Accordingly, the database (110) can store data on various rescue units that can be deployed to the site so that appropriate response measures and rescue operations can be carried out at the event site. This database (110) can obtain rescue unit data from one or more of the following: an external fire department server, an external police department server, an external rescue vehicle support server, an external rescue personnel support server, an external rescue material support server, and an external rescue facility support server. Here, rescue unit data can collectively refer to human and material resources capable of performing appropriate rescue and relief measures at various event sites. For example, it can encompass firefighters, fire trucks, police officers for on-site command, police cars for traffic control, various excavation equipment, various transport vehicles, various construction equipment (cranes, etc.), various towing vehicles, etc., which can be dispatched to the event site to perform rescue and relief measures. The rescue unit data stored in the database (110) can be used to generate three-dimensional event environment data in the three-dimensional event environment data generation unit (120) described later and to perform a rescue simulation in the rescue simulation progress unit (130).

[0038] Meanwhile, in the present invention, the term "three-dimensional event environment data" may refer to a map formed by simulating a three-dimensional (3D) virtual space that is most similar (close to) the event site captured through a camera device, and mapped onto a three-dimensional map so that it can be viewed from any angle.

[0040] The 3D event environment data generation unit (120) can generate virtual 3D event environment data corresponding to the actual event site based on information obtained by photographing the event site. More specifically, the 3D event environment data generation unit (120) can determine the location of the event site and the location of the rescue target based on the event site data and the rescue target data, and then transmit them to a mobile body capable of moving to the location of the event site and the location of the rescue target via one or more paths among a ground path, a sea path, or an aerial path, so that the event site can be photographed through a camera device placed on the mobile body. At this time, the mobile body is as follows.

[0041] The mobile body moves to a location for generating three-dimensional event environment data for an event site and plays the role of transmitting the shooting data generated by shooting the event site in real time to the three-dimensional event environment data generation unit (120) described later. This mobile body may include a camera device, a GPS device that collects the position value of the mobile body in real time, and a LiDAR sensor for determining the height between the mobile body and the ground or the distance to an object (or object) located below the mobile body when the mobile body moves to the event site.

[0042] A mobile entity may refer to a means of transportation that moves to an event site along a ground, sea, or air route. If such a mobile entity moves along a ground route, it may be a radio-controlled car (such as an RC car). If the mobile entity moves along a sea route, it may be a radio-controlled vessel. Additionally, in one embodiment, if the mobile entity moves along an air route rather than land or sea, it may be an unmanned aerial vehicle (UAV), an unmanned drone, or, in some cases, a helicopter. Furthermore, in one embodiment, the mobile entity may be a vehicle or a highly trained animal. Meanwhile, in addition to these embodiments, any means of transportation capable of moving to an event site may be applied as a mobile entity.

[0043] In addition, in one embodiment, the mobile body may include a communication module, a power module, a movement motor, a control module, and a sensor module. The communication module communicates with a GPS device and receives location information from the GPS device. In addition, the communication module may communicate with a control center server and transmit shooting data collected from a camera device to a 3D event environment data generation unit (120). The power module can supply power required for the operation of the movement motor as well as power to all components requiring power within the mobile body. Furthermore, the power module may be a rechargeable power source capable of producing high output and being small in size, such as a lithium-ion battery. The control module controls the movement of the mobile body to perform all controls for the purpose of enabling the mobile body to move to a location for generating a visualization map. The sensor module is mounted on the main body of the mobile body and can collect movement information of the mobile body (e.g., speed information, acceleration information, flight altitude, etc.) and can transmit the collected movement information to the control module.

[0044] In addition, in one embodiment, the control module can calculate the movement path or flight path of the moving body and then transmit information about it to the 3D event environment data generation unit (120) through the communication module.

[0045] A camera device positioned on a moving body moves along with the moving body to photograph the event site and transmits the resulting photographic data in real time to a 3D event environment data generation unit (120). More specifically, the camera device can photograph the situation at the location where the moving body has moved and generate photographic data including a video or multiple images. The photographic data generated in this way can be transmitted in real time to a 3D event environment data generation unit (120). To this end, the camera device may further include a wireless network communication device (not shown). The camera device can transmit the photographic data in real time to a 3D event environment data generation unit (120) through the wireless network communication device.

[0046] In addition, in one embodiment, the camera device may be in the form of a camera installed on an aircraft such as an unmanned aerial vehicle (UAV) or a drone, or may be in the form of a camera installed on a vehicle or a radio-controlled car, and in some cases, the camera device may be in the form of a camera worn or attached to a highly trained animal. That is, regardless of the type of moving object, the camera device can capture the event site and transmit the captured data to the 3D event environment data generation unit (120) in a real-time streaming manner.

[0047] Meanwhile, in one embodiment, the camera device may be connected to one side of the movable body, for example, the lower side, through a separate camera mounting module (not shown). The camera mounting module may be formed to allow the camera device to be attached and detached. In addition, the quality of the image can be improved by stably supporting and securing the camera device to one side of the camera mounting module.

[0048] In addition, in one embodiment, the camera device may stabilize the focus position to improve the quality of the generated shooting data during the process of shooting an event site. For example, the camera device may include a damping means (not shown) for dampening vibrations generated by the moving body when the moving body moves. In this case, the damping means may refer to a spring, a damping rubber pad, etc. In addition, in one embodiment, the damping means may have a plurality of corrugated shapes to absorb vibration energy caused by vibrations or waves generated by the moving body when the moving body moves. Through this, vibrations generated by the moving body can be absorbed by the damping means, thereby eliminating horizontal vibrations, and low-frequency vibrations generated by the moving body can be eliminated by the spring.

[0049] The GPS device can collect location values ​​based on GPS information about a moving object and can transmit them in real time to a 3D event environment data generation unit (120) through a communication module. Based on the location values ​​transmitted through the GPS device, the 3D event environment data generation unit (120) can accurately determine the location at the time of shooting during the process of generating 3D event environment data that matches the shooting data, thereby enabling the generation of more accurate 3D event environment data.

[0050] The lidar sensor can measure the distance value for the vertical downward height of the moving body or the distance and height from terrain and features located below to the moving body during the process in which the moving body moves to a location for generating a visualization map and shooting is performed through a camera device, and transmit this in real time to the 3D event environment data generation unit (120). More specifically, the lidar sensor (114) may refer to an aerial lidar sensor. The lidar sensor is a method of determining the coordinates based on laser irradiation and reflection between the ground object (building, tree, etc.) and the lidar sensor.

[0051] The 3D event environment data generation unit (120) receives shooting data in real time from a camera device placed on a moving object and uses it to generate 3D event environment data corresponding to the shooting data. More specifically, the 3D event environment data generation unit (120) according to the present invention may refer to a type of map generation device. After receiving shooting data from the camera device in a real-time streaming manner, this 3D event environment data generation unit (120) can generate a visualization map corresponding to the shooting data in real time based on a 3D Gaussian Splatting method.

[0052] Here, 3D Gaussian splatting can represent 3D scenes from various viewpoints using millions of particles with position, rotation, size, opacity, and color through multiple photos or videos from multiple viewpoints. Generally, since the NeRF technique uses a probability-based sampling processing technique during rendering, the amount of computation becomes excessive and noise may occur. Accordingly, in the present invention, the 3D event environment data generation unit (120) utilizes a 3D Gaussian splatting method that can drastically reduce the amount of computation and improve quality in terms of rendering computation. This 3D Gaussian splatting is not a surface model (Blender, Maya, etc.), does not use polygons, and has the characteristic of having no textures. It also has the characteristic of not using neural networks. Furthermore, this 3D Gaussian splatting can have the advantage of being usable in real-time (within tens of minutes) compared to existing image processing methods.

[0053] In addition, in one embodiment, the 3D event environment data generation unit (120) may assign a value derived by calibrating the 3D coordinate value of a specific location or object in a virtual space implemented in a visualization map generated by 3D Gaussian splatting and the 3D coordinate value of a rescue target with a location value collected through a GPS device and a distance value (LiDAR value) collected through a LiDAR sensor. At this time, the position accuracy of the specific location or object in the virtual space may change depending on the density of the derived value.

[0054] In addition, the 3D event environment data generation unit (120) according to the present invention can create a virtual space for an event site by using shooting data received in real time from a camera device, position values ​​corresponding to the movement of a moving object during the transmission of said shooting data, and distance values ​​for the vertical downward height of the moving object, so that a 3D virtual space corresponding to the 3D scan of a wide area, rather than a small-scale object, can be formed. The virtual space created at this time may refer to a space created in 3D by simulating the actual space captured through the camera device. In particular, since the 3D event environment data generation unit (120) according to the present invention can reflect position values ​​for the actual position of the moving object in the virtual space created, the placement position and size of the object created in the 3D virtual space can all be created according to a scale corresponding to the actual. In addition, the 3D event environment data generation unit (120) can implement all objects placed in the actual location in 3D correspondingly based on shooting data received in real time from the camera device, and place them in a 3D virtual space.

[0055] Additionally, the 3D event environment data generation unit (120) can implement a virtual space for the event site and, in the process of implementing rescue targets and various objects placed at the event site in 3D and placing them in the virtual space, acquire one or more of the topographic data, plant distribution data, past flood damage record data, and past flood record data for the event site from the database (110) and reflect them when generating the 3D event environment data. More specifically, the 3D event environment data generation unit (1200) can acquire actual topographic data, plant distribution data, past flood damage record data, and past flood record data for the event site from the database (110) and reflect them so that the event site can be expressed more realistically. Here, the topographic data can be used to identify the topography of the event site in the event of an earthquake or landslide, and the plant distribution data can be used to estimate the scale of the wildfire at the event site and the scale of firefighting resources that must be deployed for fire suppression in the event of a wildfire.

[0056] Additionally, the 3D event environment data generation unit (120) can implement a virtual space for the event site and, in the process of implementing objects placed at the event site in 3D and placing them in the virtual space, classify the types of various objects placed at the event site and then implement them in 3D and place them in the virtual space. More specifically, the 3D event environment data generation unit (120) may include a classification module (not shown) for classifying the types of various objects placed at the event site.

[0057] The classification module can receive detection signals for multiple objects located in an area adjacent to the event site from a radar deployed on the moving object. The classification module may include an object classification module that classifies objects as objects such as terrain or features using the received detection signals, and a biological classification module that classifies objects as living things such as trees or animals using the received detection signals. The object classification module can classify detected objects as non-moving objects by receiving and analyzing encrypted radar signals from the radar deployed on the moving object. Such an object classification module may further include a question module and a response module. The question module generates a radar signal in the form of a code propagation of a fixed format, and the response module receives and analyzes the encrypted code propagation transmitted from the target to identify whether the object is moving or not, and can classify the object as a moving living thing based on the identification result.

[0058] In addition, since the area within the shooting data is very extensive, the 3D event environment data generation unit (120) may use a method of dividing the frame-by-frame image of the shooting data into multiple areas and implementing virtual space cells for each divided area, and then merging the virtual space cells for each area into a single virtual space during the process of implementing a virtual space for the said location on a visualization map. For example, the 3D event environment data generation unit (120) may divide the frame-by-frame image constituting the shooting data into the number of areas that are best for optimization and implement virtual space cells for each divided area. After independently performing optimization on each of the multiple virtual space cells created in this way, the multiple virtual space cells for which optimization is completed can be merged together to form a single virtual space.

[0059] In this process, if there is an area that is not captured by the camera device, the particles in that uncaptured area are inevitably expressed in a large and rough form using conventional techniques. However, in order to prevent this, the 3D event environment data generation unit (120) can virtually implement the 3D event environment data and reflect it on the 3D event environment data when information about the continuous area is not found in the captured data during the process of implementing a continuous area on the visualization map.

[0060] More specifically, the 3D event environment data generation unit (120) can determine whether the similarity of boundary vector values ​​between adjacent virtual space cells exceeds a preset error range during the process of merging virtual space cells for each region into the single virtual space. If it is determined that there is an unimplemented region between adjacent virtual space cells, the 3D event environment data generation unit (120) can generate a virtual space cell corresponding to the unimplemented region based on the results of learning the unimplemented region based on deep learning, and then have it additionally reflected between adjacent virtual space cells.

[0061] Here, the 3D event environment data generation unit (120) may include a boundary vector learning model that learns the similarity of boundary vector values ​​between virtual space cells. The boundary vector learning model can be trained through deep learning-based learning, and after receiving boundary vector values ​​between adjacent virtual space cells generated based on captured data as input data, it can determine whether adjacent virtual spaces are connected similarly or correspondingly by comparing them with the training data.

[0062] Additionally, the 3D event environment data generation unit (120) can perform correction processing so that the similarity of the boundary vector values ​​of each virtual space cell corresponding to the additionally reflected unimplemented area between adjacent virtual space cells and the adjacent virtual space cells falls within a preset error range. In addition, in one embodiment, the 3D event environment data generation unit (120) can identify the dimming values ​​of the external geometry for the adjacent virtual space cells and the virtual space cells corresponding to the additionally reflected unimplemented area between adjacent virtual space cells, and if the dimming values ​​for each virtual space cell do not match, it can perform correction processing so that the similarity of the dimming values ​​for each virtual space cell falls within a preset error range. More specifically, the 3D event environment data generation unit (120) can identify the dimming values ​​of the external geometry for each of the multiple virtual space cells generated during the process of implementing virtual space cells for each divided area. Here, the dimming value of the external geometry may refer to a reference value for determining whether the lighting for each virtual space cell is consistent or not. If the lighting for each virtual space cell is uneven, floasters noise, such as black spots, may occur in a single virtual space that has been merged into one. Therefore, the 3D event environment data generation unit (120) can prevent the occurrence of floasters noise in advance by correcting the dimming value for each virtual space cell so that if the dimming value for each virtual space cell is not consistent, the similarity of the dimming value for all virtual space cells comes within a preset error range.

[0063] In addition, in one embodiment, the 3D event environment data generation unit (120) performs preprocessing to contrast the colors of objects identified within the shooting data and can generate numerical data based on object data for the preprocessed objects. At this time, the object data may include coordinate values ​​on a satellite map for the objects. If coordinate values ​​on a satellite map are not identified, the 3D event environment data generation unit (120) can infer and input coordinate values ​​on a satellite map for the objects based on the coordinate values ​​of moving objects on a satellite map, thereby ensuring that objects placed at the actual event site are not omitted.

[0064] In addition, in one embodiment, the 3D event environment data generation unit (120) is connected to a correction position value transmission module that is placed on the ground and has an absolute position value, and the correction position value transmission module can receive a position value from a satellite, output a GPS correction value of the moving body based on the position value, and then wirelessly transmit the output GPS correction value to the 3D event environment data generation unit (120). More specifically, the correction position value transmission module is placed on the ground and, while having an absolute position value, receives a position value from a satellite and then outputs a GPS correction value of the moving body to the 3D event environment data generation unit (120) based on the position value. At this time, the correction position value transmission module may be configured to cross-check and verify whether the GPS device placed on the moving body is properly identifying the position of the moving body.

[0065] To this end, the correction position value transmission module can receive a position value from a satellite while having an absolute position value, and then continuously transmit the GPS correction value of the moving body to the 3D event environment data generation unit (120) at a preset interval based on the position value. Through this, the 3D event environment data generation unit (120) can compare the position value received in real time from the GPS device with the GPS correction value wirelessly transmitted through the correction position value transmission module, and if the difference in position values ​​exceeds a preset error range, it can initiate an operation to reset the GPS device provided in the moving body while maintaining the state of receiving the GPS correction value from the correction position value transmission module. Through this, since the correction position value transmission module can continue to provide the GPS correction value of the moving body to the 3D event environment data generation unit (120) even during the process of resetting the GPS device, the 3D event environment data generation unit (120) does not lose the real-time position value even if the moving body is moving. Once the reset of the GPS device provided in the moving body is completed, the 3D event environment data generation unit (120) continues to perform the process of comparing the position value received in real time from the GPS device with the GPS correction value transmitted wirelessly through the correction position value transmission module.

[0067] The simulation execution unit (130) performs the role of conducting multiple simulations by reflecting event site location information, rescue unit data, and rescue target data on the 3D event environment data generated through the 3D event environment data generation unit (120). More specifically, the simulation execution unit (130) inputs the 3D event environment data generated in real-time through the 3D event environment data generation unit (120) described above into the simulation software, thereby enabling the application of a physical environment suitable for the purpose and allowing a simulation corresponding to that environment to be executed. In addition, the simulation software can be executed to enable the application of various physical environments, and in particular, can be implemented by inputting a physical environment suitable for the purpose of the event site.

[0068] In addition, in one embodiment, the simulation progress unit (130), while conducting the simulation, acquires weather information, ocean information, and sea surface image information regarding the event site from a connected external weather information server as input data. Then, based on the acquired input data, it inputs the weather information, ocean information, and sea surface image information regarding the event site into a machine learning-based weather change estimation model trained to output weather changes according to weather conditions, ocean conditions, and the location of the moving object, thereby acquiring image pattern information regarding the event site and additionally reflecting the image pattern information in the rescue simulation. Here, the external weather information server may refer to a weather agency server. The simulation progress unit (130) can acquire actual weather information, ocean information, and sea surface image information regarding the event site from such external weather information server. The acquired information can be input as input data into the weather change estimation model. After learning from this, the weather change estimation model can output image pattern information regarding the event site. Since the image pattern information can reflect weather changes optimized for the event site, the rescue management terminal at a remote location can understand the situation at the event site more realistically.

[0069] Additionally, in one embodiment, the rescue simulation progress unit (130) may acquire one or more of wildfire simulation data, earthquake simulation data, flood simulation data, landslide simulation data, river or stream flooding simulation data, typhoon simulation data, storm simulation data, tsunami simulation data, heavy snowfall simulation data, heat wave simulation data, cold wave simulation data, collapse simulation data, fire simulation data, infectious disease simulation data, and chemical spill simulation data corresponding to the event site from a connected external event information database, and then additionally reflect them in the rescue simulation along with the image pattern information. More specifically, the rescue simulation progress unit (130) may acquire various simulation data from an external event information database and reflect them in the rescue simulation so that the situation at the event site can be understood more realistically from a rescue management terminal at a remote location. In this case, at the event site, there may have been a wildfire, earthquake, flood, landslide, overflow of a river or stream, typhoon / storm, tsunami, heavy snowfall, heatwave / cold wave, building collapse, large fire, epidemic, or toxic chemical spill.Accordingly, the rescue simulation progress unit (130) incorporates one or more of the following into the rescue simulation based on the results of analyzing the types of events at the event site identified through the camera device, so that the various environments can be simulated and reflected in a manner similar to reality: wildfire simulation data, earthquake simulation data, flood simulation data, landslide simulation data, river or stream flooding simulation data, typhoon simulation data, storm simulation data, tsunami simulation data, heavy snowfall simulation data, heat wave simulation data, cold wave simulation data, collapse simulation data, fire simulation data, infectious disease simulation data, and chemical spill simulation data.

[0070] In addition, in one embodiment, the rescue simulation progress unit (130) may conduct multiple simulations rather than a single simulation while reflecting the event site location information, combat deployment unit information, and object information on the 3D event environment data to conduct the simulation, thereby allowing the operation scenario data provision unit (140), described later, to select the optimal simulation and then generate operation scenario data accordingly and provide it to the operation management terminal. This is described as follows.

[0072] The rescue scenario providing unit (140) can generate a rescue scenario based on the results of the rescue simulation conducted through the rescue simulation conducting unit (130) and provide it to the rescue management terminal. More specifically, the rescue scenario providing unit (140) can quantify the results of the rescue simulation conducted through the rescue simulation conducting unit (130), and generate rescue scenario data based on the rescue simulation result having the highest numerical value and provide it to the rescue management terminal. At this time, the rescue scenario data providing unit (140) can collect the results of the rescue simulation conducted as input data, and input the results of the rescue simulation conducted through the simulation conducting unit (130) into a machine learning-based rescue success rate estimation model trained to output a rescue success rate based on the collected input data, predict and quantify the rescue success rate for each rescue simulation, and then generate rescue scenario data based on the numerical value having the highest rescue success rate among the predicted multiple rescue success rates. Here, a learning processor for performing machine learning can train an artificial neural network using training data or a training set. This learning processor may train an artificial neural network by directly acquiring preprocessed input data, or by acquiring preprocessed input data.

[0073] The rescue scenario providing unit (140) can generate rescue scenario data based on the rescue simulation with the highest rescue success rate among the results of multiple rescue simulations, after quantifying the combat power for each rescue simulation. The generated rescue scenario data can be transmitted to a rescue management terminal. Meanwhile, when the rescue scenario providing unit (140) receives a request from the rescue management terminal to generate new rescue scenario data, it can receive a selection of items to be excluded during the rescue simulation from the previously transmitted rescue scenario data. Through this, the rescue scenario providing unit (140) can provide a new rescue success rate after excluding the items selected by the rescue management terminal during the process of predicting and quantifying the rescue success rate for each new rescue simulation.

[0075] Next, we will examine in order the process of generating virtual 3D event environment data corresponding to the event site based on the shooting data from the 3D event environment data generation unit (120) that was examined earlier.

[0076] FIG. 2 is a flowchart showing the process of generating virtual 3D event environment data corresponding to the event site in sequence in the 3D event environment data generation unit (120).

[0077] Referring to FIG. 2, when a moving body moves to an event site (S201), the camera device starts taking pictures of the terrain and features at the event site to generate shooting data, and transmits the generated shooting data in real time to a 3D event environment data generation unit (120) (S202). At this time, the shooting data may include both the position value of the moving body and the distance value (LiDAR value) between the terrain and features being photographed and the moving body.

[0078] When shooting data is received, the 3D event environment data generation unit (120) generates 3D event environment data by implementing a 3D virtual space corresponding to the shooting data and 3D objects for objects and multiple real objects (S203). In this process, the 3D event environment data generation unit (120) can divide the frame-by-frame image of the shooting data into multiple regions, implement virtual space cells for each divided region, and then merge the virtual space cells for each region into a single virtual space.

[0079] Additionally, the 3D event environment data generation unit (120) identifies the similarity of boundary vector values ​​between adjacent virtual space cells during the process of merging virtual space cells for each region into a single virtual space, and if there is an unimplemented region, it creates a virtual space cell corresponding to the unimplemented region based on the results of learning the unimplemented region based on deep learning and then adds it between adjacent virtual space cells (S204). In this process, the 3D event environment data generation unit (120) can correct the similarity of the boundary vector values ​​of each virtual space cell corresponding to the unimplemented region added between adjacent virtual space cells and the adjacent virtual space cells so that it falls within a preset error range, or it can identify the dimming value of the external geometry for the virtual space cell corresponding to the unimplemented region added between adjacent virtual space cells and the adjacent virtual space cells, and if the dimming values ​​for each virtual space cell do not match, it can correct the dimming value for each virtual space cell so that the similarity of the dimming value for each virtual space cell falls within a preset error range.

[0081] Although the embodiments described above have been described as utilizing aspects of the subject matter currently invented in one or more standalone computer systems, the present invention is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present invention may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.

[0082] Although the present invention has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the invention as understood by a person skilled in the art to which the invention pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification. Explanation of the symbols

[0084] 100: Event response system with video data-based real-time visualization map 110: Database 120: 3D Event Environment Data Generation Unit 130: Rescue Simulation Progress Section 140: Rescue Scenario Data Provider

Claims

Claim 1 A database (110) for storing rescue unit data that can be deployed to an event site; a 3D event environment data generation unit (120) for generating virtual 3D event environment data corresponding to the event site based on event site data obtained by photographing the event site and rescue target data for the rescue target; and a rescue simulation execution unit (130) for conducting multiple rescue simulations by reflecting the rescue unit data and the rescue target data on the generated 3D event environment data. The system includes a rescue scenario data providing unit (140) that generates rescue scenario data based on the results of the rescue simulation and provides it to a rescue management terminal; wherein the 3D event environment data generating unit (120) determines the location of the event site and the location of the rescue target based on the event site data and the rescue target data, and then transmits it to a mobile body capable of moving to the location of the event site and the location of the rescue target via one or more paths among a ground path, a sea path, or an aerial path, so that the event site is photographed through a camera device placed on the mobile body; wherein the mobile body is an unmanned aerial vehicle (UAV), and the camera device is configured to have a 360-degree camera capable of omnidirectional shooting or is arranged in multiple units to enable multi-angle shooting facing different directions, and the camera device is configured to photograph the event site to generate video data or multiple image data and transmit it in real time to the 3D event environment data generating unit (120); and the 3D event environment data generating unit (120) determines the location of the mobile body from a GPS device placed on the mobile body The position value according to movement is collected in real time, and the distance value for the vertical downward height of the moving body is collected in real time from the LiDAR sensor placed on the moving body, and the 3D event environment data generation unit (120) collects shooting data in a real-time streaming manner from the camera device placed on the moving body, andDuring the process of transmitting the above-mentioned shooting data, after receiving the position value corresponding to the position movement of the moving body and the distance value for the vertical downward height of the moving body, the above-mentioned 3D event environment data corresponding to the above-mentioned shooting data is generated based on a 3D Gaussian Splatting method, the event site within the above-mentioned shooting data is analyzed based on the above-mentioned 3D Gaussian Splatting method, a virtual space for the analyzed event site is implemented, and the rescue target placed at the event site is implemented in 3D and placed in the virtual space. The above-mentioned 3D event environment data generation unit (120), in the process of implementing the virtual space for the event site, divides the frame-by-frame image of the above-mentioned shooting data into multiple regions, implements virtual space cells for each divided region, and then merges the virtual space cells for each region into a single virtual space. If the similarity of boundary vector values ​​between adjacent virtual space cells exceeds a preset error range, it is determined that there is an unimplemented region between the adjacent virtual space cells, and the unimplemented region is learned based on deep learning. An event response system with an image data-based real-time visualization map applied, which generates virtual space cells corresponding to the aforementioned unimplemented area based on the learned results and then additionally reflects them between the adjacent virtual space cells. Claim 2 In claim 1, the database (110) is an event response system that applies a real-time visualization map based on image data to acquire the rescue unit data through external linkage or user input. Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 In claim 1, the 3D event environment data generation unit (120) implements a virtual space for the event site and implements the rescue target placed at the event site in 3D and places it in the virtual space, and in the process of obtaining one or more of the topographic data, plant distribution data, past flood damage record data, and past flood record data for the event site from the database (110), and then applies a real-time visualization map based on image data to the event response system that reflects this data when generating the 3D event environment data. Claim 7 delete Claim 8 In claim 1, the 3D event environment data generation unit (120) corrects the boundary vector value of each virtual space cell corresponding to the additionally reflected unimplemented area between the adjacent virtual space cell and the adjacent virtual space cell so that the similarity is within a preset error range, identifies the dimming value of the external geometry for the virtual space cell corresponding to the additionally reflected unimplemented area between the adjacent virtual space cell and the adjacent virtual space cell, and corrects the dimming value of each virtual space cell so that the similarity is within a preset error range when the dimming values ​​for each virtual space cell do not match, thereby creating an event response system with an image data-based real-time visualization map applied. Claim 9 In claim 8, a preprocessing step is performed to contrast the color of the rescue target identified within the above-mentioned shooting data, and numerical data is generated based on the rescue target data for the preprocessed rescue target, wherein the rescue target data includes coordinate values ​​on a satellite map for the rescue target, and if the coordinate values ​​on the satellite map are not identified, the 3D event environment data generation unit (120) infers and inputs the coordinate values ​​on the satellite map for the rescue target based on the coordinate values ​​of the moving object on the satellite map, thereby forming an event response system with an image data-based real-time visualization map applied. Claim 10 In claim 1, the 3D event environment data generation unit (120) is connected to a correction position value transmission module that is placed on the ground and has an absolute position value, and the correction position value transmission module receives a position value from a satellite, outputs a GPS correction value of the moving body based on the position value, and wirelessly transmits the outputted GPS correction value to the 3D event environment data generation unit (120), and when the GPS correction value is received, the 3D event environment data generation unit (120) compares the position value received in real time from the GPS device with the GPS correction value wirelessly transmitted through the correction position value transmission module, and if the difference in position values ​​exceeds a preset error range, maintains the state of receiving the GPS correction value from the correction position value transmission module and resets the GPS device provided in the moving body, an event response system with an image data-based real-time visualization map applied. Claim 11 In claim 1, the rescue simulation progress unit (130) acquires weather information, ocean information, and sea surface image information regarding the event site from a connected external weather information server as input data during the process of conducting the rescue simulation, and then acquires image pattern information regarding the event site by inputting the weather information, ocean information, and sea surface image information regarding the event site into a machine learning-based weather change estimation model trained to output weather conditions, ocean conditions, and weather changes according to the location of the moving object based on the acquired input data, and then additionally reflects the image pattern information in the rescue simulation. This is an event response system with an image data-based real-time visualization map applied. Claim 12 In claim 11, the rescue simulation progress unit (130) acquires one or more of the wildfire simulation data, earthquake simulation data, flood simulation data, landslide simulation data, river or stream flooding simulation data, typhoon simulation data, storm simulation data, tsunami simulation data, heavy snow simulation data, heat wave simulation data, cold wave simulation data, collapse simulation data, fire simulation data, infectious disease simulation data, and chemical spill simulation data corresponding to the event site from a connected external event information database during the process of conducting the rescue simulation, and then adds to the rescue simulation the image data-based real-time visualization map applied to the image pattern information.

Images