Electronic device and system for performing situational assessment operations based on object identification based on edge AI technology

Abstract

The present invention relates to an electronic device for performing a situation assessment operation based on information about an object, the electronic device including a memory, a processor, and a camera, wherein the processor directly processes an image processing-based artificial intelligence model, the processor identifies information about the object using an image of the object captured by the camera as an input to the image processing-based artificial intelligence model, and the processor performs the situation assessment operation based on the identified object information.

Description

[Technical Field]

[0001] The present invention relates to an electronic device and an electronic system for performing situational assessment operations based on object identification based on edge AI technology. [Background technology]

[0002] There is a high possibility of accidents occurring along the coast due to a variety of causes. Therefore, a system is needed that can immediately determine whether people are currently in danger using cameras, etc.

[0003] Therefore, in this invention, we propose a technology that identifies people through artificial intelligence, but can also identify whether movements of objects that do not have a human shape are those of an actual person.

[0004] Recently, parking spaces within buildings are no longer limited to the building's residents or visitors, but are shared with all users, allowing users to use parking spaces and the building manager to earn revenue by using excess parking spaces.

[0005] Therefore, the present invention proposes an apparatus and method for guiding a vehicle to a parking space based on reservation information of the parking space.

[0006] Conventional forest fire detection relies on manual checking of CCTV footage that monitors the mountains, which makes it difficult to immediately identify forest fires, and it is virtually impossible for people to manually check all CCTV footage.

[0007] Therefore, in this invention, we propose a technology that applies Edge AI to detect forest fires through artificial intelligence, but to avoid delays due to data transmission speed in emergency situations such as forest fires.

[0008] With the increase in the number of automobiles, the incidence of traffic accidents is increasing significantly. Therefore, it is becoming increasingly important to clarify who is responsible for an accident through an objective investigation of the accident-related facts. Generally, when a traffic accident occurs, the parties involved agree on whether or not there is fault and the degree of fault, or the police are dispatched to inspect and record the accident situation, determine the circumstances at the time of the accident, and determine whether or not there is fault and the degree of fault. However, this method has the problem of relying on the subjective judgment of the parties involved in the accident and police officers, without providing an accurate analysis of the circumstances of the traffic accident.

[0009] To address this issue, an accident video analysis system and an analysis method using the same that determines the degree of fault by comparing black box video with similar accident video are disclosed in Patent Document 1. However, the invention disclosed in Patent Document 1 still has the problem that the black box video uses spatially limited video and cannot utilize standard traffic accident information.

[0010] Furthermore, while the video surveillance devices that are currently installed and in operation are making progress in providing traffic information through the analysis of real-time road footage, they are not yet capable of analyzing the current situation of traffic accidents in real time. Summary of the Invention [Problem to be solved by the invention]

[0011] One embodiment of the present invention provides an apparatus for providing object identification and advance warning based on edge AI technology.

[0012] An embodiment of the present invention provides an apparatus and method for providing a vehicle parking reservation and guidance service based on artificial intelligence.

[0013] One embodiment of the present invention provides an apparatus and method for precise detection and response to wildfires based on edge artificial intelligence.

[0014] In order to solve the above-mentioned problems, an object of the present invention is to provide a traffic accident analysis system and method for estimating the similarity between actual traffic accident information and standard traffic accident information. [Means for solving the problem]

[0015] To achieve the above-mentioned object, an electronic device according to one embodiment of the present invention includes a memory and a processor connected to the memory, and the processor receives images captured by a camera installed on a target coast, identifies a person in the captured images via an artificial intelligence module, and determines whether the person is in a dangerous situation based on a risk level based on the location and time at which the person was identified.If it is determined that the person is in a dangerous situation, the processor visually displays and outputs the location of the person in the image captured by the camera on a display already installed, and transmits information indicating that the person is in a dangerous situation to an operator terminal used by an operator who operates the electronic device.

[0016] At this time, the processor can control the camera based on weather information so that it operates as a thermal imaging camera in weather conditions where visibility is difficult to ensure, such as fog, or at night.

[0017] At this time, the processor divides the captured image into a plurality of blocks of the size of the first block that has been set in advance, and performs an object detection process on the plurality of blocks in order via an artificial intelligence module to identify the person.

[0018] In this case, the processor receives a first size of the object to be identified, derives a second size when the object is present in the captured image based on the first size and the camera angle, derives the size of the smallest square second block that includes the second size, derives the proportion of the dangerous object and dangerous zone already set at the angle based on the camera angle, and sets the size of the first block based on the size of the second block and the proportion.

[0019] At this time, if the risk level of the first block where the identified person is located exceeds a pre-set critical risk level, the processor can determine that the person is in a dangerous situation.

[0020] In this case, the danger level of the first block can be calculated based on the already set dangerous objects and dangerous areas contained within the first block, the weather of the target coast, the time period in which the person was identified, and the number of times the first person was identified during the already set time period in the first block.

[0021] At this time, the risk level of the first block is calculated by the following formula:

[0022]

number

[0023] RoR (Rate of Risk) means the risk level, NoAPD (Number of Average Person Discrimination) means the average number of people identified during a first period already set in a second block in which a person has been identified at least once among the multiple blocks, NoPD (Number of Person Discrimination) means the number of people identified during a first period already set in the first block, PoRO (Proportion of Risk Object) means the proportion of the risk object and the risk area in the first block, FoT (Factor of Time) means the time risk factor corresponding to the time period in which the person was identified among the time risk factors already assigned from 0 to 5 by time period, and FoW (Factor of Weather) means the weather risk factor corresponding to the weather of the target coast among the weather risk factors already assigned from 0 to 5 by weather.

[0024] In this case, the object detection process extracts a first person object that matches the outline of a person in the block, generates a third block including the first person object, generates a virtual horizontal line based on the center of the third block to divide the block into an upper part and a lower part, generates a virtual vertical line based on the center of the upper part, sets the leftmost point of the first person object in the upper part as a first point, sets the point where the first point and the vertical line are perpendicular to each other as an upper reference point, and defines the time when the first point and the upper reference point are set as A first reference line is generated connecting the first point and the upper reference point at a first time point, and a first transition line is generated connecting the first point and the upper reference point at a second time point after a time interval previously set from the first time point has elapsed. A first angle between the first reference line and the first transition line is derived using the upper reference point as a reference. A rightmost point of the first human object in the upper portion is set as a second point. A second reference line is generated connecting the second point and the upper reference point at the first time point, and a second transition line is generated connecting the second point and the upper reference point at the second time point. a second change line connecting the first and second change lines is generated, a second angle between the second change line and the second reference line is derived based on the upper reference point, an upper change amount is extracted based on the difference between the first angle and the second angle, a virtual vertical line is generated based on the center of the lower portion, the leftmost point of the first human object in the lower portion is set as a third point, a point where the third point and the vertical line are perpendicular is set as a lower reference point, a third reference line connecting the third point and the lower reference point is generated at the first time point, and a second change line connecting the third point and the lower reference point is generated at the second time point. a third transition line connecting the first human object and the second human object, deriving a third angle between the third reference line and the third transition line based on the lower reference point; setting a rightmost point of the first human object in the lower portion as a fourth point; generating a fourth reference line connecting the fourth point and the lower reference point at the first time point; generating a fourth transition line connecting the fourth point and the second human object at the second time point; deriving a fourth angle between the fourth reference line and the fourth transition line based on the lower reference point; and extracting a change amount based on a difference between the third angle and the fourth angle;If the upper level change amount is within a predetermined error range from a previously set reference upper level change amount, and the lower level change amount is within the predetermined error range from a previously set reference lower level change amount, the first human object can be identified as a person.

[0025] To achieve the above-mentioned object, an electronic device according to one embodiment of the present invention includes a memory and a processor connected to the memory, and the processor receives identification information of a vehicle entering a parking lot from a first camera installed at an entrance / exit of the parking lot, confirms reservation information for the reservation time and reserved parking location in response to the identification information, confirms the location of the vehicle from a plurality of second cameras installed inside the parking lot, generates route information based on the location of the entrance / exit of the parking lot and the reserved parking location, generates guidance information based on the location of the vehicle and the route information, and provides the guidance information to a user terminal operating the vehicle.

[0026] At this time, the processor derives a vehicle image from a third camera from among the plurality of second cameras, from which the vehicle is captured, confirms the position of the vehicle based on the position of the third camera and the vehicle image, generates an augmented reality image corresponding to the position of the vehicle, the position of the third camera, and the angle of the third camera based on the route information, and generates the guidance information by superimposing the augmented reality image on the vehicle image.

[0027] In this case, the augmented reality image may be an arrow image indicating where the vehicle should go on the route information based on the position of the vehicle in the vehicle image.

[0028] At this time, the processor assigns a unique identification ID to each parking location in the parking lot, derives the last parking location of the vehicle based on the vehicle image, and compares the first identification ID corresponding to the last parking location with the second identification ID corresponding to the reserved parking location to confirm whether the vehicle has parked in the reserved parking location.

[0029] At this time, if the final parking location of the vehicle is different from the reserved parking location, the processor transmits to the user terminal information indicating that the final parking location is different from the reserved parking location and route information to the reserved parking location; if the final parking location is still different from the reserved parking location after a previously set time has elapsed, the processor generates a first surcharge that reflects the first surcharge information already set on the reserved parking fee corresponding to the reservation information and transmits it to the user terminal; if the vehicle is located at the reserved parking location beyond the reserved time, the processor calculates a first parking fee corresponding to the exceeded time and generates a second surcharge that reflects the second surcharge information already set on the first parking fee and transmits it to the user terminal.

[0030] At this time, the reserved parking fee can be calculated based on the reservation time received from the user terminal, the previously set basic parking fee per hour, the previously set time difference index for each time, and the previously set location difference index for each parking location between the reserved parking location and the parking lot.

[0031] At this time, the reserved parking fee is calculated using the following formula:

[0032]

number

[0033] RPF (Reservation Parking Fee) means the reserved parking fee, IoPS (Index of Parking Space) means the location difference index of the parking space corresponding to the reserved parking location, SPF (Standard Parking Fee) means the basic parking fee per hour that has been set, IoT (Index of Time)_n means the time difference index corresponding to the nth hour during the reserved time, and k may mean the total time (Hour) included in the reserved time.

[0034] At this time, the time difference index is calculated by the following formula:

[0035]

number

[0036] NoOPS (Number of Occupied Parking Space) means the average number of vehicles parked in the parking lot at the nth time, NoTPS (Number of Total Parking Space) means the total number of parking spaces in the parking lot, NoD (Number of Departures)_n means the number of vehicles that exited the parking lot at the nth time, and NoE (Number of Entrances)_n means the number of vehicles that entered the parking lot at the nth time.

[0037] At this time, the position difference index is calculated by the following formula:

[0038]

number

[0039] NoSPS (Number of Selection of Parking Space) means the number of times all users during a set period have selected to reserve a parking space corresponding to the reserved parking location, and NoASPS (Number of Average Selection of Parking Space) can mean the average number of times all users during a set period have selected to reserve all parking spaces in the parking lot.

[0040] At this time, if the vehicle's identification information corresponds to a vehicle for which a temporary stop has already been set, the processor can set the temporary stop location already set within the parking lot as the reserved parking location for the vehicle.

[0041] To achieve the above-mentioned object, an electronic device according to an embodiment of the present invention includes a memory and a processor connected to the memory, and the processor receives photographic data from a camera installed on a mountain, analyzes the photographic data via an artificial intelligence module to extract a gas mass object, analyzes the photographic data and the gas mass object to determine whether the gas mass object is caused by a fire, and if it is determined that the gas mass object is caused by a fire, derives a source point of the gas mass object and transmits the source point and the fire determination result to an operator terminal.

[0042] At this time, if the processor determines that the gas mass object is caused by a fire, it can send a control command to have an already installed drone photograph the source point, and transmit the drone photography data photographed by the drone to the operator terminal.

[0043] At this time, the processor divides the photographed data into a first region corresponding to a mountain and a second region corresponding to other regions, sets the narrower portion of the photographed data as the source direction reference based on the width of the upper and lower parts of the gas mass object, and if the frame of the gas mass object is entirely contained within the first region, determines that the gas mass object originated from a mountain, and if the frame of the gas mass object is in contact with the boundary between the first region and the second region and the source direction reference is located on the first region side, determines that the gas mass object originated from a mountain, and if it is determined that the gas mass object originated from a mountain, determines whether the gas mass object is a stain or a real gas such as smoke or fog, and if it is determined that the gas mass object is a real gas such as smoke or fog, receives thermal image data from the camera and determines whether the gas mass object is caused by a fire based on the thermal image data.

[0044] At this time, the processor receives information about wind direction and wind speed from an air flow meter installed on the mountain to determine whether the gas mass object is a stain or real gas such as smoke or fog, derives a first object center where a first maximum width and a first maximum height of the gas mass object meet at a first time point, derives a second object center where a second maximum width and a second maximum height of the gas mass object meet at a second time point after a previously set time has elapsed from the first time point, compares a previously set first center movement distance on the camera angle with a second center movement distance between the first object center and the second object center on the camera angle based on the wind speed, compares the wind direction with the direction of the second center movement distance, and determines whether the direction of the second center movement distance is within a previously set first error range from the wind direction. When the second center movement distance is within a predetermined second error range with the first center movement distance, the gas mass object is determined to be a real gas such as smoke or fog. When the direction of the second center movement distance is within the first error range with the wind direction but the second center movement distance is outside the second error range with the first center movement distance, one of the longest widths or longest heights that is closest to the wind direction is selected, and a gas diffusion change rate corresponding to the ratio of the selected first longest width or first longest height to the corresponding second longest width or second longest height is derived. When the gas diffusion change rate is within a predetermined third error range with a predetermined reference diffusion change rate so as to correspond to the diffusion degree of smoke or fog based on the wind speed, the gas mass object is determined to be a real gas such as smoke or fog.

[0045] At this time, in order to determine whether the gas mass object is caused by a fire, the processor generates a first block in which the gas mass object is included in the photographing data, the first block having the maximum width and maximum length of the gas mass object as its horizontal length and vertical length, divides the first region into a plurality of second blocks having the same size as the first block, divides the second region into a plurality of third blocks having the same size as the first block, derives a first maximum temperature from the first block based on the thermal imaging data, derives a second maximum temperature for each of the second blocks based on the thermal imaging data and calculates a first average value corresponding to the average of the second maximum temperatures, derives a third maximum temperature for each of the third blocks based on the thermal imaging data and calculates a second average value corresponding to the average of the third maximum temperatures, and if the first maximum temperature exceeds a previously set first critical temperature, determines that the gas mass object is caused by a fire, but compares the first maximum temperature with the first average value, and if the difference between the first maximum temperature and the first average value is within a previously set fourth error range, determines that there is a possibility of a large-scale forest fire. and transmits information indicating that a fire has occurred to a pre-set related organization or control center and the operator terminal; if a difference between the first maximum temperature and the first average value is outside the fourth error range, it determines that the fire is localized, transmits a control command to have the drone photograph the point of origin, and transmits drone photographed data photographed by the drone to the operator terminal; if the first maximum temperature is equal to or lower than the first critical temperature and exceeds a pre-set second critical temperature that is lower than the first critical temperature, it compares the first maximum temperature with the first average value and the second average value, and if a difference between the first maximum temperature, the first average value, and the second average value is within the fourth error range, it determines that the temperature rise is due to weather, and transmits the determination result to the operator terminal; if the first maximum temperature and the first average value are within the fourth error range and the first maximum temperature and the second average value are outside the fourth error range, it determines that there is a high possibility of a fire, transmits a control command to have the drone photograph the point of origin, and transmits drone photographed data photographed by the drone to the operator terminal.

[0046] In order to achieve the above-mentioned object, a traffic accident analysis system according to an embodiment of the present disclosure may include: a dataset construction unit that constructs each standard traffic accident dataset from a plurality of standard traffic accident information; an omniscient representation unit that maps video data of actual traffic accident information taken on a road onto a two-dimensional planar map; and a traffic accident estimation unit that compares the actual traffic accident information mapped by the omniscient representation unit with each standard traffic accident dataset constructed by the dataset construction unit and estimates standard traffic accident information similar to the actual traffic accident information.

[0047] The data set construction unit may classify the types of traffic accidents from each standard traffic accident information and perform labeling according to the classified traffic accident types to construct the respective standard traffic accident data sets.

[0048] The types of traffic accidents may include vehicle-to-vehicle, vehicle-to-person, vehicle-to-motorcycle, and vehicle-to-bicycle.

[0049] The dataset constructed by the dataset construction unit may be a dataset constructed to include not only the video data of the standard traffic accident information but also text data that is explanatory material explaining the traffic accident.

[0050] The omniscient representation unit may identify the frame of an object from the actual traffic accident video and display a bounding box to identify the direction and angle of the object.

[0051] The omniscient representation unit can display circular distance lines at predetermined distance intervals on the actual traffic accident video, and perform mapping on the two-dimensional planar map taking into account distance and angle using the displayed circular distance lines.

[0052] The actual traffic accident information may include information such as traffic signals at the time of the traffic accident video.

[0053] The traffic accident estimation unit may include a unit for extracting image features and a unit for classifying images in a manner that effectively recognizes and emphasizes features between adjacent images while maintaining spatial information of the image.

[0054] The traffic accident analysis system may include a video monitoring device and a management server, the video monitoring device may include the omni-intelligent representation unit and the traffic accident estimation unit, and the management server may include the dataset construction unit.

[0055] The traffic accident estimation unit of the video monitoring device can provide standard traffic accident information of a situation similar to actual traffic accident information and a similarity to the actual traffic accident information to the user terminal.

[0056] A traffic accident analysis method in a traffic accident analysis system according to another embodiment of the present disclosure may include the steps of constructing respective standard traffic accident datasets from a large number of standard traffic accident information pieces; mapping video data of actual traffic accident information taken on a road onto a two-dimensional planar map; and comparing the actual traffic accident information mapped in the mapping step with each standard traffic accident dataset constructed in the constructing step to estimate standard traffic accident information similar to the actual traffic accident information.

[0057] To solve this problem, an electronic device for performing a situation assessment operation based on information about an object is provided, which includes a memory, a processor, and a camera, wherein the processor directly processes an image processing-based artificial intelligence model, the processor uses an image of the object captured by the camera as an input to the image processing-based artificial intelligence model to identify information about the object, and the processor performs the situation assessment operation based on the identified object information.

[0058] To solve this problem, an electronic device for performing a situation assessment operation based on information about an object is provided, which includes a memory, a processor, and a camera, wherein the processor independently processes an image processing-based artificial intelligence model, the processor identifies information about the object using an image of the object captured by the camera as an input to the image processing-based artificial intelligence model, and the processor performs the situation assessment operation based on the identified object information.

[0059] To solve this problem, an electronic device for performing a situation assessment operation based on information about an object is provided, which includes a memory, a processor, and a camera, and the electronic device is installed locally rather than in a central control unit on a network and can implement artificial intelligence-based control and / or processing, the processor processes an image processing-based artificial intelligence model, the processor identifies information about the object using an image of the object captured by the camera as an input to the artificial intelligence model, and the processor performs the situation assessment operation based on the information about the identified object.

[0060] To solve this problem, the processor receives vehicle identification information and vehicle location information from the camera, checks reservation information for the vehicle in response to the received identification information, and generates route information and / or guidance information for the vehicle. The present invention provides an electronic device for performing situation judgment operations based on information about an object.

[0061] In order to solve this problem, an electronic system for performing situation assessment operations based on information about an object includes a first electronic device, a second electronic device, and a third electronic device, each including a processor that receives vehicle identification information and vehicle location information from the camera, checks reservation information for the vehicle corresponding to the received identification information, and generates route information and / or guidance information for the vehicle, wherein the processor of the first electronic device receives information about the vehicle from the camera of the first electronic device, the processor of the second electronic device checks the position of the vehicle from the camera of the second electronic device and the camera of the third electronic device, the camera of the third electronic device derives an image of the vehicle, and the processor of the third electronic device derives the final parking location of the vehicle based on the derived image.

[0062] To solve this problem, the present invention provides an electronic device for performing situation assessment operations based on information about an object, in which the processor constructs a dataset by comparing and evaluating a large amount of actual data, learns using the constructed dataset, and estimates the similarity with an image of the object captured by the camera.

[0063] To solve the problem, the processor applies a convolutional neural network to estimate the similarity between the image of the object captured by the camera and an electronic device for performing situation assessment operations based on information about the object is provided.

[0064] To solve this problem, the processor divides the image of the object captured by the camera into a plurality of blocks, performs an object detection process on the plurality of blocks, and provides an electronic device for performing a situation assessment operation based on information about the object.

[0065] To achieve this object, the processor provides an electronic device for performing situational assessment operations based on object information, the electronic device being capable of controlling the camera to operate as a thermal imaging camera.

[0066] To solve the problem, the processor identifies a person through the object detection process, and if the danger level of the block where the identified person is located exceeds a predetermined critical danger level, the processor transmits information indicating that the identified person is in a dangerous situation to a user terminal.

[0067] To solve this problem, the processor analyzes the image of the object captured by the camera to extract a gas mass object, and if it determines that the gas mass object is real gas, it receives thermal image data from the camera and provides an electronic device for performing situation assessment operations based on information about the object.

[0068] To solve this problem, the processor constructs a dataset from standard traffic accident information, maps an image of the object in the actual traffic accident information captured by the camera on a two-dimensional planar map, compares the dataset with the actual traffic accident information, and estimates the standard traffic accident information that is similar to the actual traffic accident information, thereby providing an electronic device for performing situation assessment operations based on object information. [Effects of the Invention]

[0069] As such, according to one embodiment of the present invention, a device that provides object identification and advance warning based on edge AI technology can be provided.

[0070] As described above, according to one embodiment of the present invention, an apparatus and method for providing a vehicle parking reservation and guidance service based on artificial intelligence can be provided.

[0071] As such, according to one embodiment of the present invention, an apparatus and method for precise detection and response to forest fires based on edge artificial intelligence can be provided.

[0072] With the above-described configuration, the present invention can objectively determine whether or not there is liability for negligence and the degree of negligence by analyzing actual traffic accident information by estimating similarity based on standard traffic accident information.

[0073] The present invention also enables a two-dimensional analysis of traffic accidents by mapping video data of actual traffic accident information onto a two-dimensional map, thereby enabling more accurate estimation of the type of traffic accident.

[0074] The present invention also enables safety diagnosis and prevention of traffic safety solutions by constructing a data set based on the type of traffic accident as basic data for preventing, diagnosing, responding to, and predicting traffic accidents. [Brief explanation of the drawings]

[0075] [Figure 1] 1 is a conceptual diagram of a coastal person identification and disaster advance warning device according to an embodiment of the present invention; [Figure 2] 1 is a block diagram of an electronic device according to one embodiment of the present invention. [Figure 3] 1 is a diagram illustrating an example of dividing a captured image according to an embodiment of the present invention; [Figure 4] 10 is an example diagram illustrating generating a first block size according to an embodiment of the present invention. [Figure 5] 1 is a table illustrating an example of weather risk coefficients and time risk coefficients according to an embodiment of the present invention. [Figure 6] 1 is an exemplary diagram of identifying a person according to an embodiment of the present invention; [Figure 7] FIG. 10 is a conceptual diagram of yet another embodiment of the object detection process of the present invention. [Figure 8] FIG. 10 is a conceptual diagram of yet another embodiment of the object detection process of the present invention. [Figure 9]2 is a flowchart of a coastal person identification and disaster advance warning method according to an embodiment of the present invention. [Figure 10] 1 is a conceptual diagram of an AI-based vehicle parking reservation and guidance service providing device according to an embodiment of the present invention; [Figure 11] 1 is a block diagram of an electronic device according to one embodiment of the present invention. [Figure 12] 1 is a diagram illustrating generation of route information according to an embodiment of the present invention; [Figure 13] 1 is a diagram illustrating a method for determining a vehicle location according to an embodiment of the present invention; [Figure 14] 1 is a diagram illustrating guidance information in which an augmented reality image is superimposed on a vehicle image according to an embodiment of the present invention; [Figure 15] 1 is a flowchart of a method for providing a vehicle parking reservation and guidance service based on artificial intelligence according to an embodiment of the present invention. [Figure 16] 1 is a conceptual diagram of an edge artificial intelligence-based forest fire precision detection and response device according to an embodiment of the present invention. [Figure 17] 1 is a block diagram of an electronic device according to one embodiment of the present invention. [Figure 18] 1 is a diagram illustrating an example of a captured image according to an embodiment of the present invention; [Figure 19] 1 is an exemplary diagram illustrating determining the origin of a gas mass object according to an embodiment of the present invention; [Figure 20] 10 is an exemplary diagram illustrating a method for determining whether a mass of gas object is real gas according to an embodiment of the present invention; [Figure 21] 10 is an exemplary diagram illustrating a method for determining whether a mass of gas object is real gas according to an embodiment of the present invention; [Figure 22] 10 is an exemplary diagram illustrating a process for determining whether a gas mass object is generated by a fire according to an embodiment of the present invention; [Figure 23] 1 is a flowchart of a method for accurately detecting and responding to forest fires based on edge artificial intelligence according to an embodiment of the present invention. [Figure 24]1 is a diagram illustrating a traffic accident analysis system according to an embodiment of the present disclosure. [Figure 25] 25 is a block diagram illustrating a case where the video monitoring device or the management server shown in FIG. 24 operates as a general electronic device. [Figure 26] 1 is a diagram illustrating a block diagram of a traffic accident analysis system according to another embodiment of the present disclosure. [Figure 27] 1 is a diagram illustrating an example of two-dimensional images of standard traffic accidents classified by type and / or situation according to an embodiment of the present disclosure. [Figure 28] 1 is a diagram illustrating an example of matching a camera image to a Naver map according to an embodiment of the present disclosure. [Figure 29] 1 is a diagram illustrating an example of matching a camera image to a Naver map according to an embodiment of the present disclosure. [Figure 30] 1 is a diagram illustrating an example of estimating a standard traffic accident type of an actual traffic accident situation according to an embodiment of the present disclosure. [Figure 31] 1 is a diagram illustrating an example of displaying a standard traffic accident estimation result of an actual traffic accident situation according to an embodiment of the present disclosure. [Figure 32] 10 is a flowchart illustrating a traffic accident analysis method according to another embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0076] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0077] In describing the embodiments, technical details that are well known in the technical field to which the present invention pertains and are not directly related to the present invention will be omitted in order to more clearly convey the gist of the present invention without obscuring it.

[0078] For the same reason, in the accompanying drawings, some components are exaggerated, omitted, or illustrated schematically, and the size of each component does not entirely reflect the actual size. The same or corresponding components in each drawing are given the same reference numerals.

[0079] The advantages and features of the present invention, as well as methods for achieving them, will become clearer with reference to the following detailed description of the embodiments in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and may be embodied in various different forms. The present embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully convey the scope of the invention to those skilled in the art to which the present invention pertains. The present invention is defined only by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

[0080] In this regard, each block of the process flowchart diagrams and combinations of flowchart diagrams may be implemented by computer program instructions. These computer program instructions may be loaded onto a processor in a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that the instructions, executed by the processor of the computer or other programmable data processing device, create means for performing the functions described in the flowchart blocks. These computer program instructions may also be stored in computer-usable or computer-readable memory that can direct the computer or other programmable data processing device to implement functions in a particular manner, such that the instructions stored in the computer-usable or computer-readable memory can produce an article of manufacture containing instruction means for performing the functions described in the flowchart blocks. Computer program instructions may also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable data processing device to create a computer-implemented process, causing the computer or other programmable data processing device to execute instructions that provide steps for performing the functions described in the flowchart blocks.

[0081] Also, each block may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specified logical function. Also, it should be noted that in some alternative implementations, the functions described in the blocks may occur out of order. For example, two successive blocks may be performed substantially simultaneously, or the blocks may sometimes be performed in reverse order depending on the corresponding function.

[0082] The term "module" used in this embodiment refers to software or hardware components such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC), and the "module" performs a certain function. However, the term "module" is not limited to software or hardware. The "module" may be configured to reside on an addressable storage medium or to implement one or more processors. Thus, by way of example, the "module" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided by the components and "modules" may be combined into fewer components and "modules" or further separated into additional components and "modules." Furthermore, the components and "modules" may be embodied to implement one or more CPUs within a device or a security multimedia card.

[0083] In describing the embodiments of the present invention in detail, we will focus primarily on examples of specific systems. However, the main gist of what we are trying to claim in this specification can be applied to other communication systems and services having similar technical backgrounds without departing significantly from the scope disclosed in this specification, and this would be within the judgment of a person skilled in the art. Device that provides object identification and advance warning based on edge AI technology FIG. 1 is a conceptual diagram of a coastal person identification and disaster pre-warning device 100 according to an embodiment of the present invention.

[0084] 1, a coastal person identification and disaster pre-warning device 100 according to an embodiment of the present invention can perform a situation determination operation based on object information. Furthermore, the coastal person identification and disaster pre-warning device 100 can be installed locally, rather than as a central control device on a network, and can implement control and / or processing based on artificial intelligence. Specifically, the coastal person identification and disaster pre-warning device 100 can identify people on the coast using a camera, determine whether the people are in a dangerous situation, and provide a warning about the dangerous situation to an adjacent display and an operator terminal of an operator who operates the electronic device 100 based on the determination result.

[0085] Meanwhile, the coastal person identification and disaster pre-warning device 100 may also be referred to as the "electronic device 100" in the present invention.

[0086] In this case, the operator terminal may include a communication-enabled desktop computer, laptop computer, notebook, smartphone, tablet PC, mobile phone, smart watch, smart glass, e-book reader, portable multimedia player (PMP), portable game console, navigation device, digital camera, digital multimedia broadcasting (DMB) player, digital audio recorder, digital audio player, digital video recorder, digital video player, personal digital assistant (PDA), etc.

[0087] FIG. 2 is a block diagram of an electronic device 100 according to one embodiment of the present invention.

[0088] According to an embodiment, the electronic device 100 includes a processor 110 and a memory 120. The electronic device 100 may also include the processor 110, the memory 120, and a camera. The processor 110 may perform at least one of the methods described above. The memory 120 may store information related to the methods described above or a program implementing the methods described above. The memory 120 may be a volatile memory or a non-volatile memory. The memory 120 may also be referred to as a "database," a "storage unit," etc.

[0089] The processor 110 can execute programs and control the electronic device 100. The code of the programs executed by the processor 110 may be stored in the memory 120. The electronic device 100 can be connected to an external device (e.g., a personal computer or a network) via an input / output device (not shown) to exchange data with the external device.

[0090] At this time, the processor 110 can receive images captured by cameras installed on the coast of the target area.

[0091] In this case, the camera may be a camera that functions as both a general visible light camera and a thermal imaging camera, or a camera that can perform both functions. That is, the processor 110 can control the camera to operate as a thermal imaging camera.

[0092] At this time, the processor 110 controls the camera to operate as a thermal imaging camera based on weather information when visibility is poor due to fog or at night, thereby enabling people to be identified even in poor visibility (visible light).

[0093] In addition, the processor 110 can directly process an image processing-based artificial intelligence model or can independently process an image processing-based artificial intelligence model. The processor 110 can identify information about an object by inputting an image of the object captured by a camera to the image processing-based artificial intelligence model. Specifically, the processor 110 can identify a person in the captured image through an artificial intelligence module and determine whether the person is in a dangerous situation based on a risk level based on the location and time at which the person was identified.

[0094] In this case, the artificial intelligence module can learn human images and identify people in captured images using deep learning techniques, which are a type of machine learning.

[0095] In addition, the AI ​​module can calculate weights for multiple inputs in the function through learning through deep learning. Various AI network models, such as recurrent neural networks (RNNs), deep neural networks (DNNs), and dynamic recurrent neural networks (DRNNs), can be used for such learning.

[0096] Here, RNN is a deep learning technique that simultaneously considers current data and past data, and a recurrent neural network (RNN) is a neural network in which the connections between units constituting the artificial neural network form a directed cycle. Various methods may be used to construct a recurrent neural network (RNN), including a fully recurrent network, a Hopfield network, an Elman network, an echo state network (ESN), a long short-term memory network (LSTM), a bidirectional RNN, a continuous-time RNN (CTRNN), a hierarchical RNN, and a second-order RNN. Methods such as gradient descent, Hessian-free optimization, and global optimization may be used to train a recurrent neural network (RNN).

[0097] However, the AI ​​module has a problem in that it may be difficult to distinguish between actual human movements and human-like objects (e.g., standing signs). Therefore, the present invention proposes a configuration for identifying people through their movements, which will be described later.

[0098] In addition, the processor 110 may perform a situation determination operation based on information about the identified object. Specifically, if the processor 110 determines that a person is in a dangerous situation, the processor 110 may visually display and output the position of the person in the image captured by the camera on a pre-installed display.

[0099] At this time, the display may output guidance information that can be provided in the corresponding coastal area during normal times, such as geographical guidance and weather information.

[0100] In addition, the processor 110 may perform a situation determination operation based on information about the identified object. Specifically, the processor 110 may transmit information indicating that a person is in a dangerous situation to an operator terminal used by an operator who operates the electronic device 100.

[0101] This allows the system to rescue people in danger or guide them out of the area.

[0102] FIG. 3 is an exemplary diagram illustrating how a captured image is divided according to an embodiment of the present invention.

[0103] For accurate person identification through the artificial intelligence module, the captured image needs to be divided into certain parts and analyzed intensively.

[0104] To this end, the processor 110 may divide an image of an object captured by a camera into a plurality of blocks and run an object detection processor on the plurality of blocks. Specifically, the processor 110 may divide the captured image into a plurality of blocks each having a predetermined first block size, and run an object detection process on the plurality of blocks in order via an artificial intelligence module to identify the person. At this time, the object detection process will be described below.

[0105] In addition, as will be described later, the size of the block can be determined based on the proportion of dangerous objects (tetrapods) or dangerous areas (tidal flats) in the captured image.

[0106] FIG. 4 is an example diagram illustrating how to generate the size of a first block according to an embodiment of the present invention.

[0107] Dangerous objects and dangerous areas occupy a very large proportion of the captured image, but if the size of the block is determined only based on the size of the object to be identified, unimportant parts may be searched, resulting in a waste of resources.

[0108] For this reason, the size of the first block needs to be adjusted appropriately.

[0109] To this end, the processor 110 receives a first size of the object to be identified, derives a second size of the object when it is present in the captured image based on the first size and the camera angle, derives the size of the smallest square-shaped second block that includes the second size, derives the proportion of the dangerous object and dangerous zone that have already been set from the angle based on the camera angle, and sets the size of the first block based on the size of the second block and the proportion.

[0110] At this time, the size of the first block can be set to a value obtained by multiplying the size of the second block by the reciprocal of the ratio.

[0111] In addition, the processor 110 may identify a person through an object detection process, and if the risk level of a block where the identified person is located exceeds a pre-set critical risk level, may transmit information indicating that the identified person is in a dangerous situation to the user terminal 200. Specifically, if the risk level of a first block where the identified person is located exceeds a pre-set critical risk level, the processor 110 may determine that the identified person is in a dangerous situation.

[0112] In this case, the danger level of the first block can be calculated based on the already set dangerous objects and dangerous areas contained within the first block, the weather of the target coast, the time period in which the person was identified, and the number of times the first person was identified in the already set time period in the first block.

[0113] In more detail, the risk level of the first block can be calculated by the following Equation 1.

[0114] [Formula 1]

number

[0115] In this case, RoR (Rate of Risk) means the risk level, NoAPD (Number of Average Person Discrimination) means the average number of people identified during a first period already set in a second block in which a person has been identified at least once among the multiple blocks, NoPD (Number of Person Discrimination) means the number of people identified during a first period already set in the first block, PoRO (Proportion of Risk Object) means the proportion of the risk object and the risk area in the first block, FoT (Factor of Time) means the time risk factor corresponding to the time period in which the person was identified among the time risk factors already assigned from 0 to 5 by time period, and FoW (Factor of Weather) means the weather risk factor corresponding to the weather of the target coast among the weather risk factors already assigned from 0 to 5 by weather.

[0116] At this time, the weather risk coefficient and the time risk coefficient may be arbitrarily set by the operator as shown in FIG.

[0117] FIG. 6 is an exemplary diagram of identifying a person according to an embodiment of the present invention.

[0118] As mentioned above, if a simple object having the shape of a person is identified, there is a risk of false alarms, so it is necessary to check based on movement to confirm whether it is an actual person.

[0119] To this end, as described above, the processor 110 identifies a person through the object detection process, which extracts a first person object that matches the outline of a person in the block, generates a third block including the first person object, generates a virtual horizontal line based on the center of the third block to divide the upper and lower parts, generates a virtual vertical line based on the center of the upper part, sets the leftmost point of the first person object in the upper part as a first point, and a point where the vertical line is perpendicular to the first point is set as an upper reference point; a time point at which the first point and the upper reference point are set is set as a first time point; a first reference line connecting the first point and the upper reference point is generated; a first transition line connecting the first point and the upper reference point is generated at a second time point at which a previously set time interval has elapsed from the first time point; a first angle between the first reference line and the first transition line is derived based on the upper reference point; and a rightmost point of the first human object in the upper portion is set as a second point; a second reference line connecting the second point and the upper reference point at a first time point; a second change line connecting the second point and the upper reference point at a second time point; a second angle between the second reference line and the second change line based on the upper reference point; an upper change amount is extracted based on a difference between the first angle and the second angle; a virtual vertical line is generated based on a center of the lower portion; a leftmost point of the first human object in the lower portion is set as a third point; a point where the third point and the vertical line are perpendicular is set as a lower reference point; generating a third reference line connecting the third point and the lower reference point at the first time point, generating a third change line connecting the third point and the lower reference point at the second time point, deriving a third angle between the third reference line and the third change line using the lower reference point as a reference, setting a rightmost point of the first human object in the lower portion as a fourth point, generating a fourth reference line connecting the fourth point and the lower reference point at the first time point, and generating a fourth change line connecting the fourth point and the lower reference point at the second time point;A fourth angle between the fourth reference line and the fourth change line is derived based on the lower reference point, and a change amount is extracted based on a difference between the third angle and the fourth angle. If the upper change amount is within a predetermined error range with a previously set reference upper change amount and the lower change amount is within the predetermined error range with a previously set reference lower change amount, the first human object can be identified as a person.

[0120] In this case, the reference upper change amount and the reference lower change amount may be set as average values ​​of the upper change amount and the lower change amount when a person moves.

[0121] The error range may be set arbitrarily by the operator, and may be set to, for example, 10%.

[0122] Rather than simply judging someone to be a person based on their movements, this technology focuses on the fact that when a person moves or acts, the left and right sides move in a correlated manner, making it possible to identify actual human movements and improve accuracy.

[0123] The object detection process described above can also be applied to identify people inside a building to determine their location in the event of a fire, or to determine whether people in an industrial area have put on safety equipment.

[0124] Looking more specifically at FIG. 7, an example of an AI-based building fire safety solution system is an electronic device that includes a memory and a processor connected to the memory, and the processor receives images captured by a camera installed at the entrance and exit of a building, identifies people from the images, counts the number of people entering and exiting the building, and checks the number of people remaining in the building in the event of an emergency such as a fire.

[0125] In this case, the person can be identified from the photographed image through the object identification process described above.

[0126] Additionally, via an artificial intelligence module, facial parts of the identified person may be separately analyzed to allow identification.

[0127] At this time, the identity verification database can continuously update information on new and former employees to accurately determine whether they are allowed to enter or exit the building.

[0128] In addition, in order for the system to operate smoothly even in the event of a fire, it is preferable that the cables connecting the system to the camera or the antennas for wireless connection are made of flame-retardant cables and materials.

[0129] In addition, since the power supply may be unstable in an emergency such as a fire, the system may be configured to include a UPS (Uninterruptible Power Supply system), i.e., an uninterruptible power supply, and operate on an independent power source.

[0130] In another embodiment, referring to FIG. 8, an AI-based industrial safety management system may be exemplified. The system may include a memory and a processor connected to the memory in an electronic device. The processor receives images captured by cameras installed in an industrial complex, identifies people from the images via an AI module, calculates the risk level of the industrial complex where the cameras are installed based on the proportion of dangerous objects and dangerous areas based on the camera angle, and provides a warning message to an operator (manager, management director, etc.) if a person is identified in a high-risk location.

[0131] In this case, a configuration for identifying a person from the photographed image can be implemented by identifying the person through the object detection process described above.

[0132] In addition, the AI ​​module can analyze the clothing of the person identified to check whether the person is wearing the required clothing or equipment (X-band, helmet, goggles) within the industrial complex where the camera is installed, and if the clothing or equipment is not properly worn, a warning message can be sent to the operator.

[0133] FIG. 9 is a flowchart of a coastal person identification and disaster pre-warning method according to an embodiment of the present invention.

[0134] Referring to FIG. 9, a coastal person identification and disaster advance warning method according to an embodiment of the present invention can receive a photographed image from a camera installed on a target coast (S101).

[0135] In addition, the coastal person identification and disaster advance warning method according to an embodiment of the present invention can identify people from the photographed image through an artificial intelligence module (S103).

[0136] In addition, the coastal person identification and disaster advance warning method according to one embodiment of the present invention can determine whether the person is in a dangerous situation based on the risk level based on the location and time at which the person was identified (S105).

[0137] In addition, in the coastal person identification and disaster advance warning method according to one embodiment of the present invention, if it is determined that the person is in a dangerous situation, the position of the person can be visually displayed and output on a pre-installed display in an image captured by the camera, and information indicating that the person is in a dangerous situation can be transmitted to an operator terminal used by an operator who operates the electronic device 100 (S107).

[0138] Furthermore, the coastal person identification and disaster advance warning method according to an embodiment of the present invention may be configured in the same manner as the coastal person identification and disaster advance warning device 100 disclosed in FIGS.

[0139] Apparatus and method for providing edge AI-based vehicle parking reservation and guidance service FIG. 10 is a conceptual diagram of an AI-based vehicle parking reservation and guidance service providing device according to an embodiment of the present invention.

[0140] 10, the AI-based vehicle parking reservation and guidance service providing device 100-2 according to an embodiment of the present invention may perform a situation determination operation based on object information. Furthermore, the AI-based vehicle parking reservation and guidance service providing device 100-2 may be installed locally, rather than as a central control device on a network, and may implement AI-based control and / or processing. Specifically, the AI-based vehicle parking reservation and guidance service providing device 100-2 may provide guidance to a user terminal 200 so that a vehicle of a user who has reserved a parking space and time can park in the reserved parking space, thereby reducing vehicle travel time and thereby reducing exhaust gas emissions, preventing environmental pollution, and preventing illegal parking.

[0141] Meanwhile, the AI-based vehicle parking reservation and guidance service providing device may also be referred to as the "electronic device 100-2" in the present invention. In addition, since the AI-based device is installed in various places in the parking lot (e.g., ceiling, pillars, walls, etc.) rather than as a central control device, and each device can implement AI-based control and / or processing, it may also be referred to as an "edge AI device."

[0142] FIG. 11 is a block diagram of an electronic device 100-2 according to one embodiment of the present invention.

[0143] According to one embodiment, the electronic device 100-2 includes a processor 110-2 and a memory 120-2. The electronic device 100-2 may also include a processor 110-2, a memory 120-2, and a camera.

[0144] The electronic device 100-2 may also include a server or an external device controlling the cameras (e.g., the first to third cameras) described with reference to FIGS. 10, 12, and 15. Alternatively, each of the first to third cameras may be implemented as the electronic device 100-2. For example, each of the first to third cameras may include a processor 110-2 and a memory 120-2. The processor 110-2 may receive vehicle identification information and vehicle location information from the camera, check vehicle reservation information corresponding to the received identification information, and generate vehicle route information and / or guidance information. Specifically, the processor 110-2 may implement a function for providing an edge AI-based vehicle parking reservation and guidance service. In this case, each of the first to third cameras may also perform a function of the edge AI device. The processor 110-2 may implement at least one of the methods described above. The memory 120-2 may store information related to the above-described methods or a program implementing the above-described methods. The memory 120-2 may be a volatile memory or a non-volatile memory. The memory 120-2 may be referred to as a "database" or a "storage unit."

[0145] The processor 110-2 may execute a program and control the electronic device 100-2. The code of the program executed by the processor 110-2 may be stored in the memory 120-2. The electronic device 100-2 may be connected to an external device (e.g., a personal computer or a network) via an input / output device (not shown) to exchange data. The processor 110-2 may directly process an image processing-based artificial intelligence model or independently process an image processing-based artificial intelligence model. The processor 110-2 may input an image of an object captured by a camera to the image processing-based artificial intelligence model to identify information about the object. At this time, the processor 110-2 of the first electronic device may receive vehicle information from the camera of the first electronic device. That is, the processor 110-2 of the first electronic device may receive identification information of an entering vehicle from a first camera installed at the entrance / exit of a parking lot.

[0146] When a vehicle is photographed by the first camera, the vehicle can be identified by recognizing the license plate of the vehicle, and the identification information can represent the vehicle number.

[0147] Furthermore, processor 110-2 can check reservation information for the reserved time and reserved parking location in accordance with the identification information.

[0148] The user of the vehicle corresponding to the identification information can reserve in advance the time and parking location for the vehicle, and the processor 110-2 confirms the information.

[0149] In addition, the processor 110-2 of the second electronic device can confirm the position of the vehicle from a plurality of second cameras installed inside the parking lot.

[0150] The second camera also recognizes the vehicle and its license plate from the captured image, and can confirm the location of the vehicle based on the position and angle of the second camera.

[0151] An artificial intelligence module may then be used to identify the vehicle's license plate and the associated vehicle location.

[0152] At this time, the artificial intelligence module can generate a trained machine learning model using deep learning techniques, which are a branch of machine learning, to collect information regarding the location of the vehicle's license plate and vehicle number, etc.

[0153] In addition, the processor 110-2 may generate route information based on the location of the entrance / exit of the parking lot and the reserved parking space. The generation of route information will be described in more detail with reference to Figures 12 to 14.

[0154] In addition, the processor 110-2 can generate guidance information based on the location of the vehicle and the route information, and can provide the guidance information to a user terminal operating the vehicle.

[0155] FIG. 11 is a diagram illustrating generation of route information according to an embodiment of the present invention, and FIG. 12 is a diagram illustrating confirmation of a vehicle location according to an embodiment of the present invention.

[0156] 11 and 12, the processor 110-2 of the third electronic device may derive an image of the vehicle and derive the final parking location of the vehicle based on the derived image. Specifically, the processor 110-2 of the third electronic device may derive a vehicle image of a third camera from among a plurality of second cameras that captures the vehicle, confirm the position of the vehicle based on the position of the third camera and the vehicle image, generate an augmented reality image corresponding to the position of the vehicle, the position of the third camera, and the angle of the third camera based on the route information, and generate the guidance information by superimposing the augmented reality image on the vehicle image.

[0157] Through this, the user can check which direction the vehicle should go to reach the reserved parking space.

[0158] FIG. 14 is a diagram illustrating guidance information in which an augmented reality image is superimposed on a vehicle image according to an embodiment of the present invention.

[0159] Referring to FIG. 14, the augmented reality image may be an arrow image indicating where the vehicle should go on the route information based on the position of the vehicle in the vehicle image.

[0160] Through this, the user can intuitively understand where he or she needs to go from his or her current location.

[0161] However, if the user's vehicle does not park in the reserved location, the driver must be guided to the correct parking location.

[0162] To this end, the processor 110-2 assigns a unique identification ID to each parking location in the parking lot, derives the final parking location of the vehicle based on the vehicle image, and compares the first identification ID corresponding to the final parking location with the second identification ID corresponding to the reserved parking location to confirm whether the vehicle has parked in the reserved parking location.

[0163] Therefore, if the final parking location of the vehicle is different from the reserved parking location, processor 110-2 can transmit information indicating that the final parking location is different from the reserved parking location and route information to the reserved parking location to user terminal 200.

[0164] In this case, there is no problem if the user moves to the reserved parking space, but if the user parks in a non-reserved space despite being given instructions, a disadvantage must be imposed.

[0165] Therefore, if the final parking location is different from the reserved parking location even after the previously set time has elapsed, the processor 110-2 can generate a first surcharge that reflects the first surcharge information that has already been set in the reserved parking fee corresponding to the reservation information and transmit it to the user terminal 200.

[0166] In this case, the time period is the time during which the movement can be made, and can be set arbitrarily by the operator, for example, 10 minutes, 5 minutes, etc.

[0167] In addition, the first surcharge information can be set arbitrarily by the operator, for example, 10% or 20%, and the first surcharge fee may be the reserved parking fee increased by the first surcharge information.

[0168] The reserved parking fee will be described later.

[0169] Furthermore, if a user does not move the vehicle even after the reserved time has passed, this may cause inconvenience to subsequent users. Therefore, it is necessary to penalize users who do not move the vehicle even after the reserved time has passed.

[0170] To this end, when the vehicle is parked at the reserved parking location beyond the reserved time, the processor 110-2 calculates a first parking fee corresponding to the exceeded time, generates a second surcharge reflecting the second surcharge information already set on the first parking fee, and transmits the second surcharge to the user terminal 200.

[0171] In this case, the first parking fee may be calculated based on the excess time using the method of calculating the reserved parking fee, and the second surcharge information may be set arbitrarily by the operator, for example, 10%, 20%, etc., and the second surcharge may be the first parking fee increased by the second surcharge information.

[0172] At this time, the reserved parking fee may be calculated based on the reservation time received from the user terminal 200, the previously set basic parking fee per hour, a time difference index previously set differentially for each time, and a location difference index previously set differentially for each parking location between the reserved parking location and the parking lot.

[0173] This allows for differential parking fees to be applied between times when there are many vehicles and times when there are few vehicles, and also allows for differentiating between locations with high and low user preference, such as parking spaces near entrances and exits or elevators, and applying differential parking fees to these locations.

[0174] In more detail, the reserved parking fee can be calculated by the following Equation 1.

[0175] [Formula 1]

number

[0176] At this time, the time difference index can be calculated by the following Equation 2.

[0177] [Formula 2]

number

[0178] In this case, NoOPS (Number of Occupied Parking Space) means the average number of vehicles parked in the parking lot at the nth time, NoTPS (Number of Total Parking Space) means the total number of parking spaces in the parking lot, NoD (Number of Departures)_n means the number of vehicles that exited the parking lot at the nth time, and NoE (Number of Entrances)_n means the number of vehicles that entered the parking lot at the nth time.

[0179] At this time, the position difference index can be calculated by the following Equation 3.

[0180] [Formula 3]

number

[0181] As described above, it is considered that only reserved vehicles can use the parking lot, but vehicles that visit on an ad hoc basis, such as delivery vehicles and parcel delivery vehicles, should be viewed differently.

[0182] Since the vehicle in question may disrupt traffic flow by stopping in an unregulated manner, the processor can set a parking space where the temporary stopping vehicle, such as the delivery vehicle or the like, can stop as a temporary stopping location in advance, and can set a reserved parking space for the temporary stopping vehicle, such as the delivery vehicle or the like, as the temporary stopping location so that the temporary stopping vehicle, such as the delivery vehicle or the delivery vehicle, can be guided to the temporary stopping location.

[0183] FIG. 15 is a flowchart of a method for providing a vehicle parking reservation and guidance service based on artificial intelligence according to an embodiment of the present invention.

[0184] Referring to FIG. 15, a method for providing an AI-based vehicle parking reservation and guidance service according to one embodiment of the present invention can receive identification information of a vehicle entering a parking lot from a first camera installed at the entrance / exit of the parking lot (S201).

[0185] In addition, the method for providing a vehicle parking reservation and guidance service based on AI according to an embodiment of the present invention can confirm reservation information for the reservation time and the reserved parking space according to the identification information (S203).

[0186] In addition, the method for providing a vehicle parking reservation and guidance service based on AI according to an embodiment of the present invention can confirm the location of the vehicle from a plurality of second cameras installed inside the parking lot (S205).

[0187] In addition, the method for providing a vehicle parking reservation and guidance service based on AI according to an embodiment of the present invention can generate route information based on the location of the entrance / exit of the parking lot and the reserved parking location (S207).

[0188] In addition, the method for providing a vehicle parking reservation and guidance service based on AI according to an embodiment of the present invention can generate guidance information based on the vehicle location and the route information (S209).

[0189] In addition, the method for providing a vehicle parking reservation and guidance service based on AI according to an embodiment of the present invention can provide the guidance information to a user terminal operating the vehicle (S211).

[0190] In addition, the method for providing an AI-based vehicle parking reservation and guidance service according to an embodiment of the present invention may be configured similarly to the AI-based vehicle parking reservation and guidance service providing device 100-2 disclosed in Figures 10 to 14.

[0191] Edge AI-based precise forest fire detection and response device and method FIG. 16 is a conceptual diagram of an edge artificial intelligence-based forest fire precision detection and response device 100-3 according to an embodiment of the present invention.

[0192] 16, the edge AI-based forest fire precision detection and response device 100-3 according to an embodiment of the present invention can perform a situation determination operation based on object information. Furthermore, the edge AI-based forest fire precision detection and response device 100-3 can be installed locally, rather than as a central control device on a network, and can implement AI-based control and / or processing. Specifically, the edge AI-based forest fire precision detection and response device 100-3 can determine that a fire has occurred based on video from a camera installed on a mountain, and provide information on the current presence or absence of a fire to a user terminal 200 used by a forest fire manager, i.e., an operator, to enable immediate response.

[0193] Meanwhile, the edge artificial intelligence-based forest fire precision detection and response device 100-3 may also be referred to as the "electronic device 100-3" in the present invention.

[0194] FIG. 17 is a block diagram of an electronic device 100-3 according to one embodiment of the present invention.

[0195] According to an embodiment, the electronic device 100-3 includes a processor 110-3 and a memory 120-3. The electronic device 100-3 may also include a processor 110-3, a memory 120-3, and a camera. The processor 110-3 may perform at least one of the methods described above. The memory 120-3 may store information related to the methods described above or a program implementing the methods described above. The memory 120-3 may be a volatile memory or a non-volatile memory. The memory 120-3 may also be referred to as a "database," a "storage unit," etc.

[0196] The processor 110-3 can execute programs and control the electronic device 100-3. The code of the programs executed by the processor 110-3 may be stored in the memory 120-3. The electronic device 100-3 can be connected to an external device (e.g., a personal computer or a network) via an input / output device (not shown) to exchange data.

[0197] At this time, the processor 110-3 can receive photographic data from a camera installed on the mountain.

[0198] In this case, the camera may be a general visible light camera or a thermal imaging camera capable of detecting temperature, as will be described later.

[0199] In addition, the processor 110-3 can analyze the image data through an artificial intelligence module to extract a gas mass object. The processor 110-3 can analyze the image of the object captured by the camera to extract the gas mass object, and if it determines that the gas mass object is real gas, it can receive thermal imaging data from the camera.

[0200] The processor 110-3 may also analyze the photographic data and the gas mass object to determine whether the gas mass object is caused by a fire, as will be described later.

[0201] In addition, if the gas mass object is determined to be caused by a fire, the processor 110-3 can derive the source point of the gas mass object and transmit the source point and the fire determination result to the user terminal 200.

[0202] At this time, if it is determined that the gas mass object is caused by a fire, a drone that has been set up in advance can be controlled to take pictures near the source of the fire so that the operator can more accurately determine whether or not there is a fire, and the drone photography data taken by the drone can be provided to the operator.

[0203] Looking at it in more detail, if the processor 110-3 determines that the gas mass object is caused by a fire, it can send a control command to an already installed drone to photograph the source point, and transmit the drone photography data photographed by the drone to the user terminal 200.

[0204] In this case, the source point may be set as the point with the highest temperature within the gas mass object, as will be described later.

[0205] FIG. 18 is a diagram illustrating an example of a captured image according to an embodiment of the present invention.

[0206] 18, a camera installed on a mountain can capture images of both the mountain and other non-mountain areas. As will be described later, in order to determine whether a fire has occurred on a mountain, the processor 110-3 can divide the captured image data into a first area corresponding to the mountain and a second area corresponding to other areas.

[0207] In this case, even if a gas mass object is extracted from the photographic data, it cannot be immediately determined that a fire has occurred, because it could be a stain on the camera lens or a gas mass unrelated to a fire, such as fog.

[0208] Therefore, in the present invention, it is possible to determine whether or not there is a fire by first sequentially checking whether it is correct that the gas mass object originated in the mountain, whether it is actually a gas mass, i.e., a true gas, and whether it is correct that it was generated by a fire.

[0209] FIG. 19 is an exemplary diagram illustrating determining the origin of a gas mass object according to an embodiment of the present invention.

[0210] Referring to FIG. 19, the processor 110-3 can set the narrower portion of the width of the upper and lower portions of the gas mass object in the photographic data as the source direction reference.

[0211] Furthermore, if the frame of the gas mass object is entirely contained within the first region, the processor 110-3 can determine that the gas mass object has occurred in a mountain.

[0212] In addition, when the frame of the gas mass object is in contact with the boundary between the first and second regions and the source direction reference is located on the first region side, the processor 110-3 may determine that the gas mass object is generated from a mountain. This is because gas has the property of spreading more as it rises, and therefore, the gas mass object can be considered to have been generated at a point where it is least spread out.

[0213] 20 and 21 are diagrams illustrating an example of determining whether a mass of gas object is real gas according to an embodiment of the present invention.

[0214] As mentioned above, if it is determined that a mass of gas object originates from a mountain, it is necessary to determine whether the mass of gas object is a speck or a real gas such as smoke or fog.

[0215] Therefore, referring to Figures 20 and 21, in order to determine whether the gas mass object is a stain or real gas such as smoke or fog, the processor 110-3 receives information about wind direction and wind speed from a wind meter installed on the mountain, derives a first object center where the first longest width and first longest height of the gas mass object meet at a first time point, and derives a second object center where the second longest width and second longest height of the gas mass object meet at a second time point after a set time has passed since the first time point.

[0216] In addition, the processor 110-3 may analyze the image of the object captured by the camera to extract a gas mass object, and if it determines that the gas mass object is real gas, it may receive thermal imaging data from the camera. The processor 110-3 may compare a first center movement distance that has been previously set from the angle of the camera based on the wind speed with a second center movement distance between the center of the first object and the center of the second object from the angle of the camera, compare the wind direction with the direction of the second center movement distance, and if the direction of the second center movement distance is within a first error range that has been previously set with the wind direction and the second center movement distance is within a second error range that has been previously set with the first center movement distance, it may determine that the gas mass object is real gas such as smoke or fog.

[0217] This is to confirm how the mass of gas object moves over time, because if it is actually gas, the center will move depending on the wind speed and direction.

[0218] In this case, the first center moving distance may be set as an average moving distance of the air mass object depending on the wind speed based on the entire collected database.

[0219] In this case, the first and second error ranges may be arbitrarily set by the operator, and may be set to 10%, 20%, etc.

[0220] In addition, when the direction of the second center movement distance is within a first error range relative to the wind direction but the second center movement distance is outside a second error range relative to the first center movement distance, the processor 110-3 selects one of the longest widths or longest heights that is closest to the wind direction, derives a gas diffusion change rate corresponding to the ratio between the selected first longest width or first longest height and the corresponding second longest width or second longest height, and when the gas diffusion change rate is within a third error range that is already set and a reference diffusion change rate that is already set to correspond to the diffusion degree of smoke or fog based on the wind speed, the processor 110-3 can determine that the gas mass object is real gas such as smoke or fog.

[0221] At this time, the reference diffusion rate may be set as an average gas diffusion rate of the gas mass object according to the wind speed based on the entire collected database.

[0222] In this case, the third error range may be arbitrarily set by the operator, and may be set to 10%, 20%, etc.

[0223] Through this, even if the center of the gas mass object does not move sufficiently, it is possible to determine whether it is real gas or not based on the diffusion of the gas mass.

[0224] FIG. 22 is an exemplary diagram illustrating a process for determining whether a gas mass object is generated by a fire according to an embodiment of the present invention.

[0225] As described above, if the gas mass object is determined to be a real gas such as smoke or fog, it is necessary to receive thermal image data from the camera and determine whether the gas mass object is caused by a fire based on the thermal image data.

[0226] In contrast, referring to Figure 22, in order to determine whether a gas mass object is caused by a fire, processor 110-3 generates a first block in which the gas mass object is included in the photographic data, but the longest width and longest height of the gas mass object are the horizontal and vertical lengths, respectively, and divides the first region into a plurality of second blocks having the same size as the first block, and divides the second region into a plurality of third blocks having the same size as the first block.

[0227] At this time, depending on the size of the captured image and the size of the first block, it may not be possible to divide it exactly as shown in Figure 22. In this case, blocks can be placed first based on the boundary between the first and second regions, and then blocks adjusted according to the size of the remaining part can be placed in the remaining part. In addition, the processor 110-3 can derive a first maximum temperature from the first block based on the thermal imaging data, derive a second maximum temperature for each second block based on the thermal imaging data and calculate a first average value corresponding to the average value of the second maximum temperatures, and derive a third maximum temperature for each third block based on the thermal imaging data and calculate a second average value corresponding to the average value of the third maximum temperatures.

[0228] This is because the temperature must be extracted to determine whether it is a fire, and the area is divided into uniform regions and the maximum temperature for each region is collected to accurately determine whether there is a fire or not.

[0229] At this time, if the first maximum temperature exceeds the previously set first critical temperature, the processor 110-3 determines that the gas mass object is caused by a fire, but compares the first maximum temperature with the first average value, and if the difference between the first maximum temperature and the first average value is within the previously set fourth error range, determines that there is a high possibility of a large-scale forest fire, and can send information indicating that a fire has occurred to the previously set related agency or control center and the user terminal 200.

[0230] This is because the temperature in the mountain area adjacent to Block 1 has already risen significantly, posing the possibility of a large-scale forest fire. Therefore, the operator can be notified and information about the fire can be sent immediately to relevant agencies such as 119, which can extinguish the fire, enabling immediate response.

[0231] In this case, the first critical temperature can be set as the average air temperature 20 meters above the fire's point when a fire breaks out. For example, a fire starts at a minimum temperature of 500 degrees, but the temperature may drop slightly as the fire diffuses into the air. Therefore, to detect a fire in a captured image, the first critical temperature can be set as described above. For example, it can be set to 100 degrees.

[0232] In this case, the fourth error range may be arbitrarily set by the operator, and may be set to 10%, 20%, etc.

[0233] In addition, if the difference between the first maximum temperature and the first average value is outside the fourth error range, the processor 110-3 determines that it is a localized fire, sends a control command to have the drone photograph the source point, and transmits the drone photographed data photographed by the drone to the user terminal 200.

[0234] In addition, when the first maximum temperature is equal to or lower than the first critical temperature and exceeds a second critical temperature that is lower than the first critical temperature and has already been set, the processor 110-3 compares the first maximum temperature with the first average value and the second average value, and when the difference between the first maximum temperature and the first average value and the second average value is within a fourth error range, it determines that the temperature rise is due to weather and transmits the determination result to the user terminal 200.

[0235] This is because the temperatures of the gas mass object, the mountain, and the non-mountain areas are all similar, and it can be seen that the temperature rose due to the weather, making it difficult to see that the gas mass object was caused by a fire.

[0236] In this case, the second critical temperature may be arbitrarily set by the operator, for example, at 50 degrees.

[0237] In addition, if the first maximum temperature and the first average value are within the fourth error range and the first maximum temperature and the second average value are outside the fourth error range, the processor 110-3 determines that there is a high possibility of a fire, and sends a control command to have the drone photograph the source point, and transmits the drone photographed data photographed by the drone to the user terminal 200.

[0238] Since the temperature is high only in the first area close to the gas mass object, it is difficult to conclude that the temperature rise is due to weather. Therefore, it is preferable to photograph the source point using a drone so that the operator can confirm it.

[0239] FIG. 23 is a flowchart illustrating a method for accurately detecting and responding to forest fires based on edge artificial intelligence according to an embodiment of the present invention.

[0240] Referring to FIG. 23, a method for detecting and responding to forest fires based on edge artificial intelligence according to an embodiment of the present invention can receive photographic data from a camera installed on a mountain (S301).

[0241] In addition, the edge AI-based forest fire precision detection and response method according to an embodiment of the present invention can analyze the photographic data through an AI module to extract a gas mass object (S303).

[0242] In addition, the edge AI-based forest fire precision detection and response method according to an embodiment of the present invention can analyze the photographic data and the gas mass object to determine whether the gas mass object is caused by a fire (S305).

[0243] In addition, the edge artificial intelligence-based forest fire precision detection and response method according to an embodiment of the present invention can derive the source point of the gas mass object if it is determined that the gas mass object is caused by a fire (S307).

[0244] In addition, the edge AI-based forest fire precise detection and response method according to an embodiment of the present invention can transmit the source location and the fire determination result to the user terminal 200 (S309).

[0245] In addition, the edge AI-based forest fire precision detection and response method according to an embodiment of the present invention may be configured in the same manner as the edge AI-based forest fire precision detection and response device 100-3 disclosed in Figures 16 to 22.

[0246] Traffic accident analysis system and method FIG. 24 is a diagram illustrating a traffic accident analysis system according to one embodiment of the present disclosure.

[0247] As shown in FIG. 24, the traffic accident analysis system 100-4 includes a camera 112-4, a video monitoring device 114-4, a management server 120-4, a database (DB) 130-4, and a user terminal 200.

[0248] The camera 112-4 is installed on the smart pole 110-4 and captures various traffic images occurring on the road. The camera 112-4 is, for example, a device capable of capturing still and video images, and according to an embodiment, may include one or more image sensors, lenses, an image signal processor (ISP), or a flash (e.g., an LED or xenon lamp). The camera 112-4 can generate a road traffic image signal corresponding to the input external image and output it to the image monitoring device 114-4. The image of the camera 112-4 used in the present disclosure is road traffic image captured from above by the camera installed on the smart pole 110-4, and can obtain information on a wider space than vehicle black box images.

[0249] The video monitoring device 114-4 may also be installed on the smart pole 110-4 and may monitor road traffic accidents, including vehicle accidents, pedestrian appearances, or a combination thereof, from road traffic footage captured by the camera 112-4. The video monitoring device 114-4 may calculate the distance and angle to the vehicle from the footage captured by the camera 112-4 through planar processing of the footage in the event of a traffic accident, generate a planar traffic accident video, and provide the planar traffic accident video to the management server 120-4. The video monitoring device 114-4 may also compare the planar traffic accident video with standard traffic accident videos to determine how similar the planar traffic accident video is to at least one of the standard traffic accident videos.

[0250] The management server 120-4 can manage the road traffic video received from each of the video monitoring devices 114-4 installed on the multiple smart poles 110-4, as well as the traffic accident-related video. The management server 120-4 can store the traffic accident-related video in the database 130-4, along with the received road traffic video.

[0251] The database 130-4 may store a large number of standard traffic accident videos. Accordingly, the database 130-4 may store a large number of standard traffic accident information and a large number of actual traffic accident videos. Here, the standard traffic accident information is traffic accident data provided by the Road Traffic Authority or the like, and may include fault ratios established by court precedents or the like.

[0252] The user terminal 140-4 is an electronic device for displaying the similarity of actual traffic accident footage, such as thought patterns and accident causes, and can receive and display actual traffic accident footage and standard traffic accident footage similar to the actual traffic accident footage from the video monitoring device 114-4 or the management server 120-4, and can also display the similarity and fault ratio. Here, the user terminal 140-4 may be an electronic device of the General Insurance Association of Japan or an electronic device of the person involved in the accident.

[0253] 24 shows that the traffic accident similarity is estimated in the video monitoring device 114-4, but the traffic accident similarity estimation can also be processed by the management server 120-4. Therefore, the traffic accident analysis system may be the video monitoring device 114-4 or the management server 120-4. Meanwhile, although the management server 120-4 and the database 130-4 are shown separately in FIG. 24, the database function may be embodied in the management server 120-4.

[0254] FIG. 25 is a block diagram for explaining the case where the video monitoring device or the management server shown in FIG. 24 operates as a general electronic device.

[0255] As shown in FIG. 25, an electronic device 200-4 coupled to a network is described. The electronic device 200-4 may include a bus 210-4, a processor 220-4, a memory 230-4, an input / output interface 250-4, a display 260-4, and a communication interface 270-4. In some embodiments, the electronic device 200-4 may omit at least one of the components or include other additional components. The bus 210-4 may include circuitry for coupling the components 220-4 to 270-4 and transmitting communications (e.g., control messages or data) between the components. The processor 220-4 may include one or more of a central processing unit, an application processor, or a communication processor (CP). The processor 220-4 may perform, for example, computations and data processing related to control and / or communication with at least one other component of the electronic device 200-4.

[0256] Memory 230-4 may include volatile and / or non-volatile memory. Memory 230-4 may store, for example, instructions or data related to at least one other component of electronic device 200-4. According to one embodiment, memory 230-4 may store software and / or programs 240-4. Programs 240-4 may include, for example, kernel 241-4, middleware 243-4, application programming interface (API) 245-4, and / or application program (or "application") 247-4. At least a portion of kernel 241-4, middleware 243-4, or API 245-4 may be referred to as an operating system. The kernel 241-4 may, for example, control or manage system resources (e.g., bus 210-4, processor 220-4, or memory 230-4) used to execute operations or functions embodied in other programs (e.g., middleware 243-4, API 245-4, or application program 247-4). The kernel 241-4 may also provide an interface that allows the middleware 243-4, API 245-4, or application program 247-4 to access individual components of the electronic device 200-4 and thereby control or manage system resources.

[0257] The middleware 243-4 may, for example, act as an intermediary to enable the API 245-4 or the application program 247-4 to communicate with the kernel 241-4 and exchange data. The middleware 243-4 may also process one or more work requests received from the application program 247-4 according to a priority order. For example, the middleware 243-4 may assign a priority for using system resources (e.g., the bus 210, the processor 220-4, or the memory 230-4) of the electronic device 200-4 to at least one of the application programs 247-4, and process the one or more work requests. The API 245-4 is an interface through which the application 247-4 controls functions provided by the kernel 241-4 or the middleware 243-4, and may include at least one interface or function (e.g., command) for file control, window control, video processing, character control, etc. The input / output interface 250-4 may, for example, transmit commands or data input from a user or other external device to other components of the electronic device 200-4, or output commands or data received from other components of the electronic device 200-4 to the user or other external device.

[0258] Display 260-4 may include, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a microelectromechanical system (MEMS) display, or an electronic paper display. Display 260-4 may, for example, display various content (e.g., text, images, videos, icons, and / or symbols, etc.) to a user. Display 260-4 may include a touchscreen and may receive touch, gesture, proximity, or hover input, for example, using an electronic pen or a part of the user's body.

[0259] The communication interface 270-4 may establish communication with, for example, another electronic device (not shown). For example, the communication interface 270-4 may be connected to a network via wireless or wired communication to communicate with another electronic device. Here, the wireless communication may include cellular communication using at least one of LTE, LTE Advance (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), wireless broadband (WiBro), or global system for mobile communications (GSM). According to an embodiment, the wireless communication may include at least one of wireless fidelity (WiFi), Bluetooth, Bluetooth Low Energy (BLE), Zigbee, near field communication (NFC), magnetic secure transmission, radio frequency (RF), or a body area network (BAN). According to an embodiment, the wireless communication may include GNSS. The GNSS may be, for example, the Global Positioning System (GPS), the Global Navigation Satellite System (Glonass), the Beidou Navigation Satellite System (hereinafter "Beidou"), or Galileo, the European global satellite-based navigation system. Hereinafter, in this document, "GPS" may be used interchangeably with "GNSS." The wired communication may include, for example, at least one of universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard 232 (RS-232), power line communication, or plain old telephone service (POTS), etc.Network 262-4 may include a telecommunications network, such as at least one of a computer network (eg, a LAN or WAN), the Internet, or a telephone network.

[0260] FIG. 26 is a diagram specifically showing a block diagram of a traffic accident analysis system according to an embodiment of the present disclosure, FIG. 27 is a diagram showing examples of 2D images of standard traffic accident types and / or situations according to an embodiment of the present disclosure, FIGS. 28 and 29 are diagrams showing examples of matching camera images to Naver maps according to an embodiment of the present disclosure, FIG. 30 is a diagram showing an example of estimating a standard traffic accident type of an actual traffic accident situation according to an embodiment of the present disclosure, and FIG. 31 is a diagram showing an example of displaying a standard traffic accident estimation result of an actual traffic accident situation according to an embodiment of the present disclosure.

[0261] As shown in FIG. 26, the traffic accident analysis system 100-4 may include a camera 112-4, a video monitoring device 114-4, and a management server 120-4.

[0262] The management server 120-4 may include a standard traffic accident information storage unit 310-4 and a data set construction unit 312-4. The standard traffic accident information storage unit 310-4 stores a large number of commonly used standard traffic accident information. Here, the standard traffic accident information is traffic accident data provided by the Road Traffic Authority or the like, and may include fault ratios established by court precedents or the like. FIG. 27 shows an example of two-dimensional images of standard traffic accidents categorized by type and / or situation according to an embodiment of the present disclosure.

[0263] According to the present invention, the processor 110-4 can construct a dataset from standard traffic accident information, map the image of the object in the actual traffic accident information captured by a camera onto a two-dimensional planar map, and compare the dataset with the actual traffic accident information to estimate standard traffic accident information similar to the actual traffic accident information. Specifically, the dataset construction unit 312-4 constructs a dataset to be used by the traffic accident estimation unit 330-4. To do this, the dataset construction unit 312-4 can first classify traffic accident types that may occur on roads, such as vehicle-to-vehicle (between a car and a car), vehicle-to-person (between a car and a pedestrian), vehicle-to-motorcycle (between a car and a motorcycle), and vehicle-to-bicycle (between a car and a bicycle), using the image of the standard traffic accident information. The dataset construction unit 312-4 can perform labeling on the traffic accident video using the video into which the traffic accident type has been classified. Here, when the criterion is a vehicle, the labeling may be a passenger car, a truck, a bus, an ambulance, etc., and when the criterion is a person, the labeling may be an adult, an elderly person, a child, etc. The dataset construction unit 312-4 may further use text data, which is explanatory material explaining the traffic accident, in addition to the traffic accident video. In this case, the dataset may include video and text. Meanwhile, the dataset construction unit 312-4 may introduce artificial intelligence modeling to construct a dataset through comparison and evaluation with a large number of actual traffic accident data.

[0264] The video monitoring device 114-4 may include a road traffic information receiving unit 320-4, a road traffic information storage unit 322-4, a traffic accident identification unit 324-4, a traffic accident information storage unit 326-4, an omniscient representation unit 328-4, and a traffic accident estimation unit 330-4.

[0265] The road traffic information receiving unit 320-4 receives real-time road traffic images provided by the camera 112-4.

[0266] The road traffic information storage unit 322-4 stores road traffic images received by the road traffic information receiving unit 320-4.

[0267] The video stored in the road traffic information storage unit 322-4 may be stored in a new file at predetermined time intervals, for example, every two minutes. That is, two minutes of video may be stored in a first file, the next two minutes of video may be stored in a second file, and another two minutes of video may be stored in a third file. Then, only the previous two video files may be maintained, while the remaining files may be deleted. Therefore, the files stored in the road traffic information storage unit 322-4 may include the previous two files and one file currently being recorded.

[0268] The traffic accident identification unit 324-4 identifies whether there is a traffic accident in the road traffic video being recorded. If the traffic accident identification unit 324-4 identifies a traffic accident, the traffic accident identification unit 324 can copy and store the traffic accident information stored in the road traffic information storage unit 322-4 at the time of the traffic accident occurrence in the traffic accident information storage unit 326-4. The traffic accident information stored in the traffic accident information storage unit 326-4 may be provided to the management server 120-4.

[0269] The omniscient representation unit 328-4 converts the traffic accident video from the actual traffic accident information stored in the traffic accident information storage unit 326-4 into an omniscient representation for a predetermined time, for example, 1 to 5 seconds, from the identification of the accident vehicle to the occurrence of the accident.

[0270] To perform omni-intelligent representation of the video, the omni-intelligent representation unit 328-4 may first display circular distance lines at predetermined distance intervals, for example, 1 to 10 meters, on the traffic accident video. Here, the circular distance lines may be values ​​previously obtained through video captured by a camera 112-4 installed on the smart pole 110-4. The omni-intelligent representation unit 328-4 may then display a bounding box using the frame of the identified object. Here, the bounding box for the vehicle may be used to calculate the vehicle's traveling direction and angle. The omni-intelligent representation unit 328-4 may then match the 2D map with the actual location of the accident scene. Here, the 2D planar map may be a Naver map. As a result, the vehicle position in the traffic accident video may be mapped onto the 2D planar map taking into account the distance and angle on the smart pole 110-4. The omni-intelligent representation unit 328-4 may display the accident video in omni-intelligent representation on the 2D planar map within a predetermined time, for example, 1 to 5 seconds, from the identification of the accident vehicle to the occurrence of the accident. According to an embodiment of the present disclosure, an example of camera footage is shown in FIG. 28, and an example of matching to a Neighbor map is shown in FIG.

[0271] The processor 220-4 may perform a situation assessment operation. The situation assessment operation may include the processor 220-4 constructing a dataset by comparing and evaluating a large number of actual data, learning using the constructed dataset, and estimating a similarity between the image of an object captured by the camera 112-4. Specifically, the traffic accident estimation unit 330-4 may estimate similar standard traffic accident information by learning actual traffic accident information using the constructed standard traffic accident dataset. On the other hand, the traffic accident estimation unit 330-4 may estimate standard traffic accidents similar to the traffic accident in question and their similarity by learning from the actual traffic accident dataset constructed from actual traffic accident information. In this case, the processor 220-4 may apply a convolutional neural network to estimate a similarity between the image of an object captured by the camera 112-4. The traffic accident estimation unit 330-4 may apply a convolutional neural network (CNN) to the image to perform analysis. The CNN comprises a part that extracts image features by effectively recognizing and emphasizing features between adjacent images while maintaining spatial information of the image, and a part that classifies the image. The feature extraction region may comprise a convolutional layer that searches for image features while minimizing the number of shared parameters using a filter, and a pooling layer that enhances and collects features. Figure 30 illustrates an example of estimating a standard traffic accident type for an actual traffic accident situation according to an embodiment of the present disclosure.

[0272] Therefore, the traffic accident estimation unit 330-4 can compare the two-dimensional planar traffic accident video represented by the omniscient representation unit 328-4 with standard traffic accident videos to estimate how similar the planar traffic accident video is to at least one of the standard traffic accident videos. A diagram showing an example of displaying a standard traffic accident estimation result of an actual traffic accident situation according to an embodiment of the present disclosure is shown in Figure 31.

[0273] FIG. 32 is a flowchart illustrating a traffic accident analysis method according to another embodiment of the present disclosure. The dataset construction unit 312-4 of the management server 120-4 constructs a dataset to be used by the traffic accident estimation unit 330-4 from a large number of standard traffic accident information items stored in the standard traffic accident information storage unit 310-4 (S810). To do this, the dataset construction unit 312-4 first classifies traffic accident types that may occur on roads, such as vehicle-to-vehicle (car and car), car-to-person (car and pedestrian), car-to-motorcycle (car and motorcycle), and car-to-bicycle (car and bicycle), using video of the standard traffic accident information. The dataset construction unit 312-4 can perform labeling on the traffic accident video using the video with the classified traffic accident type. Here, the labeling can be, for example, a passenger car, truck, bus, or ambulance when the vehicle is the criterion, or an adult, elderly person, or child when the person is the criterion. The dataset construction unit 312-4 can further use text data, which is explanatory material explaining the traffic accident, in addition to the traffic accident video. In this case, the data set may include video and text.

[0274] The road traffic information receiving unit 320-4 of the video monitoring device 114-4 receives real-time road traffic video provided from the camera 112-4 (S820). The road traffic information storage unit 322-4 stores the road traffic video received by the road traffic information receiving unit 320-4 (S830). The video stored in the road traffic information storage unit 322-4 may be stored in a new file every two minutes, for example.

[0275] The traffic accident identification unit 324-4 identifies whether there is a traffic accident in the road traffic video being recorded (S840). If the traffic accident identification unit 324-4 identifies a traffic accident, the traffic accident identification unit 324-4 copies and stores the traffic accident information stored in the road traffic information storage unit 322-4 at the time of the traffic accident in the traffic accident information storage unit 326-4 (S850). The traffic accident information stored in the traffic accident information storage unit 326-4 may be provided to the management server 120-4.

[0276] The omni-intelligent representation unit 328-4 converts the traffic accident video from the actual traffic accident information stored in the traffic accident information storage unit 326-4 into an omni-intelligent representation in a predetermined time interval, for example, 1 to 5 seconds, from the identification of the accident vehicle to the occurrence of the accident (S860). To generate an omni-intelligent representation of the video, the omni-intelligent representation unit 328-4 may first display circular distance lines at predetermined distance intervals, for example, 1 to 10 meters, on the traffic accident video. Here, the circular distance lines may be values ​​previously obtained through video captured by a camera 112-4 installed on a smart pole 110-4. The omni-intelligent representation unit 328-4 may then display a bounding box using the frame of the identified object. Here, the bounding box for the vehicle may be used to calculate the vehicle's traveling direction and angle. The omni-intelligent representation unit 328-4 may then match the 2D map with the actual location of the accident site. Here, the 2D planar map may be a Naver map. As a result, the vehicle positions in the traffic accident video may be mapped onto a two-dimensional planar map taking into account the distance and angle at the smart pole 110-4. The omniscient representation unit 328-4 can display the accident video on a two-dimensional planar map in an omniscient representation for a predetermined time, for example, one to five seconds, from the identification of the accident vehicle to the occurrence of the accident.

[0277] The traffic accident estimation unit 330-4 estimates similar standard traffic accident information by learning actual traffic accident information using the standard traffic accident dataset constructed in this way (S870). On the other hand, the traffic accident estimation unit 330-4 can estimate standard traffic accidents similar to the traffic accident in question and their similarity by learning from the actual traffic accident dataset constructed from actual traffic accident information. In this case, the traffic accident estimation unit 330-4 can analyze the video by applying a convolutional neural network (CNN). The CNN consists of a part that extracts image features and a part that classifies the image by effectively recognizing and emphasizing features with neighboring images while maintaining the spatial information of the image. The feature extraction region may consist of a convolutional layer that searches for image features while minimizing the number of shared parameters using a filter, and a pooling layer that enhances and collects features.

[0278] The above-described embodiments may be implemented using hardware components, software components, and / or a combination of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as a processor, controller, arithmetic logic unit (ALU), digital signal processor, microcomputer, field programmable gate array (FPGA), programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a processing device may be described as being a single device; however, those skilled in the art will recognize that a processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing device may include multiple processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

[0279] Methods according to the embodiments may be embodied in the form of program instructions that can be executed by various computer means and stored on a computer-readable medium. The computer-readable medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions stored on the medium may be specially designed and constructed for the embodiments, or may be publicly known and available to those skilled in the art of computer software. Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include not only machine code, such as produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter, for example. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, or vice versa.

[0280] Software may include a computer program, code, instructions, or a combination of one or more of these, which can configure a processing device to operate as desired or instruct the processing device, either individually or collectively. The software and / or data may be permanently or temporarily embodied in some type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed across computer systems coupled to a network, stored and executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0281] Although the embodiments have been described above with reference to limited drawings, those skilled in the art may apply various technical modifications and variations based on the foregoing. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, or may be replaced or substituted by other components or equivalents, and still achieve suitable results.

[0282] Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

[0283] The above-described embodiments may be implemented using hardware components, software components, and / or a combination of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as a processor, controller, arithmetic logic unit (ALU), digital signal processor, microcomputer, field programmable gate array (FPGA), programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a processing device may be described as being a single device; however, those skilled in the art will recognize that a processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing device may include multiple processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

[0284] Methods according to the embodiments may be embodied in the form of program instructions that can be executed by various computer means and stored on a computer-readable medium. The computer-readable medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions stored on the medium may be specially designed and constructed for the embodiments, or may be publicly known and available to those skilled in the art of computer software. Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include not only machine code, such as produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter, for example. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, or vice versa.

[0285] Software may include a computer program, code, instructions, or a combination of one or more of these, which can configure a processing device to operate as desired or instruct the processing device, either individually or collectively. The software and / or data may be permanently or temporarily embodied in some type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed across computer systems coupled to a network, stored and executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0286] Although the embodiments have been described above with reference to limited drawings, those skilled in the art may apply various technical modifications and variations based on the foregoing. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, or may be replaced or substituted by other components or equivalents, and still achieve suitable results.

[0287] Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims

1. 1. An electronic device for performing a situational judgment operation based on information about an object, comprising: Memory and a processor; Camera and Including, The processor directly processes an image processing-based artificial intelligence model, The processor uses the image of the object captured by the camera as an input to the image processing-based artificial intelligence model to identify information about the object; the processor performs the situation assessment action based on information about the identified object. An electronic device for performing situational action based on information about an object.

2. 1. An electronic device for performing a situational judgment operation based on information about an object, comprising: Memory and a processor; Camera and Including, The processor independently processes an image processing-based artificial intelligence model, The processor uses the image of the object captured by the camera as an input to the image processing-based artificial intelligence model to identify information about the object; the processor performs the situation assessment action based on information about the identified object. An electronic device for performing situational action based on information about an object.

3. 1. An electronic device for performing a situational judgment operation based on information about an object, comprising: Memory and a processor; Camera and Including, The electronic device is not a central control device on a network, but is installed locally and can implement control and / or processing based on artificial intelligence, The processor processes an image processing-based artificial intelligence model, The processor uses the image of the object captured by the camera as an input to the artificial intelligence model to identify information about the object; the processor performs the situation assessment action based on information about the identified object. An electronic device for performing situational action based on information about an object.

4. The processor: receiving vehicle identification information and vehicle location information from the camera; confirming reservation information for the vehicle in response to the received identification information, and generating route information and / or guidance information for the vehicle; An electronic device for performing a situation determination operation based on information of an object according to any one of claims 1 to 3.

5. 1. An electronic system for performing situational action based on information about an object, comprising: a first electronic device, a second electronic device, and a third electronic device according to claim 4; a processor of the first electronic device receiving information about the vehicle from a camera of the first electronic device; a processor of the second electronic device determining the location of the vehicle from a camera of the second electronic device and a camera of the third electronic device; The camera of the third electronic device captures an image of the vehicle; The processor of the third electronic device derives a final parking location of the vehicle based on the derived image. An electronic system for performing situational action based on information about an object.

6. The situation determination operation is the processor constructs a data set through comparison and evaluation with a large number of actual data; Learning using the constructed dataset; Estimating a similarity between the image of the object captured by the camera and the image of the object; An electronic device for performing a situation determination operation based on information of an object according to any one of claims 1 to 3.

7. The processor: Applying a convolutional neural network to estimate the similarity between the image of the object captured by the camera and the image of the object; 7. An electronic device for performing a situation-judging operation based on information of an object according to claim 6.

8. The processor: Dividing the image of the object captured by the camera into a plurality of blocks, and performing an object detection process on the plurality of blocks; An electronic device for performing a situation determination operation based on information of an object according to any one of claims 1 to 3.

9. The processor: The camera can be controlled to operate as a thermal imaging camera.

9. An electronic device for performing a situation-judging operation based on information of an object according to claim 8.

10. The processor: Identifying people via the object detection process; If the risk level of the block where the identified person is located exceeds a preset critical risk level, transmit information indicating that the identified person is in a dangerous situation to the user terminal.

9. An electronic device for performing a situation-judging operation based on information of an object according to claim 8.

11. The processor: Analyzing the image of the object captured by the camera to extract a gas mass object; If it is determined that the mass of gas object is real gas, receiving thermal image data from the camera; An electronic device for performing a situation determination operation based on information of an object according to any one of claims 1 to 3.

12. The processor: We build a dataset from standard traffic accident information, Mapping the image of the object of the actual traffic accident information captured by the camera onto a two-dimensional planar map, Comparing the dataset with the actual traffic accident information, estimating the standard traffic accident information similar to the actual traffic accident information; An electronic device for performing a situation determination operation based on information of an object according to any one of claims 1 to 3.

Images