Method And Apparatus for Risk Alert by Using Detection of Tilt Deformation of Structure

Abstract

The present invention relates to a method and apparatus for notifying danger by detecting tilt deformation of a structure. According to one aspect of the present disclosure, a method for notifying danger using tilt deformation detection of a structure performed by a surveillance camera equipped with edge computing capabilities comprises: acquiring a plurality of video frames; performing preprocessing on the plurality of video frames; converting the plurality of video frames into a plurality of black-and-white video frames; extracting a plurality of tilt vectors from each of the plurality of black-and-white video frames; extracting edges of the structure from the plurality of tilts; generating an edge map based on the edges; extracting linear elements (lines) of the structure from the edge map; generating virtual displacement markers based on the edge map; generating a plurality of augmented frames; classifying the degree of danger of the structure included in each augmented frame using the plurality of augmented frames as input to an object detection model; calculating the frequency of the degree of danger; determining the final degree of danger based on the frequency; and generating a danger warning for the structure based on the final degree of danger.

Description

Technology Field

[0001] The present disclosure relates to a method and apparatus for notifying danger by detecting tilt deformation of a structure. Background Technology

[0002] The following description merely provides background information related to the present embodiment and does not constitute prior art.

[0003] Recently, the adoption of Structural Health Monitoring (SHM) systems has been increasing to ensure the safety of outdoor infrastructure such as buildings, bridges, roads, traffic lights, and signs. These systems aim to monitor changes in physical stability caused by tilting, settlement, and deformation in real time, and to provide early warnings of potential risks.

[0004] Existing SHM technologies have often relied on wired sensor networks or cloud-based central processing methods. However, these approaches suffer from issues such as transmission delays, high power consumption, and dependence on continuous network connectivity. Accordingly, to meet the increasing demand for smart city safety, Edge Computing technology is being applied to SHM systems. This enables latency-free real-time data processing and system resilience, even in environments where communication bandwidth is limited or cloud access is difficult.

[0005] However, analyzing the continuous tilt deformation of structures on edge computing devices presents challenges due to the limited computational resources of edge computing devices, environmental change factors such as wind speed, rainfall, and vibration, and the complexity of distinguishing between temporary and permanent deformation. The problem to be solved

[0006] The main purpose of the present disclosure is to provide a method and apparatus for indicating danger by utilizing tilt deformation of a structure, performed by a surveillance camera equipped with edge computing capabilities.

[0007] The problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below. means of solving the problem

[0008] According to one aspect of the present disclosure, a method for notifying danger by detecting tilt deformation of a structure performed by a surveillance camera equipped with edge computing capabilities comprises: acquiring a plurality of video frames; setting the interval of the plurality of video frames to a predetermined frame interval; storing a plurality of selected video frames in a buffer for each interval; performing preprocessing to enhance the brightness and contrast of the plurality of video frames; converting the plurality of video frames into a plurality of grayscale video frames; extracting a plurality of tilt vectors from each of the plurality of grayscale video frames; extracting an edge of the structure using an edge detection algorithm based on the plurality of tilt vectors; generating an edge map based on the edge map; extracting a linear element (line) of the structure from the edge map using a geometric structure extraction algorithm; generating a virtual displacement marker that visually indicates the location where deformation of the structure is automatically detected on the grayscale video frames based on the edge map; and generating a plurality of augmented frames by overlaying the tilt vector and the virtual displacement marker on each of the grayscale video frames. A method is provided comprising: a process of classifying the risk level of a structure included in each augmented frame using multiple augmented frames as input to an object detection model; a process of storing the risk level of each augmented frame in a buffer; a process of calculating the frequency of the risk level stored in the buffer; a process of determining the risk level with the highest frequency as the final risk level based on the frequency; and a process of generating a risk warning for the structure based on the final risk level, wherein the risk warning displays the risk location of the structure as a circle or ellipse on the augmented frame.

[0009] According to another aspect of the present disclosure, at least one memory for storing instructions; and including at least one processor, wherein the at least one processor executes instructions to acquire a plurality of video frames from a surveillance camera equipped with edge computing capabilities, set the interval of the plurality of video frames to a predetermined frame interval, store the selected plurality of video frames in a buffer for each interval, perform preprocessing to enhance the brightness and contrast of the plurality of video frames, convert the plurality of video frames into a plurality of grayscale video frames, extract a plurality of gradient vectors from each of the plurality of grayscale video frames, extract the edge of a structure using an edge detection algorithm based on the plurality of gradient vectors, generate an edge map based on the edge map, extract a linear element (line) of a structure using a geometric structure extraction algorithm from the edge map, generate a virtual displacement marker that visually indicates the location where deformation of the structure on the grayscale video frame is automatically detected based on the edge map, generate a plurality of augmented frames by overlaying the gradient vectors and the virtual displacement marker on each of the plurality of grayscale video frames, and use the plurality of augmented frames as input to an object detection model, wherein the elements included in each augmented frame A device is provided that classifies the risk level of a structure, stores the risk level of each augmented frame in a buffer, calculates the frequency of the risk levels stored in the buffer, determines the risk level with the highest frequency based on the frequency as the final risk level, and generates a risk alarm for the structure based on the final risk level, wherein the risk alarm displays the risk location of the structure as a circle or ellipse on the augmented frame.

[0010] According to another aspect of the present disclosure, a computer program stored on a computer-readable recording medium is provided to execute each process included in the method described above.

[0011] According to another aspect of the present disclosure, a CCTV camera is provided that operates according to the method described above. Effects of the invention

[0012] According to one embodiment of the present disclosure, since changes in the inclination of a structure can be analyzed in real time within an edge computing device, there is an effect of detecting abnormal conditions of a structure without delay even in an environment where the network is unstable.

[0013] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0014] FIG. 1 is a schematic diagram illustrating the process of generating a danger alarm using a surveillance camera according to one embodiment of the present disclosure. FIG. 2 is a block diagram schematically showing the processing sequence between internal modules of a surveillance camera according to one embodiment of the present disclosure. FIG. 3 is a flowchart illustrating a method of notifying danger using a surveillance camera according to one embodiment of the present disclosure. FIG. 4 is a diagram illustrating the preprocessing process of a video frame according to one embodiment of the present disclosure. FIG. 5 is a schematic diagram illustrating a deep learning-based object detection model according to one embodiment of the present disclosure. FIG. 6 is a detailed diagram showing a deep learning-based object detection model according to one embodiment of the present disclosure. FIG. 7 is a schematic block diagram illustrating an exemplary computing device that can be used to implement a method or device according to the present disclosure. Specific details for implementing the invention

[0015] Some embodiments of the present disclosure will be described in detail below with reference to exemplary drawings.

[0016] It should be noted that when assigning reference numerals to the components of each drawing, the same components are assigned the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing the embodiments of this specification, if it is determined that a detailed description of related known configurations or functions could obscure the essence of the invention, such detailed description is omitted.

[0017] In describing the components of the embodiments according to the present disclosure, symbols such as first, second, i), ii), a), b), etc., may be used. These symbols are intended only to distinguish the components from other components, and the essence, order, or sequence of the components is not limited by the symbols. When a part in the specification is described as 'comprising' or 'having' a component, this means that, unless explicitly stated otherwise, it does not exclude other components but may include additional components.

[0018] The detailed description set forth below, together with the accompanying drawings, is intended to describe exemplary embodiments of the present disclosure and is not intended to represent the only embodiment in which the present disclosure can be practiced.

[0019] hereinafter, surveillance cameras equipped with edge computing capabilities will be referred to as 'surveillance cameras'.

[0020] FIG. 1 is a schematic diagram illustrating the process of generating a danger alarm using a surveillance camera according to one embodiment of the present disclosure.

[0021] A surveillance camera can acquire multiple video frames. The acquired video frames may be in color. The surveillance camera can convert multiple video frames into multiple grayscale video frames. Grayscale conversion is intended to reduce data size and computational load. The surveillance camera can convert multiple grayscale video frames into multiple augmented frames displaying danger alerts. The augmented frames may be in color. The danger alerts may be displayed as circles or ellipses and may be displayed in different colors depending on the level of danger.

[0022] FIG. 2 is a block diagram schematically showing the processing sequence between internal modules of a surveillance camera according to one embodiment of the present disclosure.

[0023] The surveillance camera (10) may include all or part of a preprocessing module (101), an edge extraction module (102), a risk level classification module (103), and a risk alarm generation module (104). Not all blocks shown in FIG. 2 are essential components, and in other embodiments, some blocks may be added, changed, or deleted. Meanwhile, the components shown in FIG. 2 represent functionally distinct elements, and at least one component may be implemented in a form that is integrated with one another in an actual physical environment.

[0024] The preprocessing module (101) can acquire video frames of a structure. The preprocessing module (101) can set the interval of the acquired video frames to a predetermined frame interval, for example, 25 to 30 frames. However, since the frame interval can be adjusted according to the placement environment of the surveillance camera (10), the present disclosure is not limited to such examples. The preprocessing module (101) can store the selected video frames according to the predetermined frame interval in a buffer. The preprocessing module (101) can perform preprocessing to improve brightness and contrast on the video frames stored in the buffer. The preprocessing module (101) can perform a lightweight image enhancement algorithm that is feasible on a system-on-chip (SoC). For example, the preprocessing module (101) may perform algorithms such as brightness correction, contrast enhancement, gamma correction, histogram equalization, and contrast limited adaptive histogram equalization (CLAHE). However, the present disclosure is not limited to these examples.

[0025] The edge extraction module (102) can extract the edges of a structure represented by a video frame using an edge detection algorithm. The edge extraction module (102) can extract the edges of a structure using a Canny edge detection algorithm. Additionally / alternatively, the edge extraction module (102) can extract the edges of a structure using a Sobel Operator, a Prewitt Operator, a Roberts Cross Operator, a Laplacian of Gaussian (LoG) filter, and a Scharr Operator.

[0026] The edge extraction module (102) can generate an edge map based on boundary lines. The edge extraction module (102) can extract linear elements (lines) of a structure from the edge map using a geometric structure extraction algorithm, such as the Hough Transform. However, the present disclosure is not limited to such examples. Here, the linear elements of the structure may include vertical and horizontal lines of the structure.

[0027] The edge extraction module (102) can generate a virtual displacement marker that visually indicates the location where deformation of the structure is automatically detected on a black-and-white video frame based on the edge map. The virtual displacement marker can be used to predict the deformation location of the structure. The color of the virtual displacement marker can be changed based on the degree of deformation of the structure. For example, the virtual displacement marker may be a deep learning-based virtual displacement marker. The degree of deformation of the structure can be calculated by a gradient vector and a linear element.

[0028] The edge extraction module (102) can generate an augmented frame. The augmented frame can be generated by overlaying a gradient vector and a virtual displacement marker onto a black-and-white video frame, respectively. During the overlay process, the transparency of the gradient vector and the virtual displacement marker can be adjusted. The augmented frame may be input image data with enhanced features for analyzing the risk level of a structure. More specifically, the augmented frame may be image data with enhanced feature information to clearly reveal the gradient deformation characteristics of the structure by performing brightness and contrast enhancement, gamma correction, black-and-white conversion, and edge extraction on the original video frame acquired from a surveillance camera.

[0029] The risk level classification module (103) receives an augmented frame generated through preprocessing and edge extraction and can classify the state of a structure represented by the augmented frame according to a predefined risk level. The risk level classification module (103) may include an object detection model. Here, the backbone of the object detection model may be changed while maintaining the header as is. For example, the backbone may be changed to GhostNet. The object detection model may be a deep learning-based object detection model, such as YOLOv11. In other words, the object detection module of the present disclosure may include a modified YOLOv11 model. However, the present disclosure is not limited to such examples.

[0030] The object detection model of the risk level classification module (103) can be trained to classify the risk level based on either the tilt angle of the structure or the degree of deformation of the structure for a plurality of input augmented frames. More specifically, the risk level classification module (103) can be trained to classify an augmented frame representing a structure into one of no risk, low risk, medium risk, and high risk.

[0031] The risk alarm generation module (104) can generate a risk alarm based on the classified risk level. The risk alarm generation module (104) can store the classified risk level in a buffer. The risk alarm generation module (104) can store all or part of the augmented frame representing the classified risk level. More specifically, the risk alarm generation module (104) can store only a part of the augmented frame representing the classified risk level according to a predetermined standard.

[0032] For example, the risk alarm generation module (104) may store only a portion of the augmented frames representing the risk level classified based on time. For example, it may assign a higher weight to the most recent augmented frames or store only the most recent N (N is a natural number) augmented frames. By doing so, the risk alarm generation module (104) can reduce the delay in determining the final risk level and increase real-time performance.

[0033] As another example, the risk alarm generation module (104) may store only a portion of the augmented frame representing the risk level classified based on reliability. For example, the risk alarm generation module (104) may not store the augmented frame even if the risk level is medium risk, if the reliability is less than 0.5.

[0034] As another example, the risk alarm generation module (104) may store only a portion of the augmented frames representing the risk level classified based on deformation persistence. For example, the augmented frames may be stored only when the same risk level appears consecutively more than a preset number of times. Through this, the risk alarm generation module (104) can eliminate instantaneous false positives, such as changes in wind speed or lighting.

[0035] As another example, the risk alarm generation module (104) may store only a portion of the augmented frames representing the degree of risk classified based on spatial consistency. For example, the risk alarm generation module (104) may store only the augmented frames representing the degree of risk that occurred at the same location of the same structure.

[0036] As another example, the risk alarm generation module (104) may store only a portion of augmented frames representing a risk level classified based on a combination of some of time, reliability, variation persistence, and spatial consistency. For example, the risk alarm generation module (104) may store only a portion of augmented frames representing a risk level classified based on time, reliability, and variation persistence. More specifically, among the last 10 augmented frames, only augmented frames in which the reliability is 0.5 or higher and the same risk level persists at least twice may be stored.

[0037] The risk alarm generation module (104) can calculate the frequency of the risk level for each augmented frame. The risk alarm generation module (104) can determine the final risk information based on the calculated frequency. The risk alarm generation module (104) can determine the reliability of the risk level for each augmented frame. The risk alarm generation module (104) can generate a risk alarm according to the final risk level. For example, the risk alarm generation module (104) may generate a risk alarm in green for no risk or not display a risk alarm. The risk alarm generation module (104) may generate a risk alarm in yellow for low risk, orange for medium risk, and red for high risk.

[0038] The danger alarm generation module (104) can display the danger location of the structure on the augmentation frame. The displayed danger location may be indicated as a circle or an ellipse. The danger alarm generation module (104) can determine the radius or the length of the major axis of the circle or ellipse according to the final level of danger. After generating the danger alarm, the danger alarm generation module (104) can initialize the augmentation frame stored in the buffer.

[0039] After generating a risk alarm, the risk alarm generation module (104) can generate a new risk alarm by accumulating augmented frames stored in a buffer. At this time, the risk alarm generation module (104) can accumulate all or part of the augmented frames stored in the buffer. The risk alarm generation module (104) can store part of the augmented frames stored in the buffer according to the same criteria as the method by which the risk alarm generation module (104) stores part of the augmented frames indicating the degree of risk according to a predetermined criterion. For example, the risk alarm generation module (104) can store only part of the augmented frames stored in the buffer based on time, reliability, deformation persistence, spatial consistency, or a combination thereof.

[0040] FIG. 3 is a flowchart illustrating a method for notifying a risk according to one embodiment of the present disclosure.

[0041] The method of Fig. 3 below is a method implemented on a system-on-chip (SoC).

[0042] The surveillance camera (10) acquires multiple video frames (S301). The surveillance camera (10) can acquire multiple video frames of the structure.

[0043] The surveillance camera (10) sets the interval between multiple video frames to a predetermined frame interval (S302). For example, the surveillance camera (10) can set the interval between multiple video frames to 25 to 30 frames.

[0044] The surveillance camera (10) stores multiple selected video frames in a buffer at intervals (S303).

[0045] The surveillance camera (10) performs preprocessing to improve the brightness and contrast of a plurality of video frames (S304). The surveillance camera (10) performs preprocessing using a lightweight image enhancement algorithm executable on a system-on-chip. The lightweight image enhancement algorithm includes any one of gamma correction, histogram equalization, and contrast-limited adaptive histogram equalization. However, the present disclosure is not limited to these examples.

[0046] The surveillance camera (10) converts multiple video frames into multiple black-and-white video frames (S305). To reduce computational complexity, the surveillance camera (10) can convert multiple video frames that have undergone preprocessing into multiple black-and-white video frames.

[0047] The surveillance camera (10) extracts multiple gradient vectors from each of the multiple black-and-white video frames (S306).

[0048] The surveillance camera (10) extracts the boundary line of the structure using an edge detection algorithm based on a plurality of gradient vectors (S307). The surveillance camera (10) may use an optimized Canny edge detection algorithm to increase computational efficiency on a system-on-chip. Here, the Canny edge detection algorithm may include fixed kernel size convolution, reduced Gaussian smoothing, and quantized gradient thresholds.

[0049] The surveillance camera (10) generates an edge map based on the boundary line (S308).

[0050] The surveillance camera (10) extracts linear elements of the structure using a geometric structure extraction algorithm from an edge map (S309). Here, the linear elements of the structure may include vertical and horizontal lines of the structure.

[0051] The surveillance camera (10) generates a displacement marker based on an edge map (S310). The surveillance camera (10) can generate a virtual displacement marker that visually indicates the location where deformation of the structure is automatically detected on a black-and-white video frame based on the edge map. The color of the virtual displacement marker may be changed based on the degree of deformation of the structure calculated by a gradient vector and a linear element.

[0052] The surveillance camera (10) generates multiple augmented frames by overlaying a gradient vector and a displacement marker onto each of the multiple black-and-white video frames (S311).

[0053] The surveillance camera (10) classifies the risk level of a structure included in each augmented frame by using a plurality of augmented frames as input to an object detection model (S312). Here, the object detection model can be trained to receive the augmented frames as input and output one of the classes of no risk, low risk, medium risk, and high risk based on the inclination angle of the structure. To reduce the amount of computation (FLOPs) and memory usage of the object detection model on a system-on-chip, the backbone may be changed to a GhostNet-based convolutional layer. For example, the object detection model may be a modified YOLOv11 model with a modified backbone. However, the present disclosure is not limited to such examples.

[0054] The surveillance camera (10) stores the risk level of each augmented frame in a buffer (S313).

[0055] The surveillance camera (10) calculates the frequency of the risk level stored in the buffer (S314).

[0056] The surveillance camera (10) determines the highest frequency risk level as the final risk level based on frequency (S315). The surveillance camera (10) can determine the class with the highest frequency as the final risk level by using a temporal voting mechanism for the output classes. If there are risk levels with the same frequency, the surveillance camera (10) can determine the risk level of the most recent augmented frame as the final risk level. Additionally / alternatively, if there are risk levels with the same frequency, the surveillance camera (10) can determine the risk level with higher reliability among the most recent augmented frames as the final risk level. Additionally / alternatively, if there are risk levels with the same frequency, the surveillance camera (10) can maintain the previously determined final risk level.

[0057] The surveillance camera (10) generates a risk warning for the structure based on the final risk level (S316). The surveillance camera (10) can generate a risk warning for the structure when the final risk level indicates a structural risk to the structure, for example, when the augmentation frame is classified as medium risk or high risk. Here, the risk warning may be displayed as a circle or ellipse on the augmentation frame indicating the risk location of the structure. The risk warning may be displayed in different colors depending on the classified risk level. For example, no risk may be displayed in green, low risk in yellow, medium risk in orange, and high risk in red. However, the present disclosure is not limited to these examples. The surveillance camera (10) may determine the radius or length of the major axis of the circle or ellipse in proportion to the final risk level.

[0058] FIG. 4 is a diagram illustrating the preprocessing process of a video frame according to one embodiment of the present disclosure.

[0059] The preprocessing module (101) can perform preprocessing of video frames. The preprocessing module (101) can correct the brightness and contrast of video frames to improve the accuracy of gradient analysis in an edge computing environment. More specifically, the preprocessing module (101) can perform brightness and contrast enhancement processing on the acquired video frames using a lightweight image enhancement algorithm, and then apply convolution operations to normalize the brightness and contrast characteristics of the image. The preprocessing module (101) can perform deconvolution operations to restore detailed information that may be lost due to convolution processing. The preprocessing module (101) can remove noise using smoothing operations using a Gaussian kernel. The preprocessing module (101) can perform preprocessing on video frames using M (M is a natural number) times convolution operations, deconvolution operations, and smoothing operations using a Gaussian kernel. Here, the width (W), height (H), number of pixel channels (D), and total number of video frames (M) of the input video frame sequence and the output video frame sequence may be the same.

[0060] FIG. 5 is a schematic diagram illustrating a deep learning-based object detection model according to one embodiment of the present disclosure.

[0061] The header of the deep learning-based object detection model is maintained, but the backbone may be changed. For example, the changed backbone may be GhostNet. The surveillance camera (10) may include the modified deep learning-based object detection model. More specific details are described below in FIG. 6.

[0062] FIG. 6 is a detailed diagram showing a deep learning-based object detection model according to one embodiment of the present disclosure.

[0063] As shown in Fig. 6, CBS can be composed of convolution, batch normalization, and sigmoid linear units (activation functions). C2PSA can be composed of two Cross Stage Partial (CSP) blocks and a Polarized Self-Attention mechanism.

[0064] Multiple input augmented frames can classify the risk level by passing through the backbone, neck, and head, which are transformed into a deep learning-based object detection model GhostNet. The classified risk level can be stored in a buffer.

[0065] FIG. 7 is a schematic block diagram illustrating an exemplary computing device that can be used to implement a method or device according to the present disclosure.

[0066] The computing device (700) may include some or all of memory (701), a processor (702), storage (703), an input / output interface (704), and a communication interface (705). The computing device (700) may be a stationary computing device such as a desktop computer, a server, an AI accelerator, etc., as well as a mobile computing device such as a laptop computer, a smartphone, etc. The computing device (700) may structurally or functionally include at least a part of the surveillance camera (10). The computing device (700) may include any specialized hardware accelerator capable of processing the computations of an artificial intelligence model in an efficient manner. For example, the computing device (700) may include a graphic processing unit (GPU), a tensor processing unit (TPU), or a neural processing unit (NPU).

[0067] Memory (701) may store a program that enables the processor (702) to perform a method or operation according to various embodiments of the present disclosure. For example, the program may include a plurality of instructions executable by the processor (702), and the aforementioned method(s) or operation(s) may be performed by executing the plurality of instructions by the processor (702).

[0068] The memory (701) may be a single memory or multiple memories. In this case, information necessary to perform a method or operation according to various embodiments of the present disclosure may be stored in a single memory or divided and stored in multiple memories. If the memory (701) is composed of multiple memories, the multiple memories may be physically separated.

[0069] The memory (701) may include at least one of volatile memory and non-volatile memory. Volatile memory includes Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), and non-volatile memory includes flash memory.

[0070] The processor (702) may include at least one core capable of executing at least one instruction. The processor (702) may execute instructions stored in memory (701). The processor (702) may be a single processor or multiple processors.

[0071] Storage (703) retains stored data even when power supplied to the computing device (700) is cut off. For example, storage (703) may include non-volatile memory and may include storage media such as magnetic tape, optical disk, and magnetic disk.

[0072] A program stored in storage (703) can be loaded into memory (701) before being executed by the processor (702). Storage (703) can store a file written in a programming language, and a program generated from the file by a compiler, etc., can be loaded into memory (701). Storage (703) can store data to be processed by the processor (702) and / or data processed by the processor (702).

[0073] The input / output interface (704) may provide an interface with input devices such as a keyboard, mouse, etc. and / or output devices such as a display device, printer, etc. The operator may use an input device to trigger the execution of a program by the processor (702) and / or use an output device to check the processing results of the processor (702).

[0074] The communication interface (705) may provide access to an external network. For example, the computing device (700) may communicate with other devices using the communication interface (705). The other devices may include an external server.

[0075] Each component of the method or device according to the present invention may be implemented in hardware or software, or in a combination of hardware and software. Additionally, the function of each component may be implemented in software, and a microprocessor may be implemented to execute the function of the software corresponding to each component.

[0076] Various embodiments of the systems and techniques described herein may be realized as digital electronic circuits, integrated circuits, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include being implemented as one or more computer programs executable on a programmable system. A programmable system comprises a storage system, at least one input device, and at least one programmable processor (which may be a special-purpose processor or a general-purpose processor) coupled to receive data and instructions from at least one output device and to transmit data and instructions to them. Computer programs (which are also known as programs, software, software applications, or code) include instructions of the programmable processor and are stored on a "computer-readable recording medium."

[0077] Computer-readable recording media include all types of recording devices in which data that can be read by a computer system is stored. Such computer-readable recording media may be non-volatile or non-transitory media such as ROM, CD-ROM, magnetic tape, floppy disk, memory card, hard disk, magneto-optical disk, and storage device, and may also include transitory media such as data transmission media. Additionally, computer-readable recording media may be distributed across networked computer systems, and computer-readable code may be stored and executed in a distributed manner.

[0078] Although the flowcharts in this specification describe each process as being executed sequentially, this is merely an illustrative explanation of the technical concept of one embodiment of the present disclosure. In other words, a person skilled in the art to which one embodiment of the present disclosure belongs may modify and adapt the flowchart in various ways, such as by changing the order described in the flowchart or executing one or more of the processes in parallel, without departing from the essential features of one embodiment of the present disclosure; therefore, the flowchart is not limited to a chronological order.

[0079] The foregoing description is merely an illustrative explanation of the technical concept of the present embodiment, and a person skilled in the art to which the present embodiment belongs would be able to make various modifications and variations within the scope of the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment.

Claims

Claim 1 A method for notifying danger by detecting tilt deformation of a structure performed by a surveillance camera equipped with edge computing capabilities, comprising: a process of acquiring multiple video frames; a process of setting the interval of the multiple video frames to a predetermined frame interval; a process of storing the selected multiple video frames in a buffer for each interval; a process of performing preprocessing to enhance the brightness and contrast of the multiple video frames; a process of converting the multiple video frames into multiple black-and-white video frames; a process of extracting multiple tilt vectors from each of the multiple black-and-white video frames; a process of extracting the edge of the structure using an edge detection algorithm based on the multiple tilt vectors; a process of generating an edge map based on the edge map; a process of extracting a linear element (line) of the structure from the edge map using a geometric structure extraction algorithm; a process of generating a virtual displacement marker that visually indicates the location where deformation of the structure is automatically detected on the black-and-white video frames based on the edge map; and a process of overlaying the tilt vector and the virtual displacement marker on each of the multiple black-and-white video frames to generate a plurality A method comprising: a process of generating augmented frames; a process of classifying the risk level of a structure included in each augmented frame using the plurality of augmented frames as input to an object detection model; a process of storing the risk level of each augmented frame in the buffer; a process of calculating the frequency of the risk level stored in the buffer; a process of determining the risk level with the highest frequency as the final risk level based on the frequency; and a process of generating a risk warning for the structure based on the final risk level, wherein the risk warning indicates the risk location of the structure as a circle or ellipse on the augmented frame. Claim 2 The method of claim 1, wherein the method is implemented on a system-on-chip (SoC). Claim 3 A method according to claim 1, wherein the process of performing preprocessing to improve the brightness and contrast of the plurality of video frames is performed using a lightweight image enhancement algorithm executable on a system-on-chip (SoC). Claim 4 In claim 3, the lightweight image enhancement algorithm comprises any one of Gamma Correction, Histogram Equalization, and Contrast-Limited Adaptive Histogram Equalization (CLAHE). Claim 5 In claim 1, the edge detection algorithm is a Canny edge detection algorithm. Claim 6 A method according to claim 1, wherein the linear elements of the structure include vertical and horizontal lines of the structure. Claim 7 A method according to claim 1, wherein the virtual displacement marker changes color based on the degree of deformation of the structure calculated by the gradient vector and the linear element. Claim 8 A method according to claim 1, wherein the object detection model is trained to output one of no risk, low risk, medium risk, and high risk based on the tilt angle of the structure with respect to the augmented frame. Claim 9 In claim 1, the object detection model is a method in which the backbone is changed to a convolutional layer based on GhostNet. Claim 10 In claim 1, the process of determining the highest frequency risk level based on the frequency as the final risk level is a method of determining the risk level of the most recent augmented frame as the final risk level when there are risk levels with the same frequency. Claim 11 In claim 10, a method for determining the risk level with higher reliability among the most recent augmented frames as the final risk level when the same risk level exists for the frequency. Claim 12 In claim 11, a method of maintaining a previously determined final risk level when the same risk level exists at the frequency. Claim 13 A method according to claim 1, wherein the radius or length of the major axis of the circle or ellipse is determined in proportion to the final risk level. Claim 14 At least one memory for storing instructions; and includes at least one processor, wherein the at least one processor, by executing the instructions, acquires a plurality of video frames from a surveillance camera equipped with edge computing capabilities, sets the interval of the plurality of video frames to a predetermined frame interval, stores the selected plurality of video frames at each interval in a buffer, performs preprocessing to enhance the brightness and contrast of the plurality of video frames, converts the plurality of video frames into a plurality of grayscale video frames, extracts a plurality of gradient vectors from each of the plurality of grayscale video frames, extracts the edge of a structure using an edge detection algorithm based on the plurality of gradient vectors, generates an edge map based on the edge map, extracts a linear element (line) of the structure using a geometric structure extraction algorithm from the edge map, generates a virtual displacement marker that visually indicates the location where deformation of the structure on the grayscale video frame is automatically detected based on the edge map, and overlays the gradient vector and the virtual displacement marker on each of the plurality of grayscale video frames to obtain a plurality of augmented frames A device that generates a plurality of augmented frames, classifies the risk level of a structure included in each augmented frame using the plurality of augmented frames as input to an object detection model, stores the risk level of each augmented frame in a buffer, calculates the frequency of the risk level stored in the buffer, determines the risk level with the highest frequency as the final risk level based on the frequency, generates a risk warning for the structure based on the final risk level, and displays the risk location of the structure as a circle or ellipse on the augmented frames. Claim 15 A computer program stored on a computer-readable recording medium to execute each process included in the method according to any one of claims 1 to 13. Claim 16 A CCTV camera that operates in accordance with any one of paragraphs 1 through 13.

Images