System and method for analyzing CCTV image

Abstract

An image analysis method according to the present invention includes the steps of: storing an image analysis event obtained by analyzing images collected from a plurality of CCTVs; outputting the image analysis event as an image analysis event item in a multi-dimensional form with reference to the plurality of CCTVs and a timeline; receiving a focus input for a specific image analysis event from a user; and performing movement to the plurality of CCTVs in the image analysis event item or movement to a timeline in a single CCTV in response to the focus input.

Description

CCTV video analysis system and method

[0001] The present invention relates to a CCTV video analysis system and method, and more particularly, to a system and method capable of storing and outputting video analysis event items in a multidimensional format.

[0002] Recently, intelligent CCTV video analysis system technology that automatically recognizes and controls events such as disasters, accidents, incidents, and accidents by analyzing CCTV videos input in real time is being actively researched.

[0003] Conventional intelligent CCTV video analysis systems operate by analyzing CCTV video in real time to detect events, and then storing, searching, and displaying these video analysis events.

[0004] In particular, when displaying saved video analysis events to the controller, the method mainly used was to display video analysis events that occurred in a specific CCTV in a list format when the controller selects the CCTV, or to display the searched events in a list format when the controller performs an event search based on a text query for a large amount of CCTV footage.

[0005] However, these conventional methods have the disadvantage of preventing controllers from viewing video analysis events generated from a large number of CCTV cameras at a glance, hindering rapid CCTV monitoring. This makes it difficult for controllers to respond quickly and potentially lead to missed or delayed recognition of important events.

[0006] Therefore, to support rapid control by controllers, a new form of CCTV video analysis event presentation is needed that allows for at-a-glance review and effective management of video analysis events generated from a large number of CCTV cameras. In other words, it is necessary to provide controllers with information in a more intuitive and visual manner, while also providing the ability to quickly respond to a large number of events.

[0007] An embodiment of the present invention provides a CCTV video analysis system and method that outputs a multidimensional list of multiple events analyzed from videos captured by multiple CCTVs, thereby allowing a user to easily check and analyze multiple CCTVs and multiple events over time at a glance.

[0008] However, the technical task that this embodiment seeks to achieve is not limited to the technical task described above, and other technical tasks may exist.

[0009] As a technical means for achieving the above-described technical task, a video analysis method according to a first aspect of the present invention includes the steps of: storing a video analysis event analyzed for videos collected from a plurality of CCTVs; outputting the video analysis event as a video analysis event item in a multidimensional format based on the plurality of CCTVs and a timeline; receiving a focus input for a specific video analysis event from a user; and performing a movement to the plurality of CCTVs or a movement to a timeline within a single CCTV within the video analysis event item in response to the focus input.

[0010] In some embodiments of the present invention, the step of outputting the video analysis event item in a multidimensional format based on the plurality of CCTVs and a timeline may store the video analysis event in a database in a multidimensional format based on the timeline.

[0011] In some embodiments of the present invention, the step of storing a video analysis event analyzed for videos collected from the plurality of CCTVs stores a video analysis event including identification information of a CCTV corresponding to the video analysis event, URL information of the corresponding video, and time information of the video analysis event, wherein the time information of the video analysis event may include occurrence time and end time of the video analysis event, and start frame information and end frame information of occurrence of the video analysis event in the video.

[0012] In some embodiments of the present invention, the step of performing movement to the plurality of CCTVs or movement to a timeline within a single CCTV in response to the focus input may, when the focus input is an input of movement up or down, move the focus to a video analysis event of a previous or subsequent channel that occurred in the same time zone as the currently focused video analysis event.

[0013] In some embodiments of the present invention, the step of performing movement to the plurality of CCTVs or movement to a timeline within a single CCTV in response to the focus input may, when the focus input is a movement input to the left or right, move the focus to a previous or subsequent video analysis event of the same channel as the currently focused video analysis event.

[0014] In addition, a video analysis system according to one embodiment of the present invention includes a communication unit that receives videos captured from a plurality of CCTVs, a memory in which a program for analyzing a video analysis event based on the collected videos and storing and outputting the video analysis event in a multidimensional form is stored, and a processor that executes the program stored in the memory, thereby storing a video analysis event analyzed for the collected videos, outputting the video analysis event as a video analysis event item in a multidimensional form based on a timeline, and, upon receiving a focus input for a specific video analysis event from a user, performs movement to the plurality of CCTVs within the video analysis event item or movement to a timeline within a single CCTV in response to the focus input.

[0015] In some embodiments of the present invention, the processor may store the image analysis event in a database in a multidimensional form based on a timeline.

[0016] In some embodiments of the present invention, the processor stores a video analysis event including identification information of a CCTV corresponding to the video analysis event, URL information of the corresponding video, and time information of the video analysis event, wherein the time information of the video analysis event may include occurrence time and end time of the video analysis event, and start frame information and end frame information of occurrence of the video analysis event in the video.

[0017] In some embodiments of the present invention, the processor may move the focus to an image analysis event in a previous or subsequent channel that occurred in the same time zone as the currently focused image analysis event, if the focus input is an up or down movement input.

[0018] In some embodiments of the present invention, the processor can move the focus to a previous or subsequent image analysis event of the same channel as the currently focused image analysis event when the focus input is a left or right movement input.

[0019] In addition, other methods for implementing the present invention, other systems, and computer-readable recording media recording a computer program for executing the above methods may be further provided.

[0020] According to one embodiment of the present invention described above, data generated from a large number of CCTV cameras can be efficiently managed and visualized. This allows controllers to display a large number of CCTV events in a multidimensional format on a single screen, enabling more intuitive understanding, rather than having to monitor multiple screens simultaneously and track events in real time. This reduces the controller's field of vision, reduces the workload in complex environments, and reduces fatigue.

[0021] Furthermore, one embodiment of the present invention can rapidly detect events occurring in real time and effectively convey that information visually to users. Controllers can quickly recognize events, take necessary actions more quickly, and prevent the spread of incidents or disasters. This can improve the efficiency of control operations and speed response times.

[0022] Furthermore, one embodiment of the present invention can effectively manage and visualize stored event data, enabling controllers to quickly search and analyze information related to past disasters, accidents, incidents, or incidents. This capability supports pre-investigation, event reenactment, evidence collection, and accurate decision-making, enabling controllers to quickly access and manage critical information.

[0023] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0024] FIG. 1 is a block diagram illustrating the configuration of a CCTV video analysis system (100) according to one embodiment of the present invention.

[0025] FIG. 2 is a flowchart illustrating steps for storing an image analysis event according to one embodiment of the present invention.

[0026] FIG. 3 is a flowchart illustrating a process for outputting image analysis event items in one embodiment of the present invention.

[0027] FIG. 4 is a diagram illustrating an example of displaying image analysis event items in a multidimensional form in one embodiment of the present invention.

[0028] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined solely by the scope of the claims.

[0029] The terminology used herein is for the purpose of describing embodiments only and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise. As used herein, the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the mentioned components. Like reference numerals refer to like components throughout the specification, and "and / or" includes each and any combination of one or more of the mentioned components. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one component from another. Therefore, it should be understood that a first component mentioned below may also be a second component within the technical spirit of the present invention.

[0030] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in their common sense to those skilled in the art to which the present invention pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0031] Hereinafter, a CCTV image analysis system (100) according to one embodiment of the present invention will be described with reference to FIG. 1.

[0032] FIG. 1 is a block diagram illustrating the configuration of a CCTV video analysis system (100) according to one embodiment of the present invention.

[0033] A CCTV video analysis system (100) according to one embodiment of the present invention includes an input unit (110), a communication unit (120), a display unit (130), a memory (140), and a processor (150). At this time, the CCTV video analysis system in the present invention may be a VMS.

[0034] The input unit (110) generates input data in response to user input of the CCTV video analysis system (100). At this time, the user input may include a selection input for each video captured by CCTV, a movement input within a video analysis event item in which a video analysis event, which is a result of CCTV video analysis, is displayed in a multidimensional manner, and other control inputs such as video playback, pause, video information request, and termination based on a generated CCTV video playlist.

[0035] The input unit (110) includes at least one input means. The input unit (110) may include a keyboard, a key pad, a dome switch, a touch panel, a touch key, a mouse, a menu button, etc.

[0036] The communication unit (120) transmits and receives data with a plurality of CCTVs, and also performs communication with external devices such as servers or data collection devices to transmit and receive data. The communication unit (120) may include both a wired communication module and a wireless communication module. The wired communication module may be implemented as a power line communication device, a telephone line communication device, a cable home (MoCA), Ethernet, IEEE1294, an integrated wired home network, and an RS-485 control device. In addition, the wireless communication module may be configured as a module for implementing functions such as WLAN (wireless LAN), Bluetooth, HDR WPAN, UWB, ZigBee, Impulse Radio, 60GHz WPAN, Binary-CDMA, wireless USB technology, wireless HDMI technology, and other 5G (5th generation communication), LTE-A (long term evolution-advanced), LTE (long term evolution), and Wi-Fi (wireless fidelity).

[0037] The display unit (130) displays display data according to the operation of the CCTV video analysis system (100). For example, the display unit (130) can display on the screen a multidimensional video analysis event item, a list of detected objects, a list of objects being tracked, a list of their history, a CCTV video playlist, and CCTV video based on the CCTV video and the CCTV video playlist.

[0038] The display unit (130) includes a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a micro electro mechanical systems (MEMS) display, and an electronic paper display. The display unit (130) may be implemented as a touch screen by being combined with the input unit (110).

[0039] The memory (140) stores programs for analyzing image analysis events based on collected images and storing and outputting the image analysis events in a multidimensional format. Here, the memory (140) is a general term for non-volatile storage devices and volatile storage devices that maintain stored information even when power is not supplied. For example, the memory (140) may include NAND flash memory such as a compact flash (CF) card, an SD (secure digital) card, a memory stick, a solid-state drive (SSD), and a micro SD card, a magnetic computer storage device such as a hard disk drive (HDD), and an optical disc drive such as a CD-ROM or DVD-ROM.

[0040] The processor (150) can control at least one other component (e.g., hardware or software component) of the CCTV video analysis system (100) by executing software such as a program, and can perform various data processing or operations.

[0041] The processor (150) stores the image analysis event according to the image analysis in a multidimensional format, outputs it as an image analysis event item in a multidimensional format, and selects the corresponding event by moving within the image analysis event item according to the user's selection input (hereinafter, focus input).

[0042] Meanwhile, in one embodiment of the present invention, the processor (150) may use at least one of a machine learning, neural network, or deep learning algorithm as an artificial intelligence (AI) algorithm for event analysis, etc. For example, at least one of a machine learning, neural network, or deep learning algorithm may be used as an artificial intelligence (AI) algorithm, and examples of a neural network may include models such as a CNN (Convolutional Neural Network), a DNN (Deep Neural Network), and an RNN (Recurrent Neural Network).

[0043] Below, a method performed by a CCTV video analysis system (100) according to one embodiment of the present invention will be described.

[0044] FIG. 2 is a flowchart illustrating steps for storing an image analysis event according to one embodiment of the present invention.

[0045] First, as multiple CCTV cameras begin operating, each camera captures images in real time (S110). Multiple CCTV cameras are installed in various locations and locations, and each captures events and situations occurring within its respective surveillance area.

[0046] Next, the video received from multiple CCTVs is analyzed to detect whether an event related to a disaster, calamity, incident, or accident has occurred (S120). For example, events related to disaster, calamity, incident, or accident are detected through video motion detection, object detection, pattern recognition, and other techniques using computer vision, deep learning algorithms, and the like. In the present invention, the detected event is referred to as a video analysis event.

[0047] Next, as a result of the event analysis, it is determined whether a video analysis event has occurred (S130), and if the determination result shows that no video analysis event has been detected, it is determined whether there has been a user's termination input (S140), and if there has been a termination input, the process of saving the video analysis event to the database is terminated.

[0048] In contrast, if a video analysis event is detected as a result of the determination, detailed information of the corresponding video analysis event is recorded in the database (S150). At this time, the processor (150) may include identification information of the CCTV corresponding to the video analysis event, URL information of the corresponding video, time information of the video analysis event, and event type information. Here, the time information of the video analysis event may include occurrence time and end time of the video analysis event (or duration of the video analysis event), start frame information and end frame information of the video analysis event occurrence in the video (or relative time at which the video analysis event occurred or ended based on the stored CCTV video).

[0049] At this time, the processor (150) may store event information in a multidimensional format when storing video analysis events in a database. In one embodiment, the multidimensional format may include identification information of multiple CCTVs and timelines of multiple video analysis events as components.

[0050] Afterwards, it is determined whether there has been a user's termination input (S140), and if there has been a termination input, the process of saving the video analysis event to the database is terminated.

[0051] FIG. 3 is a flowchart illustrating a process for outputting image analysis event items in one embodiment of the present invention. FIG. 4 is a diagram illustrating an example of displaying image analysis event items in a multidimensional format in one embodiment of the present invention.

[0052] First, multiple pre-analyzed video analysis events are acquired from the database (S210). As described above, the video analysis events may include CCTV identifiers, event occurrence time information, event type information, etc.

[0053] Next, the video analysis event is output as a video analysis event item in a multidimensional format based on multiple CCTVs and timelines (S220).

[0054] One embodiment of the present invention can output a video analysis event item with a timeline as the horizontal axis and a CCTV identification number (or name) as the vertical axis, or conversely, can output a video analysis event item with a CCTV identification number as the horizontal axis and a timeline as the vertical axis.

[0055] At this time, in one embodiment of the present invention, in the case of a video analysis event item that expresses a video analysis event based on a timeline, it is possible to express graphically or textually that no event occurred for a section in which an event did not occur.

[0056] In addition, when expressing a video analysis event based on a timeline, one embodiment of the present invention can express simple or detailed information about the event in the section where the event occurred in the form of graphics or text.

[0057] Additionally, you can view detailed information about video analysis events by focusing on specific events.

[0058] Next, a focus input for a specific video analysis event is received from the user (S230). Then, in response to the focus input, movement to multiple CCTVs within the video analysis event items or movement to a timeline within a single CCTV is performed (S240).

[0059] In one embodiment, if the user's focus input is determined to be 'up', the focus is moved to an event of a previous channel that occurred at the same time as the current focus video analysis event (S241). After the focus movement is completed, the process returns to step S230.

[0060] In one embodiment, if the user's focus input is determined to be 'ha', the focus is moved to an event of a subsequent channel that occurred at the same time as the current focus video analysis event (S242). After the focus movement is completed, the process returns to step S230.

[0061] In one embodiment, if the user's focus input is determined to be 'left', the focus is moved to the previous video analysis event of the same channel as the current focus video analysis event (S243). After the focus movement is completed, the process returns to step S230.

[0062] In one embodiment, if the user's focus input is determined to be 'right', the focus is moved to a subsequent video analysis event on the same channel as the current focus video analysis event (S244). After the focus movement is completed, the process returns to step S230.

[0063] In one embodiment, when the user's input is determined to be 'play' or 'select', the video corresponding to the currently focused video analysis event is played (S245). At this time, in order to play the video corresponding to the currently focused video analysis event, the CCTV video URL of the database and the time information of the video analysis event may be used in relation to the currently focused video analysis event. Meanwhile, Trick Play (Play, Pause, Fast Forward, Rewind, Stop) may be supported during video playback. After the video playback is completed, the process returns to step S230.

[0064] Meanwhile, while one embodiment of the present invention has been described as a two-dimensional format based on a multidimensional timeline and CCTV channel information, this is not necessarily limited to this. That is, after setting the timeline and CCTV channel information as separate axes, location, event importance, etc. can be configured as separate axes to store and output video analysis event items.

[0065] Furthermore, one embodiment of the present invention can operate by varying the position and size information of each video analysis event within the video analysis event item based on the ratio of time information occupied by each video analysis event, rather than simply outputting the timeline and CCTV channel information. For example, in FIG. 4, since the smoke event or fire event of CCTV 3 has the longest timeline among all video analysis events, the size of the corresponding event can be displayed by setting it to be proportionally larger than that of other video analysis events.

[0066] Information of each video analysis event within the video analysis event item may be configured to include a thumbnail, and in the case of the preceding embodiment, the size of the thumbnail of each video analysis event within the video analysis event item may be adjusted.

[0067] Additionally, in one embodiment of the present invention, for each video analysis event, the importance (urgency of processing, risk, importance of installation location, etc.) may be preset, and the size may be adjusted to further reflect the importance.

[0068] In addition, in one embodiment of the present invention, each video analysis event configured within the video analysis event item may be selectively configured rather than all events detected by analysis from the video being configured.

[0069] In one embodiment, in the case of a video analysis event analyzed from a video captured by a PTZ CCTV, those analyzed from a video that satisfies or exceeds the minimum viewing time of the PTZ CCTV may be included, and video analysis events analyzed from a video that satisfies or falls below the minimum viewing time may not be included. In this case, the minimum viewing time may be set separately according to the type of each event, and when multiple events occur simultaneously, the minimum viewing time may be a minimum viewing time determined by their sum or weight. Meanwhile, the minimum viewing time for each event may be set based on the average event start-end time information from the video captured by the entire CCTV and previously collected CCTV videos, and the average viewing time including the movement time of the PTZ CCTV to detect the event. It should be noted that the minimum viewing time may be periodically updated as the entire CCTV continues to operate.

[0070] Meanwhile, in the above description, steps S110 to S245 may be further divided into additional steps or combined into fewer steps, depending on the implementation of the present invention. Furthermore, some steps may be omitted as needed, and the order of steps may be changed. Furthermore, even if other details are omitted, the details described in FIG. 1 and FIGS. 2 and 3 are mutually applicable.

[0071] The embodiments of the present invention described above can be implemented as a program (or application) and stored in a medium to be executed in conjunction with a hardware server.

[0072] The above-described program may include codes coded in a computer language, such as C, C++, JAVA, or machine language, that can be read by the processor (CPU) of the computer through the device interface of the computer, so that the computer reads the program and executes the methods implemented as a program. Such codes may include functional codes related to functions that define functions necessary for executing the methods, and may include control codes related to execution procedures necessary for the processor of the computer to execute the functions according to a predetermined procedure. In addition, such codes may further include memory reference-related codes regarding which location (address address) of the internal or external memory of the computer should reference additional information or media necessary for the processor of the computer to execute the functions. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to send and receive during communication.

[0073] The above storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program can be stored in various recording media on various servers that the computer can access or in various recording media on the user's computer. In addition, the medium can be distributed across network-connected computer systems, so that computer-readable code can be stored in a distributed manner.

[0074] The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer-readable recording medium well known in the art to which the present invention pertains.

[0075] While the embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering the technical concept or essential features thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

Claims

1. In a method performed by a computer, A step of storing a video analysis event analyzed from videos collected from multiple CCTVs; A step of outputting the above video analysis event as a video analysis event item in a multidimensional format based on the plurality of CCTVs and timelines; A step of receiving a focus input for a specific video analysis event from a user; and Including a step of performing movement to the plurality of CCTVs or movement to a timeline within a single CCTV within the video analysis event item in response to the focus input. CCTV footage analysis method.

2. In paragraph 1, The step of outputting the video analysis event items in a multidimensional format based on the plurality of CCTVs and timelines is as follows: The above video analysis event is stored in a database in a multidimensional form based on a timeline. CCTV footage analysis method.

3. In paragraph 2, The step of storing the video analysis event analyzed from the video collected from the above multiple CCTVs is as follows: Store a video analysis event including the identification information of the CCTV corresponding to the above video analysis event, the URL information of the corresponding video, and the time information of the above video analysis event. The time information of the above video analysis event includes the occurrence time and end time of the video analysis event, the start frame information of the video analysis event occurrence in the video, and the end frame information. CCTV footage analysis method.

4. In paragraph 1, The step of performing movement to the plurality of CCTVs or movement to a timeline within a single CCTV within the video analysis event item in response to the above focus input is: If the above focus input is an up or down movement input, the focus moves to a video analysis event in a previous or subsequent channel that occurred in the same time zone as the currently focused video analysis event. CCTV footage analysis method.

5. In paragraph 1, The step of performing movement to the plurality of CCTVs or movement to a timeline within a single CCTV within the video analysis event item in response to the above focus input is: If the above focus input is a movement input to the left or right, the focus is moved to the previous or next video analysis event of the same channel as the currently focused video analysis event. CCTV footage analysis method.

6. A communication unit that receives video footage from multiple CCTVs. A memory storing a program for analyzing video analysis events based on the collected video and storing and outputting the video analysis events in a multidimensional format; and By executing the program stored in the above memory, a processor is included that stores the video analysis event analyzed for the collected video, outputs the video analysis event as a video analysis event item in a multidimensional form based on a timeline, and, upon receiving a focus input for a specific video analysis event from a user, performs movement to the plurality of CCTVs within the video analysis event item or movement to a timeline within a single CCTV in response to the focus input. CCTV video analysis system.

7. In paragraph 6, The above processor stores the video analysis event in a database in a multidimensional form based on a timeline. CCTV video analysis system.

8. In paragraph 7, The above processor stores a video analysis event including identification information of the CCTV corresponding to the video analysis event, URL information of the corresponding video, and time information of the video analysis event. The time information of the above video analysis event includes the occurrence time and end time of the video analysis event, the start frame information of the video analysis event occurrence in the video, and the end frame information. CCTV video analysis system.

9. In paragraph 6, The processor moves the focus to a video analysis event in a previous or subsequent channel that occurred in the same time zone as the currently focused video analysis event, if the focus input is an up or down movement input. CCTV video analysis system.

10. In paragraph 6, The processor moves the focus to a previous or subsequent video analysis event of the same channel as the currently focused video analysis event when the focus input is a movement input to the left or right. CCTV video analysis system.

Images