Apparatus and method for updating logic for detecting pattern

Abstract

The present embodiments may provide an apparatus for updating logic, the apparatus comprising a reception unit for receiving video data, a data processing unit for generating video data information by analyzing the video data by using a deep learning model, inputting streaming data including the video data information and metadata into a data stream, and storing the streaming data as replay data, and a control unit for: determining whether a first event has occurred by analyzing the streaming data being streamed by using pattern detection logic for detecting a pattern corresponding to the first event, wherein the replay data is repeatedly analyzed using the pattern detection logic; updating the pattern detection logic on the basis of whether the first event has occurred, as determined; generating a pattern corresponding to a second event that may occur due to the occurrence of the first event; and updating the pattern detection logic to detect the pattern corresponding to the second event.

Description

Logic update device and method for detecting patterns

[0001] The present invention relates to a device and method capable of updating logic for detecting a pattern.

[0002] Monitoring systems are being utilized in a variety of fields. Conventional monitoring systems required managers to monitor video in real time, identify events occurring in the monitoring environment, and take action in response to them. These systems required multiple managers, and real-time video monitoring required a high level of concentration, resulting in high levels of manager fatigue.

[0003] As technology advances, deep learning-based monitoring systems capable of recognizing and tracking distinctive objects, such as objects or people, are rapidly evolving. These systems analyze video using deep learning algorithms and determine whether an event has occurred based on patterns detected in the analyzed video. This allows a small number of administrators to efficiently monitor the monitoring environment. However, due to the nature of deep learning algorithms, repeated testing and updates are necessary to improve reliability. Consequently, there is a growing demand for devices and methods capable of testing and updating the deep learning algorithms in deep learning-based monitoring systems.

[0004] Against this backdrop, a logic update device and method capable of testing and updating logic for detecting a pattern corresponding to an event can be provided.

[0005] In one aspect, the present embodiments may provide a logic update device including a receiving unit that receives image data, a data processing unit that analyzes the image data using a deep learning model to generate image data information, inputs streaming data including the image data information and metadata into a data stream, and stores the streaming data as replay data, and a control unit that analyzes the streamed streaming data using pattern detection logic that detects a pattern corresponding to a first event to determine whether a first event has occurred, repeatedly analyzes the replay data using the pattern detection logic, updates the pattern detection logic based on the determined occurrence of the first event, generates a pattern corresponding to a second event that may occur due to the occurrence of the first event, and updates the pattern detection logic to detect a pattern corresponding to the second event.

[0006] In another aspect, a logic update method may be provided, including a step of receiving video data, a step of analyzing the video data using a deep learning model to generate video data information, a step of inputting streaming data including the video data information and metadata into a data stream, a step of storing the streaming data as replay data, and a step of analyzing the streamed streaming data using a pattern detection logic that detects a pattern corresponding to a first event to determine whether a first event has occurred, repeatedly analyzing the replay data using the pattern detection logic, updating the pattern detection logic based on the determined occurrence of the first event, generating a pattern corresponding to a second event that may occur due to the occurrence of the first event, and updating the pattern detection logic to detect a pattern corresponding to the second event.

[0007] In a deep learning-based monitoring system, a device and method can be provided for testing and updating logic for detecting patterns corresponding to events.

[0008] Figure 1 is a schematic diagram for explaining a logic update device according to the present embodiments.

[0009] Figure 2 is a drawing for explaining a data processing unit according to the present embodiments.

[0010] Figure 3 is a diagram for explaining streaming data according to the present embodiments.

[0011] FIG. 4 is a diagram illustrating a pipeline for performing monitoring according to the present embodiments.

[0012] Figure 5 is a flowchart for explaining a logic update method according to the present embodiments.

[0013] FIG. 6 is a drawing for explaining a method of generating a pattern according to the present embodiments.

[0014] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary drawings. When adding reference numerals to components in each drawing, identical components may have the same numerals as much as possible even if they are shown in different drawings. In addition, when describing the present embodiments, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the technical idea of ​​the present invention, the detailed description may be omitted. When "includes," "has," "consists of," etc. are used in this specification, other parts may be added unless "only" is used. When a component is expressed in the singular, it may include a case in which the plural is included unless specifically stated otherwise.

[0015] Additionally, terms such as first, second, A, B, (a), (b), etc. may be used to describe components of the present disclosure. These terms are only intended to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by the terms.

[0016] In a description of the positional relationship of components, when it is described that two or more components are "connected," "combined," or "connected," it should be understood that the two or more components may be directly "connected," "combined," or "connected," but that the two or more components may also be further "interposed" with another component to be "connected," "combined," or "connected." Here, the other component may be included in one or more of the two or more components that are "connected," "combined," or "connected" to each other.

[0017] In the description of the temporal flow relationship related to components, operation methods, or manufacturing methods, for example, when the temporal or flow relationship is described as “after”, “following”, “next to”, “before”, etc., it may also include cases where it is not continuous, unless “immediately” or “directly” is used.

[0018] Meanwhile, when numerical values ​​or corresponding information (e.g., levels, etc.) for components are mentioned, even without separate explicit description, the numerical values ​​or corresponding information may be interpreted as including an error range that may occur due to various factors (e.g., process factors, internal or external impact, noise, etc.).

[0019] Hereinafter, a logic update device and method for detecting a pattern according to embodiments of the present disclosure will be described with reference to related drawings.

[0020]

[0021] FIG. 1 is a schematic diagram illustrating a logic update device according to the present embodiments. FIG. 2 is a schematic diagram illustrating a data processing unit according to the present embodiments. FIG. 2 is a schematic diagram illustrating a data processing unit according to the present embodiments. FIG. 3 is a schematic diagram illustrating streaming data according to the present embodiments. FIG. 4 is a schematic diagram illustrating a pipeline for performing monitoring according to the present embodiments.

[0022] Referring to FIG. 1, a logic update device (100) may include a receiving unit (110) that receives image data, a data processing unit (120) that analyzes image data using a deep learning model to generate image data information, inputs streaming data including image data information and meta information into a data stream, and stores the streaming data as replay data, and a control unit (130) that analyzes streaming data using a pattern detection logic that detects a pattern corresponding to a first event to determine whether a first event has occurred, repeatedly analyzes replay data using the pattern detection logic, updates the pattern detection logic based on the determined occurrence of the first event, generates a pattern corresponding to a second event that may occur due to the occurrence of the first event, and updates the pattern detection logic to detect a pattern corresponding to the second event.

[0023] The configuration of the logic update device (100) illustrated in FIG. 1 is an example for explaining the present invention and is not limited thereto. The logic update device (100) may further include other components as needed, or some components may be omitted. In this case, according to an example, each component of the logic update device (100) may be combined with one another and implemented as one according to the design method, or some components may be omitted.

[0024] A logic update device (100) according to an example may be configured with the same hardware as an image capture device installed in a monitoring environment. In this case, the receiving unit (110) may receive image data captured from the image capture device.

[0025] As another example, the logic update device (100) may be configured with hardware different from the video recording device. In this case, the receiver (110) may receive video data recorded from at least one camera installed in the monitoring environment via network communication. Here, the video data may be received in the RTSP (Real-Time Streaming Protocol) format.

[0026] A video recording device refers to a device capable of acquiring video data, such as a camera or CCTV. The video data may be a typical 2D photograph, video, or a frame representing an image at a specific point in time that can be acquired from a video recording device. In addition, the video data may include objects existing in the monitoring environment. Here, the objects may include dynamic objects, static objects, and specific phenomena that can be captured in the video data. For example, video data acquired in a factory where a fire has occurred may include objects including dynamic objects (factory workers), static objects (factory facilities), and specific phenomena (smoke, flames).

[0027] The receiving unit (110) can receive a device identification number, the date, time, and location of the image capture from the image capture device. In addition, the receiving unit (110) can obtain the file format of the image data acquired from the image capture device.

[0028] Referring back to FIG. 1, the data processing unit (120) can analyze image data received from the receiving unit (110) using a deep learning model. According to one embodiment, the deep learning model can include a Vision AI model capable of performing visual data analysis including image data. That is, the data processing unit (120) can recognize objects included in the image data using the Vision AI model.

[0029] Here, the Vision AI model may be a Visual Language Model (VLM), trained based on multimodal data and capable of understanding and processing text and images together. For example, the VLM model may include at least one of a Zero-Shot Object Detection Model and a Skeleton Extraction Model.

[0030] Multimodal data refers to data that combines at least two different types of data. VLM can be pre-trained using multimodal data combining text and images. VLM can then understand and learn the relationship between images and text, internally generating an embedding space that connects the two types of data.

[0031] For example, a VLM could be a zero-shot object detection model capable of detecting objects not previously defined. That is, a zero-shot object detection model pre-trained using multimodal data can predict objects specified in text in new images.

[0032] For example, a Zero-shot Object Detection Model trained on multimodal data can convert new image data and text into embedding vectors when input. The Zero-shot Object Detection Model can then calculate the similarity between the image embedding and the text embedding, thereby scoring a similarity score between objects and text within the image. Furthermore, the Zero-shot Object Detection Model can identify the location of the object within the image based on the calculated similarity score. The Zero-shot Object Detection Model can display the location of the object as a bounding box centered on the region with the highest similarity within the image.

[0033] The above description is not limited to an example of how the data processing unit (120) recognizes objects contained in image data using a deep learning model. The deep learning model may vary depending on hardware resources and inference time, and may be configured in various ways as needed, as long as it does not contradict the technical principles of the present disclosure.

[0034] Referring to FIG. 2, the data processing unit (120) can generate image data information. The data processing unit (120) can receive object information about objects recognized from each VLM. For example, the data processing unit (120) can receive object information about objects recognized from a Zero-Shot Object Detection Model and a Skeleton Extraction Model. The data processing unit (120) can refine the object information based on the object information received from each model. In addition, the data processing unit (120) can post-process the object information received from each model to produce post-processing information including object speed, movement direction, and distance between objects. The data processing unit (120) can generate image data information including object information and post-processing information.

[0035] The data processing unit (120) can generate metadata. The data processing unit (120) can receive information from the receiving unit (110) including at least one of a device identification number, the date, time, and location of the video capture, or the file format of the video data. The metadata can include information including at least one of a device identification number, the date, time, and location of the video capture, or the file format of the video data.

[0036] Referring to FIG. 2, the data processing unit (120) can generate streaming data including image data information and meta information. As illustrated in FIG. 3, the data processing unit (120) according to one embodiment can convert image data information and meta information into JSON (JavaScript Object Notation) format. The data processing unit (120) can process the converted JSON format image data and meta information into streaming data.

[0037] The data processing unit (120) can input processed streaming data into a data stream capable of real-time data processing. According to one embodiment, the data processing unit (120) can input streaming data into a data stream platform capable of processing large amounts of data, such as KAFKA or KINESIS. The data processing unit (120) can stream streaming data using the stream platform.

[0038] The data processing unit (120) can store streaming data input into a data stream as replay data. The data processing unit (120) can convert the streaming data into a storage format to generate replay data. The data processing unit (120) can store the replay data in a predetermined data storage. Here, the data storage may be implemented as one with the logic update device (100) depending on the design method of the logic update device (100), or may be implemented by being connected to the logic update device (100) via network communication.

[0039] According to one embodiment, the data processing unit (120) can collect streaming data using AWS Firehose and transmit it to at least one external data storage system among S3, Redshift, and Elasticsearch. The data processing unit (120) can store replay data in the Apache Parquet format. Here, Apache Parquet is a columnar storage format capable of processing and compressing large amounts of data.

[0040] The data processing unit (120) can classify replay data into batch units based on predetermined criteria. The data processing unit (120) can classify replay data by dividing it into specific time periods, daily units, and specific phenomena.

[0041] According to one embodiment, the data processing unit (120) can classify replay data according to predetermined criteria using a Lambda function. The data processing unit (120) can store the classified replay data by setting each storage path.

[0042] Referring back to FIG. 1, the control unit (130) can analyze streaming data being streamed. The control unit (130) can analyze the streaming data being streamed using pattern detection logic that detects a pattern corresponding to a first event, thereby determining whether the first event has occurred.

[0043] The control unit (130) can analyze streaming data using pattern detection logic. Here, the pattern detection logic refers to logic that detects a pattern corresponding to an event. In other words, the control unit (130) can analyze streaming data using pattern detection logic and detect a pattern in the streaming data. The control unit (130) can determine whether an event has occurred based on whether a pattern has been detected.

[0044] The control unit (130) can load the stored replay data in batches. The control unit (130) can analyze the replay data loaded in batches using pattern detection logic. That is, the control unit (130) can analyze the replay data using pattern detection logic and detect a pattern in the replay data. The control unit (130) can determine whether an event has occurred based on whether a pattern has been detected.

[0045] The control unit (130) can repeatedly analyze stored replay data using pattern detection logic. The control unit (130) can repeatedly analyze replay data of the same batch unit to determine whether a pattern has been detected. Furthermore, the control unit (130) can update the pattern detection logic based on whether a pattern has been detected through repeated analysis.

[0046] In summary, the control unit (130) can repeatedly perform an operation of detecting a pattern included in replay data of the same batch unit. The control unit (130) can update the pattern detection logic based on whether or not the pattern is repeatedly detected, thereby improving the accuracy and reliability of the pattern detection logic.

[0047] Referring back to FIG. 1, the control unit (130) may generate a pattern corresponding to a second event that may occur due to the occurrence of a first event, and update the pattern detection logic to detect the pattern corresponding to the second event. In one embodiment, the control unit (130) may use the occurrence of the first event as a parameter of the pattern detection logic to detect the pattern of the second event.

[0048] For example, the pattern detection logic may be configured to detect a first event by configuring parameters such as whether a specific phenomenon occurs, the number of times a specific phenomenon occurs, and whether a specific phenomenon occurs within a batch unit time. In other words, the pattern detection logic for detecting the first event may be configured with image data information and meta information as parameters.

[0049] Here, whether the first event is detected can be configured as a parameter for detecting the second event. That is, the pattern detection logic corresponding to the second event can include the occurrence of the first event as a parameter. To illustrate an example, assume that the first event is smoke generation and the second event is fire generation.

[0050] Here, the control unit (130) can detect the first event, smoke generation, using pattern detection logic. During factory operation, smoke may be generated within the factory. However, during non-operational periods, detection of smoke within the factory indicates a fire.

[0051] The control unit (130) can detect flames based on image data information. The control unit (130) can set whether the factory is operating and whether smoke is generated as logic for detecting fire occurrence. That is, the control unit (130) can set whether the first event has occurred as a parameter to detect the second event. Accordingly, the control unit (130) can generate a pattern capable of detecting the second event that may occur based on the first event, and update the pattern detection logic to detect the second event.

[0052] The control unit (130) can test the generated pattern detection logic by loading replay data of another batch unit in which the first event was detected. Accordingly, the control unit (130) can update the pattern detection logic corresponding to the pattern of the second event.

[0053] According to an example, the control unit (130) may utilize Apache Flink, which is capable of stream processing and complex event processing, to analyze streaming data and load and analyze replay data. Here, Apache Flink is a distributed computing framework that processes large-scale data streaming and batch data. Apache Flink is a high-performance, low-latency distributed data processing engine that supports both real-time data streaming and batch processing, and can process data from various data sources. In addition, the control unit (130) may load stored replay data using the Parquet connector library and implement pattern detection logic for detecting specific events. The example described above and the method for implementing the example are examples for explaining the present invention, and are not limited thereto. The method for implementing the receiving unit (110), the data processing unit (120), and the control unit (130) may be implemented in various ways as needed, as long as it does not contradict the technical spirit of the present disclosure.

[0054] Accordingly, a logic update device (100) can be provided that can test pattern detection logic using replay data, thereby reducing testing time and costs, ensuring logic reliability and accuracy, and enhancing the stability and performance of a monitoring system. Furthermore, a logic update device (100) can be provided that can predict and respond to the occurrence of new events by generating logic that detects patterns for new events.

[0055]

[0056] FIG. 4 is a diagram illustrating a pipeline for performing monitoring according to the present embodiments.

[0057] The control unit (130) may provide a user interface (UI) including at least one of a dashboard that displays information about an event, a dashboard that executes and manages pattern detection logic, and a dashboard that can set an action according to the occurrence of an event.

[0058] Referring to FIG. 4, an example of implementing a pipeline for monitoring is described. As illustrated in FIG. 4, the control unit (130) can provide a monitoring web UI that can execute test code and check event logs. Specifically, when the control unit (130) according to an example receives an execution or termination signal through the web UI, a pattern detection logic test can be executed or terminated. Here, the Node.js server can execute or terminate a script corresponding to the signal. The Flink App can be built by retrieving the latest source code from AWS Code. In addition, the Flink App can analyze streaming data and replay data to generate event logs. The event logs can be transmitted to Amazon Kinesis Data Analytics for real-time analysis. The analyzed results can be recorded as a Validation Log DataStream.

[0059] The control unit (130) can display event information on the monitoring web UI based on the analyzed results. Furthermore, the control unit (130) can provide a dashboard for setting actions based on the occurrence of an event and a dashboard for managing pattern detection logic. Accordingly, a pipeline can be provided for controlling the logic update device (100) and managing pattern detection logic using the web UI.

[0060] The examples and methods for implementing the examples described above are merely examples for illustrating the present invention and are not intended to be limiting. Methods for implementing a logic update device may be configured in various ways as needed, as long as they do not contradict the technical principles of the present disclosure.

[0061]

[0062] Hereinafter, a logic update method capable of performing some or all of the embodiments described with reference to FIGS. 1 through 4 will be described with reference to the drawings. The above description may be omitted to avoid redundant explanation, and in this case, the omitted content may be substantially equally applied to the following description, as long as it does not contradict the technical spirit of the invention.

[0063] Figure 5 is a flowchart for explaining a logic update method according to the present embodiments.

[0064] Referring to FIG. 5, the logic update device can receive image data (S510). According to one example, the logic update device can be configured with the same hardware as the image capture device installed in the monitoring environment. In this case, the logic update device can receive image data captured from the image capture device.

[0065] As another example, the logic update device may be configured with hardware different from the video recording device. In this case, the logic update device may receive video data recorded from at least one camera installed in the monitoring environment via network communication. Here, the video data may be received via RTSP (Real-Time Streaming Protocol).

[0066] A video recording device refers to a device capable of acquiring video data, such as a camera or CCTV. The video data may be a typical 2D photograph, video, or a frame representing an image at a specific point in time that can be acquired from a video recording device. In addition, the video data may include objects existing in the monitoring environment. Here, the objects may include dynamic objects, static objects, and specific phenomena that can be captured in the video data. For example, video data acquired in a factory where a fire has occurred may include objects including dynamic objects (factory workers), static objects (factory facilities), and specific phenomena (smoke, flames).

[0067] The logic update device can receive the device identification number, the date, time, and location of the image capture from the image capture device. Additionally, the logic update device can obtain the file format of the image data acquired from the image capture device.

[0068] Referring again to FIG. 5, the logic update device can analyze image data (S520). According to one embodiment, the deep learning model may include a Vision AI model capable of performing visual data analysis including image data. That is, the logic update device can recognize objects contained in the image data using the Vision AI model.

[0069] Here, the Vision AI model may be a Visual Language Model (VLM), trained based on multimodal data and capable of understanding and processing text and images together. For example, the VLM model may include at least one of a Zero-Shot Object Detection Model and a Skeleton Extraction Model.

[0070] Multimodal data refers to data that combines at least two different types of data. VLM can be pre-trained using multimodal data combining text and images. VLM can then understand and learn the relationship between images and text, internally generating an embedding space that connects the two types of data.

[0071] For example, a VLM could be a zero-shot object detection model capable of detecting objects not previously defined. That is, a zero-shot object detection model pre-trained using multimodal data can predict objects specified in text in new images.

[0072] For example, a Zero-shot Object Detection Model trained on multimodal data can convert new image data and text into embedding vectors when input. The Zero-shot Object Detection Model can then calculate the similarity between the image embedding and the text embedding, thereby scoring a similarity score between objects and text within the image. Furthermore, the Zero-shot Object Detection Model can identify the location of the object within the image based on the calculated similarity score. The Zero-shot Object Detection Model can display the location of the object as a bounding box centered on the region with the highest similarity within the image.

[0073] The above description is not intended to be limiting and is merely an example of how a logic update device uses a deep learning model to recognize objects contained in image data. The deep learning model may vary depending on hardware resources and inference time, and may be configured in various ways as needed, as long as it does not contradict the technical principles of the present disclosure.

[0074] The logic update device can generate image data information. The logic update device can receive object information about recognized objects from each VLM. For example, the logic update device can receive object information about recognized objects from a Zero-Shot Object Detection Model and a Skeleton Extraction Model. The logic update device can refine the object information based on the object information received from each model. In addition, the logic update device can post-process the object information received from each model to produce post-processing information including object speed, movement direction, and distance between objects. The logic update device can generate image data information including object information and post-processing information.

[0075] The logic update device can generate metadata. The logic update device can obtain information including at least one of a device identification number, the date, time, and location of the video capture, or the file format of the video data. The metadata can include information including at least one of a device identification number, the date, time, and location of the video capture, or the file format of the video data.

[0076] The logic update device can generate streaming data containing image data information and metadata. As illustrated in FIG. 3, the logic update device according to one embodiment can convert image data information and metadata into JSON (JavaScript Object Notation) format. The logic update device can process the converted JSON format image data and metadata into streaming data.

[0077] The data processing unit can input processed streaming data into a data stream capable of real-time data processing. In one embodiment, the data processing unit can input the streaming data into a data stream platform capable of processing large amounts of data, such as KAFKA or KINESIS. The logic update unit can stream the streaming data using the stream platform.

[0078] A logic update device can store streaming data input into a data stream as replay data. The logic update device can convert the streaming data into a storage format to generate replay data. The logic update device can store the replay data in a predetermined data storage. Here, the data storage can be implemented as one with the logic update device, or can be implemented by being connected to the logic update device via network communication, depending on the design method of the logic update device.

[0079] According to one embodiment, a logic update device can collect streaming data using AWS Firehose and transmit it to at least one external data store, including S3, Redshift, and Elasticsearch. The data processing unit can store replay data in the Apache Parquet format. Apache Parquet is a columnar storage format capable of processing and compressing large amounts of data.

[0080] The logic update device can classify replay data into batches based on certain criteria. The logic update device can classify replay data by specific time periods, days, and specific events.

[0081] According to one embodiment, a logic update device can classify replay data based on predetermined criteria using a Lambda function. The logic update device can store the classified replay data by setting a respective storage path.

[0082] Again, referring to FIG. 5, the logic update device can detect an event contained in image data and update the pattern detection logic (S530). The logic update device can analyze the streaming data being streamed. The logic update device can analyze the streaming data being streamed using the pattern detection logic that detects a pattern corresponding to the first event, thereby determining whether the first event has occurred.

[0083] The logic update device can analyze streaming data using pattern detection logic. Here, pattern detection logic refers to logic that detects patterns corresponding to events. In other words, the logic update device can analyze streaming data using pattern detection logic to detect patterns in the streaming data. The logic update device can determine whether an event has occurred based on whether a pattern has been detected.

[0084] The logic update device can load stored replay data in batches. The logic update device can analyze the replay data loaded in batches using pattern detection logic. In other words, the logic update device can analyze the replay data using pattern detection logic to detect patterns in the replay data. The logic update device can determine whether an event has occurred based on whether a pattern has been detected.

[0085] The logic update device can repeatedly analyze stored replay data using pattern detection logic. The logic update device can repeatedly analyze replay data from the same batch to determine whether a pattern is detected. Furthermore, the logic update device can update the pattern detection logic based on the pattern detection results obtained through repeated analysis.

[0086] In summary, the logic update device can repeatedly perform the operation of detecting patterns contained in replay data of the same batch unit. The logic update device can update the pattern detection logic based on the repeatedly acquired pattern detections, thereby improving the accuracy and reliability of the pattern detection logic.

[0087]

[0088] FIG. 6 is a drawing for explaining a method of generating a pattern according to the present embodiments.

[0089] Referring to FIG. 6, the logic update device can receive replay data (S610). The logic update device can detect a first event (S620). The logic update device can generate a pattern corresponding to a second event that may occur due to the occurrence of the first event, and update the pattern detection logic to detect the pattern corresponding to the second event. In one embodiment, the logic update device can use the occurrence of the first event as a parameter of the pattern detection logic to detect the pattern of the second event.

[0090] For example, the pattern detection logic may be configured to detect a first event by configuring parameters such as whether a specific phenomenon occurs, the number of times a specific phenomenon occurs, and whether a specific phenomenon occurs within a batch unit time. In other words, the pattern detection logic for detecting the first event may be configured with image data information and meta information as parameters.

[0091] Referring to Fig. 6, the logic update device can generate a pattern corresponding to a second event (S630). Whether the first event is detected can be configured as a parameter for detecting the second event. That is, the pattern detection logic corresponding to the second event can include the occurrence of the first event as a parameter. To illustrate an example, assume that the first event is smoke generation and the second event is fire generation.

[0092] Here, the logic update device can detect the first event, smoke, using pattern detection logic. During factory operating hours, smoke may be generated within the factory. However, during non-operating hours, the detection of smoke within the factory indicates a fire.

[0093] Here, the logic update device can detect flames based on image data information. The logic update device can set whether the factory is operating and whether smoke is generated as logic for detecting fire occurrence. That is, the logic update device can set whether the first event occurs as a parameter to detect the second event. Accordingly, the logic update device can generate a pattern capable of detecting the second event that may occur based on the first event, and update the pattern detection logic to detect the second event. The logic update device can test the generated pattern detection logic by loading replay data of another batch unit in which the first event was detected. Accordingly, the logic update device can update the pattern detection logic corresponding to the pattern of the second event.

[0094] According to this, a logic update method can be provided that uses replay data to test pattern detection logic, reducing testing time and costs, ensuring logic reliability and accuracy, and enhancing the stability and performance of a monitoring system. Furthermore, by creating logic to detect patterns for new events, a logic update method can be provided that can predict and respond to the occurrence of new events.

[0095]

[0096] The above-described methods and / or various embodiments may be realized by digital electronic circuits, computer hardware, firmware, software, and / or a combination thereof. Various embodiments of the present disclosure may be implemented as a computer program that is executed by a data processing device, for example, one or more programmable processors and / or one or more computing devices, or stored on a computer-readable recording medium and / or a computer-readable recording medium. The above-described computer program may be written in any form of programming language, including a compiled language or an interpreted language, and may be distributed in any form, such as a standalone program, a module, a subroutine, etc. The computer program may be distributed through a single computing device, multiple computing devices connected through the same network, and / or multiple computing devices distributed to be connected through multiple different networks.

[0097] The methods and / or various embodiments described above may be performed by one or more processors configured to execute one or more computer programs that process, store, and / or manage any function, function, etc. by operating on the basis of input data or generating output data. For example, the methods and / or various embodiments of the present disclosure may be performed by special purpose logic circuits such as Field Programmable Gate Arrays (FPGAs) or Application Specific Integrated Circuits (ASICs), and an apparatus and / or system for performing the methods and / or embodiments of the present disclosure may be implemented as special purpose logic circuits such as FPGAs or ASICs.

[0098] The one or more processors executing the computer program may include a general-purpose or special-purpose microprocessor and / or one or more processors of any type of digital computing device. The processor may receive instructions and / or data from each of read-only memory and random-access memory, or may receive instructions and / or data from the read-only memory and the random-access memory. In the present disclosure, components of a computing device performing the methods and / or embodiments may include one or more processors for executing instructions, and one or more memory devices for storing instructions and / or data.

[0099] According to one embodiment, the computing device can transmit and receive data to and from one or more mass storage devices for storing data. For example, the computing device can receive and / or transfer data from a magnetic disc or an optical disc. A computer-readable storage medium suitable for storing instructions and / or data associated with a computer program may include, but is not limited to, any form of non-volatile memory, including semiconductor memory devices such as Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), flash memory devices, and the like. For example, the computer-readable storage medium may include a magnetic disk such as an internal hard disk or a removable disk, a magneto-optical disk, a CD-ROM, and a DVD-ROM disk.

[0100] To provide interaction with a user, a computing device may include, but is not limited to, a display device (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD), etc.) for providing or displaying information to the user, and a pointing device (e.g., a keyboard, a mouse, a trackball, etc.) for allowing the user to provide input and / or commands to the computing device. That is, the computing device may further include any other types of devices for providing interaction with the user. For example, the computing device may provide any form of sensory update to the user, including visual updates, auditory updates, and / or tactile updates, for interacting with the user. In this regard, the user may provide input to the computing device through various gestures, such as visual, vocal, or motion.

[0101] In the present disclosure, various embodiments may be implemented in a computing system that includes a backend component (e.g., a data server), a middleware component (e.g., an application server), and / or a front-end component. In this case, the components may be interconnected via any form or medium of digital data communication, such as a communications network. For example, the communications network may include a Local Area Network (LAN), a Wide Area Network (WAN), and the like.

[0102] A computing device based on the present embodiments may be implemented using hardware and / or software configured to interact with a user, including a user device, a user interface (UI) device, a user terminal, or a client device. For example, the computing device may include a portable computing device such as a laptop computer. Additionally or alternatively, the computing device may include, but is not limited to, Personal Digital Assistants (PDAs), tablet PCs, game consoles, wearable devices, Internet of Things (IoT) devices, virtual reality (VR) devices, augmented reality (AR) devices, and the like. The computing device may further include other types of devices configured to interact with a user. In addition, the computing device may include a portable communication device (e.g., a mobile phone, a smart phone, a wireless cellular phone, etc.) suitable for wireless communication over a network such as a mobile communication network. The computing device may be configured to communicate wirelessly with a network server using wireless communication technologies and / or protocols, such as Radio Frequency (RF), Microwave Frequency (MWF), and / or Infrared Ray Frequency (IRF).

[0103] The above description is merely an illustrative example of the technical idea of ​​the present disclosure, and those skilled in the art to which the present disclosure pertains will appreciate that various modifications and variations can be made without departing from the essential characteristics of the technical idea of ​​the present disclosure. In addition, the present embodiments are not intended to limit the technical idea of ​​the present disclosure but rather to explain it, and therefore the scope of the technical idea of ​​the present disclosure is not limited by these embodiments. The scope of protection of the present disclosure should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included within the scope of the rights of the present disclosure.

[0104]

[0105] CROSS-REFERENCE TO RELATED APPLICATION

[0106] This patent application claims priority under 35 USC § 119(a) to Korean Patent Application No. 10-2024-0118927, filed September 3, 2024, the entire contents of which are incorporated herein by reference. Furthermore, this patent application claims priority in countries other than the United States for the same reasons, the entire contents of which are incorporated herein by reference.

Claims

1. A receiving unit that receives video data; A data processing unit that analyzes the video data using a deep learning model to generate video data information, inputs streaming data including the video data information and meta information into a data stream, and stores the streaming data as replay data; and By analyzing the streaming data using a pattern detection logic that detects a pattern corresponding to the first event, it is determined whether the first event has occurred. The replay data is repeatedly analyzed using the above pattern detection logic, and the pattern detection logic is updated based on whether the first event has occurred. A control unit that generates a pattern corresponding to a second event that may occur due to the occurrence of the first event, and updates the pattern detection logic to detect the pattern corresponding to the second event; A logic update device including:

2. In paragraph 1, The above deep learning model is, A logic update device that is a Visual Language Model (VLM) trained based on multimodal data.

3. In paragraph 1, The above video data information is, A logic update device including object information for an object included in the above image data, wherein the object includes a dynamic object, a static object, and a specific phenomenon.

4. In paragraph 1, The above meta information is, A logic update device including at least one of a device identification number, the date, time and location of acquisition of the image data, and the file format of the image data.

5. In paragraph 1, The above replay data is, A logic update device that stores data in the Apache Parquet file format, a columnar storage format.

6. In paragraph 1, The above replay data is, A logic update device that is classified and loaded by batch unit.

7. In paragraph 1, The above control unit, A logic update device providing a user interface (UI) including at least one of a dashboard that displays information about the above event, a dashboard that executes and manages the pattern detection logic, and a dashboard that can set an action according to the occurrence of the above event.

8. Step of receiving video data; A step of analyzing the video data using a deep learning model to generate video data information, importing streaming data including the video data information and meta information into a data stream, and storing the streaming data as replay data; and By analyzing the streaming data using a pattern detection logic that detects a pattern corresponding to the first event, it is determined whether the first event has occurred. The replay data is repeatedly analyzed using the above pattern detection logic, and the pattern detection logic is updated based on whether the determined event has occurred. A step of generating a pattern corresponding to a second event that may occur due to the occurrence of the first event, and updating the pattern detection logic to detect the pattern corresponding to the second event; A logic update method including .

9. In paragraph 8, The above deep learning model is, A logic update method based on a VLM (Visual Language Model) trained on multimodal data.

10. In paragraph 8, The above video data information is, A logic update method including object information for an object included in the above image data, wherein the object includes a dynamic object, a static object, and a specific phenomenon.

11. In paragraph 8, The above meta information is, A logic update method including at least one of a device identification number, the date, time and location of acquisition of the image data, and the file format of the image data.

12. In paragraph 8, The above replay data is, A method of updating logic stored in Apache Parquet file format, a columnar storage format.

13. In paragraph 8, The above replay data is, A logic update method that is classified and loaded by batch unit.

14. In paragraph 8, The above updating steps are: A logic update method providing a user interface (UI) including at least one of a dashboard displaying information about the above event, a dashboard executing and managing the pattern detection logic, and a dashboard capable of setting an action according to the occurrence of the above event.

Images