Method for generating event-related message through image analysis and apparatus for supporting same

Abstract

A method for generating an event-related message through image analysis includes the steps of: when a predefined event occurs in a captured image, detecting a first keyword corresponding to each of an object, an event occurrence area, and an event type from a current image at the time of occurrence of the event; generating a first message related to the initial generation of the event on the basis of the detected first keyword; obtaining an image preceding the current image; detecting a second keyword corresponding to each of the time of capture of the preceding image, an area related to the occurrence of the event, and a cause of the event from the obtained preceding image; generating a second message related to the cause of the event on the basis of the detected second keyword; and determining the urgency of the event and the range of occurrence of the event on the basis of the first message and the second message.

Description

Method for generating event-related messages through video analysis and device supporting the same

[0001] This specification relates to a method for generating an event-related message, and more specifically, to a method for generating an event-related message through image analysis and a device for supporting the same.

[0002] In the past, when an event such as an emergency occurred while monitoring footage in a situation room or control center, the control personnel would check the footage and use a radio or cell phone to inform the field personnel of the emergency situation.

[0003] In other words, the conventional emergency situation communication method has the problem that the transmission results vary depending on accumulated control experience, such as when the control agent verbally transmits the situation based on the current video being filmed.

[0004] Korean Patent No. 10-2295766 proposes a method of determining whether an accident has occurred through image analysis and transmitting an accident notification message.

[0005] However, Korean Patent No. 10-2295766 does not disclose a method for automatically generating messages related to accident occurrence through image analysis.

[0006] That is, research is needed on how to automatically generate event-related messages and transmit them externally when an event such as an accident occurs.

[0007] Therefore, the purpose of this specification is to provide a method for extracting keywords related to an event through image analysis, generating event-related messages using the extracted keywords, and automatically transmitting the messages to field personnel, police officers, firefighters, etc.

[0008] Additionally, the present specification aims to provide an event marking function in the memory or storage of an image analysis device once the urgency and scope of the event are determined.

[0009] The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0010] The present specification provides a method for generating an event-related message through image analysis, comprising: a step of detecting, when a predefined event occurs in a captured image, a first keyword corresponding to an object, an area where the event occurred, and a type of the event, respectively, in a current image at the time of the event occurrence; a step of generating a first message related to the initial occurrence of the event based on the detected first keyword; a step of acquiring a previous image with respect to the current image; a step of detecting, from the acquired previous image, a second keyword corresponding to a shooting time of the previous image, an area related to the occurrence of the event, and a cause of the occurrence of the event, respectively; a step of generating a second message related to the cause of the occurrence of the event based on the detected second keyword; and a step of determining the urgency of the event and the occurrence range of the event based on the first message and the second message.

[0011] In addition, the method in the present specification is characterized by further including a step of transmitting a notification message related to the occurrence of the event based on the urgency of the determined event and the occurrence range of the event.

[0012] In addition, in the present specification, the previous image is characterized as an image including the object among images prior to the occurrence time of the event related to the current image.

[0013] In addition, the step of generating the first message in the present specification is characterized by including a step of arranging the object, the event occurrence area, and the type of the event in that order.

[0014] In addition, the step of generating the second message in the present specification is characterized by including the step of calculating a time delta by comparing the shooting time of the previous image detected by the second keyword with the shooting time of the current image; and the step of arranging the calculated time delta, the area related to the occurrence of the event, and the cause of the occurrence of the event in that order.

[0015] In addition, the urgency of the event in the present specification is characterized in that it is determined based on the difference time, the cause of occurrence of the event, and the object.

[0016] Additionally, the urgency of the event is characterized in that it is classified as high, medium, or low in this specification.

[0017] In addition, the scope of the event in this specification is characterized in that it is determined based on the event occurrence area and the object.

[0018] Additionally, the scope of the event in this specification is characterized as being broad, moderate or narrow.

[0019] In addition, the first message and the second message in this specification are characterized in that they are generated through a large language model (LLM).

[0020] In addition, the method in the present specification is characterized in that, if the language type of the first keyword and the language type of the second keyword are different, it further includes a step of converting the language of the second keyword to be the same as the language type of the first keyword.

[0021] In addition, the present specification provides an image analysis device that generates an event-related message through image analysis, comprising: a wireless communication unit for transmitting and receiving a wireless signal; a memory for storing a captured image; and a processor functionally connected to the wireless communication unit and the memory, wherein, when a predefined event occurs in the captured image, the processor detects a first keyword corresponding to an object, an event occurrence area, and a type of event in a current image at the time of occurrence of the event, respectively, and generates a first message related to the initial occurrence of the event based on the detected first keyword, acquires a previous image with respect to the current image, and detects a second keyword corresponding to a shooting time of the previous image, an area related to the occurrence of the event, and a cause of the occurrence of the event, respectively, in the acquired previous image, and generates a second message related to the cause of the occurrence of the event based on the detected second keyword, and determines the urgency of the event and the occurrence range of the event based on the first message and the second message.

[0022] In addition, in the present specification, the processor is characterized in that it determines an image including the object as a previous image among images prior to the occurrence time of the event related to the current image.

[0023] In addition, in the present specification, the processor is characterized in that it generates the first message by arranging the detected first keyword in the order of the object, the event occurrence area, and the type of the event.

[0024] In addition, in the present specification, the processor is characterized in that it compares the shooting time of the previous image detected by the second keyword with the shooting time of the current image to calculate a time delta, and generates the second message by arranging the calculated time delta, the area related to the occurrence of the event, and the cause of the occurrence of the event in that order.

[0025] This specification extracts keywords related to an event through video analysis, and automatically generates and delivers event-related messages using the extracted keywords, thereby enabling the same event occurrence situation to be delivered regardless of differences in experience of control personnel, etc., and has the effect of simultaneously and quickly notifying event occurrence to event-related locations or people, such as field personnel, police officers, and firefighters.

[0026] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

[0027] The accompanying drawings, which are incorporated in and constitute a part of the detailed description to aid in the understanding of the present invention, provide embodiments of the present invention and, together with the detailed description, explain the technical features of the present invention.

[0028] Figure 1 is a diagram showing an example of a conceptual diagram of an image monitoring system to which the method proposed in this specification can be applied.

[0029] Figure 2 is an internal block diagram illustrating an image analysis device to which the method proposed in this specification can be applied.

[0030] Figure 3 is a block diagram of an AI device to which the method proposed in this specification can be applied.

[0031] Figure 4 is a diagram showing an example of a method for generating a message proposed in this specification.

[0032] Figure 5 is a flowchart illustrating an example of a method for generating a message through image analysis proposed in this specification.

[0033] Figure 6 is a flowchart illustrating another example of a method for generating a message through image analysis proposed in this specification.

[0034] It should be noted that the technical terms used in this specification are merely used to describe specific embodiments and are not intended to limit the scope of the technology disclosed herein. Furthermore, unless specifically defined otherwise herein, the technical terms used herein should be interpreted as having a meaning generally understood by a person of ordinary skill in the art to which the technology disclosed herein pertains, and should not be interpreted in an excessively broad or narrow sense. Furthermore, if a technical term used herein is an incorrect technical term that does not accurately express the scope of the technology disclosed herein, it should be replaced with a technical term that can be correctly understood by a person of ordinary skill in the art to which the technology disclosed herein pertains. Furthermore, general terms used herein should be interpreted according to their dictionary definitions or according to the context, and should not be interpreted in an excessively narrow sense.

[0035] While terms including ordinal numbers, such as "first" and "second," used herein may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0036] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components are given the same reference numbers and redundant descriptions thereof will be omitted.

[0037] Additionally, when describing the technology disclosed in this specification, detailed descriptions of related known technologies will be omitted if they are deemed to obscure the gist of the technology disclosed in this specification. Furthermore, it should be noted that the attached drawings are intended solely to facilitate understanding of the concepts of the technology disclosed in this specification and should not be construed as limiting the scope of the technology.

[0038] An image according to an embodiment of the present invention includes both still images and moving images unless there is a special limitation.

[0039]

[0040] Figure 1 is a diagram showing an example of a conceptual diagram of an image monitoring system to which the method proposed in this specification can be applied.

[0041] Referring to FIG. 1, the image monitoring system (10) may include an image analysis device (100), a terminal device (200), etc.

[0042] The above image analysis device (100) is a device that can implement a method for automatically generating an event occurrence-related message through AI analysis of a captured image proposed in this specification, and may include a camera such as a surveillance camera (or CCTV), an edge device, an AI camera, a network camera, a storage device such as an NVR, a server, etc.

[0043] The above terminal device (200) is a device capable of receiving an event occurrence notification message, and may include a terminal carried by a field agent, a terminal related to the police (a terminal carried by a police officer, a terminal installed inside a police car, a terminal at a police station, etc.), a terminal related to firefighting (a terminal carried by a fire officer, a terminal installed inside a fire truck, a terminal at a fire station, etc.), etc.

[0044]

[0045] Internal block diagram of the video analysis device

[0046] Figure 2 is an internal block diagram illustrating an image analysis device to which the method proposed in this specification can be applied.

[0047] The image analysis device (100) may include a wireless communication unit (110), an input unit (120), an output unit (130), a memory (140), and a processor (150). The components illustrated in FIG. 2 are not essential for implementing the image analysis device, and thus the image analysis device described in this specification may have more or fewer components than the components listed above.

[0048] More specifically, among the above components, the wireless communication unit may include one or more modules that enable wireless communication between the video analysis device and the video monitoring system, and between the video analysis device and the terminal device. In addition, the wireless communication unit may include one or more modules that connect the video analysis device to one or more networks.

[0049] The wireless communication unit may include at least one of a mobile communication module, a short-range communication module, and a location information module.

[0050] The input unit may include a user input unit (e.g., a touch key, a mechanical key, etc.) for receiving information from a user. The input unit may additionally include a camera or a video input unit for inputting a video signal, a microphone for inputting an audio signal, or an audio input unit. Voice data or image data collected from the input unit may be analyzed and processed into a user's control command.

[0051] The output section is for generating output related to visual, auditory, or tactile sensations, and may include a display section, an audio output section, etc.

[0052] Additionally, the memory stores data supporting various functions of the image analysis device. The memory can store a number of application programs (or applications) running on the image analysis device, data for the operation of the image analysis device, and commands. At least some of these applications can be downloaded from an external server via wireless communication. Meanwhile, the application programs can be stored in the memory, installed on the image analysis device, and driven by the processor to perform the operations (or functions) of the image analysis device.

[0053] In addition to the operations associated with the aforementioned application, the processor (or control unit) typically controls the overall operation of the image analysis device. The processor processes signals, data, and information input or output through the components discussed above, or runs application programs stored in memory, thereby providing or processing appropriate information or functions to the user.

[0054] Additionally, the processor may control at least some of the components discussed with reference to FIG. 2 to drive an application program stored in memory. Furthermore, the processor may operate at least two or more of the components included in the image analysis device in combination to drive the application program.

[0055] In particular, when a predefined event occurs in a captured image, the processor detects a first keyword corresponding to an object, an area where the event occurred, and a type of the event in a current image at the time of the event occurrence, and generates a first message related to the initial occurrence of the event based on the detected first keyword, acquires a previous image for the current image, and detects a second keyword corresponding to the shooting time of the previous image, an area related to the occurrence of the event, and a cause of the occurrence of the event in the acquired previous image, and generates a second message related to the cause of the occurrence of the event based on the detected second keyword, and determines the urgency of the event and the occurrence range of the event based on the first message and the second message.

[0056] Additionally, the processor can determine an image including the object as a previous image among images prior to the occurrence of the event related to the current image.

[0057] Additionally, the processor can generate the first message by arranging the detected first keywords in the order of the object, the event occurrence area, and the type of the event.

[0058] In addition, the processor may compare the shooting time of the previous image detected by the second keyword with the shooting time of the current image to calculate a time delta, and generate the second message by arranging the calculated time delta, the area related to the occurrence of the event, and the cause of the occurrence of the event in that order.

[0059] At least some of the above components may operate cooperatively with each other to implement the operation, control, or control method of the image analysis device according to various embodiments described below. In addition, the operation, control, or control method of the image analysis device may be implemented on the image analysis device by driving at least one application program stored in the memory.

[0060]

[0061] AI device block diagram

[0062] Figure 3 is a block diagram of an AI device to which the method proposed in this specification can be applied.

[0063] The AI ​​device (20) may be included as part of at least a portion of the image analysis device (100) illustrated in FIG. 1 or FIG. 2 and may be provided to perform at least a portion of the AI ​​processing.

[0064] The above AI device (20) may include an AI processor (21), a memory (25), and / or a communication unit (27). If the AI ​​device is included in an image analysis device, the communication unit (27) may be omitted.

[0065] The above AI device (20) is a computing device capable of learning a neural network, and can be implemented as various electronic devices such as a server, desktop PC, notebook PC, tablet PC, etc., and can perform the method proposed in this specification by loading the LLM model described in FIGS. 5 and 6.

[0066] The AI ​​processor (21) can learn a neural network using a program stored in the memory (25). In particular, the AI ​​processor (21) can learn a neural network for recognizing image-related data. Here, the neural network for recognizing image-related data can be designed to simulate the structure of the human brain on a computer, and can include a plurality of network nodes having weights that simulate neurons of the human neural network. The plurality of network modes can each exchange data according to a connection relationship so as to simulate the synaptic activity of neurons that exchange signals through synapses. Here, the neural network can include a deep learning model developed from a neural network model. In the deep learning model, a plurality of network nodes are located in different layers and can exchange data according to a convolution connection relationship. Examples of neural network models include various deep learning techniques such as deep neural networks (DNNs), convolutional deep neural networks (CNNs), recurrent Boltzmann machines (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), and deep Q-networks, which can be applied to fields such as computer vision, speech recognition, natural language processing, and speech / signal processing.

[0067] Meanwhile, the processor performing the functions described above may be a general-purpose processor (e.g., CPU), but may also be an AI-specific processor for artificial intelligence learning (e.g., GPU).

[0068] The memory (25) can store various programs and data required for the operation of the AI ​​device (20). The memory (25) can be implemented as a non-volatile memory, a volatile memory, a flash memory, a hard disk drive (HDD), a solid state drive (SDD), etc. The memory (25) is accessed by the AI ​​processor (21), and data reading / recording / modifying / deleting / updating, etc. can be performed by the AI ​​processor (21). In addition, the memory (25) can store a neural network model (e.g., a deep learning model (26), a Re-ID model (28)) generated through a learning algorithm for image analysis according to one embodiment of the present invention.

[0069] Meanwhile, the AI ​​processor (21) may include a data learning unit (22) that learns a neural network for image analysis. The data learning unit (22) may learn criteria regarding which learning data to use for determining image analysis and how to classify and recognize data using the learning data. The data learning unit (22) may acquire learning data to be used for learning and apply the acquired learning data to the deep learning model, thereby learning the deep learning model.

[0070] The data learning unit (22) may be manufactured in the form of at least one hardware chip and mounted on the AI ​​device (20). For example, the data learning unit (22) may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as a part of a general-purpose processor (CPU) or a graphics processor (GPU) and mounted on the AI ​​device (20). In addition, the data learning unit (22) may be implemented as a software module. When implemented as a software module (or a program module including instructions), the software module may be stored in a non-transitory computer readable medium that can be read by a computer. In this case, at least one software module may be provided by an operating system (OS) or an application.

[0071] The data learning unit (22) may include a learning data acquisition unit (23) and a model learning unit (24).

[0072] The learning data acquisition unit (23) can acquire learning data required for a neural network model for image analysis. For example, the learning data acquisition unit (23) can acquire image data and / or sample data for input into a neural network model as learning data.

[0073] The model learning unit (24) can use the acquired learning data to learn the neural network model to have a judgment criterion on how to classify a given data. At this time, the model learning unit (24) can train the neural network model through supervised learning that uses at least some of the learning data as a judgment criterion. Alternatively, the model learning unit (24) can train the neural network model through unsupervised learning that discovers a judgment criterion by learning on its own using the learning data without guidance. In addition, the model learning unit (24) can train the neural network model through reinforcement learning using feedback on whether the result of the situation judgment according to the learning is correct. In addition, the model learning unit (24) can train the neural network model using a learning algorithm including error back-propagation or gradient descent.

[0074] Once the neural network model is trained, the model training unit (24) can store the trained neural network model in memory. The model training unit (24) can also store the trained neural network model in the memory of a server connected to the AI ​​device (20) via a wired or wireless network.

[0075] The data learning unit (22) may further include a learning data preprocessing unit (not shown) and a learning data selection unit (not shown) to improve the analysis results of the recognition model or to save resources or time required for creating the recognition model.

[0076] The learning data preprocessing unit can preprocess the acquired data so that it can be used for learning to determine situations. For example, the learning data preprocessing unit can process the acquired data into a preset format so that the model learning unit (24) can utilize the acquired learning data for learning image recognition.

[0077] In addition, the learning data selection unit can select data required for learning from among the learning data acquired by the learning data acquisition unit (23) or the learning data preprocessed by the preprocessing unit. The selected learning data can be provided to the model learning unit (24). For example, the learning data selection unit can select only data for objects included in a specific area as learning data by detecting a specific area among images acquired through a camera.

[0078] Additionally, the data learning unit (22) may further include a model evaluation unit (not shown) to improve the analysis results of the neural network model.

[0079] The model evaluation unit inputs evaluation data into the neural network model, and if the analysis results output from the evaluation data do not satisfy a predetermined standard, it can cause the model learning unit (22) to relearn. In this case, the evaluation data may be predefined data for evaluating the recognition model. For example, the model evaluation unit can evaluate that the predetermined standard is not satisfied if the number or ratio of evaluation data with inaccurate analysis results among the analysis results of the learned recognition model for the evaluation data exceeds a preset threshold.

[0080] The communication unit (27) can transmit the AI ​​processing result by the AI ​​processor (21) to an external device.

[0081] The AI ​​device (20) illustrated in FIG. 3 is functionally divided into an AI processor (21), a memory (25), a communication unit (27), etc., but it should be noted that the aforementioned components may be integrated into one module and referred to as an AI module.

[0082]

[0083] Next, a method for automatically generating an event occurrence-related message through AI analysis of a captured video suggested in this specification will be examined with reference to FIGS. 4 and 5.

[0084] FIG. 4 is a diagram showing an example of a method for generating a message proposed in this specification, and FIG. 5 is a flowchart showing an example of a method for generating a message through image analysis proposed in this specification.

[0085] First, the video analysis device determines whether an event (or a specific event) has occurred in the video currently being filmed (hereinafter referred to as the “current video”) (S510).

[0086] The above event may be a predefined event, for example, an event related to human behavior such as a person wandering, an intrusion into a specific place, an abandonment of an object or thing, a fight between people, arson, a person falling down, etc., or an event related to a disaster such as a fire, a collapse, an explosion, a traffic accident, a chemical, biological, radiological, or environmental accident, etc., or an event related to a disaster such as a typhoon, a flood, heavy rain, heavy snow, a storm, a drought, a tsunami, an earthquake, etc.

[0087] Next, when the occurrence of the above event is detected, the image analysis device analyzes the current image to extract a first keyword related to the event (S520).

[0088] The above first keyword may include an object, an event type, an area where the event occurred, the current time when the event occurred, etc.

[0089] The above object can be extracted from the current image using an object detection algorithm, and the object to be included in the first keyword is necessarily a person, and if there is no person, it can be an object associated with the occurred event, such as a weapon, a bag, a suitcase, etc. Alternatively, the object can include an object associated with the event, such as a weapon, a bag, a suitcase, etc. mentioned above, along with a person, but whether or not to include an object associated with the event can be optional.

[0090] And, the type of the above event refers to the type, type, etc. of the event that occurred in the current video and caused the extraction of the first keyword.

[0091] And, the current time at which the above event occurred means the time when the event occurred, that is, the time at which the current video was filmed.

[0092] In addition, the occurrence area of ​​the above event can be defined in advance as the place, region, space, etc. where the event occurred.

[0093] Next, the image analysis device generates an event initial message (or first message) composed of the extracted first keyword (S530). The first message may refer to a message initially generated by the image analysis device through detection of the first keyword when an event occurs.

[0094] More specifically, the image analysis device can generate the first message by sequentially arranging the extracted first keywords in the order of object, area, and event type, and the arrangement of the first keywords constituting the first message can be implemented through an LLM (Large Language Model) model.

[0095] As illustrated in FIG. 4, for example, if the object of the first keyword is 'a male suspected of being a patient', the area of ​​the first keyword is 'parking lot entrance', and the event type of the first keyword is 'falling down', the image analysis device can use the LLM model in the order of the object, area, and event type to generate the first message 'a male suspected of being a patient is falling down at the parking lot entrance'.

[0096] Additionally, the image analysis device generates the first message by combining and / or processing the extracted first keywords to conform to the standard Korean grammar when the extracted first keyword is Korean, or by combining and / or processing the extracted first keywords to conform to the standard English grammar when the extracted first keyword is English.

[0097] That is, the image analysis device can arrange keywords in accordance with the arrangement order of the extracted first keywords, determine the language type of the extracted keywords using the arranged keywords, and finally generate the first message in accordance with the standard grammar corresponding to the language type of the extracted keywords.

[0098] The above image analysis device can store standard grammar for each language in advance in the memory of the image analysis device to generate the first message according to the standard grammar for each type of language.

[0099] Next, the image analysis device generates an event addition message (or second message) based on the generated first message.

[0100] More specifically, the image analysis device searches for a previous image related to the current image in a memory or storage device and extracts (or finds or obtains or acquires) the previous image.

[0101] Here, the image analysis device can obtain a previous image related to the current image from the memory or storage device based on the following criteria.

[0102] First, (1) images prior to a predetermined time from the time of the current image are primarily filtered, and images including 'object' and / or 'area' among the first keywords among the primarily filtered images are determined as the previous image, or (2) images including 'object' among the first keywords are primarily filtered, and when a predetermined number or more of the primarily filtered images exist, the previous image is determined by considering 'area' and / or 'time' among the first keywords.

[0103] Here, when considering the 'time' of the first keyword, only videos within a certain time (e.g., 30 minutes ago, 1 hour ago, 4 hours ago, 24 hours ago, etc.) from the time of the current video can be considered.

[0104] And, if the image analysis device cannot determine or specify the previous image through both of the methods for determining the previous image discussed above, it determines the image from which the 'cause or characteristic of the event' among the second keywords constituting the second message to be described later can be extracted as the previous image among the images filtered by the method (1) or (2).

[0105] The above image analysis device extracts a second keyword from the determined previous image (S540).

[0106] The above second keyword may include an event cause or event characteristic, the time at which the previous video was shot (hereinafter referred to as “previous time”), an area, etc.

[0107] The image analysis device generates the second message including the extracted second keyword (S550).

[0108] More specifically, the image analysis device first calculates the 'difference time (or time delta)' before generating the second message through the extracted second keyword.

[0109] The above difference time refers to the time difference between the time the current video was filmed and the previous time. For example, if the current video was filmed at 9:10 PM and the previous time was 9:05 PM, the difference time is 5 minutes.

[0110] Then, the image analysis device sequentially arranges the calculated difference time, area, and cause of the event (or characteristic of the event) in order and applies the LLM model to generate the second message.

[0111] Here, the second message can be generated in accordance with a standard grammar that matches the language type of the second keyword applied when generating the first message.

[0112] For example, if the time difference of the second keyword is '5 minutes ago', the area is 'parking lot entrance', and the cause of the event is 'hit by a passing vehicle', the second message may generate a second message that says 'hit by a vehicle passing the parking lot entrance 5 minutes ago'.

[0113] Here, the language types of the first message and the second message may be the same, but if the language types of the first message and the language types of the second message are different, the image analysis device translates or converts the language type of the second message to match the language type of the first message.

[0114] Next, the image analysis device determines the urgency of the event and the scope of the event based on the generated first message and the second message (S560).

[0115] Here, the urgency of the above event is a parameter that determines the transmission range to managers, control personnel, police stations, fire stations, etc. depending on the type of event, and can be classified as high, medium, or low.

[0116] The urgency of the above event can be determined by using the difference time from the first keyword or the second keyword, the cause of the event, and the object as factors for determining the urgency.

[0117] That is, the urgency of the above event can be classified as high, medium, or low based on factors such as the process of occurrence of the event, the degree of impact of the event, the possibility of damage caused by the event, the predictability of the event, the turning point of the event, the controllability of the event, and the continuity of the event.

[0118] In addition, the scope of the above event is a factor that determines the location of damage, the size of damage, the height of damage, etc. according to the occurrence of the event, and can be determined as wide, normal, or narrow, and can be determined using area, objects, etc. among the first or second keywords as a judgment factor, and can be determined using the same situation exposure in conjunction with surveillance cameras such as surrounding CCTV.

[0119] Next, the image analysis device transmits an event occurrence-related notification message to the outside according to the urgency and scope of the determined event (S580).

[0120] In addition, the video analysis device records or displays a mark on the event according to the urgency and scope of the determined event (S570) and stores it in a memory or storage device, thereby extracting keywords from the video and generating event-related messages more accurately through the extracted keywords.

[0121] The above external locations may be field agents' terminals, police stations, fire stations, etc.

[0122]

[0123] FIG. 6 is a flowchart illustrating another example of a method for generating a message through image analysis proposed in this specification.

[0124] First, when a predefined event occurs in a captured video, the video analysis device detects a first keyword corresponding to an object, an event occurrence area, and an event type in the current video at the time of the event occurrence (S610).

[0125] Next, the image analysis device generates a first message related to the initial occurrence of the event based on the detected first keyword (S620).

[0126] The above first message can be generated by arranging the object, the event occurrence area, and the type of the event in that order.

[0127] Next, the image analysis device obtains a previous image for the current image (S630).

[0128] Here, the previous image may be an image that includes the object among images prior to the occurrence of the event related to the current image.

[0129] Next, the image analysis device detects a second keyword corresponding to the shooting time of the previous image, the area related to the occurrence of the event, and the cause of the occurrence of the event, respectively, from the acquired previous image (S640).

[0130] Next, the image analysis device generates a second message related to the cause of the event occurrence based on the detected second keyword (S650).

[0131] The second message may be generated by comparing the shooting time of the previous image detected by the second keyword with the shooting time of the current image to calculate a time delta, and arranging the calculated time delta, the area related to the occurrence of the event, and the cause of the occurrence of the event in that order.

[0132] Next, the image analysis device determines the urgency of the event and the occurrence range of the event based on the first message and the second message (S660).

[0133] The urgency of the above event can be determined based on the difference time, the cause of the event, and the object.

[0134] The urgency of the above event can be classified as high, medium, or low.

[0135] The scope of the above event can be determined based on the event occurrence area and the object.

[0136] The scope of the above event can be classified as broad, moderate or narrow.

[0137] Additionally, the image analysis device can transmit a notification message related to the occurrence of the event based on the urgency of the determined event and the occurrence range of the event.

[0138] Additionally, the first message and the second message can be generated through a large language model (LLM).

[0139] Additionally, the image analysis device can convert the language of the second keyword into the same language as the language of the first keyword when the language type of the first keyword is different from the language type of the second keyword.

[0140]

[0141] The embodiments described above are combinations of components and features of the present invention in a predetermined form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented without being combined with other components or features. Furthermore, it is also possible to form an embodiment of the present invention by combining some components and / or features. The order of operations described in the embodiments of the present invention may be changed. Some components or features of one embodiment may be included in another embodiment or may be replaced with corresponding components or features of another embodiment. It is self-evident that claims that do not have an explicit citation relationship in the patent claims may be combined to form an embodiment or may be incorporated as a new claim through a post-application amendment.

[0142] Embodiments of the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of hardware implementation, an embodiment of the present invention may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.

[0143] When implemented via firmware or software, an embodiment of the present invention may be implemented in the form of a module, procedure, function, or the like that performs the functions or operations described above. The software code may be stored in memory and executed by a processor. The memory may be located within or external to the processor and may exchange data with the processor via various known means.

[0144] It will be apparent to those skilled in the art that the present invention can be embodied in other specific forms without departing from the essential characteristics thereof. Therefore, the above detailed description should not be construed as limiting in any respect, but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the scope of equivalents of the present invention are intended to be included within the scope of the present invention.

[0145] The method of generating event-related messages through image analysis of the present invention has been described with a focus on examples applied to surveillance systems, but it can also be applied to various other image systems.

Claims

1. A method for generating event-related messages through video analysis, When a predefined event occurs in a captured video, a step of detecting a first keyword corresponding to an object, an event occurrence area, and an event type in the current video at the time of the event occurrence; A step of generating a first message related to the initial occurrence of an event based on the first keyword detected above; A step of obtaining a previous image for the current image; A step of detecting a second keyword corresponding to the shooting time of the previous image, the area related to the occurrence of the event, and the cause of the occurrence of the event, respectively, from the acquired previous image; A step of generating a second message related to the cause of the event occurrence based on the detected second keyword; and A method characterized by comprising the step of determining the urgency of the event and the scope of occurrence of the event based on the first message and the second message.

2. In paragraph 1, A method characterized by further comprising the step of transmitting a notification message related to the occurrence of the event based on the urgency of the determined event and the occurrence range of the event.

3. In paragraph 1, A method characterized in that the above previous image is an image including the object among the images prior to the occurrence time of the event related to the current image.

4. In paragraph 3, The step of generating the above first message is: A method characterized by including a step of arranging in the order of the object, the event occurrence area, and the type of the event.

5. In paragraph 4, The step of generating the second message is: A step of calculating a time delta by comparing the shooting time of the previous image detected by the second keyword with the shooting time of the current image; and A method characterized by comprising the step of arranging the calculated difference time, the area related to the occurrence of the event, and the cause of the occurrence of the event in that order.

6. In paragraph 5, A method characterized in that the urgency of the above event is determined based on the difference time, the cause of occurrence of the above event, and the above object.

7. In paragraph 6, A method characterized in that the urgency of the above event is classified as high, medium, or low.

8. In paragraph 7, A method characterized in that the scope of the above event is determined based on the event occurrence area and the object.

9. In paragraph 8, A method characterized in that the scope of the above event is classified as wide, medium or narrow.

10. In paragraph 9, A method characterized in that the first message and the second message are generated through a large language model (LLM).

11. In Article 10, A method characterized by further comprising a step of converting the language of the second keyword into the same language type as the first keyword, if the language type of the first keyword and the language type of the second keyword are different.

12. In a video analysis device that generates event-related messages through video analysis, A wireless communication unit for transmitting and receiving wireless signals; Memory for storing captured images; and A processor functionally connected to the wireless communication unit and the memory, wherein the processor comprises: An image analysis device characterized in that, when a predefined event occurs in a captured image, a first keyword corresponding to an object, an area where the event occurs, and a type of the event is detected from a current image at the time of the event occurrence, a first message related to the initial occurrence of the event is generated based on the detected first keyword, a previous image with respect to the current image is acquired, a second keyword corresponding to the shooting time of the previous image, an area related to the occurrence of the event, and a cause of the occurrence of the event is detected from the acquired previous image, a second message related to the cause of the occurrence of the event is generated based on the detected second keyword, and the urgency of the event and the occurrence range of the event are determined based on the first message and the second message.

13. In the 12th paragraph, the processor, An image analysis device characterized in that it determines an image including the object as a previous image among images prior to the occurrence of the event related to the current image.

14. In the 12th paragraph, the processor, An image analysis device characterized in that it generates the first message by arranging the detected first keywords in the order of the object, the event occurrence area, and the type of the event.

15. In the 12th paragraph, the processor, An image analysis device characterized in that it compares the shooting time of the previous image detected by the second keyword with the shooting time of the current image to calculate a time difference (time delta), and generates the second message by arranging the calculated time difference, the area related to the occurrence of the event, and the cause of the occurrence of the event in that order.

Images