Surveillance camera system and event notification method thereof

Abstract

The present invention provides a surveillance camera system comprising: an image processing device for acquiring video data; and a server for acquiring the video data from the image processing device, generating a notification on the basis of a preset first notification rule in response to an event of the video data, analyzing a user response pattern or event information acquired in response to the notification, and generating a second notification rule that is distinct from the first notification rule, on the basis of the response pattern or the event information.

Description

Surveillance camera system and event notification method for surveillance camera system

[0001] Embodiments of the present invention relate to a surveillance camera system and an event notification method for a surveillance camera system.

[0002] Recently, surveillance camera systems equipped with multiple video processing units are being widely utilized for security and safety management in various environments, including public spaces, industrial facilities, and buildings. These systems collect video from numerous cameras distributed across different locations and perform tasks such as image recognition, analysis, and storage through video processing units. Central control centers use integrated management software, such as a Video Management System (VMS), to monitor video transmitted from each device in real time and perform functions such as searching for recorded footage and providing event notifications.

[0003] The present invention aims to provide a surveillance camera system and a method for notifying events of a surveillance camera system. However, these objectives are exemplary and do not limit the scope of the present invention.

[0004] According to one aspect of the present invention, a surveillance camera system is provided, comprising: an image processing device for acquiring video data; and a server that acquires video data from the image processing device, generates a notification based on a first notification rule that is preset in response to an event of the video data, analyzes a user's response pattern or event information acquired in response to the notification, and generates a second notification rule that is distinguished from the first notification rule based on the response pattern or the event information.

[0005] The above server can display the analysis result of the above response pattern and the above second notification rule on the user interface.

[0006] The above server can generate a notification based on the first notification rule that generates a notification in response to a motion detection event, and display the generated notification on the user interface.

[0007] The server above can modify the first notification rule corresponding to the motion detection event to generate the second notification rule corresponding to the object-based motion detection event in which the motion detection unit has been changed.

[0008] The above server can generate a notification based on the first notification rule that generates a notification in response to a temperature detection event, and display the generated notification on the user interface.

[0009] The server above can modify the first notification rule corresponding to the temperature detection event to generate the second notification rule with a changed detection temperature range.

[0010] The above server can analyze whether the user plays the content obtained during a preset time in response to the above notification, and can analyze the event information including the event frequency.

[0011] According to one aspect of the present invention, an event notification method for a surveillance camera system is provided, comprising the steps of: acquiring video data from an image processing device; generating a notification based on a first notification rule that is preset in response to an event of the video data; analyzing a user's response pattern or event information acquired in response to the notification; and generating a second notification rule that is distinguished from the first notification rule based on the response pattern or the event information.

[0012] An event notification method according to one embodiment of the present invention may further include the step of displaying the analysis result of the corresponding pattern and the second notification rule on a user interface.

[0013] The step of generating the above notification may include generating a notification based on the first notification rule that generates a notification in response to a motion detection event, and displaying the generated notification on a user interface.

[0014] The step of generating the second notification rule may include modifying the first notification rule corresponding to the motion detection event to generate the second notification rule corresponding to the object-based motion detection event in which the motion detection unit has been changed.

[0015] The step of generating the above notification may include generating a notification based on the first notification rule that generates a notification in response to a temperature detection event, and displaying the generated notification on a user interface.

[0016] The step of generating the second notification rule may include modifying the first notification rule corresponding to the temperature detection event to generate the second notification rule with a changed detection temperature range.

[0017] The step of analyzing the above-mentioned response pattern or event information may include a step of analyzing whether the user plays the content obtained during a preset time in response to the above-mentioned notification, and a step of analyzing the above-mentioned event information including the event frequency.

[0018] According to one aspect of the present invention, a computer program stored in a computer-readable recording medium is provided to execute the method described above using a computer.

[0019] Other aspects, features, and advantages other than those described above will become clear from the following specific details, claims, and drawings for implementing the invention.

[0020] According to one embodiment of the present invention as described above, there is an effect of providing event notifications optimized for the user through user pattern analysis. Of course, the scope of the present invention is not limited by this effect.

[0021] FIG. 1 is a schematic diagram showing a surveillance camera system according to one embodiment.

[0022] FIGS. 2 and FIGS. 3 are schematic block diagrams of an image processing device according to one embodiment.

[0023] FIG. 4 is a schematic block diagram of a network server according to one embodiment.

[0024] FIG. 5 is a schematic diagram showing a surveillance camera system according to one embodiment.

[0025] FIGS. 6a and FIGS. 6b are schematic drawings illustrating a surveillance camera system according to one embodiment.

[0026] FIG. 7 is a schematic diagram showing an AI server according to one embodiment.

[0027] FIG. 8 is a schematic diagram showing a cloud-based surveillance camera system according to one embodiment.

[0028] FIG. 9 is a schematic diagram showing a monitoring system according to one embodiment.

[0029] FIG. 10 is a flowchart illustrating an event notification method according to one embodiment.

[0030] FIGS. 11 and FIGS. 12 are drawings for explaining an event notification method according to one embodiment.

[0031] FIG. 13 is a diagram illustrating a software update method according to one embodiment.

[0032] FIGS. 14 and FIGS. 15 are drawings for illustrating an event notification method according to another embodiment.

[0033] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components regardless of drawing symbols will be assigned the same reference number, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not inherently possess distinct meanings or roles. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the invention.

[0034] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0035] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0036] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0037] In embodiments of the present invention, terms such as "comprising" or "having" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not excluding in advance the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0038] Additionally, in embodiments of the present invention, at least one of the plurality of components refers to all of the plurality of components, as well as each of the plurality of components excluding the remainder, or any combination thereof. Furthermore, "configured to" may be used interchangeably with, depending on the context, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." "Configured to" does not necessarily mean only that which is "specifically designed to" in hardware. Instead, in some situations, the expression "device configured to" may mean that the device is "capable of" together with other devices or components. For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing said operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or an application processor) capable of performing said operations by executing one or more software programs stored in a memory device.

[0039] In the present invention, the term "image" is used to encompass discontinuous video, still image, and MJPEG.

[0040] FIG. 1 is a schematic diagram showing a surveillance camera system according to one embodiment.

[0041] Referring to FIG. 1, a surveillance camera system (10) according to one embodiment may include an image processing device (100) and an external device (300). In one embodiment, the surveillance camera system (10) may further include a network server (200) capable of performing wired and wireless communication with the image processing device (100) and the external device (300).

[0042] The image processing device (100) may be an electronic device for capturing images and audio of a surveillance area by capturing a surveillance area. The image processing device (100) may capture a surveillance area in real time for surveillance or security purposes and may be provided with one or more devices. The image processing device (100) may be a camera placed at a fixed position in a specific location, a camera that can move automatically or manually along a certain path, or a camera that can be moved by a person or a robot, etc. The image processing device (100) may be a PTZ camera having pan, tilt, and zoom functions. The image processing device (100) may be an analog camera, or an IP camera or network camera used by connecting to a wired or wireless internet. Depending on the function and use of the camera, the image processing device (100) may have various shapes and sizes, such as dome type, box type, bullet type, camouflage type, etc.

[0043] In one embodiment, the image processing device (100) is a visible light camera and can generate a visible image according to the luminance distribution of an object by acquiring image information by detecting light. In one embodiment, the image processing device (100) is an infrared light camera (or thermal camera) and can generate a thermal image that displays different colors depending on the intensity by detecting radiant energy (thermal energy) emitted by an object in the form of an infrared wavelength, which is a type of electromagnetic wave, and measuring the intensity of the thermal energy.

[0044] The image processing device (100) may have the function of recording the surveillance area or taking a photograph. The image processing device (100) may have the function of recording sounds occurring in the surveillance area. The image processing device (100) may have the function of generating an alert or performing recording or taking a photograph when a change, such as movement or sound, occurs in the surveillance area.

[0045] The image processing device (100) can analyze the captured image itself and / or an image obtained by editing the image. For example, the image processing device (100) can detect objects in the image by using an object detection algorithm. The object detection algorithm may be an AI-based algorithm, and may detect objects by applying a pre-trained artificial neural network model.

[0046] The image processing device (100) can analyze an image to generate metadata and index information for said metadata. The image processing device (100) can analyze the image and / or audio included in the image together or separately to generate metadata and index information for said metadata.

[0047] The network server (200) can provide various functions to the image processing device (100) and the external device (300) through the network (400). In FIG. 1, the network server (200) is provided independently from the image processing device (100), but the embodiments of the present invention are not limited thereto. For example, the network server (200) may be embedded in the image processing device (100). A description of the network server (200) will be provided later.

[0048] The external device (300) may include a server (3001) and a client (3002).

[0049] The server (3001) may be a device that includes a DVR (Digital Video Recorder), NVR (Network Video Recorder), VMS (Video Management System), etc., and provides video storage, search, and management functions. The server (3001) may receive and store video and / or audio from the video processing device (100) and index the video and / or audio for user search. The server (3001) may analyze the video and / or audio received from the video processing device (100). In one embodiment, a plurality of servers (3001) built at physically different locations may be federated with each other and managed and controlled by a single software. In one embodiment, the server (3001) may be built in a government office, police station, hospital, central control center, central control center, comprehensive situation room, etc.

[0050] The client (3002) is a terminal including a desktop PC, tablet PC, slate PC, notebook computer, mobile device such as a smartphone, etc., and may include software and / or applications for performing video monitoring and search, etc. by a user (operator). The client (3002) may be connected to the image processing device (100) and / or server (3001) via a network. In one embodiment, the client (3002) may manage and operate the image processing device (100) and server (3001) based on the cloud.

[0051] In one embodiment, the client (3002) transmits an information provision request signal to the image processing device (100) or server (3001) requesting the provision of all or part of the image and / or audio, and may receive all or part of the image and / or audio from the image processing device (100) or server (3001). The client (3002) transmits an information provision request signal to the image processing device (100) or server (3001) requesting metadata and / or index information for the metadata obtained by analyzing the image and / or audio, and may receive metadata and / or index information from the image processing device (100) or server (3001).

[0052] The image processing device (100), network server (200), and external device (300) can perform wired and wireless communication through the network (400). The network (400) may include, for example, wired networks such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), ISDNs (Integrated Service Digital Networks), wireless internet such as 3G, 4G (LTE), 5G, WiFi, Wibro, Wimax, wireless LANs, CDMA, satellite communication, and wireless networks including short-range communication such as Bluetooth, RFID (Radio Frequency Identification), infrared communication (IrDA, infrared Data Association), UWB (Ultra Wideband), ZigBee, and NFC (Near Field Communication). In the case of wireless mobile communication, the network (400) may additionally include components such as a base station (BTS), a mobile switching center (MSC), a home location register (HLR), an access gateway that enables the transmission and reception of wireless packet data, and a Packet Data Serving Node (PDSN). The scope of this specification is not limited thereto.

[0053] FIGS. 2 and FIGS. 3 are schematic block diagrams of an image processing device according to one embodiment.

[0054] In one embodiment, the image processing device (100) may be a camera (101). The camera (101) may be an IP camera or a network camera. The camera (101) may acquire an image, compress the acquired image, and transmit it to an external device (300) (see FIG. 1) using a predetermined transmission protocol. For example, the camera (101) may acquire an image, analyze the acquired image in real time to detect / identify / track / analyze / search for a target or moving object, and provide appropriate information or functions to the user.

[0055] Referring to FIGS. 2 and FIGS. 3, the camera (101) may include a sensing unit (1010), a data processing unit (1030), and a communication unit (1050).

[0056] The sensor unit (1010) may be a sensing means including an optical system (110) and an image sensor (120).

[0057] The optical system (110) can optically process light from a subject. The optical system (110) may include at least one lens, such as a zoom lens that controls the field of view to narrow or widen according to the focal length, and a focus lens that focuses. In one embodiment, the lens may include at least one glass lens and / or at least one liquid lens. A liquid lens is a lens that can control zoom and focus by controlling the thickness (shape (curvature) of the liquid lens) using a fluid liquid. Using a liquid lens has the advantage of reducing the number of lenses and the size of the lens module, and also allows for fast focusing at a desired focal length.

[0058] The optical system (110) may further include an optical low-pass filter (OLPF), an infrared cut filter (IRCF), an iris for controlling light intensity, etc.

[0059] The image sensor (120) can perform the function of capturing a surveillance area to acquire an image. In one embodiment, the image sensor (120) can be implemented as a CCD (Charge-Coupled Device) sensor, a CMOS (Complementary Metal-Oxide-Semiconductor) sensor, etc. The image sensor (120) can convert an optical signal transmitted through the optical system (110) into an electrical signal.

[0060] The data processing unit (1030) may be an information processing means implemented with various number of hardware or / and software configurations that execute specific functions. For example, the data processing unit (1030) may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by code or instructions included in a program. The data processing device embedded in hardware may include, for example, a microprocessor, a central processing unit (CPU), an image signal processor (ISP), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), and other processing devices, but the scope of the present invention is not limited thereto. The data processing unit (1030) may be implemented with at least two of the above-described processing devices and processors as a single integrated configuration (e.g., a single chip), or each as an independent configuration (e.g., multiple chips).

[0061] In one embodiment, the data processing unit (1030) may further include a neural processing unit (NPU). In one embodiment, the neural processing unit may be included in the data processing unit (1030) in a form inherent within the aforementioned processing unit and processor. The neural processing unit is a processor specialized in processing artificial intelligence models, and the artificial intelligence model may be generated through machine learning. The artificial intelligence model may include a plurality of artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may additionally or substantially include a software structure.

[0062] In one embodiment, the artificial intelligence model can be trained by collecting and labeling suitable training data for determining object detection / classification / recognition, etc., and applying it to an artificial neural network. The trained artificial intelligence model can be mounted on a camera (101) and used for inference operations such as object detection / classification / recognition through a neural network processing device of the camera (101) or the aforementioned data processing device.

[0063] In one embodiment, the artificial intelligence model can be further trained using a cloud server to suit the monitoring environment or user requirements even after being installed once on the camera (101), and the newly trained model can be redeployed to the camera (101) for use.

[0064] A neural network processing unit may be mounted on the camera (101) in the form of at least one hardware chip. For example, the neural network processing unit may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as part of a general-purpose processor (CPU) or a graphics-dedicated processor (GPU) and mounted on the camera (101). Additionally, the neural network processing unit may be implemented as a software module. If the neural network processing unit is implemented as a software module (or a program module containing instructions), the software module may be stored on a non-transitory computer-readable media. In this case, at least one software module may be provided by an operating system (OS) or by an application. The data processing unit (1030) may include an image processing unit (130), a streaming unit (140), an analysis unit (150), an event unit (160), a system management unit (170), and a camera control unit (180).

[0065] The image processing unit (130) can improve image quality by adjusting the brightness, contrast, color amount, contrast ratio, sharpness, etc. of the image. Each parameter value for image quality improvement can be fixed as an initial value or adjusted by the user and pre-set. The image processing unit (130) can perform image signal processing for image quality improvement, such as noise reduction, gamma correction, color filter array interpolation, color matrix, color correction, and color enhancement. The image processing unit (130) can apply various methods for image quality improvement, such as edge enhancement algorithms, histogram-based image quality improvement algorithms such as histogram equalization or histogram stretching, and hue control algorithms. In one embodiment, the image processing unit (130) may be an image signal processor (ISP). The image processing unit (130) may include an exposure control unit (1311), a backlight correction unit (1312), a balance correction unit (1313), a gamma correction unit (1314), a sharpness control unit (1315), a shake correction unit (1316), a focus control unit (1317), a zoom control unit (1318), a filter control unit (1319), an OSD control unit (1320), and a noise control unit (1321).

[0066] The exposure control unit (1311) can control the brightness of the image by controlling the amount of light entering the image sensor (120) using at least one of a shutter, an iris, and gain or ISO sensitivity. The exposure control unit (1311) can control the amount of light by adjusting the shutter speed. The exposure control unit (1311) can control the amount of light by adjusting the degree of opening and closing of the iris. The exposure control unit (1311) can control the degree of light amplification by adjusting the gain or ISO sensitivity. In one embodiment, the exposure control unit (1311) can control the shutter, the iris, and the gain / ISO sensitivity according to user settings. In one embodiment, when the function is set, the exposure control unit (1311) can perform an auto exposure algorithm according to the illuminance of the environment in which the camera is installed.

[0067] The backlight correction unit (1312) can improve the quality of an image by applying a backlight correction algorithm to the image so that dark areas are clearly visible when the image is too bright overall. The backlight correction algorithm may include a Back Light Compensation (BLC) algorithm, a Wide Dynamic Range (WDR) algorithm, a Digital Wide Dynamic Range (D-WDR) algorithm, and a High Dynamic Range (HDR) algorithm. The backlight correction unit (1312) can increase the dynamic range using the WDR algorithm, the D-WDR algorithm, and the HDR algorithm, thereby enabling both bright and dark areas in the image to be clearly visible simultaneously. In one embodiment, the backlight correction unit (1312) can block backlighting in a specific area to make the image of other areas appear brighter. For example, when car headlights shine strongly at a dark parking lot entrance or a gas station entrance at night, only the headlight light can be blocked to identify the vehicle license plate. The function of the backlight correction unit (1312) can be set by the user (on / off).

[0068] The balance correction unit (1313) can improve the color identification ability of an image by adjusting the white balance of the image to correct color differences according to the light source (illumination). In one embodiment, the balance correction unit (1313) can adjust the white balance appropriately for the selected light source according to user settings. In one embodiment, the balance control unit (1313) can perform an Auto White Balance algorithm according to the environment around the camera when its function is set. In one embodiment, the balance control unit (1313) can adjust the white balance of the image to a white balance value manually set by the user.

[0069] The gamma correction unit (1314) can increase visibility by changing the contrast of the image through gamma correction.

[0070] The sharpness control unit (1315) can increase the resolution by adjusting the sharpness of the subject boundary (contour).

[0071] The shake correction unit (1316) can stabilize the image by applying a shake correction algorithm to the image to correct image shake caused by camera shake. The shake correction algorithm may include an Optical Image Stabilization (OIS) algorithm, an Electronic Image Stabilization (EIS) algorithm, and a Digital Image Stabilization (DIS) algorithm.

[0072] The focus control unit (1317) can control the focus of the camera using a focus control algorithm. In one embodiment, the focus control unit (1317) can move the position of the focus lens by controlling the focus motor according to a control signal for focus control. The focus control algorithm may include known autofocus algorithms such as a hill climbing method. For example, the focus control unit (1317) can move the position of the focus lens from a near distance to a far distance (infinity) in real time, compare the focus values ​​before and after to check the increase or decrease in the focus value, the tilt, and the change in the tilt, determine the peak point with the largest focus value as the in-focus position, and control the driving of the focus motor to move the focus lens to the in-focus position. In one embodiment, the focus control unit (1317) can control the focus by controlling the curvature of the liquid lens according to a control signal for focus control.

[0073] The zoom control unit (1318) can perform optical zoom and / or digital zoom functions according to a control signal for controlling the zoom magnification. In one embodiment, the zoom control unit (1318) can adjust the zoom magnification by moving the position of the zoom lens to zoom in or zoom out. In one embodiment, the zoom control unit (1318) can control the zoom magnification by controlling the curvature of the liquid lens. In one embodiment, the zoom control unit (1318) can provide a more magnified image through digital zoom exceeding the optical zoom. The angle of view or field of view of the camera can be controlled by the zoom magnification.

[0074] The filter control unit (1319) can improve image identification capabilities during the day and night by adjusting the infrared cut filter (IRCF). The filter control unit (1319) can block infrared rays and allow visible light to pass through by positioning the infrared cut filter (IRCF) in front of the image sensor (120). The filter control unit (1319) can allow infrared rays to pass through in addition to visible light by removing the infrared cut filter (IRCF). The filter control unit (1319) can perform a day / night mode function by automatically turning the infrared cut filter (IRCF) on / off according to the ambient light level.

[0075] The OSD (On-screen display) control unit (1320) can overlay a mask of text and / or polygons on the image. For example, the OSD (On-screen display) control unit (1320) can display the date, time, manufacturer, etc. at a predetermined location on the image.

[0076] The noise control unit (1321) can perform a digital noise reduction algorithm to reduce color noise in an image captured in a low-light environment. The noise reduction algorithm may include 2D-NR (Noise Reduction) and 3D-NR.

[0077] The streaming unit (140) can compress the video received from the video processing unit (130) and provide it to an external device via a network. The streaming unit (140) may include an encoder (1411), an audio processing unit (1412), and an RTSP server (1413).

[0078] The encoder (1411) can encode the image and compress the image. The encoder (1411) can compress the raw image input from the image sensor (120) or the image processed by the image processing unit (130). In one embodiment, the raw image may be used for artificial intelligence (AI) algorithms, etc.

[0079] The encoder (1411) can compress video using an intra-frame compression method and / or an inter-frame compression method. Depending on the network environment or monitoring purpose, the encoder (1411) can compress video into compression formats such as H.264, H.265 (HEVC (High Efficiency Video Coding)), JPEG, and MPEG-4.

[0080] The encoder (1411) can compress video according to encoding settings to generate at least one video stream. Encoding settings may include compression format, video quality (high quality / medium quality / low quality), resolution, bit rate, frame rate, etc. The encoder (1411) may compress video using a constant bit rate (CBR) or a variable bit rate (VBR). A profile of the video stream may be determined according to the encoding settings. The profile may represent the specifications of the video stream transmitted from the camera (101) to an external device. The profile may include setting values ​​such as compression format, video quality (high quality / medium quality / low quality), resolution, frame rate, bit rate, etc. For example, the profile may include setting values ​​such as "MJPEG, low quality, 640X480, 5fps, 2Mbps". In one embodiment, a profile of a video stream output from an encoder (1411) may be pre-set according to the network environment and the video receiving device (external device). For example, a profile suitable for streaming purposes may be pre-set, such as a profile for a video stream for storage, a profile for a video stream for monitoring screen output, a profile for a video stream for a mobile device, or a profile for a video stream for a web viewer. The encoder (1411) can compress the video using the setting values ​​of the selected profile.

[0081] The audio processing unit (1412) includes an audio codec and can generate at least one audio stream by processing and compressing audio input from an audio sensor and / or a stored sound source. Depending on the microphone structure, the audio sensor may be a dynamic microphone, a condenser microphone, a ribbon microphone, etc., and depending on the directionality, a directional microphone, an omnidirectional microphone, a super-directional microphone, etc., may be used. The audio sensor may be provided separately from the image sensor (120) or integrally, such as being embedded in the image sensor (120).

[0082] The RTSP server (1413) may be a server operated for real-time video streaming. The RTSP server (1413) may transmit video streams and audio streams to external devices connected to the RTSP port via a network according to the Real-time Streaming Protocol (RTSP).

[0083] The analysis unit (150) can analyze the image processed by the image processing unit (130) in real time. The analysis unit (150) can perform background area detection, foreground and object detection, object counting, camera tampering detection, face detection, etc. Additionally, the analysis unit (150) can calculate brightness, color, texture, and shape information of the image. The analysis unit (150) can analyze audio input from an audio sensor in real time. The results of the image and / or audio analysis can be generated as metadata. In one embodiment, the analysis unit (150) can detect / recognize / identify objects using artificial intelligence (AI) based technology.

[0084] The analysis unit (150) may include a motion detection unit (1511), an audio detection unit (1512), an audio classification unit (1513), a tempering detection unit (1514), and a defocus detection unit (1515).

[0085] The motion detection unit (1511) can detect the motion of an object within an image. The motion detection unit (1511) can detect the motion of an object from an image using a motion detection (MD) algorithm. In one embodiment, the motion detection unit (1511) can detect motion using an artificial neural network (ANN).

[0086] The audio detection unit (1512) can detect audio input from the audio sensor.

[0087] The audio classification unit (1513) can classify the detected audio. In one embodiment, the audio can be classified into vocal, which is a sound produced from a person's throat, and non-vocal, which is other sounds. For example, vocal can be classified into conversation, scream, crying, etc., and non-vocal can be classified into footsteps, glass breaking, explosion, crash, gunshot, etc. It goes without saying that vocal and non-vocal are not limited to the types mentioned above and can be set in various ways depending on the monitoring area and system design. In one embodiment, the audio classification unit (1513) can learn audio features and classify the audio using an Artificial Neural Network (ANN).

[0088] The tempering detection unit (1514) can detect camera movement or camera obstruction situations that last for a predetermined period of time. For example, the tempering detection unit (1514) can detect situations where spray is applied to the lens, where the lens is covered, etc.

[0089] The defocus detection unit (1515) can detect focus distortion.

[0090] The event unit (160) may receive video and / or audio from the analysis unit (150). The event unit (160) may receive metadata indicating the analysis results of the video and / or audio from the analysis unit (150). The event unit (160) may detect an event based on the metadata and notify the detection of the event. The event unit (160) may generate an event rule to perform an action on the device when a specific event occurs. The event rule may include conditions and actions. The conditions may be detailed rules for detecting the event. The actions may be operations performed when the occurrence of the event is determined. The event unit (160) may associate an action to be taken in response to the occurrence of the event. In one embodiment, the event conditions and the action associated with the event may be predefined by the user.

[0091] The event unit (160) may include an event generation unit (1611), a handover unit (1612), an alarm unit (1613), an FTP upload unit (1614), a storage control unit (1615), and an MQTT unit (1616).

[0092] The event generation unit (1611) can generate conditions by combining various situations detected as a result of analyzing video and / or audio. The event generation unit (1611) can detect events based on conditions set according to the type of event.

[0093] In one embodiment, the event generating unit (1611) may generate a video event when the analysis result of the video satisfies a set event condition. Here, the video event may include the appearance and / or disappearance of an object within the screen, the occurrence of a specific image (e.g., the appearance of a face that cannot be recognized) by the user, the change in screen color, the detection of motion of an object, the detection of motion by the user within a specific area, the detection of motion by the user in a specific direction, the turning off of the video, tampering, the distortion of focus, etc.

[0094] In one embodiment, the event generating unit (1611) may generate an audio event when the result of analyzing the audio satisfies a set event occurrence condition. The audio event may include events set by the user, such as the occurrence of a specific audio signal such as conversation, scream, crying, or shouting; the occurrence of abnormal audio signals such as footsteps, alarm, crash, car tire skid, glass breaking sound, explosion, or gunshot; or the occurrence of a voice exceeding a threshold value.

[0095] In one embodiment, the event generation unit (1611) may generate an event when a combination of video analysis results and / or audio analysis results satisfies a set event occurrence condition. It goes without saying that video events and audio events are not limited to the types described above and can be set in various ways depending on the monitoring area and system design.

[0096] The handover unit (1612) can transmit the generated event or event generation conditions to another camera, or receive the event or event generation conditions from another camera, so that the camera (101) can perform a set action when an event occurs, such as panning, tilting, zooming, or automatic object tracking.

[0097] The alarm unit (1613) can output an alarm signal when an event occurs. The alarm unit (1613) can trigger input from various sensors outside the camera or output to an actuator to output an alarm signal. The alarm unit (1613) can notify of the occurrence of an event through audio playback, email transmission, etc.

[0098] The FTP upload unit (1614) can upload the video and / or audio in which the event is detected to a File Transfer Protocol (FTP) server on the network when an event occurs. The video may include still images and video images. In one embodiment, the FTP upload unit (1614) can upload the video and / or audio in which the event is detected to an FTP server at a set upload period and a set upload speed when an event occurs.

[0099] The storage control unit (1615) can store (record) video and / or audio in a storage device. In one embodiment, the storage control unit (1615) can store video and / or audio and metadata received from the analysis unit (150) in a storage device. The metadata may include object information (movement, sound, intrusion into a designated area, etc.), object identification information (person, car, license plate, face, hat, clothing, etc.) detected from the video and / or audio, and detected location information (coordinates, size, etc.). In one embodiment, when an event occurs, the storage control unit (1615) can store video and / or audio and metadata for a predetermined period of time before and after the event in a storage device. The storage control unit (1615) can store video according to a set profile.

[0100] The storage device can store various programs and data required for the operation of the camera (101). Data reading, recording, modification, deletion, and updating can be performed by each component including the image processing unit (130) of the storage device. Additionally, the storage device can store a neural network model (e.g., a deep learning model) generated through a learning algorithm for data recognition / classification. The storage device may include internal memory and / or external storage media such as an SD card. The storage device may include a Video Management System (VMS), a Network Video Recorder (NVR), etc. The storage device may include a Network-Attached Storage (NAS) and a Storage Area Network (SAN), such as web storage and a cloud server, which perform storage functions over the Internet.

[0101] The MQTT unit (1616) can transmit (notify) events or receive notifications of event occurrences using the MQTT (Message Queuing Telemetry Transport) protocol. In one embodiment, the MQTT unit (1616) includes an MQTT client and can transmit events in real time by publishing MQTT messages to an MQTT broker (server) using the MQTT protocol through the MQTT client. In one embodiment, the MQTT unit (1616) can subscribe to MQTT messages published to the MQTT broker (server) to receive notifications of event occurrences in real time and start video recording.

[0102] The system management unit (170) may include a firmware upgrade unit (1711), a system information management unit (1712), a power management unit (1713), an RTC management unit (1714), and a network management unit (1715).

[0103] The firmware upgrade unit (1711) can upgrade the firmware (F / W) of the camera. The firmware upgrade unit (1711) can upgrade the old version firmware by receiving new firmware (F / W) from the firmware server via wired or wireless connection.

[0104] The system information management department (1712) can manage product information such as the camera model name and serial number.

[0105] The power management unit (1713) can convert AC or DC power supplied from an external power source into power required for the operation of each component of the camera. The power management unit (1713) may include an auxiliary power source, such as a rechargeable built-in or replaceable battery. The power management unit (1713) can manage power according to the class of the Ethernet Power over Ethernet (PoE). The power management unit (1713) can manage and control the auxiliary power source and the external power source in maximum power mode and low power mode.

[0106] The RTC management unit (1714) can manage the camera's internal time by synchronizing with a network server such as NTP.

[0107] The network management unit (1715) can manage general network functions such as IP address settings, port settings, and DDNS server settings.

[0108] The camera control unit (180) can control the camera's posture by moving the camera to a physical position corresponding to the Pan / Tilt / Rotate / ZoomFocus value according to the object's motion information, user input, or preset. The camera control unit (180) may include a pan control unit (1811), a tilt control unit (1812), a rotation control unit (1813), a preset control unit (1814), and a sequence action control unit (1815).

[0109] The fan control unit (1811) can control the rotation of the camera in the horizontal direction. In one embodiment, the fan control unit (1811) can rotate the camera in the horizontal direction by controlling the driving of the fan motor based on the fan value.

[0110] The tilt control unit (1812) can control the rotation of the camera in the vertical direction. In one embodiment, the tilt control unit (1812) can rotate the camera in the vertical direction by controlling the driving of the tilt motor based on the tilt value.

[0111] The rotation control unit (1813) can control the rotation of the camera in a clockwise or counterclockwise direction of the image. The rotation control unit (1813) can rotate the lens in a clockwise or counterclockwise direction of the image with the center of the camera as the axis.

[0112] The preset control unit (1814) can quickly move the camera to a physical position corresponding to a pre-specified Pan / Tilt / Rotate / ZoomFocus value (preset). The preset control unit (1814) can change the camera's orientation by moving or rotating the camera according to at least one specified preset.

[0113] The sequence action control unit (1815) can control the camera's operation repeatedly by scheduling operations that are pre-specified, such as presets.

[0114] The communication unit (1050) may be an interface that transmits a stream by connecting to a network server (200) and an external device (300) for wired or wireless communication. The communication unit (1050) may be configured to include TCP / IP, HTTP(S) / RTP / RTSP, FTP, MQTT protocols, etc.

[0115] To prevent the features of the present embodiment from being obscured, only the components related to the present embodiment are illustrated. Accordingly, it can be understood by those skilled in the art related to the present embodiment that other general components may be included in addition to the components illustrated in FIGS. 2 and FIGS. 3.

[0116] FIG. 4 is a schematic block diagram of a network server according to one embodiment.

[0117] Referring to FIG. 4, a network server (200) according to one embodiment may include an FTP server (2011), a web server (2013), an authentication server (2015), etc. The network server (200) may be provided independently on the network (400) separately from the camera, or may be embedded in the camera.

[0118] The FTP server (2011) can receive video and / or audio in which an event is detected from the FTP upload unit (1614) of the video processing device (100) when an event occurs.

[0119] The web server (2013) can provide HTTP(S) access to the image processing device (100) and the external device (300), or be embedded in the image processing device (100) or the server (3001) to provide HTTP(S) access to the external device (300).

[0120] The authentication server (2015) can perform user authentication when attempting to connect to the image processing device (100) of an external device (300) using RTP / RTSP / HTTP / TCP, etc.

[0121] FIG. 5 is a schematic diagram showing a surveillance camera system according to one embodiment.

[0122] Referring to FIG. 5, a surveillance camera system (20) according to one embodiment may include an image processing device (100), a network server (200), an external device (300), and an image analysis server (500). The following description will focus on the differences from the surveillance camera system (10) of FIG. 1.

[0123] The video analysis server (500) may be a device that receives and stores the video itself captured through the video processing device (100) and / or the video obtained by editing the video. The video analysis server (500) may perform the video and / or audio analysis function of the video processing device (100) of FIG. 1. For example, the video analysis server (500) may analyze the video received in real time from the video processing device (100) and / or the stored video to correspond to the intended use. The video analysis server (500) may analyze the video to generate metadata and index information for the metadata. The video analysis server (500) may analyze the video and / or the audio included in the video together or separately to generate metadata and index information for the metadata. The video analysis server (500) may be an AI server in which an artificial intelligence (AI) video analysis function is implemented.

[0124] The external device (300) may transmit an information provision request signal to the video analysis server (500) requesting the provision of all or part of the video and / or audio. The external device (300) may transmit an information provision request signal to the video analysis server (500) requesting metadata and / or index information for the metadata obtained by analyzing the video and / or audio.

[0125] FIGS. 6a and 6b are schematic diagrams illustrating a surveillance camera system according to one embodiment. FIG. 7 is a schematic diagram illustrating an AI server according to one embodiment.

[0126] Referring to FIG. 6a, a plurality of image processing devices (100) may include heterogeneous cameras with different functions. For example, the plurality of cameras may include at least one IP camera and at least one analog camera, and at least one IP camera may include an AI camera that includes a neural network processing unit (NPU). Hereinafter, for convenience of explanation, the AI ​​camera is referred to as the first camera (1001), the IP camera without AI function as the second camera (1003), and the analog camera as the third camera (1005).

[0127] The first camera (1001) and the second camera (1003) may have additional functions by including at least one third-party application of a specific function installed through an open platform. The first camera (1001) and the second camera (1003) can analyze video to detect / track / classify objects in real time and provide a real-time notification function when a set event occurs. The first camera (1001) can analyze video using AI in a deep learning manner and generate attribute information of the video.

[0128] The server (3001) can monitor and manage a plurality of image processing devices (100). The server (3001) can monitor images acquired by the image processing device (100) and search for stored images in response to a request for information provision from a client (3002). In one embodiment, the server (3001) may be a VMS (Video Management System).

[0129] The server (3001) may receive notification of an event (EV) when an event occurs from the first camera (1001) and the second camera (1003), and may receive images and / or metadata (MD). The images may be original images, and detection images (DetShot) or best images (BestShot). The server (3001) may search for objects and attributes based on the best images (BestShot) received from the first camera (1001). The server (3001) may search for objects based on detection images (DetShot) received from the second camera (1003). The best images (BestShot) may be an image selected from at least one detection image (DetShot) or an object image that is part of the selected image.

[0130] The server (3001) can receive video from the third camera (1005). The server (3001) can analyze the video received from the third camera (1005). The third camera (1005) may not perform video analysis functions.

[0131] Referring to FIG. 6b, the surveillance camera system may further include an AI server (5001) connected to a second camera (1002) and a third camera (1003). The AI ​​server (5001) receives images from the second camera (1003) and the third camera (1005) and can provide image analysis results at the same or similar level as the first camera (1001) using AI image analysis technology. The AI ​​server (5001) may be connected to a cloud server.

[0132] As illustrated in FIG. 7, the AI ​​server (5001) may include a decoder (5020) and at least one third-party application (5040). The AI ​​server (5001) may decode an encoded image (EIMG) received from at least one camera using the decoder (5020) and analyze the decoded image (DIMG) through an application (5040) that utilizes an artificial intelligence model. The at least one application (5040) may include a behavior analysis application, a fire / smoke detection application, etc.

[0133] FIG. 8 is a schematic diagram showing a cloud-based surveillance camera system according to one embodiment.

[0134] Referring to FIG. 8, in a cloud-based surveillance system (30), a camera and a client can transmit and receive data through the cloud. The cloud may include at least one cloud server including an application (App) and a database (DB). In one embodiment, the camera may be directly connected to the cloud via wired or wireless means by installing a cloud application, or it may be connected to the cloud via wired or wireless means through bridge equipment such as a gateway. The client can access the cloud via wired or wireless means to monitor stored video or receive notifications from the cloud regarding events resulting from video analysis.

[0135] FIG. 9 is a schematic diagram showing a monitoring system according to one embodiment.

[0136] Referring to FIG. 9, a monitoring system (40) according to one embodiment may include an event source (301) and a client (306).

[0137] An event source (301) is installed in a predetermined area and is equipped with a detection means, an information processing means, and a communication means, and can detect an event in the set area. In one embodiment, the event source (301) may be a surveillance camera, such as a visual camera, a thermal camera, or a special purpose camera, that includes an image sensor as the detection means. In addition to the image sensor, the event source (301) may further include a motion sensor, an audio sensor, an olfactory sensor, a temperature sensor, and a humidity sensor, an access control system, or a terminal as the detection means.

[0138] The event source (301) may include a storage device capable of inputting and outputting information, such as a hard disk drive, a solid state drive (SSD), a flash memory, a compact flash card (CF card), a secure digital card (SD card), a smart media card (SM card), a multi-media card (MMC card), or a memory stick. The event source (301) may further include a storage device such as a digital video recorder (DVR), a network video recorder (NVR), or a video management system (VMS). The storage device may receive and store detection data from a detection means. For example, the detection data may include video data, motion data, audio data, temperature data, access request data, etc.

[0139] The information processing means may be implemented with various numbers of hardware and / or software configurations that execute specific functions. In one embodiment, the information processing means may generate metadata (MD) based on detection data and generate events based on the metadata (MD). For example, the information processing means may analyze an image to recognize an object, extract metadata (MD) for the object, and store the event image containing the object combined with the metadata (MD). The information processing means may perform image analysis, object recognition and extraction, object classification and attribute determination, and metadata extraction using deep learning image analysis algorithms, etc. The metadata (MD) may include text-based metadata and image-based metadata such as blob images of motion regions and background models. Text-based metadata may be implemented in various forms that can be interpreted by the system. For example, the metadata (MD) may be implemented in a text format having rules such as XML or JSON, or in a binary form defined by a proprietary protocol.

[0140] Metadata (MD) may include information extracted from the detection data. For example, if the detection data is image data, the metadata (MD) may include information such as image identification information, the appearance / disappearance of objects (people, cars, etc.), object attributes (e.g., type, color, size, location, shape, movement, trajectory, etc.), object situation information (e.g., getting on or off), the occurrence of specific images (e.g., appearance of specific targets), and face detection. If the detection data is audio data, the metadata (MD) may include information such as the occurrence of abnormal sound sources (e.g., car tire friction sound (skid), glass breaking sound, alarm sound, collision sound, shouting, screaming, crying sound, etc.) and the occurrence of sounds exceeding a threshold.

[0141] An event source (301) is registered with at least one client (306), receives a request for information provision from the registered client (306), and can provide video and / or audio corresponding to the request for information provision to the client (306) based on metadata (MD). The event source (301) can provide metadata (MD) along with video (IMG) and / or audio to the client (306). For example, the event source (301) can notify the client (306) of an event and transmit event information when an event occurs according to a set event rule.

[0142] The client (306) can register at least one event source (301). The client (306) can receive event information from the registered event source (301).

[0143] FIG. 10 is a flowchart illustrating an event notification method according to one embodiment. An event notification method according to one embodiment of the present invention can be performed by a server (3001) shown in FIG. 1.

[0144] Referring to FIG. 10, in the event notification method, at step S110, the server (3001) can obtain video data from an image processing device.

[0145] In step S120, the server (3001) can generate a notification based on a preset first notification rule in response to an event of the video data.

[0146] A server (3001) according to one embodiment of the present invention can generate a notification based on the first notification rule that generates a notification in response to a motion detection event and display the generated notification on a user interface.

[0147] A server (3001) according to one embodiment of the present invention can generate a notification based on the first notification rule that generates a notification in response to a temperature detection event and display the generated notification on a user interface.

[0148] In step S130, the server (3001) can analyze the user's response pattern obtained in response to the notification. For example, the server (3001) according to one embodiment of the present invention can analyze whether the user plays the notification for a preset period of time in response to the notification.

[0149] In step S140, the server (3001) can generate a second notification rule that is distinct from the first notification rule based on the corresponding pattern.

[0150] A server (3001) according to one embodiment of the present invention can modify the first notification rule corresponding to the motion detection event to generate the second notification rule corresponding to the object-based motion detection event in which the motion detection unit has been changed.

[0151] A server (3001) according to one embodiment of the present invention can modify the first notification rule corresponding to the temperature detection event to generate the second notification rule in which the detection temperature range is changed.

[0152] A server (3001) according to one embodiment of the present invention can display the analysis result of the corresponding pattern and the second notification rule on a user interface.

[0153] FIGS. 11 and FIGS. 12 are drawings for explaining an event notification method according to one embodiment.

[0154] Referring to FIG. 11, an example of the result of analyzing whether a user plays a video in response to a notification generated according to an event notification rule is illustrated. For example, as shown in FIG. 11, a user may play video data in response to a notification by notification rule B and notification rule C, but may not play video data in response to a notification by notification rule A.

[0155] Referring to FIG. 12, an embodiment is illustrated in which a new notification rule is provided to a user based on the results of an analysis of the user's response pattern. For example, as illustrated in FIG. 12, a recommendation message (12) suggesting a new notification rule for notification rule A may be provided based on the user's response pattern during a preset accumulated time or during an accumulated number of notifications.

[0156] According to the present invention, notification rules optimized for the user can be generated and provided to the user through user pattern analysis.

[0157] For example, referring to FIG. 11, whether a user plays or confirms during an accumulated period of time or an accumulated number of notifications can be analyzed as a user response pattern. However, the present invention is not limited thereto, and various user response pattern analyses regarding video data or settings of an image processing device may be utilized.

[0158] According to one embodiment of the present invention, an optimized notification rule can be generated and provided to a user through the analysis of event information. For example, according to the present invention, an optimized notification rule can be generated and provided to a user when the event frequency is greater than a preset number of times.

[0159] According to the present invention, unnecessary event notifications to the user can be minimized, allowing the user to receive only the important event notifications intensively. For example, when an event occurs, a notification is set to be sent to a desktop client or a mobile notification. However, if a pattern is observed where the user does not play and check the video after receiving the notification, it is determined that the event does not require immediate verification or action, and it may be suggested to exclude the notification setting. In this case, even if the notification setting is excluded, the event log record can be stored.

[0160] In one embodiment of the present invention, when unwanted motion detection events, such as leaf swaying, frequently occur with an outdoor camera set to detect motion events, the event setting may be automatically changed from full-screen motion detection to object-based motion detection, or a suggestion message may be provided to the user to change the event setting to object-based motion detection. For example, the installation of an app or a version update for object-based motion detection may be suggested.

[0161] In one embodiment of the present invention, when a temperature detection event of a thermal camera occurs frequently, the temperature detection threshold setting may be automatically changed to above or below a specific temperature, or a suggestion message may be provided to the user to change the temperature detection threshold setting.

[0162] FIG. 13 is a diagram illustrating a software update method according to one embodiment.

[0163] Referring to FIG. 13, as illustrated in FIG. 13(a), the recording function was interrupted during software updates such as conventional S / W Update 1 (e.g., new device linkage), S / W Update 2 (e.g., bug fix), and S / W Update 3 (e.g., addition of new functions). However, according to the present invention, as illustrated in FIG. 13(b), it is possible to preserve recording data that was lost during updates by updating by component.

[0164] According to the present invention, the software of a Video Management System (VMS) is divided into a plurality of components according to function, and software updates can be performed for each component. In this case, recording is not interrupted when updating components (e.g., components related to user permission management, event log management, etc.) excluding components related to recording functions (e.g., components related to recording, transmitting, or storing data), thereby enabling the preservation of recorded data.

[0165] For example, a VMS may include multiple components listed in Table 1 and Table 2 below. For instance, components that must operate to maintain the recording function may be classified as shown in Table 2 below. In this case, if the components in Table 2 stop operating, recording cannot continue; conversely, even if the components in Table 1 stop operating, recording can continue as long as only the components in Table 2 continue operating.

[0166] Component Name Description appliance_application Management of resources specialized for embedded devices (firewall, POE, etc.) (*Exclusive to appliance products) assignment_application Support module for permissions within the VMS system audit_log_application Management of records regarding user operations / usage auth_application Management of user permissions backchannel_application Responsible for transmitting data to cameras backup_application Responsible for saving / loading VMS configuration information cloud_application Cloud function module cluster_application Clustering function module customer_support_application Customer support service related module device_application Management of external devices such as cameras managed by VMS event_application Management of events generated from devices / VMS, etc. gateway_application Management of external VMS communication connections group_application Management of user permissions http_service_application Management of external VMS communication commands log_application Management of logs for events occurring during VMS operation network_application Management of network resources (*Exclusive to appliance products) product_application Management of embedded product information (*Exclusive to appliance products Dedicated) proxy_application: Responsible for proxy (inter-server forwarding) functions resource_application: Manages data resources managed internally by the VMS (maps, schedules, layout information, etc.) server_application: Manages server-level functions and resources within the VMS system system_application: Manages system-level functions and resources update_application: Software upgrade function module

[0167] Component Name Description database_application Responsible for recording, editing, and deleting VMS internal database functions media_application Core functions such as transmitting and storing multimedia data including video and audio storage_application VMS storage media management

[0168] FIGS. 14 and FIGS. 15 are drawings for illustrating an event notification method according to another embodiment.

[0169] Referring to FIGS. 14 and 15, the direction of the audio can be displayed on a map based on an audio detection event. For example, audio can be detected using an image processing device equipped with an audio receiving device.

[0170] According to the present invention, it is possible to detect specific audio events and generate alarms, thereby enabling rapid incident identification and response. For example, in areas where privacy protection is critical, situation monitoring may be possible solely through audio detection without video monitoring. In this case, specific audio clips of sound events can be saved individually without recording the entire audio stream.

[0171] For example, for audio detection events, audio classification types including footsteps, gunshots, screams, glass breaking, tire slipping, car horns, aggression / shouting, loud noise, etc., can be preset as event triggers.

[0172] For example, when an audio detection event occurs, the server (3001) may generate and store metadata for the detected audio. For example, the metadata may include the audio classification type, event time, audio download link, confidence score, volume (dB), sound direction, etc.

[0173] For example, as illustrated in FIG. 14, when an audio detection event occurs, attributes (e.g., event name, event time, classification type, reliability, volume, sound direction, etc.) for the audio event detected by each audio detection device (e.g., image processing device, etc.) can be displayed on a map basis.

[0174] For example, when an audio detection event is detected, the server (3001) can store an audio clip that can be played through live event monitoring and event logs.

[0175] For example, as illustrated in FIG. 15, audio information including the audio direction and classification type information of the audio and the image processing device where the audio detection event occurred can be displayed on a map showing the locations where a plurality of image processing devices equipped with an audio detection function are installed. For example, if an audio event classified as the sound of glass breaking is detected by the first image processing device (16) and the second image processing device (17), the direction of the sound and the audio classification type can be displayed on the map. Additionally, if an audio event classified as footsteps is detected by the third image processing device (15), the direction of the sound and the audio classification type can be displayed on the map. In this case, the user can check the preview image (19) of the fourth image processing device (18) based on the audio detection event information of the first image processing device (16), the second image processing device (17), and the third image processing device (15) displayed on the map. In this case, when an audio detection event occurs, rapid situation verification is possible based on the map.

[0176] One or more of the surveillance cameras, autonomous vehicles, user terminals, and servers may be linked with artificial intelligence modules, robots, augmented reality (AR) devices, virtual reality (VT) devices, devices related to 5G services, etc.

[0177] For the purpose of understanding the present invention, reference numerals have been used in the preferred embodiments illustrated in the drawings, and specific terms have been used to describe the embodiments; however, the present invention is not limited by said specific terms, and the present invention may include all components that are conventionally conceivable by those skilled in the art.

[0178] The present invention may be represented by functional block configurations and various processing steps. These functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, the present invention may employ direct circuit configurations such as memory, processing, logic, look-up tables, etc., which can execute various functions by the control of one or more microprocessors or other control devices. Similar to how the components of the present invention may be implemented as software programming or software elements, the present invention may be implemented in programming or scripting languages ​​such as C, C++, Java, assembler, etc., including various algorithms implemented as combinations of data structures, processes, routines, or other programming configurations. Functional aspects may be implemented as algorithms executed on one or more processors. Additionally, the present invention may employ prior art for electronic configuration, signal processing, and / or data processing, etc. Terms such as "mechanism," "element," "means," and "configuration" may be used broadly and are not limited to mechanical and physical configurations. The above terms may include the meaning of a series of software processes (routines) in conjunction with processors, etc.

[0179] The specific embodiments described in this invention are examples and do not limit the scope of the invention in any way. For the sake of brevity of the specification, descriptions of prior electronic configurations, control systems, software, and other functional aspects of said systems may be omitted. Additionally, the connections of lines or connecting members between components shown in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be replaced or additionally represented as various functional connections, physical connections, or circuit connections in actual devices. Furthermore, unless specifically stated as "essential," "importantly," etc., a component may not be strictly necessary for the application of the invention.

[0180] In the specification of the present invention (particularly in the claims), the use of the term "above" and similar descriptive terms may be in both singular and plural. Furthermore, where a range is described in the present invention, it is implied to include an invention applying individual values ​​belonging to said range (unless otherwise stated), and is equivalent to describing each individual value constituting said range in the detailed description of the invention. Finally, regarding the steps constituting the method according to the present invention, unless explicitly stated or otherwise stated, said steps may be performed in a suitable order. The present invention is not necessarily limited by the order in which said steps are described.

Claims

1. In a surveillance camera system, An image processing device for acquiring video data; and A server that acquires video data from an image processing device, generates a notification based on a preset first notification rule in response to an event of the video data, analyzes a user's response pattern or event information acquired in response to the notification, and generates a second notification rule distinguished from the first notification rule based on the response pattern or event information; A surveillance camera system including 2. In Paragraph 1, The above server is a surveillance camera system that displays the analysis result of the above response pattern and the above second notification rule on a user interface.

3. In Paragraph 1, A surveillance camera system in which the above server generates a notification based on the first notification rule that generates a notification in response to a motion detection event, and displays the generated notification on a user interface.

4. In Paragraph 3, A surveillance camera system in which the server modifies the first notification rule corresponding to the motion detection event to generate the second notification rule corresponding to the object-based motion detection event in which the motion detection unit has been changed.

5. In Paragraph 1, A surveillance camera system in which the above server generates a notification based on the first notification rule that generates a notification in response to a temperature detection event, and displays the generated notification on a user interface.

6. In Paragraph 5, The above server is a surveillance camera system that modifies the first notification rule corresponding to the temperature detection event to generate the second notification rule with a changed detection temperature range.

7. In Paragraph 1, The above server is a surveillance camera system that analyzes whether a user plays the content obtained during a preset time in response to the above notification, and analyzes the event information including the event frequency.

8. In the event notification method of a surveillance camera system, A step of acquiring video data from an image processing device; A step of generating a notification based on a preset first notification rule in response to an event of the above video data; A step of analyzing user response patterns or event information obtained in response to the above notification; and A step of generating a second notification rule distinct from the first notification rule based on the above corresponding pattern or the above event information; An event notification method including 9. In Paragraph 8, An event notification method further comprising the step of displaying the analysis result of the above-mentioned corresponding pattern and the above-mentioned second notification rule on a user interface.

10. In Paragraph 8, An event notification method comprising the step of generating the above notification, which generates the notification based on the first notification rule that generates the notification in response to a motion detection event, and the step of displaying the generated notification on a user interface.

11. In Paragraph 10, An event notification method comprising the step of generating the second notification rule, wherein the first notification rule corresponding to the motion detection event is modified to generate the second notification rule corresponding to the object-based motion detection event in which the motion detection unit has been changed.

12. In Paragraph 8, An event notification method comprising the step of generating the above notification, which generates the notification based on the first notification rule that generates the notification in response to a temperature detection event, and the step of displaying the generated notification on a user interface.

13. In Paragraph 12, An event notification method comprising the step of generating the second notification rule, wherein the step of generating the second notification rule in which the detection temperature range is changed is modified by modifying the first notification rule corresponding to the temperature detection event.

14. In Paragraph 8, The step of analyzing the above-mentioned corresponding pattern or event information is, A step of analyzing whether the user plays the content obtained during a preset time in response to the above notification; and An event notification method comprising the step of analyzing the above event information including the event frequency.

Images