Wireless monitoring device, method for recording image of wireless monitoring device

Abstract

The present invention relates to a method for recording video of a wireless surveillance device, comprising: a step of acquiring video captured through a camera; a step of performing recording of the video in a normal mode; a step of determining whether a first set time has elapsed in the normal mode; a step of determining whether an event has been detected from the video after the first set time has elapsed; and a step of adjusting the recording speed of the video when it is determined that the event has not been detected after the first set time has elapsed.

Description

Technology Field

[0001] The present invention relates to wireless surveillance technology, and in particular to wireless surveillance technology suitable for recording images of a remote location using an artificial neural network. Background Technology

[0003] Recently, it has become common practice to install surveillance camera systems inside or outside buildings and on streets for various purposes, such as crime prevention, security, and store management. Such surveillance camera systems can perform the functions of network cameras by connecting multiple cameras to each other via a wired or wireless network.

[0004] Recent surveillance camera systems utilize network cameras that perform intelligent video analysis to wirelessly or via wired transmission of various video analysis information acquired from the surveillance area, along with the video and audio signals of the area. For example, there is an increasing demand for intelligent surveillance services in recent surveillance camera systems that go beyond simply storing video of the surveillance area to analyzing surveillance footage to recognize abnormal situations. Conventionally, representative examples of such intelligent surveillance services involve methods that detect and track objects recognized within the surveillance area and analyze abnormal behavior.

[0005] However, existing surveillance camera systems have the disadvantage of recording only when the movement of an object (person or animal) is detected. Therefore, more efficient and systematic video surveillance technology is required.

[0006] The aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot be considered as publicly known technology disclosed to the general public prior to the filing of the present invention. Prior art literature

[0008] Published Patent Application No. 10-2021-0094862 (July 30, 2021) The problem to be solved

[0009] In an embodiment of the present invention, we propose a wireless surveillance technology that can increase recording efficiency by performing recording in normal mode under normal conditions and adjusting the recording speed when no movement is detected for a certain period of time.

[0010] The problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned problems to be solved will be clearly understood by those skilled in the art to which the present invention pertains from the descriptions below. means of solving the problem

[0012] According to an embodiment of the present invention, a method for recording video of a wireless surveillance device may be provided, comprising: a step of acquiring a video captured through a camera; a step of performing recording in normal mode for the video; a step of determining whether a first set time has elapsed in the normal mode; a step of determining whether an event has been detected from the video after the first set time has elapsed; and a step of adjusting the recording speed of the video when it is determined that the event has not been detected after the first set time has elapsed.

[0013] Here, the adjusting step may include the step of recording the video in a low-power mode by performing fast-forward recording.

[0014] Additionally, the low-power mode is performed until a request to stop the monitoring function is made by the user, and the method may further include a step of feeding back to a step of determining whether the event has been detected if there is no request to stop the monitoring function.

[0015] Additionally, the above method may further include a step of providing feedback to the step of performing recording in normal mode when it is determined that the first setting time has not elapsed or that the event has been detected.

[0016] In addition, the step of determining whether an event is detected from the image may include the step of outputting an event detection result for the image using a pre-trained artificial neural network.

[0017] In addition, the artificial neural network may include a pre-trained artificial neural network that takes training image data as input, uses a preset image change rate and a preset voice change rate as label data, and outputs the event detection result.

[0018] According to an embodiment of the present invention, a wireless monitoring device may be provided comprising: an acquisition unit for acquiring an image captured through a camera; and a processing unit for performing recording in normal mode for the image and storing the recorded image in a storage unit; wherein the processing unit, when a first set time elapses in the normal mode and the event is not detected from the image, adjusts the recording speed of the image to perform recording of the image in a low-power mode and outputs an event detection result for the image using a pre-trained artificial neural network.

[0019] Here, the wireless monitoring device may further include an input unit including a wireless microphone; and an output unit including a lighting unit with varying illumination levels according to ambient brightness and an alarm unit having a siren function.

[0020] Additionally, the low-power mode may include at least one mode among a mode that reduces the frame rate of the image, a mode that lowers the resolution of the image, and a mode that compresses the image.

[0021] According to an embodiment of the present invention, a computer-readable recording medium storing a computer program may be provided, wherein the computer program includes instructions for a processor to perform a video recording method of a wireless surveillance device, and the method comprises: a step of acquiring a video captured through a camera; a step of performing recording in normal mode for the video; a step of determining whether a first set time has elapsed in the normal mode; a step of determining whether an event has been detected from the video after the first set time has elapsed; and a step of adjusting the recording speed of the video when it is determined that the event has not been detected after the first set time has elapsed.

[0022] According to an embodiment of the present invention, a computer program stored on a computer-readable recording medium may be provided, wherein the computer program includes instructions for a processor to perform a video recording method of a wireless surveillance device, and the method comprises: a step of acquiring a video captured through a camera; a step of performing recording in normal mode for the video; a step of determining whether a first set time has elapsed in the normal mode; a step of determining whether an event has been detected from the video after the first set time has elapsed; and a step of adjusting the recording speed of the video when it is determined that the event has not been detected after the first set time has elapsed. Effects of the invention

[0024] According to an embodiment of the present invention, recording is performed in normal mode under normal conditions, and when no movement is detected for a certain period of time, the recording speed of the video is adjusted, for example, by fast-forwarding recording, recording with reduced frame rate, recording with reduced resolution, or recording with compressed video, thereby improving recording efficiency such as optimizing the storage space of the surveillance system and reducing power consumption. Brief explanation of the drawing

[0026] FIG. 1 is a block diagram illustrating the function of a wireless monitoring device according to an embodiment of the present invention. Figure 2 is a block diagram illustrating the detailed functions of the processing unit in the wireless monitoring device of Figure 1. FIG. 3 is a flowchart illustrating, in an exemplary manner, a video recording method of a wireless surveillance device according to an embodiment of the present invention. FIG. 4 is a diagram specifically illustrating the event detection step of FIG. 3, and is a block diagram for exemplarily illustrating the process of training an artificial neural network within a storage unit to output the event detection result of the wireless monitoring device of FIG. 1. FIG. 5 is a diagram specifically illustrating the event detection step of FIG. 3, and is a block diagram for exemplarily illustrating the process of outputting an event detection result using an artificial neural network that has been trained in FIG. 4. Specific details for implementing the invention

[0027] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the scope of the present invention is defined only by the claims.

[0028] In describing the embodiments of the present invention, specific descriptions of known functions or configurations will be omitted unless actually necessary for describing the embodiments of the present invention. Furthermore, the terms described below are defined in consideration of the functions in the embodiments of the present invention, and these may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification.

[0029] Existing surveillance camera systems have the disadvantage of recording only when the movement of an object (person or animal) is detected.

[0030] Accordingly, in an embodiment of the present invention, we propose a wireless surveillance technology that can increase recording efficiency by performing recording in normal mode under normal circumstances and adjusting the recording speed when no movement is detected for a certain period of time.

[0031] Specifically, in an embodiment of the present invention, a video recording method for a remote wireless surveillance device equipped with an AI function may include a process of performing recording in a normal mode for 24 hours, performing fast-forward recording (low-power mode) when there is no event (movement of an object) through a video analysis function, and then performing recording in a normal mode from that time until there is no event when an event is detected.

[0032] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0033] FIG. 1 is a block diagram for explaining the function of a wireless monitoring device (100) according to an embodiment of the present invention.

[0034] The wireless surveillance device (100) is an electronic device for recording video in a wireless surveillance environment, and in an embodiment of the present invention, the electronic device may include, for example, a network video recording device, a mobile device, etc., and is not limited to a specific device.

[0035] As illustrated in FIG. 1, the wireless monitoring device (100) may include an acquisition unit (110), a processing unit (120), a storage unit (130), an input unit (140), and an output unit (150).

[0036] The acquisition unit (110) can acquire surrounding images captured through a camera (not shown). The acquisition unit (110) may include communication equipment, network equipment, etc., connected to the camera wirelessly or via a wire. The communication equipment or network equipment is equipment that connects the camera and the acquisition unit (110). Such equipment may be built based on short-range communication such as Bluetooth, Zigbee, Wi-Fi, or UWB (ultra-wide band), broadband communication such as the Internet or mobile communication networks, and is not limited to a specific communication or network.

[0037] The processing unit (120) can perform normal mode recording on the image acquired through the acquisition unit (110) and store the recorded image in the storage unit (130).

[0038] At this time, the processing unit (120) can perform recording of the video in low-power mode by adjusting the recording speed of the video when the first set time elapses in normal mode and no event is detected from the video. Here, the processing unit (120) can determine whether an event is detected by outputting an event detection result for the video using an artificial neural network that has been trained in the storage unit (130). The processing unit (120) can process to acquire a captured video through the acquisition unit (110) by executing a command in the storage unit (130), and can process to adjust the recording speed based on the event detection result output through the artificial neural network in the storage unit (130). At this time, the event detection result may include the captured video acquired through the acquisition unit (110), and the event detection result including such captured video may be transmitted to a management server via a network or used as training data when retraining the artificial neural network. This processing unit (120) may include, for example, a microprocessor-based processing device, and the specific operation and function description of the processing unit (120) will be described in detail in FIG. 3.

[0039] The storage unit (130) may include commands for outputting event detection results from a captured image (real-time image) through a pre-trained artificial neural network. To this end, the artificial neural network may be pre-trained to output event detection results by taking training image data as input and using a pre-set image change rate and a pre-set voice change rate as label data. Any commands within the storage unit (130) may be stored in the form of an application, a program, etc., and any stored commands may be selected and executed by the processing unit (130). Such a storage unit (130) may include a recording medium such as memory (RAM, read-only memory), a local disk connected via a network, or storage, for example, and there is no need to be limited to a specific recording medium in implementing an embodiment of the present invention.

[0040] The input unit (140) may provide a UI environment for inputting a user's selection signal. This input unit (140) may include input means such as, for example, a key pad or a touch pad, and may also be included together with the output unit (150) described later. Additionally, the input unit (140) may further include a wireless microphone for recording ambient sound in addition to video recording in the wireless surveillance device (100).

[0041] The output unit (150) may provide a UI environment for outputting a menu screen corresponding to a user selection signal input through the input unit (140), a captured image acquired through the acquisition unit (110), etc. Such an output unit (150) may include display means such as, for example, an LCD (liquid crystal display) or an OLED (organic light emitting diodes), and is not limited to a specific display means. In addition, the output unit (150) may be equipped with an AI lighting function that changes illumination according to ambient brightness, an alarm function, etc.

[0042] FIG. 2 is a block diagram for explaining the detailed functions of the processing unit (120) in the wireless monitoring device (100) of FIG. 1.

[0043] As illustrated in FIG. 2, the processing unit (120) may include an event detection unit (122) and a speed adjustment unit (124).

[0044] The event detection unit (122) can detect an event by determining whether an object (person or animal) is moving. For example, the event detection unit (122) can detect that an event has occurred when the rate of change in the image due to the movement of the object exceeds a threshold. Alternatively, the event detection unit (122) can detect that an event has occurred when the rate of change in the voice due to ambient noise exceeds a threshold.

[0045] The speed adjustment unit (124) can adjust the recording speed of the video currently being recorded based on the event detection result of the event detection unit (122).

[0046] In an embodiment of the present invention, the video recording speed can be adjusted in the following manner.

[0047] 1) Adjust Frame Rate (FPS): You can change the recording speed by decreasing or increasing the number of frames saved per second. A lower FPS saves storage space but may make movement appear less smooth.

[0048] 2) Change resolution: Lowering the resolution reduces the amount of data stored, which indirectly provides a speed control effect.

[0049] 3) Recording mode setting: You can set a continuous recording mode that records every moment, or an event-based recording mode that records only when motion is detected.

[0050] 4) Adjust compression settings: Data processing speed and storage capacity can be optimized by changing the compression method (H.264, H.265, etc.).

[0051] Hereinafter, together with the configuration described above, a video recording method of a wireless surveillance device (100) according to an embodiment of the present invention will be explained in more detail with reference to the flowchart of FIG. 3 attached.

[0052] As illustrated in FIG. 3, the wireless monitoring device (100) acquires an image captured through a camera (S100) and can perform recording in normal mode on the acquired image (S102).

[0053] In the process of performing recording in such normal mode, the wireless monitoring device (100) can determine whether a first set time, for example, 3 to 5 seconds, has elapsed (S104). If the first set time has not elapsed, the wireless monitoring device (100) can continue to perform recording in normal mode (S102).

[0054] Afterward, the wireless monitoring device (100) can determine whether an event has been detected from the acquired image after the first set time has elapsed (S106).

[0055] If, as a result of the judgment in step (S106), an event is detected, the wireless monitoring device (100) feeds back to step (S102) to perform recording in normal mode, but if it is determined that the first set time has elapsed and no event is detected, the recording speed of the video can be adjusted (S108).

[0056] Here, the adjusting step (S108) may include the step of performing fast-forward recording to record video in low-power mode (S110).

[0057] These low-power modes may include, for example, a mode with reduced frame rate, a mode with reduced resolution, or a mode with applied compression. For the mode with reduced frame rate, for example, if the normal mode is 120 FPS, a mode adjusted to 60 FPS may be applied in the first adjustment step, a mode adjusted to 30 FPS may be applied in the second adjustment step, and a mode adjusted to 24 FPS may be applied in the third adjustment step.

[0058] This low-power mode is performed until a request to stop the monitoring function is made by the user, and if there is no request to stop the monitoring function, it may further include a step of feeding back to a step (S106) for determining whether an event has been detected (S112).

[0059] FIG. 4 is a diagram specifically illustrating the event detection step (S106) of FIG. 3, and is a block diagram for exemplarily illustrating the process of training an artificial neural network (132) in a storage unit (130) to output the event detection result of the wireless monitoring device (100) of FIG. 1.

[0060] As illustrated in FIG. 4, the artificial neural network (132) may include an artificial neural network that has been trained to take training image data (X) as input, and to output an event detection result (y) by using a preset image change rate and a preset voice change rate as label data (y).

[0061] FIG. 5 is a diagram specifically illustrating the event detection step (S106) of FIG. 3, and is a block diagram for exemplarily illustrating the process of outputting an event detection result using the artificial neural network (132) that has been trained in FIG. 4.

[0062] As illustrated in FIG. 5, when a real-time image (X), that is, a captured image acquired through the acquisition unit (110), is input to the artificial neural network (132), the artificial neural network (132) detects an event based on previously learned data ( Can output ).

[0063] Meanwhile, the artificial neural network (132) in this specification may refer to any form of computer program that operates based on a network function, a neural network, etc. Throughout this specification, the terms model, neural network, network function, and neural network may be used interchangeably. A neural network is formed in which one or more nodes are interconnected through one or more links to form an input node and output node relationship within the neural network. The characteristics of the neural network may be determined by the number of nodes and links within the neural network, the relationships between the nodes and links, and the values ​​of the weights assigned to each of the links. A neural network may be composed of a set of one or more nodes. A subset of nodes constituting the neural network may form a layer.

[0064] A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to an input layer and an output layer. Deep neural networks may include convolutional neural networks (CNN), recurrent neural networks (RNN), autoencoders, Generative Adversarial Networks (GAN), restricted Boltzmann machines (RBM), deep belief networks (DBN), Q networks, U networks, Siamese networks, Generative Adversarial Networks (GAN), transformers, etc. The description of deep neural networks described above is merely illustrative and the present disclosure is not limited thereto.

[0065] A neural network can be trained in at least one of supervised learning, unsupervised learning, semi-supervised learning, self-supervised learning, or reinforcement learning. The training of a neural network may be a process of applying knowledge to the neural network to perform a specific operation. However, in the embodiment of the present invention, since the conditions for determining whether an event is detected can be specified to some extent, it would be preferable to apply supervised learning using a label input method to the artificial neural network (132).

[0066] Neural networks can be trained to minimize output errors. The training process involves repeatedly inputting training data into the network, calculating the error between the network's output and the target for the training data, and updating the weights of each node by backpropagating the error from the output layer to the input layer in a direction that reduces the error. In supervised learning, data with correct answers labeled is used for each training point, whereas in unsupervised learning, data without correct answers can be used. The amount of change in the connection weights of each updated node can be determined by the learning rate. The neural network's calculation of the input data and the backpropagation of the error can constitute a training cycle (epoch). The learning rate can be applied differently depending on the number of iterations of the neural network's training cycle. In addition, to prevent overfitting, methods such as increasing training data, regularization, dropout (which disables some nodes), and batch normalization layers can be applied.

[0067] In one embodiment, the model may borrow at least a part of a transformer. The transformer may be composed of an encoder that encodes embedded data and a decoder that decodes the encoded data. The transformer may have a structure that receives a series of data and outputs a series of data of different types after undergoing encoding and decoding steps. In one embodiment, the series of data may be processed into a form that the transformer can compute. The process of processing the series of data into a form that the transformer can compute may include an embedding process. Expressions such as data token, embedding vector, embedding token, etc., may refer to data embedded in a form that the transformer can process.

[0068] To encode and decode a series of data, the encoders and decoders within the transformer can be processed using an attention algorithm. An attention algorithm can refer to an algorithm that calculates the similarity between one or more keys for a given query, applies this similarity to the values ​​corresponding to each key, and then calculates an attention value by performing a weighted sum of the similarity-applied values.

[0069] Various types of attention algorithms can be classified depending on how the query, key, and value are configured. For example, if attention is calculated by setting the query, key, and value identically, this can be referred to as a self-attention algorithm. If attention is calculated by reducing the dimensionality of embedding vectors to process a series of input data in parallel and determining an individual attention head for each partitioned embedding vector, this can be referred to as a multi-head attention algorithm.

[0070] In one embodiment, the transformer may be composed of modules that perform a plurality of multi-head self-attention algorithms or multi-head encoder-decoder algorithms. In one embodiment, the transformer may also include additional components other than attention algorithms, such as embeddings, normalization, and softmax. A method for constructing the transformer using an attention algorithm may include the method disclosed in Vaswani et al., Attention Is All You Need, 2017 NIPS, which is incorporated herein by reference.

[0071] A transformer can be applied to various data domains, such as embedded natural language, segmented image data, and audio waveforms, to convert a series of input data into a series of output data. To convert data with various data domains into a series of data that can be input to the transformer, the transformer can embed the data. The transformer can process additional data that represents the relative positional or phase relationships between the series of input data. Alternatively, the series of input data may be embedded by additionally reflecting vectors that represent the relative positional or phase relationships between the input data. In one example, the relative positional relationships between the series of input data may include, but are not limited to, word order within a natural language sentence, the relative positional relationships of each segmented image, and the temporal order of segmented audio waveforms. The process of adding information that represents the relative positional or phase relationships between the series of input data may be referred to as positional encoding.

[0072] In one embodiment, the model may include, but is not limited to, at least one of a Recurrent Neural Network (RNN), a Long Short Term Memory (LSTM) network, a Deep Neural Network (DNN), a Convolutional Neural Network (CNN), and a Bidirectional Recurrent Deep Neural Network (BRDNN).

[0073] In one embodiment, the model may be a model trained using a transfer learning method. Here, transfer learning refers to a learning method in which a pre-trained model having a first task is obtained by pre-training a large amount of unlabeled training data using a semi-supervised or self-learning method, and the pre-trained model is fine-tuned to be suitable for a second task and then trained on labeled training data using a supervised learning method to implement a target model.

[0074] According to the embodiment of the present invention described above, recording is performed in normal mode under normal conditions, and when no movement is detected for a certain period of time, the recording speed of the video is adjusted, such as fast-forward recording, recording with reduced frame rate, recording with reduced resolution, or recording with compressed video. This is expected to improve recording efficiency, such as optimizing the storage space of the surveillance system and reducing power consumption.

[0075] Meanwhile, combinations of each block of the attached block diagram and each step of the flowchart may be executed by computer program instructions. Since these computer program instructions can be loaded into the processor of a general-purpose computer, a specialized computer, or other programmable data processing equipment, the instructions executed through the processor of the computer or other programmable data processing equipment create a means to perform the functions described in each block of the block diagram.

[0076] Since these computer program instructions may be stored in a computer-available or computer-readable recording medium (or memory), etc., which can be directed toward a computer or other programmable data processing equipment to implement a function in a specific way, the instructions stored in the computer-available or computer-readable recording medium (or memory) may also be used to produce a manufactured item containing instruction means that perform the function described in each block of the block diagram.

[0077] And, since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that perform a series of operation steps on a computer or other programmable data processing equipment to create a process executed by a computer and perform the computer or other programmable data processing equipment may also provide steps for executing the functions described in each block of the block diagram.

[0078] Additionally, each block may represent a module, segment, or part of code containing at least one executable instruction for executing a specified logical function(s). Also, it should be noted that in some alternative embodiments, the functions mentioned in the blocks may occur out of order. For example, two blocks shown in succession may actually be executed substantially simultaneously, or the blocks may be executed in reverse order according to the corresponding function. Explanation of the symbols

[0080] 100: Wireless monitoring device 110: Acquisition Department 120: Processing unit 122: Event detection unit 124: Speed ​​control section 130: Storage section 132: Artificial Neural Networks 140: Input section 150: Output section

Claims

Claim 1 A method for recording video of a wireless surveillance device, comprising: a step of acquiring video captured through a camera; a step of performing recording of the video in a normal mode; a step of determining whether a first set time has elapsed in the normal mode; a step of determining whether an event has been detected from the video after the first set time has elapsed; a step of adjusting the recording speed of the video when it is determined that the event has not been detected after the first set time has elapsed; and a step of outputting an event detection result for the video using a pre-trained artificial neural network for the step of determining whether an event has been detected from the video, wherein the adjusting step includes a step of performing fast-forward recording to record the video in a low-power mode, wherein the low-power mode is performed until a request to stop the surveillance function is made by a user, and further comprises a step of feeding back to the step of determining whether an event has been detected when there is no request to stop the surveillance function. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 delete

Images