Safety management device, method, and program based on de-identified IoT sensor

Abstract

The present invention relates to a safety management device, method, and program based on a de-identified IoT sensor, which may detect a surrounding environment on the basis of the de-identified IoT sensor to provide a notification for predicting and preventing the occurrence of an accident in real time. The present invention may: preprocess de-identified data and sound data when the de-identified data and the sound data acquired by imaging a surrounding environment are input; input the preprocessed data to a pre-trained neural network model to determine an abnormal behavior of an object on the basis of the movement and the sound of the object included in the preprocessed data; predict the occurrence of an accident related to the abnormal behavior of the object; and generate a notification corresponding to the abnormal behavior of the object to transmit the notification to an external device when the occurrence of the accident is expected.

Description

Non-identifiable IoT sensor-based safety management device, method, and program

[0001] The present invention relates to a safety management device for accident prevention, and more specifically, to a non-identifiable IoT sensor-based safety management device, method, and program capable of detecting the surrounding environment based on a non-identifiable IoT sensor and providing real-time notifications for predicting and preventing accidents.

[0002] Generally, surveillance cameras are used for various purposes, including industrial, educational, medical, traffic control, and pest control.

[0003] The effectiveness of surveillance cameras is being empirically proven, including their crime prevention and deterrence effects, ease of detection and arrest of criminals, reduction of citizens' fear of crime, and the supplementation of limited police resources through installation in key areas.

[0004] However, existing surveillance systems, including surveillance cameras, are installed in public places and can identify the movements or objective behaviors of objects included in video data; yet, they have problems in that they cannot predict or prevent accidents because they cannot determine whether the objects within the video data are exhibiting abnormal behavior.

[0005] In addition, existing surveillance systems had problems where personal information about objects included in video data was exposed, leading to privacy violations.

[0006] Therefore, there is a need for the development of a safety management device that can accurately and rapidly analyze abnormal object behavior based on non-identifiable data to predict and prevent accidents in advance by providing notifications.

[0007] The objective of the present invention, which aims to solve the problems described above, is to provide a safety management device, method, and program capable of protecting personal information using distance and temperature-based non-identification data, accurately determining abnormal behavior by precisely recognizing the movement of an object, and rapidly providing notifications regarding abnormal behavior by determining abnormal behavior of an object based on distance- and temperature-based non-identification data and sound data regarding the surrounding environment through a pre-trained neural network model.

[0008] The problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below.

[0009] A safety management device according to an embodiment of the present invention for solving the above-mentioned problem comprises a distance non-identification sensor module that captures a surrounding environment to acquire distance non-identification data, a sound sensor module that senses surrounding sound to acquire sound data, a communication module that communicates with an external device, and a processor including one or more cores. The processor is characterized by preprocessing the distance non-identification data and sound data when the distance non-identification data and sound data are input, inputting the preprocessed data into a pre-trained neural network model to determine abnormal behavior of the object based on the movement and sound of the object included in the preprocessed data, predicting the occurrence of an accident regarding the abnormal behavior of the object, and controlling the communication module to generate an alert corresponding to the abnormal behavior of the object and transmit it to the external device when the occurrence of the accident is predicted.

[0010]

[0011] In an embodiment, the non-identifiable data includes distance data with respect to the object located within the surrounding environment, and the processor, when preprocessing the distance data, matches a color corresponding to the depth value of the distance data based on a color map when the distance data is input, and applies the matched color to the distance data to preprocess the distance data into color distance data.

[0012]

[0013] In an embodiment, the processor is characterized by performing preprocessing when preprocessing the distance data, wherein when the distance data is input, it identifies the presence or absence of an object from the distance data, and when the object is identified, it separates only the object region located at the object from the distance data to extract an object segment.

[0014]

[0015] In an embodiment, the processor is characterized by performing preprocessing to remove noise frequencies by analyzing the frequency corresponding to the sound data when the sound data is input, when the sound data is preprocessed.

[0016]

[0017] In an embodiment, when determining abnormal behavior of the object, the processor extracts movement features of the object from the preprocessed data, calculates a similarity score by comparing the extracted movement features with preset abnormal behavior features, and determines the movement features as abnormal behavior of the object if the calculated similarity score is greater than or equal to a reference score.

[0018]

[0019] In an embodiment, the processor is characterized by, when calculating the similarity score, extracting sound features of the object from the preprocessed data, analyzing the association between the extracted sound features and the preset abnormal behavior features to calculate an association score, and adding the calculated association score as a weight to the calculated similarity score to calculate a final similarity score.

[0020]

[0021] In an embodiment, when the processor predicts the occurrence of an accident regarding the abnormal behavior of the object, it classifies a pattern type for the abnormal behavior of the object upon determining the abnormal behavior of the object, calculates an accident occurrence probability value corresponding to the classified pattern type, and predicts the occurrence of an accident regarding the abnormal behavior of the object based on the calculated accident occurrence probability value.

[0022]

[0023] In an embodiment, the processor groups the abnormal behavior type of the object and the predicted accident occurrence type according to the location where the distance non-identification sensor module and the sound sensor module are deployed, generates and stores safety management statistical information based on the accident occurrence data for each grouped location, and when a request for a safety management information provision service for a specific location is received from the external device, transmits a safety management information provision service for the specific location to the external device based on the previously stored safety management statistical information.

[0024]

[0025] A safety management method according to one embodiment of the present invention is a safety management method of a safety management device comprising a distance non-identification sensor module that captures a surrounding environment to acquire non-identification data and a sound sensor module that acquires sound data, the method comprising: receiving the distance non-identification data and sound data; preprocessing the distance non-identification data and sound data; inputting the preprocessed data into a pre-trained neural network model to determine abnormal behavior of the object based on the movement and sound of the object included in the preprocessed data; predicting the occurrence of an accident regarding the abnormal behavior of the object; and, if the occurrence of an accident is predicted, generating an alert corresponding to the abnormal behavior of the object and transmitting it to an external device.

[0026]

[0027] A computer program stored on a computer-readable storage medium according to one embodiment of the present invention, when executed on one or more processors, performs the following operations for a safety management method, wherein the operations include: receiving non-identifiable data and sound data obtained by photographing a surrounding environment; preprocessing the non-identifiable data and sound data; inputting the preprocessed data into a pre-trained neural network model to determine abnormal behavior of an object based on the movement and sound of an object included in the preprocessed data; predicting the occurrence of an accident regarding the abnormal behavior of the object; and, if the occurrence of an accident is predicted, generating a notification corresponding to the abnormal behavior of the object and transmitting it to an external device.

[0028] In addition to this, other methods for implementing the present invention, other systems, and computer-readable recording media for recording a computer program for executing said method may be further provided.

[0029] As described above, the present invention provides a notification by determining abnormal behavior of an object based on sound data and non-identifiable data based on at least one of the distance between objects within a surrounding environment and temperature through a pre-trained neural network model, thereby protecting personal information by utilizing non-identifiable data, accurately determining abnormal behavior by precisely recognizing the object's movements, and rapidly providing a notification regarding abnormal behavior.

[0030] In addition, the present invention analyzes behavioral patterns by utilizing only low-resolution data, such as distance values, rather than personal information through non-identifiable data analysis processing technology, thereby allowing users to use the space with peace of mind.

[0031] In addition, the present invention utilizes deep learning algorithms and preprocessing technology to recognize motion more precisely than existing motion detection systems, thereby enabling the real-time detection of drugs and assault by subdividing human motion patterns, and can be connected to an immediate warning system.

[0032] In addition, the present invention minimizes energy consumption through an ultra-lightweight design and allows for remote monitoring of the system status via OTA (Over The Air) technology, making maintenance easy and inexpensive.

[0033] In addition, the present invention is applicable not only to public restrooms but also to various special spaces such as facilities for the disabled, large shopping malls, and parks, and can be adjusted to meet environmental requirements and prioritize user experience.

[0034] In addition, the present invention processes collected data based on the cloud and analyzes it in real time, enabling an immediate response and allowing for rapid action to be taken in the event of an emergency.

[0035] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0036] FIG. 1 is a drawing for explaining the overall operation of a safety management device according to one embodiment of the present invention.

[0037] FIG. 2 is a drawing for explaining the configuration of a safety management device according to one embodiment of the present invention.

[0038] FIGS. 3 and FIGS. 4 are drawings for explaining a detection module of a safety management device according to an embodiment of the present invention.

[0039] FIG. 5 is a diagram illustrating a notification generation process corresponding to the determination of abnormal behavior of a safety management device according to an embodiment of the present invention.

[0040] FIGS. 6 to 8 are drawings for explaining the data preprocessing process of a safety management device according to an embodiment of the present invention.

[0041] FIGS. 9 and FIGS. 10 are drawings for explaining the process of determining abnormal behavior of a safety management device according to an embodiment of the present invention.

[0042] FIG. 11 is a diagram illustrating the process of predicting accident occurrence of a safety management device according to one embodiment of the present invention.

[0043] FIG. 12 is a flowchart for explaining a safety management method of a safety management device according to one embodiment of the present invention.

[0044] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined only by the scope of the claims.

[0045] The terms used in this specification are for describing embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more. Although terms such as "first," "second," etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, the first component mentioned below may be the second component within the technical scope of the invention.

[0046] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0047] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0048] Prior to the explanation, the meanings of the terms used in this specification are briefly explained. However, since the explanation of terms is intended to aid in understanding this specification, it should be noted that they are not used to limit the technical scope of the invention unless explicitly stated to be a limiting factor.

[0049] In this specification, neural networks, artificial neural networks, and network functions may often be used interchangeably.

[0050] Additionally, throughout this specification, the terms neural network, neural network, and network function may be used interchangeably. A neural network may consist of a set of interconnected computational units that may generally be referred to as “nodes.” These “nodes” may also be referred to as “neurons.”

[0051] A neural network is composed of at least two nodes. The nodes (or neurons) constituting the neural networks may be interconnected by one or more “links.”

[0052] FIG. 1 is a drawing for explaining the overall operation of a safety management device according to one embodiment of the present invention.

[0053] As illustrated in FIG. 1, the safety management device (10) of the present invention may include a first non-identification sensor module (100) that obtains distance data as non-identification data obtained by photographing at least one person located within the surrounding environment, and a second sound sensor module (200) that obtains sound data by sensing surrounding sound.

[0054] Here, the non-identification sensor module (100) may include a distance sensor such as a Time of Flight (ToF) sensor and a temperature sensor for temperature detection, and the non-identification data may include at least one of distance data between at least one person sensed from the distance sensor and temperature data of at least one person sensed from the temperature sensor.

[0055] The above distance sensor can measure distance data for each of multiple parts of a person (object) located within the surrounding environment, and in particular, can measure distance data for each of multiple parts of a person performing abnormal behavior, such as a person (20) who uses drugs.

[0056] Additionally, the temperature sensor may include a thermal imaging camera and can measure the temperature of a person located in the surrounding environment, and in particular, can measure temperature data for a person performing abnormal behavior, such as a person using drugs (20), and can measure temperature data for each of the multiple parts of the body.

[0057] Additionally, when the sound sensor module (200) senses ambient sound, it can sense ambient environmental sound that includes the object sound of a person performing abnormal behavior, such as a person using drugs (20).

[0058] The safety management device (10) of the present invention preprocesses the non-identification data and sound data when non-identification data and sound data are input from the non-identification sensor module (100) and the sound sensor module (200), inputs the preprocessed data into a pre-trained neural network model to determine abnormal behavior of an object based on the movement and sound of the object included in the preprocessed data, predicts the occurrence of an accident regarding the abnormal behavior of the object, and when an accident is predicted, generates a notification corresponding to the abnormal behavior of the object and transmits it to an external device (500).

[0059] For example, the safety management device (10) of the present invention can extract an object (20) of a person performing abnormal behavior, such as a person using drugs, from non-identifiable data and sound data, determine the abnormal behavior of the extracted object (20), and if the abnormal behavior is predicted to be drug use, generate a notification about it and transmit it to an external agency.

[0060] The safety management device (10) of the present invention is connected to an external device (500) for communication and can transmit a notification corresponding to the abnormal behavior of an object to the external device (500).

[0061] Here, the external device (500) may include both a standing device such as a PC (Personal Computer), Network TV, HBBTV (Hybrid Broadcast Broadband TV), Smart TV, and IPTV (Internet Protocol TV), and a mobile device or handheld device such as a smartphone, Tablet PC, Notebook, and PDA (Personal Digital Assistant).

[0062] In addition, the network connecting the external device (500) and the safety management device (10) includes both wired and wireless networks, and is a general term for a communication network that supports various communication standards or protocols for pairing or / and data transmission and reception between the external device (500) and the safety management device (10).

[0063] These wired and wireless networks include all communication networks currently or to be supported in the future by standards, and can support all one or more communication protocols for them.

[0064] These wired / wireless networks can be formed by networks for wired connections such as USB (Universal Serial Bus), CVBS (Composite Video Banking Sync), Component, S-Video (analog), DVI (Digital Visual Interface), HDMI (High Definition Multimedia Interface), RGB, and D-SUB, and communication standards or protocols for them, and networks for wireless connections such as Bluetooth, RFID (Radio Frequency Identification), infrared communication (IrDA: infrared Data Association), UWB (Ultra Wideband), ZigBee, DLNA (Digital Living Network Alliance), WLAN (Wireless LAN) (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed ​​Downlink Packet Access), LTE / LTE-A (Long Term Evolution / LTE-Advanced), and Wi-Fi Direct, and communication standards or protocols for them.

[0065]

[0066] In addition, the safety management device (10) can transmit and receive data by communicating with non-identification sensor modules (100) and sound sensor modules (200) installed at multiple locations via wired or wireless connection.

[0067] Here, the non-identification sensor module (100) and the sound sensor module (200) may be installed as a single unit or as separate units at multiple locations.

[0068] At this time, the safety management device (10) can collect data from non-identification sensor modules (100) and sound sensor modules (200) installed at multiple locations, and store the collected data by grouping it by location.

[0069]

[0070] In addition, the safety management device (10) of the present invention, as non-identifying data, can perform a first preprocessing of the distance data into color distance data by matching a color corresponding to the distance value of the distance data based on a color map and applying the matched color to the distance data, and can identify the presence or absence of an object from the color distance data, and if an object is identified, perform a second preprocessing of extracting an object segment by separating only the object area located at the object from the color distance data.

[0071]

[0072] In addition, the safety management device (10) of the present invention extracts movement features of an object (20) from preprocessed data, calculates a similarity score by comparing the extracted movement features with a preset abnormal behavior feature, extracts sound features of an object (20) from preprocessed data, calculates an association score by analyzing the correlation between the extracted sound features and the preset abnormal behavior feature, calculates a final similarity score by adding the calculated association score as a weight to the calculated similarity score, and if the final similarity score with the association score added as a weight is greater than or equal to a reference score, the movement features can be determined as abnormal behavior of the object.

[0073]

[0074] In addition, the safety management device (10) of the present invention can classify pattern types for abnormal behavior of an object (20) when determining abnormal behavior of an object (20), calculate an accident occurrence probability value corresponding to the classified pattern type, and predict the occurrence of an accident for abnormal behavior of an object (20) based on the calculated accident occurrence probability value.

[0075]

[0076] Additionally, the safety management device (10) of the present invention can generate a notification including at least one of the location and time where the object (20) is located, the pattern type of abnormal behavior of the object (20), the probability of an accident occurring of the object (20), and detailed prediction information regarding the occurrence of an accident of the object (20).

[0077]

[0078] Additionally, the safety management device (10) of the present invention groups the abnormal behavior type of an object (20) and the predicted accident occurrence type according to the location where the non-identification sensor module (100) and the sound sensor module (200) are placed, generates and stores safety management statistical information based on the accident occurrence data for each grouped location, and when a request for a safety management information provision service for a specific location is received from an external device (500), it can transmit a safety management information provision service for a specific location to the external device (500) based on the safety management statistical information stored in advance.

[0079] In this way, the present invention provides a notification by determining abnormal behavior of an object based on non-identifiable data and sound data regarding the surrounding environment through a pre-trained neural network model, thereby protecting personal information using non-identifiable data, accurately determining abnormal behavior by precisely recognizing the object's movements, and rapidly providing a notification of abnormal behavior.

[0080]

[0081] FIG. 2 is a drawing for explaining the configuration of a safety management device according to one embodiment of the present invention.

[0082] As illustrated in FIG. 2, the safety management device of the present invention may include a non-identification sensor module (100) that captures the surrounding environment to acquire distance data, a sound sensor module (200) that senses surrounding sound to acquire sound data, a communication module (300) that communicates with an external device, and a processor (400) that includes one or more cores.

[0083] Additionally, the non-identification sensor module (100) and the sound sensor module (200) can be arranged integrally on a single support plate to form a single integrated detection module.

[0084] Here, the support plate is connected to the rotary drive and can rotate, and the processor (400) can check the direction of sound generation when the sound sensor module (200) acquires sound data and control the rotation of the rotary drive so that the front surface of the integrated detection module including the non-identification sensor module (100) and the sound sensor module (200) faces the direction of sound generation.

[0085] In some cases, the non-identification sensor module (100) and the sound sensor module (200) may be configured as a separate detection module in which they are each placed separately on different support plates.

[0086] Here, the support plate may include a first support plate that supports a non-identification sensor module (100) and a second support plate that supports a sound sensor module (200).

[0087] At this time, the first support plate is connected to the first rotary drive and rotates, and the second support plate is connected to the second rotary drive and can rotate.

[0088] And, when the sound sensor module (200) acquires sound data, the processor (400) can check the direction of sound generation and control the rotational movement of the first rotary drive so that the front surface of the non-identification sensor module (100) faces the direction of sound generation, and at the same time control the rotational movement of the second rotary drive so that the front surface of the sound sensor module faces the direction of sound generation.

[0089]

[0090] Additionally, when the sound sensor module (200) acquires sound data, the processor (400) checks the direction of sound generation and controls the rotational movement of the first rotary drive so that the front surface of the non-identification sensor module (100) faces the direction of sound generation, and may not control the rotational movement of the second rotary drive so that the front surface of the sound sensor module (200) is fixed in the same direction.

[0091] In another case, the support plate includes a first support plate that supports a non-identification sensor module (100) and a second support plate that supports a sound sensor module (200), the first support plate is connected to a rotary drive and rotates, and the second support plate may be fixed.

[0092] Here, when the sound sensor module (200) acquires sound data, the processor (400) can determine the direction of sound generation and control the rotational movement of the rotary drive body so that the front surface of the non-identification sensor module (100) faces the direction of sound generation.

[0093]

[0094] Meanwhile, the processor (400) can control the communication module (300) to preprocess the non-identified data and sound data when non-identified data and sound data are input, input the preprocessed data into a pre-trained neural network model to determine the abnormal behavior of an object based on the movement and sound of the object included in the preprocessed data, predict the occurrence of an accident regarding the abnormal behavior of the object, and, if an accident is predicted, generate a notification corresponding to the abnormal behavior of the object and transmit it to an external device.

[0095] Here, when the processor (400) preprocesses distance data as non-identifiable data, when distance data is input, it matches a color corresponding to the distance value of the distance data based on a color map and applies the matched color to the distance data to preprocess the distance data into color distance data.

[0096] In some cases, when the processor (400) preprocesses non-identified data, if non-identified data is input, it may identify whether an object exists from the non-identified data, and if an object is identified, it may perform preprocessing to extract an object segment by separating only the object region located on the object from the non-identified data.

[0097] In another case, when the processor (400) preprocesses distance data as non-identifiable data, when distance data is input, it matches a color corresponding to the distance value of the distance data based on a color map, applies the matched color to the distance data to perform a first preprocessing of the distance data into color distance data, identifies whether there is an object from the color distance data, and if an object is identified, performs a second preprocessing of extracting an object segment by separating only the object region located at the object from the color distance data.

[0098]

[0099] Additionally, when the processor (400) preprocesses sound data, when sound data is input, it can perform preprocessing to remove noise frequencies by analyzing the frequency corresponding to the sound data.

[0100] Here, when removing noise frequencies, the processor (400) checks whether there is a specific frequency among the frequencies corresponding to the sound data that is outside the range of human voice frequencies when sound data is input, and if there is a specific frequency, it recognizes the specific frequency as a noise frequency and can remove the noise frequency.

[0101] In some cases, when the processor (400) removes noise frequencies, if sound data is input, the sound data may be input into a pre-trained noise classification model to classify the noise frequencies and remove the noise frequencies.

[0102] Next, when determining abnormal behavior of an object, the processor (400) extracts movement features of the object from preprocessed data, compares the extracted movement features with preset abnormal behavior features to calculate a similarity score, and if the calculated similarity score is greater than or equal to a reference score, the movement features can be determined as abnormal behavior of the object.

[0103] Here, when the processor (400) calculates the similarity score, it may extract sound features of an object from preprocessed data, analyze the association between the extracted sound features and a preset abnormal behavior feature to calculate an association score, and add the calculated association score as a weight to the calculated similarity score to calculate the final similarity score.

[0104] At this time, the processor (400) can determine that the movement feature is normal behavior of the object if the calculated similarity score is less than the reference score.

[0105] Next, when determining abnormal behavior of an object, the processor (400) extracts movement features of the object from preprocessed data, calculates a similarity score by comparing the extracted movement features with pre-set abnormal behavior features, extracts sound features of the object from preprocessed data, calculates an association score by analyzing the association between the extracted sound features and pre-set abnormal behavior features, calculates a final similarity score by adding the calculated association score as a weight to the calculated similarity score, and if the final similarity score with the association score added as a weight is greater than or equal to a reference score, the movement features can be determined as abnormal behavior of the object.

[0106] Here, the processor (400) can determine that the motion feature is normal behavior of the object if the final similarity score is less than the reference score.

[0107] And, when the processor (400) predicts the occurrence of an accident regarding the abnormal behavior of an object, it determines the abnormal behavior of the object, classifies the pattern type of the abnormal behavior of the object, calculates the probability value of the accident occurrence corresponding to the classified pattern type, and predicts the occurrence of an accident regarding the abnormal behavior of the object based on the calculated probability value of the accident occurrence.

[0108] Here, when the processor (400) classifies the pattern type of abnormal behavior of an object, if the pattern type of abnormal behavior of the object is a new pattern type different from the existing pattern type, the processor classifies the pattern type of abnormal behavior of the object as a new pattern type, collects abnormal behavior information corresponding to the new pattern type from an external server, and can predict the occurrence of an accident regarding the abnormal behavior of the object based on the abnormal behavior information corresponding to the new pattern type collected from the external server.

[0109] Next, when the processor (400) generates a notification corresponding to the abnormal behavior of an object, it may generate a notification that includes at least one of the location and time where the object is located, the pattern type of the object's abnormal behavior, the probability of an accident occurring in the object, and detailed prediction information regarding the occurrence of an accident in the object.

[0110] Here, the processor (400) may also generate a notification that includes a distance image based on distance data containing an object, if the non-identifiable data is distance data.

[0111] For example, the processor (400) can generate a notification in at least one of text type, sound type, and video type.

[0112] Next, the processor (400) groups the types of abnormal behavior of objects and the types of predicted accidents by location where the non-identification sensor module (100) and the sound sensor module (200) are placed, generates and stores safety management statistical information based on the accident data for each grouped location, and when a request for a safety management information provision service for a specific location is received from an external device, it can transmit a safety management information provision service for a specific location to the external device based on the previously stored safety management statistical information.

[0113]

[0114] Meanwhile, the aforementioned neural network model may be a deep neural network. Throughout this specification, neural network, network function, and neural network may be used interchangeably.

[0115] A deep neural network (DNN) can refer to a neural network that includes multiple hidden layers in addition to input and output layers. By using a deep neural network, one can identify the latent structures of data. That is, one can identify the latent structures of photos, text, videos, voice, and music (for example, what objects are in a photo, what the content and emotions of the text are, what the content and emotions of the voice are, etc.).

[0116] Deep neural networks may include convolutional neural networks (CNN), recurrent neural networks (RNN), restricted Boltzmann machines (RBM), deep belief networks (DBN), Q networks, U networks, Siamese networks, etc.

[0117] Convolutional neural networks are a type of deep neural network that includes a neural network containing convolutional layers. Convolutional neural networks are a type of multilayer perceptron designed to use minimal preprocessing.

[0118] CNNs can be composed of one or more convolutional layers and artificial neural network layers combined with them. CNNs can additionally utilize weights and pooling layers. Thanks to this structure, CNNs can fully utilize two-dimensional input data.

[0119] Convolutional neural networks can be used to recognize objects in images. Convolutional neural networks can process image data by representing it as a matrix with dimensions. For example, in the case of image data encoded in RGB (red-green-blue), each R, G, and B color can be represented as a 2-dimensional matrix (e.g., in the case of a 2-dimensional image). That is, the color value of each pixel in the image data can become an element of the matrix, and the size of the matrix can be equal to the size of the image. Therefore, image data can be represented as three 2-dimensional matrices (a 3-dimensional data array).

[0120] In a convolutional neural network, the convolutional process (input and output of the convolutional layer) can be performed by moving the convolutional filter and multiplying the matrix components of the convolutional filter and the image at each location.

[0121] A convolutional filter can be composed of an n*n matrix. Generally, a convolutional filter can be composed of a fixed-form filter smaller than the total number of pixels in an image. That is, when an m*m image is input into a convolutional layer (e.g., a convolutional layer with a size of n*n convolutional filter), a matrix representing n*n pixels containing each pixel of the image can be multiplied by the convolutional filter and its components (i.e., multiplying the components of the matrix together). Through multiplication with the convolutional filter, components matching the convolutional filter can be extracted from this image. For example, a 3*3 convolutional filter for extracting vertical linear components from an image can be composed as [[0,1,0], [0,1,0], [0,1,0]].

[0122] When a 3x3 convolutional filter is applied to an input image to extract vertical linear components, vertical linear components matching the convolutional filter can be extracted and output from the image. A convolutional layer can apply a convolutional filter to each matrix for each channel representing the image (i.e., R, G, B colors in the case of an R, G, B coded image). By applying a convolutional filter to the input image, the convolutional layer can extract features that match the convolutional filter from the input image. The filter values ​​of the convolutional filter (i.e., the values ​​of each component of the matrix) can be updated by backpropagation during the training process of the convolutional neural network.

[0123] A subsampling layer may be connected to the output of a convolutional layer to simplify the output of the convolutional layer, thereby reducing memory usage and computational load. For example, when the output of a convolutional layer is input to a pooling layer having a 2x2 max pooling filter, the image can be compressed by outputting the maximum value contained in each patch for every 2x2 patch from each pixel of the image. The aforementioned pooling may be a method of outputting the minimum value from a patch or the average value of a patch, and any pooling method may be included in the present invention.

[0124] A convolutional neural network may include one or more convolutional layers and subsampling layers. A convolutional neural network can extract features from an image by repeatedly performing convolutional processes and subsampling processes (e.g., the aforementioned max pooling). Through repeated convolutional and subsampling processes, the neural network can extract global features of the image.

[0125] The output of a convolutional layer or a subsampling layer can be input to a fully connected layer. A fully connected layer is a layer in which all neurons in one layer are connected to all neurons in an adjacent layer.

[0126] A fully connected layer can refer to a structure in a neural network where every node in each layer is connected to every node in another layer.

[0127]

[0128] Meanwhile, according to one embodiment of the present invention, the processor (400) of the safety management device may be composed of one or more cores and may include a processor for data analysis and deep learning, such as a central processing unit (CPU) of a computing device, a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU).

[0129] The processor (400) can read a computer program stored in memory and perform data processing for machine learning according to an embodiment of the present invention. The processor (400) can perform operations for learning a neural network. The processor (400) can perform calculations for learning a neural network, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating errors, and updating the weights of the neural network using backpropagation. At least one of the CPU, GPGPU, and TPU of the processor (130) can process the learning of a network function. For example, the CPU and GPGPU can together process the learning of a network function and data classification using the network function. In addition, in an embodiment of the present invention, processors of a plurality of computing devices can be used together to process the learning of a network function and data classification using the network function. In addition, a computer program executed in a computing device according to an embodiment of the present invention may be a CPU, GPGPU, or TPU executable program.

[0130]

[0131] Additionally, the communication module (300) of the safety management device (100) may include a wireless internet module, a short-range communication module, a location information module, etc.

[0132] A wireless internet module refers to a module for wireless internet access, configured to transmit and receive wireless signals in a communication network based on wireless internet technologies.

[0133] Wireless internet technologies include, for example, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc., and the above wireless internet module transmits and receives data according to at least one wireless internet technology within a range that includes internet technologies not listed above.

[0134] From the perspective that wireless internet access via WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A, etc. is achieved through a mobile communication network, a wireless internet module that performs wireless internet access through a mobile communication network may be understood as a type of mobile communication module.

[0135] A short-range communication module is for short-range communication and can support short-range communication by using at least one of Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0136] A location information module is a module for obtaining the server's location (or current location), and representative examples include the GPS (Global Positioning System) module or the WiFi (Wireless Fidelity) module.

[0137] In addition, the memory of the safety management device may include at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk.

[0138]

[0139] The safety management device of the present invention may operate in conjunction with web storage that performs memory storage functions on the internet. The description of the database described above is merely an example and is not limited thereto.

[0140] In addition, the safety management device of the present invention may further include an output unit and an input unit.

[0141] Here, the output unit can output information of any form generated or determined by the processor (400) and information of any form received by the communication module (300).

[0142] For example, the output section may include at least one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, and a 3D display. Some of these display modules may be configured to be transparent or light-transmitting so that the outside can be seen through them. This may be referred to as a transparent display module, and representative examples of transparent display modules include TOLED (Transparent OLED).

[0143]

[0144] Additionally, the input unit may receive user input. The input unit may include keys and / or buttons on a user interface or physical keys and / or buttons for receiving user input. A computer program according to embodiments of the present disclosure may be executed in accordance with user input through the input unit.

[0145] In addition, the input unit may receive a signal by detecting the user's button operation or touch input, or receive the voice or movements of the user, etc., through a camera or microphone and convert them into an input signal. For this purpose, speech recognition technology or motion recognition technology may be used.

[0146] Additionally, the input unit may be implemented as an external input device connected to a safety management device. For example, the input device may be at least one of a touchpad, a touch pen, a keyboard, or a mouse for receiving user input, but this is merely an example and is not limited thereto.

[0147] For example, the input unit may recognize user touch input. In some cases, the input unit may have the same configuration as the output unit. The input unit may be composed of a touch screen implemented to receive user selection input. The touch screen may use any one of the following methods: contact capacitance method, infrared light detection method, surface ultrasonic (SAW) method, piezoelectric method, or resistive method. The detailed description of the touch screen described above is merely an example according to one embodiment of the present disclosure, and various touch screen panels may be employed in the platform server (100). The input unit composed of a touch screen may include a touch sensor. The touch sensor may be configured to convert changes such as pressure applied to a specific part of the input unit or capacitance occurring in a specific part of the input unit into an electrical input signal. The touch sensor may be configured to detect not only the location and area of ​​the touch but also the pressure at the time of touch. When there is a touch input to the touch sensor, the corresponding signal(s) are sent to the touch controller. The touch controller can process the signal(s) and then transmit the corresponding data to the processor. This enables the processor to recognize whether or not a specific area of ​​the input part has been touched.

[0148]

[0149] The safety management device of the present invention may include other components for performing a server environment of a server. The safety management device may include any type of device. The safety management device may be a digital device equipped with a processor and memory and having computational capabilities, such as a laptop computer, notebook computer, desktop computer, web pad, or mobile phone.

[0150] The safety management device of the present invention may be a computing system capable of generating a user interface according to embodiments of the present invention and providing information to a user terminal via a network. The safety management device of the present invention may be a cloud server. The safety management device may be a web server that processes services. The types of safety management devices described above are merely examples and are not limited thereto.

[0151] In this way, the present invention provides a notification by determining abnormal behavior of an object based on non-identifiable data and sound data regarding the surrounding environment through a pre-trained neural network model, thereby protecting personal information using non-identifiable data, accurately determining abnormal behavior by precisely recognizing the object's movements, and rapidly providing a notification of abnormal behavior.

[0152]

[0153] In addition, the present invention analyzes behavioral patterns using only low-resolution data, such as distance values, rather than personal information through non-identifiable data analysis processing technology, so users can use the space with peace of mind.

[0154]

[0155] In addition, the present invention utilizes deep learning algorithms and preprocessing technology to recognize motion more precisely than existing motion detection systems, thereby enabling the real-time detection of drugs and assault by subdividing human motion patterns, and can be connected to an immediate warning system.

[0156]

[0157] In addition, the present invention minimizes energy consumption through an ultra-lightweight design and allows for remote monitoring of the system status via OTA (Over The Air) technology, making maintenance easy and inexpensive.

[0158]

[0159] In addition, the present invention is applicable not only to public restrooms but also to various special spaces such as facilities for the disabled, large shopping malls, and parks, and can be adjusted to meet environmental requirements and prioritize user experience.

[0160]

[0161] In addition, the present invention processes collected data based on the cloud and analyzes it in real time, enabling an immediate response and allowing for rapid action to be taken in the event of an emergency.

[0162] FIGS. 3 and 4 are drawings for explaining a detection module of a safety management device according to an embodiment of the present invention.

[0163] As illustrated in FIGS. 3 and 4, the safety management device of the present invention may include a non-identification sensor module (100) that captures the surrounding environment to acquire non-identification data, and a sound sensor module (200) that senses surrounding sound to acquire sound data.

[0164] As shown in FIG. 3, the non-identification sensor module (100) and the sound sensor module (200) can be arranged integrally on a single support plate (600) to form a single integrated detection module.

[0165] Here, the support plate (600) is connected to the rotary drive (700) and can rotate, and the processor (400) can determine the direction of sound generation when the sound sensor module (200) acquires sound data regarding an object (20), such as a person exhibiting abnormal behavior like drugs, and control the rotation of the rotary drive (700) so that the front surface of the integrated detection module including the non-identification sensor module (100) and the sound sensor module (200) faces the direction of sound generation.

[0166] That is, when the present invention acquires sound from an object (20) located in the surrounding environment, it identifies the point of sound generation and rotates the non-identification sensor module (100) so that it faces the direction of sound generation to accurately acquire non-identification data for the object (20), and at the same time, rotates the sound sensor module (200) so that it faces the direction of sound generation to accurately acquire sound data for the object (20) more significantly.

[0167] Accordingly, since the present invention can acquire optimized non-identifiable data [including at least one of distance data from the object (20) and temperature data of the object (20)] and sound data, the recognition rate of the object (20) is greatly improved, and the accuracy of the analysis of abnormal behavior of the object (20) can be improved.

[0168] As another embodiment, as shown in FIG. 4, a non-identification sensor module (100) and a sound sensor module (200) may be configured as a separable detection module in which they are each placed separately on different support plates (610, 620).

[0169] Here, the support plate may include a first support plate (610) that supports a non-identification sensor module (100) and a second support plate (620) that supports a sound sensor module (200).

[0170] At this time, the first support plate (610) is connected to the first rotary drive and rotates, and the second support plate (620) is connected to the second rotary drive and rotates.

[0171] And, when the sound sensor module (200) acquires sound data for an object (20), such as a person exhibiting abnormal behavior like drugs, the processor (400) can determine the direction of sound generation and control the rotational movement of the first rotary drive so that the front surface of the non-identification sensor module (100) faces the direction of sound generation, and at the same time control the rotational movement of the second rotary drive so that the front surface of the sound sensor module faces the direction of sound generation.

[0172] Additionally, when the sound sensor module (200) acquires sound data regarding an object (20), such as a person exhibiting abnormal behavior like drugs, the processor (400) checks the direction of sound generation, controls the rotational movement of the first rotary drive so that the front surface of the non-identification sensor module (100) faces the direction of sound generation, and may not control the rotational movement of the second rotary drive so that the front surface of the sound sensor module (200) is fixed in the same direction.

[0173] In another case, the support plate includes a first support plate (610) that supports a non-identification sensor module (100) and a second support plate (620) that supports a sound sensor module (200), the first support plate (610) is connected to a rotary drive and rotates, and the second support plate (620) may be fixed.

[0174] Here, when the sound sensor module (200) acquires sound data, the processor (400) can determine the direction of sound generation and control the rotational movement of the rotary drive body so that the front surface of the non-identification sensor module (100) faces the direction of sound generation.

[0175]

[0176] FIG. 5 is a diagram illustrating a notification generation process corresponding to the determination of abnormal behavior of a safety management device according to an embodiment of the present invention.

[0177] As illustrated in FIG. 5, the present invention can perform a preprocessing process (810) for preprocessing non-identified data and sound data when non-identified data and sound data are input.

[0178] Here, the preprocessing process (810) may include a process for preprocessing non-identifiable data and a process for preprocessing sound data.

[0179] For example, as the above-mentioned non-identifiable data, the process of preprocessing distance data involves matching a color corresponding to the distance value of the distance data based on a color map when the distance data is input, and applying the matched color to the distance data to preprocess the distance data into color distance data.

[0180] As another example, the process of preprocessing distance data may involve identifying the presence or absence of objects from the distance data when the data is input, and, if an object is identified, performing preprocessing to extract object segments by separating only the object region located at the object from the distance data.

[0181] As another example, the process of preprocessing distance data may involve, when distance data is input, matching a color corresponding to the distance value of the distance data based on a color map, applying the matched color to the distance data to perform a first preprocessing of the distance data into color distance data, identifying the presence or absence of an object from the color distance data, and, if an object is identified, performing a second preprocessing to extract an object segment by separating only the object region located on the object from the color distance data.

[0182] In addition, the sound data preprocessing process can perform preprocessing by analyzing the frequencies corresponding to the sound data and removing noise frequencies when sound data is input.

[0183] For example, in the process of preprocessing sound data, when sound data is input, it checks whether there are specific frequencies among the frequencies corresponding to the sound data that fall outside the range of human voice frequencies, and if specific frequencies exist, recognizes them as noise frequencies and can remove the noise frequencies.

[0184] As another example, in the process of preprocessing sound data, when sound data is input, it can be fed into a pre-trained noise classification model to classify noise frequencies and remove them.

[0185] Furthermore, the present invention can perform an abnormal behavior determination process (820) by inputting preprocessed data into a pre-trained neural network model to determine abnormal behavior of an object based on the movement and sound of the object included in the preprocessed data.

[0186] For example, the abnormal behavior judgment process (820) extracts movement features of an object from preprocessed data, calculates a similarity score by comparing the extracted movement features with a preset abnormal behavior feature, and if the calculated similarity score is greater than or equal to a reference score, the movement features can be judged as abnormal behavior of the object.

[0187] Here, the abnormal behavior judgment process (820) can calculate a similarity score by extracting sound features of an object from preprocessed data when calculating the similarity score, analyzing the association between the extracted sound features and a preset abnormal behavior feature to calculate an association score, and adding the calculated association score as a weight to the calculated similarity score to calculate a final similarity score.

[0188] As another example, the abnormal behavior judgment process (820) extracts movement features of an object from preprocessed data, calculates a similarity score by comparing the extracted movement features with a preset abnormal behavior feature, extracts sound features of an object from preprocessed data, calculates an association score by analyzing the association between the extracted sound features and the preset abnormal behavior feature, calculates a final similarity score by adding the calculated association score as a weight to the calculated similarity score, and if the final similarity score with the association score added as a weight is greater than or equal to the reference score, the movement features can be judged as abnormal behavior of the object.

[0189] Here, the abnormal behavior judgment process (820) can determine the movement feature as normal behavior of the object if the final similarity score is less than the reference score.

[0190] Next, the present invention can perform an accident occurrence prediction process (830) that predicts the occurrence of an accident regarding abnormal behavior of an object.

[0191] Here, the accident occurrence prediction process (830) determines the abnormal behavior of an object, classifies the pattern type of the abnormal behavior of the object, calculates the accident occurrence probability value corresponding to the classified pattern type, and can predict the occurrence of an accident regarding the abnormal behavior of the object based on the calculated accident occurrence probability value.

[0192] For example, the accident occurrence prediction process (830) classifies the pattern type of abnormal behavior of an object as a new pattern type if the pattern type of abnormal behavior of the object is a new pattern type different from the existing pattern type, collects abnormal behavior information corresponding to the new pattern type from an external server, and can predict the occurrence of an accident regarding the abnormal behavior of the object based on the abnormal behavior information corresponding to the new pattern type collected from the external server.

[0193] In addition, the present invention can perform a notification generation process (840) that generates a notification corresponding to the abnormal behavior of an object and transmits it to an external device when an accident is predicted to occur.

[0194] Here, the notification generation process (840) can generate a notification that includes at least one of the location and time where the object is located, the pattern type of abnormal behavior of the object, the probability of an accident occurring of the object, and detailed prediction information regarding the occurrence of an accident of the object.

[0195] In some cases, the notification generation process (840) may generate a notification that further includes a distance image based on distance data containing an object.

[0196] For example, the notification generation process (840) can generate the notification as at least one of text type, sound type, and video type.

[0197]

[0198] Additionally, the notification generation process (840) groups the types of abnormal behavior of objects and predicted accident occurrence types according to the locations where the non-identifying sensor module and the sound sensor module are placed, generates and stores safety management statistical information based on the accident occurrence data for each grouped location, and when a request for a safety management information provision service for a specific location is received from an external device, it can transmit a safety management information provision service for a specific location to the external device based on the previously stored safety management statistical information.

[0199]

[0200] FIGS. 6 to 8 are drawings for explaining the data preprocessing process of a safety management device according to an embodiment of the present invention.

[0201] As illustrated in FIG. 6, the present invention can perform a first preprocessing process (812) to convert distance data into color distance data by matching a color corresponding to the distance value of the distance data based on a color map when distance data is input, and applying the matched color to the distance data. It can also perform a second preprocessing process (814) to identify the presence or absence of an object from the color distance data, and if an object is identified, to separate only the object region located at the object from the color distance data to extract an object segment.

[0202]

[0203] In addition, as illustrated in FIG. 7, the present invention can perform preprocessing when preprocessing sound data, including a frequency analysis process that analyzes the frequency corresponding to the sound data when sound data is input, and a noise frequency removal process (818) that removes the noise frequency.

[0204] For example, when removing noise frequencies, the present invention checks whether there is a specific frequency among the frequencies corresponding to the sound data that falls outside the range of human voice frequencies when sound data is input, and if the specific frequency exists, the specific frequency is recognized as a noise frequency and the noise frequency can be removed.

[0205]

[0206] In addition, as illustrated in FIG. 8, when removing noise frequencies, the present invention may remove noise frequencies by inputting sound data into a pre-trained noise classification model (819) to classify the noise frequencies.

[0207]

[0208] FIGS. 9 and FIGS. 10 are drawings for explaining the process of determining abnormal behavior of a safety management device according to one embodiment of the present invention.

[0209] As illustrated in FIG. 9, when determining abnormal behavior of an object, the present invention can sequentially perform a motion feature extraction process (822) for extracting motion features of an object from preprocessed data, a similarity score calculation process (824) for calculating a similarity score by comparing the extracted motion features with preset abnormal behavior features, and an abnormal behavior determination process (825) for determining motion features as abnormal behavior of an object if the calculated similarity score is greater than or equal to a reference score.

[0210] For example, when calculating a similarity score, the present invention extracts sound features of an object from preprocessed data, analyzes the association between the extracted sound features and a preset abnormal behavior feature to calculate an association score, and calculates a final similarity score by adding the calculated association score as a weight to the calculated similarity score.

[0211] Here, the present invention can determine that a movement feature is normal behavior of an object if the calculated similarity score is less than a reference score.

[0212]

[0213] Additionally, as illustrated in FIG. 10, the present invention may include a motion extraction process (822) for extracting motion features of an object from preprocessed data when determining abnormal behavior of an object, a similarity score calculation process (824) for calculating a similarity score by comparing the extracted motion features with a preset abnormal behavior feature, a sound extraction process (826) for extracting sound features of an object from preprocessed data, an association score calculation process (827) for calculating an association score by analyzing the association between the extracted sound features and a preset abnormal behavior feature, a final similarity score calculation process (828) for calculating a final similarity score by adding the calculated association score as a weight to the calculated similarity score, and an abnormal behavior determination process (829) for determining motion features as abnormal behavior of an object if the final similarity score with the association score added as a weight is greater than or equal to a reference score.

[0214] Here, the present invention can determine that a motion feature is normal behavior of an object if the final similarity score is less than a reference score.

[0215]

[0216] FIG. 11 is a diagram illustrating the process of predicting accident occurrence of a safety management device according to one embodiment of the present invention.

[0217] As illustrated in FIG. 11, the present invention may include a type classification process (832) for classifying pattern types for abnormal behavior of an object when determining abnormal behavior of an object, an accident occurrence probability value calculation process (834) for calculating an accident occurrence probability value corresponding to the classified pattern type, and an accident occurrence prediction process (836) for predicting the occurrence of an accident for abnormal behavior of an object based on the calculated accident occurrence probability value.

[0218] For example, when classifying pattern types for abnormal behavior of an object, the present invention classifies the pattern type for abnormal behavior of an object as a new pattern type if the pattern type for abnormal behavior of the object is a new pattern type different from an existing pattern type, collects abnormal behavior information corresponding to the new pattern type from an external server, and can predict the occurrence of an accident regarding the abnormal behavior of the object based on the abnormal behavior information corresponding to the new pattern type collected from the external server.

[0219]

[0220] FIG. 12 is a flowchart illustrating a safety management method of a safety management device according to an embodiment of the present invention.

[0221] As illustrated in FIG. 12, the present invention can receive non-identifiable data and sound data (S10).

[0222] Next, the present invention can preprocess input non-identifiable data and sound data (S20).

[0223] Here, the present invention can perform a first preprocessing of the distance data as non-identifying data by, when the distance data is input, matching a color corresponding to the distance value of the distance data based on a color map and applying the matched color to the distance data to perform a first preprocessing of the distance data into color distance data, identifying the presence or absence of an object from the color distance data, and, if an object is identified, performing a second preprocessing of extracting an object segment by separating only the object region located at the object from the color distance data.

[0224] In addition, when preprocessing sound data, the present invention can perform a frequency analysis process that analyzes frequencies corresponding to sound data when sound data is input, and a preprocessing process that removes noise frequencies.

[0225] Next, the present invention inputs the preprocessed data into a pre-trained neural network model to determine abnormal behavior of an object based on the movement and sound of the object included in the preprocessed data (S30).

[0226] Here, the present invention extracts motion features of an object from preprocessed data, calculates a similarity score by comparing the extracted motion features with preset abnormal behavior features, extracts sound features of an object from preprocessed data, calculates an association score by analyzing the association between the extracted sound features and preset abnormal behavior features, calculates a final similarity score by adding the calculated association score as a weight to the calculated similarity score, and if the final similarity score with the association score added as a weight is greater than or equal to a reference score, the motion features can be determined as abnormal behavior of the object.

[0227] And, the present invention can predict the occurrence of an accident regarding abnormal behavior of an object (S40).

[0228] Here, the present invention determines abnormal behavior of an object, classifies pattern types for abnormal behavior of the object, calculates accident occurrence probability values ​​corresponding to the classified pattern types, and can predict accident occurrence regarding the abnormal behavior of the object based on the calculated accident occurrence probability values.

[0229] Next, the present invention can generate a notification corresponding to the abnormal behavior of the object when an accident occurs (S50).

[0230] Here, the present invention can generate a notification including at least one of the location and time where the object is located, the pattern type of abnormal behavior of the object, the predicted probability of an accident occurring of the object, and detailed prediction information regarding the occurrence of an accident of the object.

[0231] In some cases, the present invention may generate a notification that further includes a distance image based on distance data containing an object.

[0232] Next, the present invention can transmit the generated notification to an external device (S60).

[0233]

[0234] In addition, the present invention groups types of abnormal object behavior and predicted accident occurrence types according to locations where non-identification sensor modules and sound sensor modules are deployed, generates and stores safety management statistical information based on accident occurrence data for each grouped location, and when a request for a safety management information provision service for a specific location is received from an external device, transmits a safety management information provision service for the specific location to the external device based on the previously stored safety management statistical information.

[0235] In this way, the present invention provides a notification by determining abnormal behavior of an object based on non-identifiable data and sound data regarding the surrounding environment through a pre-trained neural network model, thereby protecting personal information using non-identifiable data, accurately determining abnormal behavior by precisely recognizing the object's movements, and rapidly providing a notification of abnormal behavior.

[0236]

[0237] In addition, the present invention analyzes behavioral patterns by utilizing only low-resolution data, such as distance values, rather than personal information through non-identifiable data analysis processing technology, thereby allowing users to use the space with peace of mind.

[0238]

[0239] In addition, the present invention utilizes deep learning algorithms and preprocessing technology to recognize motion more precisely than existing motion detection systems, thereby enabling the real-time detection of drugs and assault by subdividing human motion patterns, and can be connected to an immediate warning system.

[0240]

[0241] In addition, the present invention minimizes energy consumption through an ultra-lightweight design and allows for remote monitoring of the system status through OTA (Over The Air) technology, making maintenance easy and inexpensive.

[0242]

[0243] In addition, the present invention is applicable not only to public restrooms but also to various special spaces such as facilities for the disabled, large shopping malls, and parks, and can be adjusted to meet environmental requirements and prioritize user experience.

[0244]

[0245] In addition, the present invention processes collected data based on the cloud and analyzes it in real time to enable an immediate response, and allows for rapid action to be taken in the event of an emergency.

[0246] The method according to one embodiment of the present invention described above may be implemented as a program (or application) and stored on a medium to be executed in combination with a server, which is hardware.

[0247] The aforementioned program may include code encoded in a computer language such as C, C++, JAVA, or machine language, which can be read by the computer's processor (CPU) through the computer's device interface, in order for the computer to read the program and execute the methods implemented in the program. Such code may include functional code related to functions that define the necessary functions for executing the methods, and may include control code related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure.

[0248] Additionally, such code may further include memory reference-related code regarding where (address) in the internal or external memory of the computer additional information or media required for the processor of the computer to execute the above functions should be referenced.

[0249] In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the above functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to transmit or receive during communication.

[0250] The above-mentioned storage medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the above-mentioned storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the above-mentioned program may be stored on various recording media on various servers that the computer can access, or on various recording media on the user's computer. Additionally, the above-mentioned medium may be distributed across networked computer systems, and computer-readable code may be stored in a distributed manner.

[0251] The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium well known in the art to which the present invention belongs.

[0252] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive.

Claims

1. A distance non-identification sensor module that captures the surrounding environment to acquire distance non-identification data; A sound sensor module that acquires sound data by sensing ambient sound; A communication module that communicates with an external device; and, It includes a processor comprising one or more cores, and The above processor is, A safety management device characterized by controlling a communication module to preprocess the distance-nonidentification data and sound data when the above distance-nonidentification data and sound data are input, input the preprocessed data into a pre-trained neural network model to determine abnormal behavior of the object based on the movement and sound of the object included in the preprocessed data, predict the occurrence of an accident regarding the abnormal behavior of the object, and, when the occurrence of the accident is predicted, generate an alert corresponding to the abnormal behavior of the object and transmit it to the external device.

2. In Paragraph 1, The above non-identifiable data includes distance data with the object located within the surrounding environment, and The above processor is, A safety management device characterized by, when preprocessing the distance data, matching a color corresponding to the depth value of the distance data based on a color map when the distance data is input, and applying the matched color to the distance data to preprocess the distance data into color distance data.

3. In Paragraph 2, The above processor is, A safety management device characterized by performing preprocessing when preprocessing the above distance data, wherein when the above distance data is input, the presence or absence of an object is identified from the above distance data, and when the object is identified, only the object region located at the object is separated from the above distance data to extract an object segment.

4. In Paragraph 3, The above processor is, A safety management device characterized by performing preprocessing to remove noise frequencies by analyzing the frequency corresponding to the sound data when the sound data is input, when the sound data is preprocessed.

5. In Paragraph 1, The above processor is, A safety management device characterized by, when determining abnormal behavior of the object, extracting movement features of the object from the preprocessed data, calculating a similarity score by comparing the extracted movement features with a preset abnormal behavior feature, and determining the movement features as abnormal behavior of the object if the calculated similarity score is greater than or equal to a reference score.

6. In Paragraph 5, The above processor is, A safety management device characterized by, when calculating the similarity score, extracting sound features of the object from the preprocessed data, analyzing the correlation between the extracted sound features and the preset abnormal behavior features to calculate a correlation score, and adding the calculated correlation score as a weight to the calculated similarity score to calculate a final similarity score.

7. In Paragraph 5, The above processor is, A safety management device characterized by, when predicting the occurrence of an accident regarding abnormal behavior of the object, classifying a pattern type for the abnormal behavior of the object upon determining the abnormal behavior of the object, calculating an accident occurrence probability value corresponding to the classified pattern type, and predicting the occurrence of an accident regarding the abnormal behavior of the object based on the calculated accident occurrence probability value.

8. In Paragraph 1, The above processor is, A safety management device characterized by grouping the abnormal behavior types of the object and the predicted accident occurrence types according to the locations where the distance non-identification sensor module and the sound sensor module are deployed, generating and storing safety management statistical information based on the accident occurrence data for each grouped location, and, when a request for a safety management information provision service for a specific location is received from the external device, transmitting a safety management information provision service for the specific location to the external device based on the previously stored safety management statistical information.

9. A safety management method of a safety management device comprising a distance sensor module that captures a surrounding environment to acquire non-identifiable data and a sound sensor module that acquires sound data, wherein A step of receiving the above distance non-identification data and sound data; A step of preprocessing the above distance non-identification data and sound data; A step of inputting the above-mentioned preprocessed data into a pre-trained neural network model to determine abnormal behavior of the object based on the movement and sound of the object included in the above-mentioned preprocessed data; A step of predicting the occurrence of an accident regarding abnormal behavior of the above object; and A safety management method characterized by including the step of generating a notification corresponding to the abnormal behavior of the object and transmitting it to an external device when the occurrence of the above accident is predicted.

10. A computer program stored on a computer-readable storage medium, wherein, when executed on one or more processors, the computer program performs the following operations for a safety management method, said operations being: An operation to receive non-identifiable data and sound data obtained by photographing the surrounding environment; Operation of preprocessing the above distance non-identification data and sound data; An operation of inputting the above-mentioned preprocessed data into a pre-trained neural network model to determine abnormal behavior of the object based on the movement and sound of the object included in the above-mentioned preprocessed data; An action for predicting the occurrence of an accident regarding abnormal behavior of the above object; and A computer program stored on a computer-readable storage medium, characterized by including an operation of generating a notification corresponding to the abnormal behavior of the object and transmitting it to an external device when the occurrence of the above accident is predicted.

Images