Infant and toddler motion analysis device and method thereof

Abstract

The present disclosure relates to an infant and toddler motion analysis device and to a method therefor. The device comprises: a memory storing a process for analyzing motions of an infant or toddler; and a processor for executing the process, wherein the processor may determine whether a motion occurs in each of a plurality of body parts of the infant or toddler on the basis of acceleration data measured from sensors respectively attached to the plurality of body parts, and determine whether the motions of the infant or toddler are normal on the basis of the acceleration data measured for the plurality of body parts in which the motions are determined to have occurred.

Description

Device for analyzing movement of infants and toddlers and method therefor

[0001] The present disclosure relates to a device and method for analyzing the movements of infants and toddlers.

[0002] Many devices are available that allow parents and others to monitor the condition and behavior of infants and toddlers through audio and video. Their primary purpose is to monitor infants and toddlers' condition in real time, enabling rapid response to dangerous situations such as falls or rollovers.

[0003] To overcome the limitations of simple monitoring of the above conventional technology, a technology is needed that can continuously monitor the daily movements of infants and toddlers, analyze movement patterns, and predict future developmental problems in infants and toddlers. However, such technology is not currently available.

[0004] The purpose of the embodiments disclosed in this disclosure is to provide a device and method capable of analyzing the movements of infants and toddlers.

[0005] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0006] An electronic device for analyzing the movement of infants and toddlers according to the present disclosure for solving the above-described technical problem includes a memory storing a process for analyzing the movement of infants and toddlers; and a processor executing the process, wherein the processor determines whether a movement occurs for a plurality of body parts of the infant and toddler based on acceleration data measured from sensors respectively attached to the plurality of body parts of the infant and toddler, and determines whether the movement of the infant and toddler is normal based on the acceleration data measured for the plurality of body parts where the movement occurs based on the determined result.

[0007] At this time, the processor can detect a motion area from the acceleration data based on a preset threshold value to determine whether the motion has occurred, and can determine whether the motion of the infant or toddler is normal based on the acceleration data of the detected motion area.

[0008] In addition, the processor can output a diagnosis result on whether the movement of the infant or toddler is normal based on the type of abnormal movement, the time point and frequency of occurrence of the abnormal movement among the movements of the infant or toddler determined based on acceleration data of the detected movement area.

[0009] In addition, the processor determines whether the movement of the infant or toddler is normal based on the acceleration data using a pre-learned artificial intelligence model, and the artificial intelligence model may be learned based on learning data including a general movement assessment (GMA) for a plurality of infants or toddlers, brain imaging findings for the plurality of infants or toddlers, neurological examination findings, and occurrence of a neurological disease for the plurality of infants or toddlers.

[0010] Additionally, the above artificial intelligence model can be used to predict the neurodevelopmental prognosis of the infant or toddler.

[0011] In addition, the electronic device further includes a camera that photographs the infant or toddler, and the processor can check whether the recognition result of the infant or toddler's movement in the photographed image of the camera and the measured acceleration data match at preset intervals.

[0012] In addition, the processor can classify acceleration data determined to be CS (Cramped-Synchronized) among acceleration data measured for the plurality of body parts, and derive a CS ratio among the measured acceleration data based on the classification result.

[0013] In addition, a method for analyzing the movement of an infant or toddler performed by a processor of an electronic device according to the present disclosure for solving the above-described technical problem includes a step of determining whether a movement has occurred in each body part of the infant or toddler based on acceleration data measured from sensors attached to each of the plurality of body parts of the infant or toddler; and a step of determining whether the movement of the infant or toddler is normal based on the acceleration data measured for the plurality of body parts where the movement has occurred according to the determination result.

[0014] At this time, the normality determination step includes a step of detecting a motion area from the acceleration data based on a preset threshold value to determine whether the motion has occurred; and a step of determining whether the motion of the infant or toddler is normal based on the acceleration data of the detected motion area.

[0015] In addition, the above normality determination step may further include a step of outputting a diagnosis result on whether the movement of the infant or toddler is normal based on the type of abnormal movement, the time point and frequency of occurrence of the abnormal movement among the movements of the infant or toddler determined based on the acceleration data of the detected movement area.

[0016] In addition, the above normality judgment step uses a pre-learned artificial intelligence model to judge whether the movement of the infant or toddler is normal based on the acceleration data, and the artificial intelligence model may be learned based on learning data including a general movement assessment (GMA) for a plurality of infants or toddlers, brain imaging findings for the plurality of infants or toddlers, neurological examination findings, and occurrence of a neurological disease for the plurality of infants or toddlers.

[0017] In addition, the method for analyzing infant / toddler movement according to the present disclosure can predict the neurodevelopmental prognosis of the infant / toddler using the artificial intelligence model.

[0018] In addition, the method for analyzing movements of infants and toddlers according to the present disclosure may further include a step of photographing the infant and toddler through a camera and a step of checking whether the recognition result of the movement of the infant and toddler in the captured image of the camera and the measured acceleration data match at preset intervals.

[0019] In addition, the infant / toddler movement analysis method according to the present disclosure may further include a step of classifying acceleration data determined to be CS (Cramped-Synchronized) among acceleration data measured for the plurality of body parts, and a step of deriving a CS ratio among the measured acceleration data based on the classification result.

[0020] In addition, a computer program stored in a computer-readable recording medium for executing a method for implementing the present disclosure may be further provided.

[0021] In addition, a computer-readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

[0022] According to the aforementioned problem solving means of the present disclosure, it provides the effect of being able to analyze the movements of infants and toddlers.

[0023] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0024] FIG. 1 is a schematic diagram of a movement analysis system for infants and toddlers according to an embodiment of the present disclosure.

[0025] Figure 2 is a drawing illustrating sensors attached to each body part of the infant or toddler in Figure 1.

[0026] Figure 3 is a block diagram of a device for analyzing movement of infants and toddlers according to an embodiment of the present disclosure.

[0027] Figure 4 is a flowchart of a method for analyzing the movement of infants and toddlers according to an embodiment of the present disclosure.

[0028] Figure 5 is a diagram illustrating a process in which a processor analyzes and classifies acceleration data.

[0029] Figure 6 is a diagram illustrating a process in which a processor analyzes acceleration data of raw data.

[0030] Fig. 7 is a diagram illustrating detection of a motion area from acceleration data in Fig. 6.

[0031] Figure 8 is a diagram illustrating the application of a composite filter to determine similarity in Figure 6.

[0032] Figure 9 is a diagram illustrating the use of the Euclidean distance method to determine similarity in Figure 6.

[0033] Figure 10 is a diagram illustrating the use of a dynamic time joining technique for similarity determination in Figure 6.

[0034] Figure 10 is a drawing illustrating a synchronization area in each operation area of ​​Figure 6.

[0035] Figure 11 is a diagram illustrating detection of ternary data.

[0036] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and any content that is common in the technical field to which this disclosure pertains or that overlaps between embodiments is omitted. The terms "part, module, element, block" used in the specification may be implemented in software or hardware, and depending on the embodiments, multiple "parts, modules, elements, blocks" may be implemented as a single component, or a single "part, module, element, block" may include multiple components.

[0037] Throughout the specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0038] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0039] Throughout the specification, when we say that an element is "on" another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0040] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0041] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0042] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0043] The operating principle and embodiments of the present disclosure are described below with reference to the attached drawings.

[0044] As used herein, the term "electronic device according to the present disclosure" encompasses a variety of devices capable of performing computational processing and providing results to a user. For example, an electronic device according to the present disclosure may include a computer, a server device, and a portable terminal, or may be any one of them.

[0045] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0046] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0047] The above portable terminal may include, for example, all kinds of handheld-based wireless communication devices such as PCS, GSM, PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smart phones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMD).

[0048] The artificial intelligence-related functions according to the present disclosure are operated through a processor and a storage unit. The processor may be composed of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a Digital Signal Processor (DSP), a graphics-only processor such as a GPU or a Vision Processing Unit (VPU), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in the storage unit. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0049] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is learned by a learning algorithm using a plurality of learning data, thereby creating the predefined operation rules or artificial intelligence models set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0050] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weights, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q-network.

[0051] According to an exemplary embodiment of the present disclosure, a processor can implement artificial intelligence. Artificial intelligence refers to a machine learning method based on an artificial neural network that mimics human neurons (biological neurons) to enable machines to learn. Artificial intelligence methodologies can be categorized into supervised learning, where input and output data are provided together as training data, thereby determining the solution (output data) to a problem (input data); unsupervised learning, where only input data is provided without output data, so that the solution (output data) to a problem (input data) is not determined; and reinforcement learning, where a reward is provided from an external environment each time an action is taken in the current state, and learning proceeds in a direction that maximizes this reward. Furthermore, artificial intelligence methodologies can be categorized by the architecture of the learning model. The architectures of widely used deep learning technologies can be categorized into convolutional neural networks, recurrent neural networks, transformers, and generative adversarial networks.

[0052] The device may include an artificial intelligence model. The artificial intelligence model may be a single artificial intelligence model or may be implemented as multiple artificial intelligence models. The artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model in general that has problem-solving capabilities by changing the binding strength of synapses through learning, formed by artificial neurons (nodes) that form a network by combining synapses. The neurons of the neural network may include a combination of weights or biases. The neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a desired outcome from an arbitrary input by changing the weights of the neurons through learning.

[0053] The processor can create a neural network, train (or learn) a neural network, perform a calculation based on received input data, generate an information signal based on the calculation result, or retrain the neural network. The models of the neural network can include various types of models such as CNN, R-CNN, RPN, RNN, S-DNN, S-SDNN, Deconvolution Network, DBN, RBM, Fully Convolutional Network, LSTM Network, Classification Network, etc., such as GoogleNet, AlexNet, VGG Network, etc., but are not limited thereto. The processor can include one or more processors for performing calculations according to the models of the neural network. For example, the neural network can include a deep neural network.

[0054] Neural networks include CNN, RNN, perceptron, multilayer perceptron, Feed Forward (FF), Radial Basis Network (RBF), Deep Feed Forward (DFF), Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), Auto Encoder (AE), Variational Auto Encoder (VAE), Denoising Auto Encoder (DAE), Sparse Auto Encoder (SAE), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network), LSM (Liquid State Machine), ELM (Extreme Learning) Machine), ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Network) It will be understood by those skilled in the art that the neural network may include any neural network, including but not limited to a Neural Computer (NN), a Neural Turning Machine (NTM), a Capsule Network (CN), a Kohonen Network (KN), and an Attention Network (AN).

[0055] According to an exemplary embodiment of the present disclosure, the processor may be configured to perform a process for generating a CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, Region with Convolution Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restrcted Boltzman Machine (RBM), Fully Convolutional Network, Long Short-Term Memory (LSTM) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA for natural language processing, Text Analysis, Dialog System, GPT-3, GPT-4, Visual Analytics for vision processing, Visual Understanding, Video Synthesis, ResNet for data intelligence, Anomaly Detection, Prediction, Time-Series Forecasting, Various artificial intelligence structures and algorithms, including optimization, recommendation, and data creation, can be utilized, but are not limited thereto. Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0056] FIG. 1 is a schematic diagram of a movement analysis system for infants and toddlers according to an embodiment of the present disclosure.

[0057] Figure 2 is a drawing illustrating sensors attached to each body part of the infant or toddler in Figure 1.

[0058] However, in some embodiments, the system (10) may include fewer or more components than those illustrated in FIG. 1.

[0059] Referring to FIG. 1, the infant movement analysis system (10) according to the embodiment of the present disclosure can receive acceleration data (30) measured from sensors attached to each body part of an infant (20) by an electronic device (100), analyze the data, and output diagnostic data (40) including the result of determining whether the movement of the infant (20) is normal.

[0060] As shown in FIG. 2, at least one sensor can be attached to each body part of the infant or toddler, and as shown in the drawing, the sensor unit (140) can include a sensor (141) attached to the left arm, a sensor (142) attached to the right arm, a sensor (143) attached to the left leg, a sensor (144) attached to the right leg, a sensor (145) attached to the head, and a sensor (146) attached to the torso.

[0061] At this time, each body part may have multiple sensors attached to improve measurement accuracy.

[0062] Referring to FIG. 2, according to an embodiment, the electronic device (100) may further include a photographing unit (160) including at least one camera for photographing an infant or toddler, and the processor (110) of the electronic device (100) may compare an image captured through the camera with acceleration data measured and received from a sensor.

[0063] The processor (110) of the electronic device (100) can check whether the recognition result of the movement of the infant or toddler in the captured image of the camera matches the acceleration data measured through the sensor unit (140) at preset intervals.

[0064] Below, with reference to other drawings, a more detailed description will be given of an infant movement analysis device, a control method of the device, and a program according to an embodiment of the present disclosure.

[0065] Figure 3 is a block diagram of a device for analyzing movement of infants and toddlers according to an embodiment of the present disclosure.

[0066] Referring to FIG. 3, the infant / toddler movement analysis device according to the embodiment of the present disclosure includes a processor (110), a communication unit (120), a memory (130), a sensor unit (140), a detection unit (150), a photographing unit (160), and an output unit (170).

[0067] However, in some embodiments, the electronic device (100) may include fewer or more components than those illustrated in FIG. 3.

[0068] The processor (110) may be implemented as a storage unit that stores data regarding an algorithm for controlling the operation of components within the device or a program that reproduces the algorithm, and at least one processor (110) that performs the aforementioned operation using the data stored in the storage unit. In this case, the storage unit and the processor (110) may each be implemented as separate chips. Alternatively, the storage unit and the processor (110) may be implemented as a single chip.

[0069] In addition, the processor (110) can control any one or a combination of the components described above to implement various embodiments according to the present disclosure described in the drawings below on the device.

[0070] In addition to operations related to the above-described application, the processor (110) can typically control the overall operation of the device. The processor (110) can process signals, data, information, etc. input or output through the components described above, or run application programs stored in the storage unit, thereby providing or processing appropriate information or functions to the user.

[0071] In addition, the processor (110) may control at least some of the components of the device to run an application program stored in the storage unit. Furthermore, the processor (110) may operate at least two or more of the components included in the device in combination to run the application program.

[0072] The processor (110) may be implemented as one or more. Hereinafter, even if the processor (110) is expressed as singular, it may be considered as plural. The processor (110) may control the configurations of the electronic device (100). The processor (110) may refer to a data processing device built into hardware that has a physically structured circuit to perform a function expressed by a code or command included in a program. As such, the processor (110) is an example of a data processing device built into hardware, and may encompass processing devices such as a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA), but the scope of the present invention is not limited thereto. The processor (110) may separately include a learning processor (110) for performing artificial intelligence operations, or may include a learning processor (110) on its own.

[0073] In various embodiments, the processor (110) may include one or more of a central processing unit (CPU), an application processor (AP), or a communication processor (CP). At least a portion of the processor (110) may be hardware, access memory (130), and perform functions related to instructions stored in the memory (130).

[0074] The communication unit (120) may include one or more modules that connect the electronic device (100) to one or more networks.

[0075] The communication unit (120) may include one or more components that enable communication with an external device, and may include, for example, at least one of a broadcast reception module, a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0076] The wired communication module may include various wired communication modules such as a Local Area Network (LAN) module, a Wide Area Network (WAN) module, or a Value Added Network (VAN) module, as well as various cable communication modules such as a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), a Digital Visual Interface (DVI), RS-232 (recommended standard 232), power line communication, or plain old telephone service (POTS).

[0077] The wireless communication module may include a wireless communication module that supports various wireless communication methods such as GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G, in addition to a WiFi module and a Wireless Broadband module.

[0078] The wireless communication module may include a wireless communication interface including an antenna and a transmitter for transmitting communication signals. Furthermore, the wireless communication module may further include a signal conversion module that modulates a digital control signal output from the processor (110) through the wireless communication interface into an analog wireless signal under the control of the processor (110).

[0079] The short-range communication module is for short-range communication, and is based on Bluetooth. TM ), RFID (Radio Frequency Identification), infrared communication (Infrared Data Association; IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies can be used to support short-range communication.

[0080] The communication unit (120) may also use the name of the communication interface.

[0081] The communication interface can establish communication between the electronic device (100) and an external device. For example, the communication interface can communicate with the external device via wireless communication (e.g., Wi-Fi (Wireless Fidelity), Bluetooth, NFC (Near Field Communication), MST (Magnetic Stripe Transmission), etc.) or wired communication.

[0082] The memory (130) can store data supporting various functions of the device. The memory (130) can store a plurality of application programs (or applications) running on the device, data for the operation of the device, and commands. At least some of these application programs may exist for the basic functions of the device. Meanwhile, the application programs can be stored in the memory (130), installed on the device, and driven to perform operations (or functions) by the processor (110).

[0083] The memory (130) can store data supporting various functions of the device and programs for the operation of the processor (110), input / output data (e.g., music files, still images, moving images, etc.) can be stored, and a plurality of application programs (or applications) run on the device, data for the operation of the device, and commands can be stored. At least some of these application programs can be downloaded from an external server via wireless communication.

[0084] The memory (130) may include at least one type of storage medium among a flash memory (130) type, a hard disk type, an SSD (Solid State Disk type), an SDD (Silicon Disk Drive) type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory (130) may be a database that is separate from the device but is connected by wire or wirelessly.

[0085] The memory (130) may be electrically connected to the processor (110) and may store at least one code executed by the processor (110). The memory (130) may collectively refer to various types of storage devices. The memory (130) may store information necessary for performing operations using artificial intelligence, machine learning, and artificial neural networks.

[0086] The memory (130) can store various learning models. The learning models stored in the memory (130) can infer result values ​​for new input data other than learning data, and the inferred values ​​can be used as a basis for judgment to perform a certain action. The learning models stored in the memory (130) can perform learning based on label information, and various backpropagation algorithms can be applied so that the loss function has a target value to increase the accuracy of learning.

[0087] Additionally, the memory (130) may have multiple processes for the electronic device (100).

[0088] The sensor unit (140) may include at least one sensor attached to each body part of an infant or toddler as described with reference to FIG. 2, and may transmit measured acceleration data to the communication unit (120) of the electronic device (100) via a wired / wireless communication method.

[0089] The sensor unit (140) senses at least one of the internal information of the device, the surrounding environmental information surrounding the device, and the user information, and generates a sensing signal corresponding thereto. Based on this sensing signal, the processor (110) can control the operation or behavior of the device, or perform data processing, functions, or operations related to an application program installed in the device.

[0090] As described above, the sensor unit (140) may include at least one of a proximity sensor, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, a G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor (IR sensor), a fingerprint recognition sensor, an ultrasonic sensor, an optical sensor (e.g., a camera), a microphone, an environmental sensor (e.g., at least one of a barometer, a hygrometer, a thermometer, a radiation detection sensor, a heat detection sensor, and a gas detection sensor), and a chemical sensor (e.g., a healthcare sensor, a biometric recognition sensor, etc.). Meanwhile, the present device may utilize information sensed by at least two or more of these sensors in combination.

[0091] The detection unit (150) can detect a motion area where the infant or toddler is judged to have actually moved from the acceleration data measured and received through the sensor unit (140). To this end, the memory (130) can store threshold values ​​and algorithms for detecting the motion area.

[0092] The camera unit (160) may include at least one camera for photographing infants and toddlers, and may transmit the photographed image to the communication unit (120) of the electronic device (100).

[0093] The camera processes image frames, such as still images or video, obtained by the image sensor in shooting mode. The processed image frames can be displayed or stored in a storage device.

[0094] The camera can capture still images and videos. For example, when the electronic device (100) is mounted on the HMD device, the camera can capture images of at least the front area of ​​the HMD device. In one embodiment, the camera can be activated after a specified time has elapsed since the electronic device (100) is mounted on the HMD device and the HMD device starts operating. In various embodiments, the camera can be activated from the time the electronic device (100) is mounted on the HMD device. Alternatively, the camera can be activated from the time the user puts on the HMD device.

[0095] In various embodiments, the camera may include, for example, at least one depth camera (e.g., a Time Of Flight (TOF) type or a structured light type) and a color camera (e.g., an RGB camera). In addition, the camera may further include at least one sensor (e.g., a proximity sensor), a light source (e.g., an LED array), etc., in relation to performing a function. In various embodiments, the at least one sensor may be configured as a separate module from the camera and may perform sensing of at least a front area of ​​the HMD device. For example, the sensor (e.g., a proximity sensor) module may perform sensing of an object by emitting infrared light (or transmitting ultrasonic waves) to the front area of ​​the HMD device and receiving infrared light (or receiving ultrasonic waves) reflected from an object. In this case, the camera may be activated from the time when at least one object is sensed by the sensor module.

[0096] The output unit (170) is for generating output related to visual, auditory, or tactile sensations, and may include at least one of a display unit, an audio output unit (170), a haptic module, and an optical output unit (170). The display unit may be formed as a layer structure with a touch sensor or formed as an integral part, thereby implementing a touch screen. This touch screen may function as a user input unit that provides an input interface between the device and a user, and at the same time, provide an output interface between the device and the user.

[0097] The processor (110) can output the judgment result and diagnosis result regarding the infant's movement through the output unit (170).

[0098] However, the output unit (170) does not necessarily have to be included in the configuration of the electronic device (100), and the electronic device (100) may transmit and provide the judgment result or diagnosis result determined based on the movement of the infant or toddler to the terminal of the guardian of the infant or toddler, or to a medical staff device.

[0099] In some embodiments, the electronic device (100) may further include components such as an input unit and an interface unit as follows.

[0100] The input unit is for inputting video information (or signal), audio information (or signal), data, or information input from a user, and may include at least one camera, at least one microphone, and at least one user input unit. Voice data or image data collected from the input unit may be analyzed and processed into a user control command.

[0101] The input unit is for receiving information from the user, and when information is input through the input unit, the processor (110) can control the operation of the device to correspond to the input information. The input unit may include a hardware physical key (e.g., a button located on at least one of the front, rear, and side of the device, a dome switch, a jog wheel, a jog switch, etc.) and a software touch key. As an example, the touch key may be a virtual key, a soft key, or a visual key displayed on a touch screen type display unit through software processing, or may be a touch key disposed on a part other than the touch screen. Meanwhile, the virtual key or visual key may have various forms and be displayed on the touch screen, and may be, for example, formed of a graphic, text, an icon, a video, or a combination thereof.

[0102] The interface unit serves as a passageway for various types of external devices connected to the device. The interface unit may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module (SIM), an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port. The device may perform appropriate control related to the external device connected to the interface unit.

[0103] The input / output interface may serve as an interface that can transmit commands or data input from a user or other external device to other components of the electronic device (100). In addition, the input / output interface may output commands or data received from other components of the electronic device (100) to the user or other external device.

[0104] Figure 4 is a flowchart of a method for analyzing the movement of infants and toddlers according to an embodiment of the present disclosure.

[0105] FIGS. 5 to 11 are various exemplary drawings for explaining a device, method, and program for analyzing movement of infants and toddlers according to an embodiment of the present disclosure.

[0106] Referring to FIGS. 4 to 11, a device, method, and program for analyzing movement of infants and toddlers according to an embodiment of the present disclosure will be described.

[0107] The processor (110) receives acceleration data measured from the sensor unit (140). (S410)

[0108] In an embodiment of the present disclosure, the sensor unit (140) includes at least one sensor attached to each of a plurality of body parts of the infant or toddler.

[0109] The multiple body parts include a left arm, a right arm, a left leg, and a right leg. However, this is only one example, and the body parts of infants and toddlers for measuring acceleration are not limited to these.

[0110] That is, at least one sensor is attached to each of the left arm, right arm, left leg, and right leg of the infant or toddler, and acceleration data measured from each sensor can be transmitted to the electronic device (100) through the communication unit (120).

[0111] The processor (110) determines whether movement occurs in each body part of the infant or toddler based on the acceleration data received from S410. (S420)

[0112] The processor (110) checks whether a motion area is detected from the acceleration data. (S430)

[0113] The processor (110) can determine whether motion occurs in each body part based on acceleration data measured from sensors attached to each of multiple body parts of the infant or toddler.

[0114] The processor (110) can detect a motion area from acceleration data based on a preset threshold value to determine whether motion has occurred.

[0115] At this time, the preset threshold may be set differently depending on the information of the infant or toddler.

[0116] The processor (110) determines whether the infant's movements are normal based on acceleration data in the movement area. (S440)

[0117] The processor (110) can determine whether the infant's movement is normal based on the acceleration data measured for multiple body parts where the movement is determined to have occurred according to the judgment result of S430.

[0118] In one embodiment, the processor (110) can determine whether the infant's movement is normal or abnormal based on acceleration data corresponding to the movement area.

[0119] At this time, the processor (110) can determine whether the infant's movement is normal based on the acceleration data measured for two body parts where the movement is determined to have occurred.

[0120] In detail, the processor (110) can determine whether the infant's movements are normal based on acceleration data measured for two body parts among the infant's left arm, right arm, left leg, and right leg.

[0121] The processor (110) outputs the diagnosis results for the infant's movements. (S450)

[0122] In detail, the processor (110) can determine the form of abnormal movement, the time and frequency at which the abnormal movement occurs, and the infant's movement based on acceleration data of the detected movement area, and output a diagnosis result for the infant's movement based on the determined form of abnormal movement, the time and frequency at which the abnormal movement occurs.

[0123] In one embodiment, the electronic device (100) can diagnose infants and toddlers using artificial intelligence.

[0124] In detail, it is possible to determine whether the infant's movements are normal based on the acceleration data using a pre-learned artificial intelligence model.

[0125] At this time, the artificial intelligence model is characterized in that it is learned based on learning data including at least one of a general movement assessment (GMA) for a plurality of infants and toddlers, brain imaging (e.g., ultrasound, MRI) findings and neurological examination findings for the plurality of infants and toddlers, and whether or not the plurality of infants and toddlers have a neurological disease.

[0126] The artificial intelligence model can analyze the received images of the plurality of infants and toddlers along with the General Movement Assessment (GMA) to calculate the acceleration of each body part according to the movement pattern in the images of the plurality of infants and toddlers, and learn the General Movement Assessment and the calculated acceleration data. The processor (110) can measure the acceleration of each body part based on the three-dimensional position coordinates and frame rates of each body part of the plurality of infants and toddlers from the plurality of image frames in which the plurality of infants and toddlers were captured.

[0127] That is, the processor (110) can input the acceleration data of the infant or toddler into the artificial intelligence model learned in this way and obtain the possibility of the infant or toddler developing a neurological disease.

[0128] The processor (110) can input acceleration data of the infant or toddler into the artificial intelligence model learned in this manner and obtain a predicted value for the neurodevelopmental prognosis of the infant or toddler as an output value.

[0129] In addition, the processor (110) can input acceleration data of the infant or toddler into the artificial intelligence model learned in this manner and obtain the probability of the infant or toddler developing each of a plurality of neurological diseases as an output value.

[0130] Figure 5 is a diagram illustrating a process in which a processor (110) analyzes and classifies acceleration data.

[0131] Referring to FIG. 5, the processor (110) performs a process (510) of analyzing acceleration data determined to be a motion area from acceleration data measured and received through the sensor unit (140), and classifies acceleration data determined to be an abnormal motion.

[0132] And, the processor (110) can input the classified acceleration data into the first classifier (520) and classify it into three types: PR (Poor Repertoire), CS (Cramped Synchronized), and CH (Chaotic).

[0133] And finally, the processor (110) can input the classified data into the second classifier (530) to derive the ratio (%) that can be determined as CS in the acceleration data.

[0134] The CS ratio (540) derived in this way can be used by medical staff as a guide to determine whether the newborn's movements correspond to CS.

[0135] The second classifier (530) may be a CS classifier and may target a combination of data for two body parts among the left arm, right arm, left leg, and right leg.

[0136] Figure 6 is a diagram illustrating a process in which a processor (110) analyzes acceleration data of raw data.

[0137] The processor (110) can classify raw data (610) received from the sensor unit (140) into first data (620) and second data (630), perform filtering on the first data (620) at a level that is not distorted by a filter such as a moving average, and perform a process (640) of detecting an operation area, and perform filtering on the second data (630) at a level that is not distorted by a filter such as a moving average, and perform a process (650) of detecting an operation area.

[0138] Then, a process (670) is performed to filter the first data (620) and the second data (630), extract similarity using a similarity judgment tool, and then determine a synchronized section.

[0139] Next, the processor (110) performs a result analysis process (680) based on the results obtained by performing processes 640, 650, and 670.

[0140] Fig. 7 is a diagram illustrating detection of a motion area from acceleration data in Fig. 6.

[0141] (A) of Fig. 7 is acceleration data measured and received from a sensor attached to the left leg of the target, and (B) is acceleration data measured and received from a sensor attached to the right leg of the target.

[0142] As shown in Fig. 7, the processor (110) detects an area having an acceleration exceeding a preset threshold value (5 in Fig. 7) from the acceleration data measured and received from the sensor unit (140) as an operation area.

[0143] In an embodiment of the present disclosure, one or more most suitable similarity judgment tools among various composite filters may be applied to determine similarity.

[0144] For example, the electronic device (100) can perform image analysis-based similarity analysis.

[0145] The processor (110) can generate an image based on acceleration data acquired for two different body parts and perform similarity analysis by comparing the two images.

[0146] In one embodiment, the processor (110) determines the similarity between two images based on brightness, contrast, and structural differences using the SSIM (Structural Similarity Index Measure) algorithm.

[0147] Figure 8 is a diagram illustrating the application of a composite filter to determine similarity in Figure 6.

[0148] Figure 9 is a diagram illustrating the use of the Euclidean distance method to determine similarity in Figure 6.

[0149] (A) of Fig. 8 illustrates the application of a moving average and a bandpass filter to acceleration data measured and received from a sensor attached to the left leg of a target, and (B) illustrates the application of a moving average and a bandpass filter to acceleration data measured and received from a sensor attached to the right leg of a target.

[0150] The processor (110) can apply a composite filter (such as an N-layered Moving Average) that successively applies moving averages with different window sizes, and then apply a bandpass filter (BPF) to remove high-frequency noise and low-frequency offset components. Thereafter, the processor (110) can apply a similarity measurement method, such as the Euclidean distance method as shown in FIG. 9, to the first data and the second data to derive / produce similarity.

[0151] In (A) of Fig. 9, a time-based graph (910) of high similarity for each data is illustrated, and a time-based difference value (920) for this is depicted.

[0152] In (B) of Fig. 9, a time-based graph (930) of low similarity for each data is illustrated, and the time-based difference value (940) for this is depicted.

[0153] The processor (110) can measure the similarity between two time series data using the dynamic time warping (DTW) technique.

[0154] Figure 10 is a drawing illustrating a synchronization area in each operation area of ​​Figure 6.

[0155] Figure 11 is a diagram illustrating the derivation of ternary data.

[0156] Referring to Fig. 11, a ternary transformation is illustrated, and Act + sync (1210), Synchro Section (1220), and Action Section (1230) are illustrated on a graph, respectively, and in order to facilitate visual distinction, levels 0, 1, and 2 are illustrated as levels 1, 3, and 6, respectively.

[0157] After performing the above processes, the processor (110) can set a synchronization area as in the Synchro Section (1110) of FIG. 10.

[0158] And, the processor (110) can convert three types of binary trajectory data, such as those in FIG. 10, into ternary data according to the following criteria.

[0159] The processor (110) can convert the remaining data into ternary data, excluding data determined to be normal from the acceleration data measured through the sensor unit (140).

[0160] ① Data i operation area F & synchronization area F

[0161] ② Data i operation area T & synchronization area F

[0162] ③ Data i operation area T & synchronization area T

[0163] (At this time, i = 1, 2, T means True, F means False.)

[0164] The ternary data can be derived for the first data and the second data as shown in Fig. 11. The processor (110) can obtain the ratio of each ternary data as a percentage from this conversion, and based on this ratio, can quantitatively determine how close the motion of the upper and lower extremities (or other measurement parts), which are inputs to the second classifier, is to CS or, conversely, to Normal.

[0165] The method according to one embodiment of the present disclosure described above can be implemented as a program (or application) and stored in a medium to be executed in combination with a hardware server.

[0166] The above-described program may include codes coded in a computer language, such as C, C++, JAVA, or machine language, that can be read by the processor (CPU) of the computer through the device interface of the computer, so that the computer reads the program and executes the methods implemented as a program. Such codes may include functional codes related to functions that define functions necessary to execute the methods, and may include control codes related to execution procedures necessary for the processor of the computer to execute the functions according to a predetermined procedure. In addition, such codes may further include memory reference-related codes regarding which location (address address) of the internal or external memory of the computer should reference additional information or media necessary for the processor of the computer to execute the functions. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to send and receive during communication.

[0167] The above storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program can be stored in various recording media on various servers that the computer can access or in various recording media on the user's computer. In addition, the medium can be distributed across network-connected computer systems, so that computer-readable code can be stored in a distributed manner.

[0168] The steps of a method or algorithm described in connection with the embodiments of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer-readable recording medium well known in the art to which the present disclosure pertains.

[0169] While the embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will appreciate that the present disclosure can be implemented in other specific forms without altering the technical spirit or essential features thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

Claims

1. Memory storing the process for analyzing the movements of infants and toddlers; and Includes a processor for executing the above process, The above processor, Based on acceleration data measured from sensors attached to each of the multiple body parts of the infant or toddler, it is determined whether a movement has occurred for the multiple body parts. Based on the acceleration data measured for the plurality of body parts where the movement occurred according to the above-determined result, it is characterized in that it is determined whether the movement of the infant or toddler is normal. Electronic devices.

2. In paragraph 1, The above processor, In order to determine whether the above motion has occurred, the motion area is detected from the acceleration data based on a preset threshold value, Characterized in that it determines whether the movement of the infant or toddler is normal based on the acceleration data of the detected movement area. Electronic devices.

3. In paragraph 2, The above processor, A diagnostic result is output as to whether the movement of the infant or toddler is normal based on the type of abnormal movement, the time point and frequency of occurrence of the abnormal movement among the movements of the infant or toddler determined based on the acceleration data of the detected movement area. Electronic devices.

4. In paragraph 1, The above processor, Using a pre-learned artificial intelligence model, the normality of the infant's movements is determined based on the acceleration data. The above artificial intelligence model is, A method characterized by learning based on learning data including a general movement assessment (GMA) for multiple infants and toddlers, brain imaging findings for the multiple infants and toddlers, neurological examination findings, and occurrence of neurological diseases for the multiple infants and toddlers. Electronic devices.

5. In paragraph 4, The above processor, Characterized in that the above artificial intelligence model is used to predict the neurodevelopmental prognosis of the infant or toddler. Electronic devices.

6. In paragraph 1, Further comprising a camera for photographing the infant or toddler; The above processor, It is characterized in that it is checked whether the recognition result of the movement of the infant or toddler in the captured image of the camera and the measured acceleration data match at each preset time. Electronic devices.

7. In paragraph 1, The above processor, Among the acceleration data measured for the above multiple body parts, the acceleration data judged to be CS (Cramped-Synchronized) is classified, A method characterized in that the CS ratio is derived from the measured acceleration data based on the above classified results. Electronic devices.

8. A method for analyzing the movement of infants and toddlers, performed by a processor of an electronic device, A step of determining whether a movement occurs in each body part based on acceleration data measured from sensors attached to each of multiple body parts of an infant or toddler; and Based on the acceleration data measured for the plurality of body parts where the movement occurred according to the above judgment result, it is characterized in that it is determined whether the movement of the infant or toddler is normal. method.

9. In paragraph 8, The above normal judgment step is: A step of detecting a motion area from the acceleration data based on a preset threshold value to determine whether the motion has occurred; and A method characterized by comprising a step of determining whether the movement of the infant or toddler is normal based on acceleration data of the detected movement area. method.

10. In paragraph 9, The above normal judgment step is: A method characterized in that it further includes a step of outputting a diagnosis result on whether the movement of the infant or toddler is normal based on the type of abnormal movement, the time point and frequency of occurrence of the abnormal movement among the movements of the infant or toddler determined based on the acceleration data of the detected movement area. method.

11. In paragraph 9, The above normality judgment step uses a pre-learned artificial intelligence model to judge whether the infant's movements are normal based on the acceleration data. The above artificial intelligence model is, A method characterized by learning based on learning data including a general movement assessment (GMA) for multiple infants and toddlers, brain imaging findings for the multiple infants and toddlers, neurological examination findings, and occurrence of neurological diseases for the multiple infants and toddlers. method.

12. In paragraph 11, Characterized in that the above artificial intelligence model is used to predict the neurodevelopmental prognosis of the infant or toddler. method.

13. In paragraph 8, The step of photographing the infant or toddler through a camera; and It is characterized by further including a step of checking whether the recognition result of the movement of the infant or toddler in the captured image of the camera and the measured acceleration data match at each preset time. method.

14. In paragraph 8, A step of classifying acceleration data determined to be CS (Cramped-Synchronized) among acceleration data measured for the above multiple body parts; and A method characterized in that it further includes a step of deriving a CS ratio among the measured acceleration data based on the classified results. method.

Images