Device for analyzing movements of infants and toddlers and method for controlling the same

Abstract

The present disclosure relates to a motion analysis device for infants and toddlers, which determines whether motion occurs in each body part based on acceleration data measured from sensors attached to each of the infant's multiple body parts, and determines whether the motion of the infant is normal based on the acceleration data measured for the multiple body parts where motion is determined to have occurred according to the determination result.

Description

Technology Field

[0001] The present disclosure relates to a device for analyzing the movements of infants and toddlers. Background Technology

[0002] Numerous devices are widely available that allow parents and others to monitor the condition or behavior of infants and toddlers using audio and video. The primary purpose of these devices is to assess the child's situation in real time, enabling a rapid response to dangerous situations such as falls or overturns.

[0003] To overcome the limitations of simple monitoring in conventional technologies as described above, there is a need for technology that can continuously monitor the daily movements of infants and toddlers and analyze movement patterns to predict potential risks in advance; however, such technology is currently not publicly available. Prior art literature

[0004] Korean Patent Publication No. 10-2277967, (Registration Date: July 9, 2021) The problem to be solved

[0005] The purpose of the embodiments disclosed in this disclosure is to provide a device capable of analyzing the movements of infants and toddlers.

[0006] The problems that this disclosure aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below. means of solving the problem

[0007] An electronic device according to one embodiment of the present disclosure for solving the above-described problem comprises: a memory storing a process for analyzing the movement of an infant; and a processor for executing said process, wherein the processor determines whether movement has occurred in each body part based on acceleration data measured from sensors attached to each of a plurality of body parts of the infant, and determines whether the movement of said infant is normal based on said acceleration data measured for a plurality of body parts where movement is determined to have occurred according to the result of said determination.

[0008] In addition, the processor can detect an operating region in the acceleration data based on a preset threshold value to determine whether an operation has occurred, and determine whether the movement of the infant is normal based on the acceleration data of the detected operating region.

[0009] In addition, the processor can output a diagnosis result for the movement of the infant based on the type of abnormal movement, the timing and frequency of abnormal movement, and the acceleration data of the detected movement area.

[0010] In addition, the processor can determine whether the corresponding movement of the infant is normal or abnormal based on acceleration data corresponding to the operating area.

[0011] In addition, the processor determines whether the movement of the infant is normal based on the acceleration data using a pre-trained artificial intelligence model, and the artificial intelligence model is trained based on training data including General Movement Assessment (GMA) for a plurality of first infants, brain imaging (ultrasound, MRI) findings and neurological examination findings for the first infants, and whether a neurological disease has occurred in the first infants.

[0012] In addition, the processor can predict the neurodevelopmental prognosis of the infant using the artificial intelligence model.

[0013] Additionally, the electronic device further includes a camera for photographing the infant, and the processor can check at preset intervals whether the recognition result of the infant's movement in the captured video of the camera matches the measured acceleration data.

[0014] Additionally, the plurality of body parts include a left arm, a right arm, a left leg, and a right leg, and at least one sensor is attached to each of the infant's left arm, right arm, left leg, and right leg, and the processor can determine whether the infant's movement is normal based on acceleration data measured for two of the infant's left arm, right arm, left leg, and right leg.

[0015] In addition, the processor can classify acceleration data determined to be Cramped-Synchronized (CS) among the acceleration data measured for the plurality of body parts, and derive the CS ratio among the measured acceleration data based on the classified result.

[0016] In addition, a control method for an electronic device according to one embodiment of the present disclosure for solving the above-described problem is a method performed by a computing device, comprising: a step of determining whether motion occurs in each body part based on acceleration data measured from sensors attached to each of a plurality of body parts of an infant; and a step of determining whether the motion of the infant is normal based on the acceleration data measured for the plurality of body parts determined to have motion according to the determination result.

[0017] In addition, the computing device can detect an operating region in the acceleration data based on a preset threshold value to determine whether an operation has occurred, and can determine whether the movement of the infant is normal based on the acceleration data of the detected operating region.

[0018] Additionally, the method may further include a step of outputting a diagnosis result for the infant's movement based on the type of abnormal movement, the timing and frequency of the abnormal movement, and the acceleration data of the detected movement area.

[0019] In addition, the computing device can determine whether the corresponding movement of the infant is normal or abnormal based on acceleration data corresponding to the operating area.

[0020] In addition, the computing device determines whether the movement of the infant is normal based on the acceleration data using a pre-trained artificial intelligence model, and the artificial intelligence model is trained based on learning data including General Movement Assessment (GMA) for a plurality of first infants, brain imaging (ultrasound, MRI) findings and neurological examination findings for the first infants, and whether a neurological disease has occurred in the first infants.

[0021] In addition, the computing device can predict the neurodevelopmental prognosis of the infant using the artificial intelligence model.

[0022] Additionally, the computing device further includes a camera for photographing the infant, and the step of determining whether the motion occurs may further include a step of checking whether the recognition result of the movement of the infant within the captured video of the camera matches the measured acceleration data at preset time intervals.

[0023] Additionally, the plurality of body parts includes a left arm, a right arm, a left leg, and a right leg, and at least one sensor is attached to each of the infant's left arm, right arm, left leg, and right leg, and the step of determining normality can determine whether the movement of the infant is normal based on acceleration data measured for two of the infant's left arm, right arm, left leg, and right leg.

[0024] In addition, the computing device can classify acceleration data determined to be Cramped-Synchronized (CS) among the acceleration data measured for the plurality of body parts, and detect the CS ratio among the measured acceleration data based on the classified result.

[0025] In addition, a computer program stored on a computer-readable recording medium for executing a method for implementing the present disclosure may be further provided.

[0026] In addition, a computer-readable recording medium for recording a computer program for executing a method for implementing the present disclosure may be further provided. Effects of the invention

[0027] According to the aforementioned means for solving the problem of the present disclosure, the effect of being able to analyze the movements of infants and toddlers is provided.

[0028] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0029] FIG. 1 is a schematic diagram of a movement analysis system for infants and toddlers according to an embodiment of the present disclosure. Figure 2 is a diagram illustrating sensors attached to each body part of the infant in Figure 1. FIG. 3 is a block diagram of a motion analysis device for infants and toddlers according to an embodiment of the present disclosure. FIG. 4 is a flowchart of a method for analyzing the movement of an infant according to an embodiment of the present disclosure. Figure 5 is a diagram illustrating a process in which a processor analyzes and classifies acceleration data. Figure 6 is a diagram illustrating a process in which a processor analyzes acceleration data of raw data. Figure 7 is a diagram illustrating the detection of an operating region from acceleration data in Figure 6. Figure 8 is a diagram illustrating the application of a composite filter for similarity determination in Figure 6. Figure 9 is a diagram illustrating the use of the Euclidean distance method for similarity determination in Figure 6. Figure 10 is a diagram illustrating the use of a dynamic time joining technique for similarity determination in Figure 6. FIG. 10 is a diagram illustrating the synchronization area shown in each operation area of ​​FIG. 6. Figure 11 is a diagram illustrating the detection of ternary data. Specific details for implementing the invention

[0030] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and general content in the art to which this disclosure pertains or content that overlaps between embodiments is omitted. The terms 'part, module, component, block' as used in the specification may be implemented in software or hardware, and depending on the embodiments, a plurality of 'parts, modules, components, blocks' may be implemented as a single component, or a single 'part, module, component, block' may include a plurality of components.

[0031] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are directly connected but also cases where they are indirectly connected, and indirect connections include connections made via a wireless communication network.

[0032] Furthermore, when it is stated that a part "includes" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0033] Throughout the specification, when it is stated that a component is located "on" another component, this includes not only cases where a component is in contact with another component, but also cases where another component exists between the two components.

[0034] Terms such as "first," "second," etc., are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0035] Singular expressions include plural expressions unless there is an obvious exception in the context.

[0036] In each step, identification codes are used for convenience of explanation and do not describe the order of the steps; the steps may be performed differently from the specified order unless a specific order is clearly indicated in the context.

[0037] The operating principles and embodiments of the present disclosure will be described below with reference to the attached drawings.

[0038] In this specification, the term "electronic device according to the present disclosure" includes all various devices capable of performing computational processing and providing results to a user. For example, the electronic device according to the present disclosure may include all of a computer, a server device, and a portable terminal, or may take the form of any one of them.

[0039] Here, the computer may include, for example, a laptop, desktop, notebook, tablet PC, slate PC, etc. equipped with a web browser.

[0040] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0041] The above portable terminal may include, for example, all types of handheld-based wireless communication devices such as PCS, GSM, PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smartphones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs).

[0042] The artificial intelligence-related functions according to the present disclosure are operated through a processor and a storage unit. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in the storage unit. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0043] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a basic artificial intelligence model is trained using a number of training data by a learning algorithm, thereby creating predefined rules of operation or artificial intelligence models configured to perform desired characteristics (or objectives). Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0044] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weights and performs neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights can be updated so that the loss value or cost value obtained by the artificial intelligence model is reduced or minimized during the learning process. Artificial neural networks may include deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), or Deep Q-Networks, but are not limited to the examples mentioned above.

[0045] According to an exemplary embodiment of the present disclosure, a processor can implement artificial intelligence. Artificial intelligence refers to a machine learning method based on artificial neural networks that enables a machine to learn by mimicking human biological neurons. Methodologies of artificial intelligence can be classified according to the learning method into supervised learning, where input data and output data are provided together as training data and the solution (output data) to the problem (input data) is predetermined; unsupervised learning, where only input data is provided without output data and the solution (output data) to the problem (input data) is not predetermined; and reinforcement learning, where a reward is given from the external environment whenever an action is taken in the current state, and learning proceeds in a direction that maximizes such reward. Additionally, methodologies of artificial intelligence may be classified according to the architecture, which is the structure of the learning model. The architectures of widely used deep learning technologies can be classified into convolutional neural networks, recurrent neural networks, transformers, generative adversarial networks, etc.

[0046] The device may include an artificial intelligence model. The artificial intelligence model may be a single model or may be implemented as multiple models. The artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model that possesses problem-solving capabilities by having artificial neurons (nodes) that form a network through synaptic connections and change the strength of synaptic connections through learning. The neurons of a neural network may include combinations of weights or biases. A neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a result to be predicted from an arbitrary input by changing the weights of the neurons through learning.

[0047] The processor can create a neural network, train or learn a neural network, perform operations based on received input data, generate information signals based on the results of the operations, or retrain the neural network. The neural network models may include, but are not limited to, various types of models such as CNN, R-CNN, RPN, RNN, S-DNN, S-SDNN, Deconvolution Network, DBN, RBM, Fully Convolutional Network, LSTM Network, and Classification Network, such as GoogleNet, AlexNet, and VGG Network. The processor may include one or more processors for performing operations according to the neural network models. For example, the neural network may include a deep neural network.

[0048] Neural networks include CNN, RNN, percept, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network), LSM (Liquid State Machine), ELM (Extreme Learning) Machine), ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Neural Network) A person skilled in the art will understand that any neural network may be included, but is not limited to, Computer (NTM), Neural Turning Machine (NTM), Capsule Network (CN), Kohonen Network (KN), and Attention Network (AN).

[0049] According to an exemplary embodiment of the present disclosure, the processor comprises a Convolutional Neural Network (CNN) such as GoogleNet, AlexNet, VGG Network, Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restructured Boltzmann Machine (RBM), Fully Convolutional Network, Long Short-Term Memory (LSTM) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA, Text Analysis, Dialog System, GPT-3, GPT-4 for Natural Language Processing, Visual Analytics, Visual Understanding, Video Synthesis for Vision Processing, Anomaly Detection, Prediction, Time-Series Forecasting, Optimization for ResNet Data Intelligence, Various artificial intelligence structures and algorithms, such as recommendation and data creation, may be used, but are not limited thereto. Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0050] FIG. 1 is a schematic diagram of a movement analysis system for infants and toddlers according to an embodiment of the present disclosure.

[0051] Figure 2 is a diagram illustrating sensors attached to each body part of the infant in Figure 1.

[0052] However, in some embodiments, the system (10) may include fewer or more components than the components shown in FIG. 1.

[0053] Referring to FIG. 1, the movement analysis system (10) of an infant according to an embodiment of the present disclosure can receive acceleration data (30) measured from sensors attached to each body part of an infant (20) and analyze the data to output diagnostic data (40) including a result of determining whether the movement of the infant (20) is normal.

[0054] As shown in FIG. 2, at least one sensor may be attached to each body part of an infant, and as illustrated in the drawing, the sensor part (140) may include a sensor (141) attached to the left arm, a sensor (142) attached to the right arm, a sensor (143) attached to the left leg, a sensor (144) attached to the right leg, a sensor (145) attached to the head, and a sensor (146) attached to the torso.

[0055] At this time, multiple sensors may be attached to each body part to improve measurement accuracy.

[0056] Referring to FIG. 2, according to an embodiment, the electronic device (100) may further include a shooting unit (160) including at least one camera for shooting an infant, and the processor (110) of the electronic device (100) may compare the image captured through the camera with acceleration data measured and received from a sensor.

[0057] The processor (110) of the electronic device (100) can check whether the recognition result of the movement of the infant in the captured video of the camera and the acceleration data measured through the sensor unit (140) match at preset times.

[0058] Below, with reference to other drawings, a motion analysis device for infants and toddlers, a control method for the device, and a program according to an embodiment of the present disclosure will be described in more detail.

[0059] FIG. 3 is a block diagram of a motion analysis device for infants and toddlers according to an embodiment of the present disclosure.

[0060] Referring to FIG. 3, the motion analysis device for infants according to an embodiment of the present disclosure includes a processor (110), a communication unit (120), a memory (130), a sensor unit (140), a detection unit (150), a shooting unit (160), and an output unit (170).

[0061] However, in some embodiments, the electronic device (100) may include fewer or more components than the components shown in FIG. 3.

[0062] The processor (110) may be implemented as a storage unit that stores data for an algorithm or a program that reproduces the algorithm for controlling the operation of components within the device, and at least one processor (110) that performs the aforementioned operation using the data stored in the storage unit. In this case, the storage unit and the processor (110) may each be implemented as separate chips. Alternatively, the storage unit and the processor (110) may be implemented as a single chip.

[0063] In addition, the processor (110) can control one or a combination of the components described above in order to implement various embodiments according to the present disclosure described in the drawings below on the device.

[0064] In addition to operations related to the application, the processor (110) can generally control the overall operation of the device. The processor (110) can provide or process appropriate information or functions to the user by processing signals, data, information, etc. that are input or output through the components described above, or by running an application stored in the storage unit.

[0065] Additionally, the processor (110) can control at least some of the components of the device to run an application stored in the storage unit. Furthermore, the processor (110) can operate at least two or more of the components included in the device in combination with each other to run the application.

[0066] The processor (110) may be implemented in one or more ways. Hereinafter, even if the processor (110) is expressed in the singular, it may be considered as plural. The processor (110) can control the configurations of the electronic device (100). The processor (110) may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by code or instructions included in a program. As such, the processor (110) may encompass processing devices such as a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA) as examples of data processing devices embedded in hardware, but the scope of the present invention is not limited thereto. The processor (110) may separately provide a learning processor (110) for performing artificial intelligence computations, or it may provide a learning processor (110) itself.

[0067] In various embodiments, the processor (110) may include one or more of a Central Processing Unit (CPU), an Application Processor (AP), or a Communication Processor (CP). At least a portion of the processor (110) may be hardware capable of accessing memory (130) and performing functions related to instructions stored in memory (130).

[0068] The communication unit (120) may include one or more modules that connect the electronic device (100) to one or more networks.

[0069] The communication unit (120) may include one or more components that enable communication with an external device, for example, at least one of a broadcast receiving module, a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0070] The wired communication module may include various wired communication modules such as a Local Area Network (LAN) module, a Wide Area Network (WAN) module, or a Value Added Network (VAN) module, as well as various cable communication modules such as USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), DVI (Digital Visual Interface), RS-232 (recommended standard 232), power line communication, or POTS (plain old telephone service).

[0071] In addition to Wi-Fi modules and WiBro (Wireless broadband) modules, the wireless communication module may include wireless communication modules that support various wireless communication methods such as GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G.

[0072] The wireless communication module may include a wireless communication interface comprising an antenna and a transmitter that transmit a communication signal. Additionally, the wireless communication module may further include a signal conversion module that modulates a digital control signal output from the processor (110) through the wireless communication interface into an analog wireless signal under the control of the processor (110).

[0073] A short-range communication module is for short-range communication and can support short-range communication by using at least one of Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0074] The communication unit (120) may use the name of the communication interface.

[0075] A communication interface can establish communication between an electronic device (100) and an external device. For example, the communication interface can communicate with the external device via wireless communication (e.g., Wi-Fi (Wireless Fidelity), Bluetooth, NFC (Near Field Communication), MST (magnetic stripe transmission), etc.) or wired communication.

[0076] The memory (130) can store data that supports various functions of the device. The memory (130) can store a number of application programs (or applications) running on the device, data for the operation of the device, and instructions. At least some of these application programs may exist for the basic functions of the device. Meanwhile, the application programs may be stored in the memory (130), installed on the device, and driven by the processor (110) to perform operations (or functions).

[0077] The memory (130) can store data supporting various functions of the device and programs for the operation of the processor (110), and can store input / output data (e.g., music files, still images, videos, etc.), and can store a number of application programs (or applications) running on the device, data for the operation of the device, and instructions. At least some of these application programs can be downloaded from an external server via wireless communication.

[0078] The memory (130) may include at least one type of storage medium among flash memory (130) type, hard disk type, SSD type (Solid State Disk type), SSD type (Silicon Disk Drive type), multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (random access memory; RAM), SRAM (static random access memory), ROM (read-only memory; ROM), EEPROM (electrically erasable programmable read-only memory), PROM (programmable read-only memory), magnetic memory, magnetic disk, and optical disk. Additionally, the memory (130) may be a database that is separated from the device but connected via wired or wireless connection.

[0079] The memory (130) can be electrically connected to the processor (110) and can store at least one code executed by the processor (110). The memory (130) can refer to various types of storage devices. The memory (130) can store information necessary to perform calculations using artificial intelligence, machine learning, or artificial neural networks.

[0080] The memory (130) can store various learning models. The learning models stored in the memory (130) can infer result values ​​for new input data other than the learning data, and the inferred values ​​can be used as a basis for judgment to perform an action. Learning can be performed on the learning models stored in the memory (130) based on label information, and various backpropagation algorithms can be applied so that the loss function has a target value in order to increase the accuracy of the learning.

[0081] Additionally, the memory (130) may have a plurality of processes for the electronic device (100).

[0082] The sensor unit (140) may include at least one sensor attached to each body part of the infant as described with reference to FIG. 2, and may transmit the measured acceleration data to the communication unit (120) of the electronic device (100) via wired / wireless communication.

[0083] The sensor unit (140) senses at least one of internal information of the device, surrounding environment information surrounding the device, and user information, and generates a corresponding sensing signal. Based on this sensing signal, the processor (110) can control the operation or function of the device, or perform data processing, functions, or operations related to an application program installed on the device.

[0084] The sensor unit (140) as described above may include at least one of a proximity sensor, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, a gravity sensor (G-sensor), a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor (IR sensor: infrared sensor), a fingerprint sensor, an ultrasonic sensor, an optical sensor (e.g., a camera), a microphone, an environmental sensor (e.g., including at least one of a barometer, a hygrometer, a thermometer, a radiation detection sensor, a heat detection sensor, a gas detection sensor), and a chemical sensor (e.g., a healthcare sensor, a biometric sensor, etc.). Meanwhile, the device may combine and utilize information sensed from at least two of these sensors.

[0085] The detection unit (150) can detect a motion area in which the infant is determined to have actually moved from acceleration data measured and received through the sensor unit (140). To this end, the memory (130) may store a threshold value and an algorithm for detecting the motion area.

[0086] The shooting unit (160) may include at least one camera for shooting infants and toddlers, and may transmit the captured image to the communication unit (120) of the electronic device (100).

[0087] The camera processes image frames, such as still images or video, obtained by the image sensor in shooting mode. The processed image frames may be displayed on or stored in a storage unit.

[0088] The camera can capture still images and video. For example, when the electronic device (100) is mounted on the HMD device, the camera can capture images of at least the front area of ​​the HMD device. In one embodiment, the camera may be activated after a specified time has elapsed since the start of operation of the HMD device, after the electronic device (100) is mounted on the HMD device. In various embodiments, the camera may be activated from the time the electronic device (100) is mounted on the HMD device. Alternatively, the camera may be activated from the time the user wears the HMD device.

[0089] In various embodiments, the camera may include, for example, at least one depth camera (e.g., Time Of Flight (TOF) method or structured light method) and a color camera (e.g., RGB camera). Additionally, the camera may further include at least one sensor (e.g., proximity sensor), a light source (e.g., LED array), etc., in relation to performing the function. In various embodiments, the at least one sensor may be configured as a module separate from the camera and may perform sensing of at least the front area of ​​the HMD device. For example, the sensor (e.g., proximity sensor) module may perform sensing of an object by emitting infrared light (or transmitting ultrasound) to the front area of ​​the HMD device and receiving infrared light (or receiving ultrasound) reflected back from the object. In this case, the camera may be activated from the point when at least one object is sensed by the sensor module.

[0090] The output unit (170) is intended to generate output related to sight, hearing, or touch, and may include at least one of a display unit, an audio output unit (170), a haptic module, and an optical output unit (170). The display unit may form a layered structure with a touch sensor or be formed integrally to implement a touch screen. Such a touch screen functions as a user input unit that provides an input interface between the device and the user, and at the same time can provide an output interface between the device and the user.

[0091] The processor (110) can output the result of judgment and diagnosis regarding the movement of the infant through the output unit (170).

[0092] However, the output unit (170) is not necessarily required to be included in the configuration of the electronic device (100), and the electronic device (100) may transmit and provide the judgment result and diagnosis result determined based on the movement of the infant to the infant's guardian's terminal or transmit and provide it to the medical staff's device.

[0093] In some embodiments, the electronic device (100) may further include components such as an input section and an interface section as follows.

[0094] The input unit is for inputting video information (or signal), audio information (or signal), data, or information input by a user, and may include at least one of at least one camera, at least one microphone, and a user input unit. Voice data or image data collected by the input unit may be analyzed and processed into a user control command.

[0095] The input unit is intended to receive information from a user, and when information is input through the input unit, the processor (110) can control the operation of the device to correspond to the input information. Such an input unit may include a hardware physical key (e.g., a button, dome switch, jog wheel, jog switch, etc. located on at least one of the front, rear, and side of the device) and a software touch key. As an example, the touch key may be composed of a virtual key, soft key, or visual key displayed on a touchscreen-type display unit through software processing, or may be composed of a touch key placed on a part other than the touchscreen. Meanwhile, the virtual key or visual key may be displayed on the touchscreen in various forms, and may be composed of, for example, a graphic, text, an icon, a video, or a combination thereof.

[0096] The interface section serves as a passage for various types of external devices connected to the present device. This interface section may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with a SIM, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port. The present device can perform appropriate control related to the external device connected to the interface section.

[0097] The input / output interface receives commands or data input from a user or other external device

[0098] It can serve as an interface capable of transmitting to other components of the electronic device (100). Additionally, the input / output interface can output commands or data received from other components of the electronic device (100) to a user or another external device.

[0099] FIG. 4 is a flowchart of a method for analyzing the movement of an infant according to an embodiment of the present disclosure.

[0100] FIGS. 5 to 11 are various exemplary drawings for explaining an infant movement analysis device, method, and program according to an embodiment of the present disclosure.

[0101] With reference to FIGS. 4 to 11, a movement analysis device, method, and program for infants and toddlers according to an embodiment of the present disclosure will be described.

[0102] The processor (110) receives acceleration data measured by the sensor unit (140). (S410)

[0103] In an embodiment of the present disclosure, the sensor portion (140) includes at least one sensor attached to each of a plurality of body parts of an infant.

[0104] Multiple body parts include the left arm, right arm, left leg, and right leg. However, this is one embodiment and the body parts of the infant for measuring acceleration are not limited thereto.

[0105] That is, at least one sensor is attached to each of the infant's left arm, right arm, left leg, and right leg, and acceleration data measured from each sensor can be transmitted to an electronic device (100) through a communication unit (120).

[0106] The processor (110) determines whether motion occurs in each body part of the infant based on acceleration data received from S410. (S420)

[0107] The processor (110) checks whether an operating region is detected in the acceleration data. (S430)

[0108] The processor (110) can determine whether motion occurs in each body part based on acceleration data measured from sensors attached to each of the multiple body parts of the infant.

[0109] The processor (110) can detect an operating region in acceleration data based on a preset threshold value to determine whether an operation has occurred.

[0110] In this case, the pre-set threshold value may be set differently depending on the information of the infant.

[0111] The processor (110) determines whether the movement of the infant is normal based on acceleration data of the movement area. (S440)

[0112] The processor (110) can determine whether the movement of an infant is normal based on acceleration data measured for multiple body parts where the movement is determined to have occurred according to the judgment result of S430.

[0113] In one embodiment, the processor (110) can determine whether the corresponding movement of the infant is normal or abnormal based on acceleration data corresponding to the movement area.

[0114] At this time, the processor (110) can determine whether the movement of the infant is normal based on acceleration data measured for two body parts where the movement is determined to have occurred.

[0115] In detail, the processor (110) can determine whether the movement of the infant is normal based on acceleration data measured for two body parts of the infant's left arm, right arm, left leg, and right leg.

[0116] The processor (110) outputs a diagnosis result regarding the movement of the infant. (S450)

[0117] In detail, the processor (110) can output a diagnosis result for the infant's movement based on the type of abnormal movement, the time and frequency of abnormal movement occurring among the infant's movements, based on acceleration data of the detected movement area.

[0118] In one embodiment, the electronic device (100) can diagnose an infant using artificial intelligence.

[0119] More specifically, using a pre-trained artificial intelligence model, it is possible to determine whether the movement of the infant is normal based on the acceleration data.

[0120] At this time, the artificial intelligence model is characterized by being trained based on learning data that includes at least one of a General Movement Assessment (GMA) for a plurality of first infants, brain imaging (ultrasound, MRI) findings and neurological examination findings for the first infants, and whether a neurological disease has occurred in the first infants.

[0121] The artificial intelligence model can analyze the video of the first infant received along with the general motion assessment (GMA) to calculate the acceleration of each body part according to the movement pattern within the video of the first infant, and learn the general motion assessment and the calculated acceleration data. The processor (110) can measure the acceleration of each body part based on the 3D position coordinates and frame rate of each body part of the first infant from multiple video frames in which the first infant is filmed.

[0122] That is, the processor (110) inputs acceleration data of an infant into an artificial intelligence model learned in this way and can obtain the likelihood of neurological disease development in the infant.

[0123] The processor (110) can input acceleration data of an infant into an artificial intelligence model learned in this way and obtain a predicted value for the neurodevelopmental prognosis of the infant as an output value.

[0124] Additionally, the processor (110) can input acceleration data of an infant into an artificial intelligence model learned in this way and obtain the probability of the infant developing each of a plurality of neurological diseases as an output value.

[0125] FIG. 5 is a diagram illustrating a process in which a processor (110) analyzes and classifies acceleration data.

[0126] Referring to FIG. 5, the processor (110) performs a process (510) of analyzing acceleration data determined to be in an operating region from acceleration data measured and received through the sensor unit (140), and classifies acceleration data determined to be abnormal motion.

[0127] And, the processor (110) inputs the classified acceleration data into the first classifier (520) to classify it into three types: PR (Poor Repertoire), CS (Cramped Synchronized), and CH (Chaotic).

[0128] And finally, the processor (110) inputs the classified data into the second classifier (530) to derive the percentage (%) of acceleration data that can be identified as CS.

[0129] The CS ratio (540) derived in this way can be used by medical staff as a guide to determine whether the newborn's movements correspond to CS.

[0130] The second classifier (530) may be a CS classifier and may target data combinations for two body parts among the left arm, right arm, left leg and right leg.

[0131] FIG. 6 is a diagram illustrating a process in which a processor (110) analyzes acceleration data of raw data.

[0132] The processor (110) can classify raw data (610) received from the sensor unit (140) into first data (620) and second data (630), perform filtering on the first data (620) at a level not distorted by a filter such as a moving average, and perform a process (640) to detect an operating area, and perform filtering on the second data (630) at a level not distorted by a filter such as a moving average, and perform a process (650) to detect an operating area.

[0133] Then, a process (670) is performed to filter the first data (620) and the second data (630), extract similarity using a similarity judgment tool, and determine the synchronized section.

[0134] Next, the processor (110) performs a result analysis process (680) based on the results obtained by performing processes 640, 650, and 670.

[0135] Figure 7 is a diagram illustrating the detection of an operating region from acceleration data in Figure 6.

[0136] (A) of FIG. 7 is an example of acceleration data measured and received from a sensor attached to the left leg of the target, and (B) is an example of acceleration data measured and received from a sensor attached to the right leg of the target.

[0137] As shown in FIG. 7, the processor (110) detects as an operating area an area having acceleration exceeding a preset threshold value (5 in FIG. 7) in the acceleration data measured and received from the sensor unit (140).

[0138] In the embodiments of the present disclosure, one or more similarity judgment tools most suitable among various composite filters may be applied for similarity judgment.

[0139] For example, the electronic device (100) can perform similarity analysis based on image analysis.

[0140] The processor (110) can generate an image based on acceleration data obtained for two different body parts and perform similarity analysis by comparing the two images.

[0141] In one embodiment, the processor (110) determines similarity based on the luminance, contrast, and structural differences of two images using the SSIM (Structural Similarity Index Measure) algorithm.

[0142] Figure 8 is a diagram illustrating the application of a composite filter for similarity determination in Figure 6.

[0143] Figure 9 is a diagram illustrating the use of the Euclidean distance method for similarity determination in Figure 6.

[0144] Figure 8 (A) illustrates the application of a moving average and a bandpass filter to acceleration data measured and received from a sensor attached to the left leg of the target, and (B) illustrates the application of a moving average and a bandpass filter to acceleration data measured and received from a sensor attached to the right leg of the target.

[0145] The processor (110) can apply a composite filter (such as an N-layered Moving Average) that continuously applies a moving average with different window sizes, and then apply a bandpass filter (BPF) to remove high-frequency noise and low-frequency offset components. Afterward, the processor (110) can derive / calculate similarity by applying a similarity measurement method, such as the Euclidean Distance Method shown in FIG. 9, to the first data and the second data.

[0146] In Fig. 9 (A), a time-series graph (910) of high similarity for each data is exemplified, and a time-series difference value (920) for it is shown.

[0147] In Fig. 9 (B), a time-series graph (930) of Low similarity for each data is exemplified, and a time-series difference value (940) is shown.

[0148] The processor (110) can measure the similarity between two time series data using a dynamic time warping (DTW) technique.

[0149] FIG. 10 is a diagram illustrating the synchronization area shown in each operation area of ​​FIG. 6.

[0150] Figure 11 is a diagram illustrating the derivation of ternary data.

[0151] Referring to FIG. 11, a ternary conversion is illustrated, and the processor (110) is illustrated as in FIG. 11, with Act + sync (1210), Synchro Section (1220), and Action Section (1230) each plotted on a graph. To facilitate visual distinction, levels 0, 1, and 2 are plotted as levels 1, 3, and 6, respectively.

[0152] After performing the above processes, the processor (110) can set up a synchronization area as shown in the Synchro Section (1110) of FIG. 10.

[0153] And, the processor (110) can convert three types of binary trajectory data, such as Fig. 10, into ternary data according to the following criteria.

[0154] The processor (110) can convert the remaining data into ternary data, excluding the data determined to be normal from the acceleration data measured through the sensor unit (140).

[0155] ① Data i Operation Area F & Synchronization Area F

[0156] ② Data i Operation Area T & Synchronization Area F

[0157] ③ Data i Operation Area T & Synchronization Area T

[0158] (Here, i = 1, 2, T means True, and F means False.)

[0159] The ternary data can be derived for the first data and the second data as shown in FIG. 11. The processor (110) can obtain a percentage of each ternary data from this conversion, and based on this percentage, can quantitatively determine how close the operation of the upper and lower limbs (or other measurement parts) that are the input of the second classifier is to CS or, conversely, to Normal.

[0160] The method according to one embodiment of the present disclosure described above may be implemented as a program (or application) and stored on a medium to be executed in combination with a server, which is hardware.

[0161] The aforementioned program may include code encoded in a computer language such as C, C++, JAVA, or machine language, which can be read by the computer's processor (CPU) through the computer's device interface, in order for the computer to read the program and execute the methods implemented in the program. Such code may include functional code related to functions that define the necessary functions for executing the methods, and may include control code related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure. Additionally, such code may further include memory reference code regarding where (address) additional information or media necessary for the computer's processor to execute the functions should be referenced in the computer's internal or external memory. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the above functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to transmit or receive during communication.

[0162] The above-mentioned storage medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the above-mentioned storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the above-mentioned program may be stored on various recording media on various servers that the computer can access, or on various recording media on the user's computer. Additionally, the above-mentioned medium may be distributed across networked computer systems, and computer-readable code may be stored in a distributed manner.

[0163] The steps of the method or algorithm described in connection with the embodiments of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium well known in the art to which the present disclosure belongs.

[0164] Although embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will understand that the present disclosure may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. Explanation of the symbols

[0165] 10: System 100: Electronic device 110: Processor 120: Communications Department 130: Memory 140: Sensor section 150: Detector 160: Filming Department 170: Output section

Claims

Claim 1 An electronic device comprising: a memory storing a process for analyzing the movement of an infant; and a processor for executing said process, wherein the processor determines whether movement occurs in each body part based on acceleration data measured from sensors attached to each of the infant's multiple body parts, determines whether the movement of said infant is normal based on said acceleration data measured for the multiple body parts determined to have occurred according to said determination result, detects the section where actual movement occurs as an action section from the acceleration data determined to have occurred, determines the similarity between said acceleration data measured in different body parts by determining the structural similarity between images generated based on said acceleration data, determines the similarity between said acceleration data measured in different body parts, determines the section where the movements of said multiple body parts are synchronized with each other as a synchro section using said similarity determination result, converts said acceleration data into ternary data using said action section and said synchro section determination result, and calculates the proportion of said data corresponding to CS (Cramped Synchronized) having abnormal movement characteristics among said ternary data. Claim 2 An electronic device according to claim 1, wherein the processor detects an operating region in the acceleration data based on a preset threshold value to determine whether an operation occurs, and determines whether the operation of the infant is normal based on the acceleration data of the detected operating region. Claim 3 An electronic device according to paragraph 2, wherein the processor outputs a diagnosis result for the movement of the infant based on the type of abnormal movement, the time and frequency at which the abnormal movement occurs, and the acceleration data of the detected movement area. Claim 4 delete Claim 5 An electronic device according to claim 1, wherein the processor determines whether the movement of the infant is normal based on the acceleration data using a pre-trained artificial intelligence model, and the artificial intelligence model is characterized by being trained based on learning data including a General Movement Assessment (GMA) for a plurality of first infants, brain imaging (ultrasound, MRI) findings and neurological examination findings for the first infants, and whether a neurological disease has occurred in the first infants. Claim 6 An electronic device according to claim 5, wherein the processor predicts the neurodevelopmental prognosis of the infant using the artificial intelligence model. Claim 7 An electronic device according to claim 1, wherein the electronic device further includes a camera for photographing the infant, and the processor checks whether the recognition result of the movement of the infant within the captured image of the camera matches the measured acceleration data at preset intervals. Claim 8 An electronic device according to claim 1, wherein the plurality of body parts includes a left arm, a right arm, a left leg, and a right leg, and at least one sensor is attached to each of the infant's left arm, right arm, left leg, and right leg, and the processor determines whether the movement of the infant is normal based on acceleration data measured for two of the infant's left arm, right arm, left leg, and right leg. Claim 9 delete Claim 10 A method for controlling an electronic device, comprising: a step of determining whether motion occurs in each body part based on acceleration data measured from sensors attached to each of a plurality of body parts of an infant based on the result of the determination; and a step of determining whether the motion of the infant is normal based on the acceleration data measured for the plurality of body parts determined to have motion according to the result of the determination, wherein the computing device detects the section where actual movement occurs as an action section from the acceleration data determined to have motion, determines the similarity between the acceleration data measured in the different body parts by determining the structural similarity between images generated based on the acceleration data measured in the different body parts, determines the section where the movements of the plurality of body parts are synchronized with each other as a synchro section using the result of the similarity determination, converts the acceleration data into ternary data using the result of the determination regarding the action section and the synchro section, and calculates the proportion of data corresponding to CS (Cramped Synchronized) having abnormal motion characteristics among the ternary data. Claim 11 A method for controlling an electronic device according to claim 10, wherein the computing device detects an operating region in the acceleration data based on a preset threshold value to determine whether an operation occurs, and determines whether the operation of the infant is normal based on the acceleration data of the detected operating region. Claim 12 A method for controlling an electronic device according to claim 11, further comprising the step of outputting a diagnostic result for the movement of the infant based on the type of abnormal movement, the time and frequency at which the abnormal movement occurs, based on acceleration data of the detected movement area. Claim 13 delete Claim 14 A method for controlling an electronic device according to claim 10, wherein the computing device determines whether the movement of the infant is normal based on the acceleration data using a pre-trained artificial intelligence model, and the artificial intelligence model is characterized by being trained based on learning data including a General Movement Assessment (GMA) for a plurality of first infants, brain imaging (ultrasound, MRI) findings and neurological examination findings for the first infants, and whether a neurological disease has occurred in the first infants. Claim 15 A method for controlling an electronic device according to claim 14, wherein the computing device predicts the neurodevelopmental prognosis of the infant using the artificial intelligence model. Claim 16 A method for controlling an electronic device according to claim 10, wherein the computing device further includes a camera for photographing the infant, and the step of determining whether the operation occurs further includes a step of checking whether the recognition result of the movement of the infant within the captured image of the camera matches the measured acceleration data at preset time intervals. Claim 17 A method for controlling an electronic device according to claim 10, wherein the plurality of body parts include a left arm, a right arm, a left leg, and a right leg, and at least one sensor is attached to each of the infant's left arm, right arm, left leg, and right leg, and the step of determining normality is characterized by determining normality of the infant's movement based on acceleration data measured for two of the infant's left arm, right arm, left leg, and right leg. Claim 18 delete Claim 19 A computer program stored on a computer-readable storage medium to execute the method of any one of claims 10 to 12 and claims 14 to 17 using a computing device.

Images