Edge sysetm and method for analyzing infant behavior

Abstract

An edge system for analyzing the behavior of infants and toddlers is provided. The system comprises a communication unit that receives a plurality of temporally consecutive video frames containing at least one object among an infant or toddler and a caregiver, a memory in which a program for analyzing the video frames to generate behavior analysis result information of the infant or toddler is stored, and a processor that executes the program stored in the memory. The processor, upon executing the program, tracks an object recognized from the video frames to estimate posture information, estimates behavior information based on the posture information estimated from the plurality of video frames, generates behavior analysis result information by analyzing the behavior information based on predetermined conditions, and provides it to a user terminal corresponding to the object.

Description

Technology Field

[0001] The present invention relates to an edge system and method for analyzing the behavior of infants and toddlers, and in particular, to analyzing the behavior of infants and toddlers through video footage acquired from CCTVs that are mandatory in childcare institutions for infants and toddlers, such as daycare centers and kindergartens. Background Technology

[0002] Childcare institutions such as daycare centers and kindergartens are required to install video information processors (CCTV and NVR (Network Video Recorder)) for purposes such as managing safety accidents involving infants and toddlers, monitoring situations of violence between teachers and infants and toddlers, and managing internal and external facilities (Ministry of Health and Welfare, Amendment to the Infant and Toddler Care Act, 2015). CCTV captures video, and NVR receives CCTV video via a network or coaxial cable method to store the video, and when a specific incident occurs, it is used by an administrator to verify the incident by searching and playing the video.

[0003] Meanwhile, as artificial intelligence technology advances, the technology for recognizing human behavior from CCTV footage is also advancing. However, in actual childcare facilities, CCTV footage is currently utilized merely to verify whether childcare workers have committed violence against infants and toddlers; the reality is that it is not being used to recognize children or analyze their behavior by integrating AI technology. Prior art literature

[0004] Registered Patent Publication No. 10-2459894 (October 24, 2022) The problem to be solved

[0005] The problem that the present invention aims to solve is to provide an edge system and method for analyzing the behavior of infants and toddlers, which analyzes specific behaviors of infants and toddlers from CCTV footage based on a pre-trained artificial intelligence model, reprocesses the analysis results, and provides them to a user.

[0006] However, the problems that the present invention aims to solve are not limited to those described above, and other problems may exist. means of solving the problem

[0007] An edge system for analyzing the behavior of infants and toddlers according to a first aspect of the present invention for solving the aforementioned problem comprises a communication unit that receives a plurality of temporally consecutive video frames containing at least one object among an infant / toddler and a caregiver, a memory in which a program for analyzing the video frames to generate behavior analysis result information of the infant / toddler is stored, and a processor that executes the program stored in the memory. At this time, as the processor executes the program, it tracks an object recognized from the video frames to estimate posture information, estimates behavior information based on the posture information estimated from the plurality of video frames, generates behavior analysis result information by analyzing the behavior information based on predetermined conditions, and provides it to a user terminal corresponding to the object.

[0008] In some embodiments of the present invention, the processor can estimate pose information by defining feature points based on each joint point of the recognized object.

[0009] In some embodiments of the present invention, the processor can estimate behavior information including at least one of a solitary behavior of the recognized object, a dangerous behavior, and an interactive behavior in which the recognized object interacts with other objects.

[0010] In some embodiments of the present invention, the processor may analyze whether there is a secondary interaction based on the unique ID of a plurality of objects, the type of behavior, the location of the behavior, and the distance information between a plurality of objects, with respect to at least one estimated behavior information.

[0011] In some embodiments of the present invention, the processor may secondarily analyze whether there is an interaction based on behavior time information, including the degree of agreement between the start and end time intervals of a plurality of objects' behaviors and whether the duration of the behavior exceeding a predetermined threshold is satisfied, with respect to the estimated behavior information.

[0012] In some embodiments of the present invention, the duration of the action may be predefined for each action information and stored in a database.

[0013] In some embodiments of the present invention, the processor may extract the unique ID, type of behavior, whether play, start time of behavior, and end time of behavior of a plurality of objects from the estimated behavior information and store them in a database.

[0014] In some embodiments of the present invention, the processor may generate behavior analysis result information by aggregating the estimated behavior information according to preset time units.

[0015] In some embodiments of the present invention, the processor may provide the result of comparing the behavior information of the recognized infant object with the behavior information of other infants or their average behavior information as the behavior analysis result information.

[0016] A computer program according to another aspect of the present invention for solving the above-mentioned problem is combined with a computer, which is hardware, to execute a method for analyzing the behavior of infants and toddlers, and is stored in a computer-readable recording medium.

[0017] Other specific details of the present invention are included in the detailed description and drawings. Effects of the invention

[0018] Since CCTVs installed in childcare facilities have been used primarily to monitor the behavior of childcare teachers, there has been significant resistance in the field since the initial mandatory introduction, and reports indicate that childcare teachers still experience considerable stress from feeling under surveillance.

[0019] According to one embodiment of the present invention described above, CCTV footage of a childcare institution, which has been used simply for surveillance purposes, can be utilized in the childcare field to assist the childcare duties of childcare teachers or administrators (directors).

[0020] For example, real-time CCTV footage can be analyzed to detect dangerous behaviors or violent situations involving infants and toddlers in real time and send alerts. Additionally, by analyzing the children's various daily routines, it can be used to automatically generate daily notices or childcare logs that caregivers are required to write. Furthermore, by analyzing the diverse behaviors of individual children and comparing them with others, it can be utilized to analyze relative activity levels, behavioral patterns (such as intimacy with other children or teachers, and active / inactive tendencies), and individual preferences (such as play preferences).

[0021] Furthermore, according to one embodiment of the present invention, a childcare teacher can receive assistance with childcare and administrative tasks by utilizing artificial intelligence that analyzes CCTV, thereby increasing concentration on childcare duties and further improving the quality of childcare services for children. In addition, the manager of the childcare institution can effectively respond to dangerous situations such as safety accidents. Moreover, parents can receive objective behavioral analysis results of the child analyzed by artificial intelligence, and through this, they can receive satisfaction and trust in the childcare institution along with various childcare services provided by the institution (daily notices, observation logs, individual counseling, etc.).

[0022] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0023] FIG. 1 is a block diagram of an edge system according to one embodiment of the present invention. FIG. 2 is a diagram showing the functional blocks of an edge system according to one embodiment of the present invention. FIG. 3 is a diagram illustrating a behavior estimation unit in one embodiment of the present invention. FIG. 4 is a diagram illustrating a behavior event detection unit in one embodiment of the present invention. FIGS. 5 and FIGS. 6 are drawings for explaining a behavior analysis result providing unit in one embodiment of the present invention. FIG. 7 is a flowchart of a method for analyzing infant behavior in an edge system according to one embodiment of the present invention. Specific details for implementing the invention

[0024] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined only by the scope of the claims.

[0025] The terms used in this specification are for describing embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more. Although terms such as "first," "second," etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, the first component mentioned below may be the second component within the technical scope of the invention.

[0026] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0027] In the following, the background of the conception of the present invention will be explained first to aid the understanding of a person skilled in the art, and then the present invention will be described in detail.

[0028] Generally, on-site NVRs provide the function of collecting, digitizing, and storing video received from multiple CCTV channels. They also offer features that allow users to search for stored footage based on time and location, as well as functions for playback and backup. Some of the latest NVRs are equipped with AI-based video analysis capabilities to detect intruders or objects in specific areas in real time and notify administrators. This functionality can be used to detect situations where children enter dangerous areas or zones within childcare facilities, and can also be applied to day and night facility management.

[0029] Meanwhile, as CCTVs installed in childcare facilities have primarily been used to monitor the behavior of childcare teachers, there has been significant resistance in the field since the initial mandatory implementation; reports indicate that childcare teachers continue to experience considerable stress due to the perception of being monitored. Under these circumstances, there are growing demands to actively utilize the footage from these CCTVs for practical application in the childcare field.

[0030] These requirements aim to assist childcare teachers or administrators (such as directors) with various tasks by utilizing CCTV video data. For example, real-time CCTV footage can be analyzed to detect dangerous behaviors or violent situations involving infants and toddlers in real time and send alerts. Additionally, by analyzing the children's various daily routines, the system can be used to automatically generate daily notices or childcare logs that teachers would otherwise have to write manually. Furthermore, it can be used to analyze the diverse behaviors of individual children and compare them with others to determine relative activity levels, behavioral patterns (such as intimacy with other children or teachers, and active / inactive tendencies), and individual preferences (such as play preferences).

[0031] Furthermore, childcare teachers caring for infants and toddlers must perform significant administrative tasks in addition to their childcare duties, such as writing various logs (education, safety, play, etc.), child behavior observation reports, and daily notices (manually written or via app). Therefore, by utilizing artificial intelligence technology that analyzes CCTV footage from childcare institutions to automate or assist with administrative tasks, teachers can increase their focus on childcare duties and thereby improve the quality of childcare services for infants and toddlers. Additionally, the results of CCTV analysis can serve as a supplementary observer to assist teachers in childcare environments where they must care for multiple children, providing insights into situations they may have missed. For instance, when writing a daily log for a child with a quiet and calm personality, teachers can refer to the summarized daily activities and behaviors analyzed by the AI.

[0032] Meanwhile, machine learning and artificial intelligence technologies for recognizing human behavior in CCTV footage are advancing. These AI technologies recognize human objects within the video and apply them to pre-trained AI models to analyze human actions moving across multiple video frames and determine what kind of behavior is being performed.

[0033] The performance of AI technology for behavioral analysis is determined by learning models based on various human behaviors. With the introduction of lightweight AI modeling techniques and improvements in hardware performance, the technology is evolving to the point where AI runs on edge systems (equipped with standalone GPUs or AI acceleration chips) rather than server-grade systems (equipped with expensive GPU clusters). Against this backdrop of lightweight AI models and falling hardware prices, an environment is being created where even daycare centers and kindergartens can adopt behavioral analysis AI systems at a low cost.

[0034] Accordingly, an edge system and method for analyzing infant behavior according to one embodiment of the present invention is characterized by analyzing specific behaviors of infants from CCTV footage based on a pre-trained artificial intelligence model, reprocessing the analysis results, and providing them to a user, thereby enabling their use in various childcare tasks such as evaluating infant development, analyzing behavior, and detecting safety accidents.

[0036] Hereinafter, an edge system (100, hereinafter edge system) for analyzing infant behavior according to an embodiment of the present invention will be described with reference to FIGS. 1 to 6.

[0037] FIG. 1 is a block diagram of an edge system (100) according to one embodiment of the present invention.

[0038] An edge system (100) according to one embodiment of the present invention includes an input unit (110), a communication unit (120), a display unit (130), a memory (140), and a processor (150).

[0039] The input unit (110) generates input data in response to user input of the edge system (100). For example, user input may include control input related to playback of captured images or previously collected and stored images, input for checking analysis results, selection, etc. The input unit (110) includes at least one input means. The input unit (110) may include a keyboard, a keypad, a dome switch, a touch panel, a touch key, a mouse, a menu button, etc.

[0040] The communication unit (120) receives a plurality of temporally consecutive video frames containing at least one object among infants and caregivers from a CCTV, NVR, external storage device, etc., and transmits and receives data with a user terminal such as a caregiver, guardian, etc. Such a communication unit (120) may include both a wired communication module and a wireless communication module. The wired communication module may be implemented as a power line communication device, a telephone line communication device, a cable home (MoCA), Ethernet, IEEE1294, an integrated wired home network, and an RS-485 control device. In addition, the wireless communication module may be composed of a module for implementing functions such as WLAN (wireless LAN), Bluetooth, HDR WPAN, UWB, ZigBee, Impulse Radio, 60GHz WPAN, Binary-CDMA, wireless USB technology and wireless HDMI technology, and other technologies such as 5G (5th generation communication), LTE-A (long term evolution-advanced), LTE (long term evolution), and Wi-Fi (wireless fidelity).

[0041] The display unit (130) displays display data according to the operation of the edge system (100). For example, the display unit (130) can display captured or stored CCTV footage, behavior information, and behavior analysis result information. The display unit (130) includes a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a micro electro mechanical systems (MEMS) display, and an electronic paper display. The display unit (130) can be combined with the input unit (110) to be implemented as a touch screen.

[0042] Memory (140) stores a program for generating behavioral analysis result information of infants and toddlers by analyzing video frames. Here, memory (140) is a general term for non-volatile storage devices and volatile storage devices that retain stored information even when power is not supplied. For example, memory (140) may include NAND flash memory such as compact flash (CF) cards, SD (secure digital) cards, memory sticks, solid-state drives (SSD) and micro SD cards, magnetic computer memory devices such as hard disk drives (HDD), and optical disc drives such as CD-ROMs and DVD-ROMs.

[0043] The processor (150) can execute software, such as a program, to control at least one other component (e.g., a hardware or software component) of the edge system (100) and can perform various data processing or operations.

[0044] FIG. 2 is a diagram showing the functional blocks of an edge system (100) according to one embodiment of the present invention.

[0045] An edge system (100) according to one embodiment of the present invention may include an image collection unit (121), a behavior recognition unit (151), a behavior estimation unit (152), a behavior event detection unit (153), a behavior estimation result aggregation unit (154), a database (141), and a behavior analysis result provision unit (122). At this time, the image collection unit (121) and the behavior analysis result provision unit (122) are composed of a communication unit (120), the behavior recognition unit (151), the behavior estimation unit (152), the behavior event detection unit (153), and the behavior estimation result aggregation unit (154) operate according to the program execution of a processor (150), and the database (141) may be stored on a memory (140).

[0046] Meanwhile, the edge system (100) refers to a computing system capable of completing video analysis by operating within the childcare institution. This is a system that is differentiated from the method of operating a cloud system (or cloud server) that operates at a remote location other than the childcare institution. The existing video analysis method operated by cloud systems involves receiving and processing CCTV footage from the childcare institution via the Internet, which poses a problem of a high risk of exposure of the personal information of infants and toddlers. Therefore, unlike the prior art, one embodiment of the present invention is characterized by performing the process from the video collection stage to the analysis stage without external transmission of CCTV footage through the edge system (100) installed within the childcare institution, and then transmitting the analysis results externally.

[0047] First, the video collection unit (121) receives a plurality of temporally consecutive video frames containing at least one object among infants and toddlers and caregivers. At this time, the entity transmitting the video frames may receive them from a plurality of CCTVs installed within the childcare institution. Video reception from the CCTVs may also be received in a streaming format via a network or coaxial cable. Additionally, the video collection unit (121) may receive video stored in an NVR, and may receive video by reading the stored video in the form of an external file such as a USB.

[0048] Next, the action recognition unit (151) tracks an object recognized from a video frame and estimates pose information. Specifically, the action recognition unit (151) may include an object recognition module (1511), an object tracking module (1512), and a pose estimation module (1513).

[0049] The object recognition module (1511) reads out a video that is being streamed or stored and recognizes an object within the video frame. At this time, the object is a person, and at least one of an infant or a caregiver is included. The object recognition module (1511) can detect the area where the object exists and extract the video of that area. For example, the object recognition module (1511) may apply a YOLO-based artificial intelligence model, and various other versions (YoloR, YoloV5, YoloV7, etc.) may be used.

[0050] The object tracking module (1512) tracks recognized objects, and can track the same object when there are multiple recognized objects within a single image and the movement paths of the multiple objects overlap. The object tracking module (1512) recognizes multiple infants and toddlers who are actively moving, and performs the function of tracking them as separate objects when their movement paths overlap and part or all of the human objects appear to overlap. For example, the object tracking module (1512) may use algorithms such as ByteTrack, Sort, and TrackTor based on image processing technology, or models such as DeepSort, which is an artificial intelligence model.

[0051] The pose estimation module (1513) estimates pose information by defining feature points based on each joint point of the recognized object. That is, the pose estimation module (1513) recognizes an object (person) in the video and estimates pose information for analyzing the behavior of each object after tracking multiple moving objects. In one embodiment, the pose estimation module (1513) can estimate the pose of a person by defining feature points (Key Point, skeleton, bone structure) based on each joint point (head, neck, both shoulders, elbows, wrists, waist, hip joints, knees, ankles, etc.) for the person object. In this case, since one embodiment of the present invention uses feature point-based skeleton data, which is much smaller in volume than the entire video data of the person, it is useful for behavior analysis processes that require analyzing multiple video frames. As an example, the pose estimation module (1513) may use artificial intelligence models such as deeppose, mobilenet, and shufflenet that estimate the skeleton from the video of a person object.

[0052] Next, the behavior estimation unit (152) estimates the behavior information of an object based on pose information estimated from multiple video frames. For example, the behavior estimation unit (152) can estimate behavior information based on a model that has learned various behaviors in advance, and can use various artificial intelligence models such as ACRN, LFB, ST-GCN, and PoseC3D.

[0053] FIG. 3 is a diagram illustrating a behavior estimation unit (152) in one embodiment of the present invention.

[0054] In one embodiment, the behavior estimation unit (152) can estimate behavior information including at least one of a solitary behavior, a dangerous behavior, and an interactive behavior in which the recognized object interacts with other objects, in order to provide meaningful behavior analysis result information of the infant to guardians and childcare teachers in terms of developmental evaluation and childcare assistance of the infant.

[0055] In this context, solitary behaviors refer to actions such as walking, sitting, running, lying down, crawling, jumping, and eating (eating alone). Interactive behaviors refer to actions between infants and toddlers or between an infant / toddler and a teacher, including actions such as hugging, eating (feeding or being fed), passing (giving and receiving objects or toys), and caring (stroking, patting, changing clothes, etc.). Threatening behaviors refer to actions such as falling, bumping, punching, kicking, grabbing, pushing, pulling, and throwing.

[0056] Meanwhile, a single solitary behavior can include various detailed behaviors. For example, walking may include detailed behaviors such as walking alone, walking while holding an object, climbing stairs, and descending stairs. Jumping may include detailed behaviors such as jumping in place, jumping on two feet, and jumping on one foot. When eating is classified as a solitary behavior, it may include detailed behaviors such as eating alone or drinking alone. Furthermore, as described below, solitary behaviors can also be classified as interactive behaviors; in this case, eating may include detailed behaviors such as being fed by a teacher or another infant or toddler, or accepting food.

[0057] In addition, the behavior estimation unit (152) may analyze various toys together from video frames and analyze the type of toy and play time together when sitting with a specific toy.

[0058] One embodiment of the present invention can determine appropriate behavioral development situations according to the age of infants and toddlers through solitary behavior analysis, or analyze activity levels and activity volume according to daily routines. Additionally, one embodiment of the present invention can analyze the degree of physical interaction between childcare teachers and infants and toddlers through reciprocal behavior analysis, and can also analyze the level of intimacy among infants and toddlers. Furthermore, one embodiment of the present invention can detect safety accidents in real time through risk behavior analysis, thereby enabling rapid response to emergency situations. In particular, by marking video footage of specific safety accidents, post-incident response (e.g., disclosure to parents) can be facilitated. Moreover, one embodiment of the present invention can utilize toys and other items as reference material for analyzing the tastes and tendencies of infants and toddlers.

[0059] Furthermore, one embodiment of the present invention can be utilized as reference material for national developmental screening tests for infants and toddlers based on their age. For example, in the case of dual-income couples who entrust their infants and toddlers to childcare facilities for the entire day, it may be difficult to answer screening test questions regarding the child's detailed fine motor skills and gross motor skills due to limited activity time at home. One embodiment of the present invention analyzes and provides various detailed behaviors corresponding to age-specific developmental screening test questions, thereby providing information that enables guardians to answer regarding behaviors that were not observed at home.

[0060] Referring again to FIG. 2, next, the behavior event detection unit (153) performs the function of secondarily analyzing whether at least one estimated behavior information is a reciprocal behavior, that is, a reciprocal behavior.

[0061] To this end, the behavior event detection unit (153) can secondarily analyze whether it is an interaction based on behavior time information including the unique ID of a plurality of objects, the type of behavior, the location of the behavior and the distance between the plurality of objects, the degree of agreement between the time intervals for the start and end of the behavior of the plurality of objects, and whether the duration of the behavior exceeding a predetermined threshold is satisfied.

[0062] FIG. 4 is a diagram illustrating a behavior event detection unit (153) in one embodiment of the present invention.

[0063] When at least one estimated behavior information is input, the behavior event detection unit (153) determines whether there is a correlation with mutual behavior for each behavior information. If there is a correlation, it measures distance information by setting and comparing reference points between each object, compares the start and end time intervals of the behavior, and determines whether the duration of the behavior exceeds a preset threshold, thereby detecting it as mutual behavior. Thus, in one embodiment of the present invention, the behavior estimation unit (152) first estimates behavior information that distinguishes between solitary behavior, dangerous behavior, and mutual behavior, and secondarily, the behavior event detection unit (153) can additionally determine whether there is mutual behavior even for solitary behavior and dangerous behavior. That is, in one embodiment of the present invention, solitary behavior and dangerous behavior can also be detected as mutual behavior according to the analysis by the behavior event detection unit (153).

[0064] For example, in the case of a hugging action, whether it is a hugging action (an action of the object being hugged) or a hugging action (an action of the person doing the hugging) is detected by the action estimation unit (152), and the action event detection unit (153) can detect it as a mutual action based on the estimation result and using the distance information and detection time of the two people. For example, if the distance between the two people is far apart based on their neck joints, or if the timing of the mutual action does not coincide, it will not be detected as a mutual action. As another example, for actions such as handing over toys or objects, or feeding each other, the object and subject of the action are detected by the action estimation unit (152), and the action event detection unit (153) can detect it as a mutual action based on the distance between the two people and the time of the action occurrence.

[0065] In addition, the behavior event detection unit (153) can also determine that actions such as punching, kicking, grabbing, and pushing among dangerous actions are mutual actions (dangerous actions that occur between each other) based on the distance between the two people and the time of occurrence.

[0066] As another example, the solitary behavior of an infant is analyzed by the behavior event detection unit (153) to determine whether it is a reciprocal behavior. The behavior event detection unit (153) can analyze that three children are sitting together in a reciprocal relationship (reciprocal behavior) when four children sitting and two children running around are recognized as objects, and only three children satisfy the distance information according to FIG. 4, the degree of agreement between the time intervals for the start and end of the behavior, and a predetermined threshold for the duration of the behavior. Additionally, if a plaything (blocks, toys, etc.) is detected between the hands or between the three children, the behavior event detection unit (153) can analyze that the three children are playing together. If there is a situation where one child is sitting and another child is running around, the reference points of the two children may be close or the start times of the behavior may coincide, but because the duration of the behavior does not satisfy the threshold according to the determination conditions of the behavior event detection unit (153), it can be determined that it is not a reciprocal behavior.

[0067] Meanwhile, each threshold value for the reference point (reference distance) between objects analyzed by the behavior event detection unit (153), the time of occurrence of the behavior (start time, end time, whether sequential or simultaneous, etc.), and the duration of the behavior may be predefined for each behavior information and stored in the database (141). For example, the threshold value for two children sitting and playing together to be recognized as an interactive behavior may be set to 60 seconds or more, and the threshold value for one child pushing and one child falling down to be recognized as an interactive behavior may be set to within 5 seconds. At this time, each threshold value may be set as an initial value based on the results of behavior analysis by the user or by artificial intelligence, and the threshold value may be updated as the number of analyses increases.

[0068] The behavior estimation result aggregation unit (154) performs the function of aggregating the results processed by the behavior estimation unit (152) and the behavior event detection unit (153) for each behavior. Since a single behavior may correspond to a single behavior and an interactive behavior, or it may correspond to all three behaviors, the behavior estimation result aggregation unit (154) may be equipped with a data processing module for aggregating this.

[0069] Additionally, the behavior estimation result aggregation unit (154) can extract the unique ID, type of behavior, whether play, start time of behavior, and end time of behavior of multiple objects from the estimated behavior information and store them in the database (141).

[0070] Additionally, the behavior estimation result aggregation unit (154) can generate behavior analysis results by aggregating the estimated behavior information according to pre-set time units (e.g., weekly, monthly, quarterly, yearly, etc.) and also performs the function of updating the database (141) for each time unit.

[0071] FIGS. 5 and FIGS. 6 are drawings for explaining a behavior analysis result providing unit (122) in one embodiment of the present invention.

[0072] In one embodiment, the behavior analysis result providing unit (122) can provide data from the database (141) that has been processed and stored by the behavior estimation result aggregation unit (154), i.e., behavior analysis result information, in the form of numerical values ​​and graphs, to user terminals corresponding to childcare teachers, directors, parents, etc., in the form of web and app.

[0073] The behavior analysis result providing unit (122) can provide the amount of behavior for a child's individual behavior on a daily / weekly / monthly / quarterly / yearly basis as shown in FIG. 5, and can also provide information on infants and toddlers who engage in mutual behavior together as shown in FIG. 6. In addition, the behavior analysis result providing unit (122) can provide the result of comparing the behavior information of the recognized infant / toddler object with the behavior information of other infants and toddlers or their average behavior information as the behavior analysis result information, and can also provide information on preferred toys, etc. This behavior analysis result information can be used as an aid for childcare teachers and childcare managers (directors) to understand the child's behavioral tendencies, and can also be used as material for guardians at home to understand the child's behavior at the childcare institution.

[0074] Meanwhile, the behavior analysis result providing unit (122) may be implemented as a separate web server so that a user can access a web page through a terminal, and may be implemented so that the user can receive behavior analysis result information through an application (app) on the user terminal by providing an external web server interface such as a RestAPI.

[0075] In addition, an edge system (100) according to one embodiment of the present invention may include an NVR function. In this case, the edge system (100) may include a separate additional storage for storing original CCTV footage, a user GUI for video search and playback, and may include additional hardware resources such as processes or memory for this purpose. Additionally, it is possible to separately store marking footage of specific behaviors, such as dangerous behaviors analyzed by the edge system (100), thereby providing a function that allows the user to immediately search for the situation in which a specific behavior occurred from the NVR.

[0076] Meanwhile, artificial intelligence models that analyze human behavior from video operate using significant computing resources (CPU, memory, GPU). Childcare facilities are equipped with at least 8 to 16 CCTV channels, and since 5 to 8 people are captured on a single screen, a method is required to process this using a server-class system equipped with a GPU cluster. However, server-class systems are not suitable for childcare facilities because they must be installed in a remote server room due to various factors such as high costs, power supply, noise, and size, and they operate by receiving and processing video via a network.

[0077] Accordingly, the edge system (100) according to the present invention must be manufactured in a standard NVR size (1U to 2U size of a 19-inch rack), installed and configured inside a childcare institution, and be inexpensive. In addition, although the behavior recognition unit (151) and behavior estimation unit (152) in the present invention were generally run on a high-cost GPU cluster, with the recent development of optimization technologies for artificial intelligence learning models (TensorRT, autoTVM compiler, etc.) and data pipeline optimization technologies, they can be implemented on a single GPU or an AI acceleration module (chip type) cluster, thus making it possible to configure the edge system (100) and providing the advantage of being easy to apply to childcare institutions.

[0078] Additionally, to further reduce the hardware costs of the edge system (100) installed inside the childcare institution, it is possible to operate the behavior event detection unit (153) and the behavior estimation result aggregation unit (154), which process result data (video frames composed of KeyPoints) of the behavior recognition unit (151) for which personal information has been anonymized, by separating them to an external cloud. The edge system (100) and the cloud are connected via the internet to exchange data, and by transmitting the analyzed result value to the edge system (100), the problem of personal information leakage caused by the transmission of the original video image to the cloud can be resolved, thereby reducing hardware costs.

[0079] FIG. 7 is a flowchart of a method for analyzing infant behavior in an edge system (100) according to one embodiment of the present invention.

[0080] First, a plurality of temporally consecutive video frames containing at least one object among an infant and a caregiver are received (S110).

[0081] Next, pose information is estimated by tracking an object recognized from a video frame (S120), and behavior information is estimated based on the pose information estimated from multiple video frames (S130).

[0082] Next, behavior analysis result information is generated by analyzing behavior information based on predetermined conditions (S140) and provided to a user terminal corresponding to an object (S150).

[0083] Meanwhile, in the above description, steps S110 to S150 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Also, some steps may be omitted as necessary, and the order between steps may be changed. Furthermore, the contents of FIGS. 1 to 6 and the contents of FIG. 7 may be applied to each other.

[0084] The infant behavior analysis method according to one embodiment of the present invention described above may be implemented as a program (or application) and stored on a medium to be executed in combination with a computer, which is hardware.

[0085] The aforementioned program may include code encoded in computer languages ​​such as C, C++, JAVA, Ruby, and machine language, which can be read by the computer's processor (CPU) through the computer's device interface, in order for the computer to read the program and execute the methods implemented in the program. Such code may include functional code related to functions that define the necessary functions for executing the methods, and may include control code related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure. Additionally, such code may further include memory reference code regarding where (address) additional information or media necessary for the computer's processor to execute the functions should be referenced in the computer's internal or external memory. Additionally, if the processor of the computer needs to communicate with any other computer or server (300) located remotely in order to execute the above functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to transmit or receive during communication.

[0086] The above-mentioned storage medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the above-mentioned storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the above-mentioned program may be stored on various recording media on various servers that the computer can access, or on various recording media on the user's computer. Additionally, the above-mentioned medium may be distributed across networked computer systems, and computer-readable code may be stored in a distributed manner.

[0087] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0088] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0089] 100: Edge System 110: Input section 120: Communications Department 130: Display unit 140: Memory 150: Processor

Claims

Claim 1 An edge system for analyzing the behavior of infants and toddlers comprises: a communication unit that receives a plurality of temporally consecutive original CCTV video frames containing at least one object among an infant / toddler and a caregiver from a plurality of CCTVs installed within a childcare institution; a memory in which a program for analyzing the original CCTV video frames to generate behavior analysis result information of the infant / toddler is stored; and a processor that executes the program stored in the memory. The processor, upon executing the program, tracks an object recognized from the original CCTV video frames to estimate posture information, estimates behavior information based on the posture information estimated from the plurality of original CCTV video frames, and then, based on at least one behavior information included in the estimated behavior information, secondarily analyzes whether it is an interactive behavior based on behavior time information including the unique ID of the plurality of objects, type of behavior, behavior point, distance information between the plurality of objects, degree of agreement between the behavior start time and behavior end time intervals of the plurality of objects, and whether the duration of the behavior exceeding a predetermined threshold is satisfied, and generates the behavior analysis result information based on the analysis result and provides it to a user terminal corresponding to the object. Claim 2 An edge system for analyzing infant behavior according to claim 1, wherein the processor estimates posture information by defining feature points based on each joint point of the recognized object. Claim 3 An edge system for analyzing infant behavior according to claim 1, wherein the processor estimates behavior information including at least one of a solitary behavior of the recognized object, a dangerous behavior, and an interactive behavior in which the recognized object interacts with other objects. Claim 4 An edge system for analyzing infant behavior according to claim 1, wherein the predetermined threshold includes a threshold for a reference distance based on reference points between objects, a time of occurrence of a behavior, and a duration of a behavior, and wherein the threshold is predefined for each behavior information and stored in a database. Claim 5 An edge system for analyzing infant behavior according to claim 4, wherein the threshold value is set as an initial value based on the results of behavior analysis by a user or by artificial intelligence, and is updated as the number of analyses increases. Claim 6 An edge system for analyzing infant behavior according to claim 1, wherein the processor analyzes whether the plurality of objects are playing together based on the hands of the plurality of objects or toys detected between the objects, and includes the play status in the behavior analysis result information. Claim 7 delete Claim 8 delete Claim 9 delete

Images