Child development test analysis method and apparatus

Abstract

The present invention relates to a child development test analysis method. The child development test analysis method comprises the steps of: generating image data by capturing motions of a first user and a second user by using a plurality of image sensors; generating voice data by recording voices of the first user and the second user by using a microphone sensor; extracting a first set of a plurality of key points from major body points of the first user on the basis of the image data; determining a first frame corresponding to the first user by connecting the extracted first set of the plurality of key points; extracting a first voice spectrogram of the first user on the basis of the voice data; and detecting behavior and emotion of the first user by using the first frame and the first voice spectrogram.

Description

Child Development Test Analysis Method and Device

[0001] The present invention relates to a method and device for analyzing child development tests, and more particularly, to a method and device for analyzing the behavior and emotions of a child undergoing a development test.

[0002]

[0003] Child development assessments can be conducted in a variety of ways, but the most common method involves placing parents and children in the same room and observing them. For example, researchers observe how parents and children play or converse, and then score relevant indicators to conduct a developmental assessment. However, this type of developmental assessment method can be problematic because the results can be influenced by factors such as the researcher's background or personality, leading to a lack of objectivity.

[0004]

[0005] The present invention provides a child development test analysis method for solving the above-mentioned problems, a computer program stored in a computer-readable medium, a computer-readable medium storing the computer program, and a device (system).

[0006]

[0007] The present invention can be implemented in various ways, including a method, a device (system), a computer program stored in a computer-readable medium, or a computer-readable medium having a computer program stored therein.

[0008] According to one embodiment of the present invention, a child development test analysis method performed by at least one processor includes the steps of capturing motions of a first user and a second user using a plurality of image sensors to generate image data, recording voices of the first user and the second user using a microphone sensor to generate voice data, extracting a first set of a plurality of key points from major points of the first user's body based on the image data, connecting the extracted first set of a plurality of key points to determine a first frame corresponding to the first user, extracting a first voice spectrogram of the first user based on the voice data, and detecting a behavior and emotion of the first user using the first frame and the first voice spectrogram.

[0009] According to one embodiment of the present invention, the method further includes the steps of extracting a second set of a plurality of key points from major points of a second user's body based on image data, connecting the extracted second set of a plurality of key points to determine a second frame corresponding to the second user, and classifying the first frame or the second frame into an adult class or a child class based on a distance between a first key point and a second key point constituting the first frame and a distance between a third key point and a fourth key point constituting the second frame.

[0010] According to one embodiment of the present invention, the method further includes the steps of: extracting a second set of a plurality of key points from major points of a second user's body based on image data; connecting the extracted second set of a plurality of key points to determine a second frame corresponding to the second user; and comparing the sizes of the first frame and the second frame to classify the first frame or the second frame into an adult class or a child class.

[0011] According to one embodiment of the present invention, the method further includes a step of extracting a second voice spectrogram of a second user based on voice data, and a step of comparing the first voice spectrogram and the second voice spectrogram to classify the first user and the second user into an adult class or a child class.

[0012] According to one embodiment of the present invention, the step of detecting the behavior and emotion of the first user using the first frame and the first voice spectrogram includes the step of extracting a first detailed frame corresponding to the torso of the first user among the first frames, the step of extracting a first coordinate value according to time of the extracted first detailed frame, and the step of providing the extracted first coordinate value to a first machine learning model trained to analyze the behavior to generate a first result value representing the behavior of the first user.

[0013] According to one embodiment of the present invention, the step of detecting the action and emotion of the first user using the first frame and the first voice spectrogram includes the step of extracting a second detailed frame corresponding to the face of the first user among the first frames, the step of extracting a second coordinate value according to time of the extracted second detailed frame, and the step of providing the extracted second coordinate value to a second machine learning model trained to analyze facial expressions to generate a second result value indicating the emotion of the first user.

[0014] According to one embodiment of the present invention, the step of detecting the behavior and emotion of the first user using the first frame and the first voice spectrogram includes the step of providing the extracted first voice spectrogram to a third machine learning model trained to analyze emotions to generate a third result value representing the emotion of the first user.

[0015] According to one embodiment of the present invention, the step of detecting the action and emotion of the first user using the first frame and the first voice spectrogram includes the step of extracting a second detailed frame corresponding to the face of the first user among the first frames, the step of extracting a second coordinate value according to time of the extracted second detailed frame, the step of providing the extracted second coordinate value to a second machine learning model trained to analyze facial expressions to generate a second result value indicating the emotion of the first user, and the step of providing the generated second result value and the third result value to an ensemble model to generate a fourth result value indicating the emotion of the first user.

[0016] According to one embodiment of the present invention, the plurality of image sensors include at least one fixed camera sensor and at least one PTZ camera sensor.

[0017] A computer program stored in a computer-readable recording medium is provided to execute a method according to one embodiment of the present invention on a computer.

[0018] A computing device according to one embodiment of the present invention includes a communication module, a memory, and at least one processor connected to the memory and configured to execute at least one computer-readable program contained in the memory. The at least one program includes commands for capturing motions of a first user and a second user using a plurality of image sensors to generate image data, recording voices of the first user and the second user using a microphone sensor to generate voice data, extracting a first set of a plurality of key points from major body points of the first user based on the image data, connecting the extracted first set of a plurality of key points to determine a first frame corresponding to the first user, extracting a first voice spectrogram of the first user based on the voice data, and detecting an action and emotion of the first user using the first frame and the first voice spectrogram.

[0019]

[0020] In various embodiments of the present invention, a computing device can automatically analyze and extract the behaviors and emotions of a child and parent performing a child development test, and obtain objective test analysis results.

[0021] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary knowledge in the technical field to which the present invention belongs (referred to as “one of ordinary skill in the art”) from the description of the claims.

[0022]

[0023] Embodiments of the present invention will be described below with reference to the accompanying drawings, wherein like reference numerals represent similar elements, but are not limited thereto.

[0024] FIG. 1 is a functional block diagram showing a software configuration of a computing device according to one embodiment of the present invention.

[0025] FIG. 2 is a flowchart illustrating an example of detecting a user's behavior according to one embodiment of the present invention.

[0026] FIG. 3 is a flowchart illustrating an example of detecting a user's emotions according to one embodiment of the present invention.

[0027] Figure 4 is a block diagram showing the hardware configuration of a computing device according to one embodiment of the present invention.

[0028]

[0029] Hereinafter, specific details for implementing the present invention will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions of widely known functions or configurations will be omitted if they may unnecessarily obscure the gist of the present invention.

[0030] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Furthermore, in the description of the embodiments below, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0031] The advantages and features of the disclosed embodiments, and the methods for achieving them, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure the completeness of the present invention and to fully inform those skilled in the art of the scope of the invention.

[0032] The terms used in this specification will be briefly explained, followed by a detailed description of the disclosed embodiments. The terms used in this specification have been selected from widely used, current terms, taking into account the functions of the present invention. However, these terms may vary depending on the intentions of engineers working in the relevant fields, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined based on their meanings and the overall content of the present invention, rather than simply their names.

[0033] In this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, plural expressions include singular expressions unless the context clearly indicates otherwise. When a part of the specification is said to include a component, this does not exclude other components, but rather implies that other components may be included, unless otherwise specifically stated.

[0034] In the present invention, terms such as “comprise”, “comprising”, etc. may indicate the presence of features, steps, operations, elements, and / or components, but such terms do not exclude the addition of one or more other functions, steps, operations, elements, components, and / or combinations thereof.

[0035] In the present invention, when a specific component is referred to as being "coupled," "combined," "connected," or "reacting" with any other component, the specific component may be directly coupled, combined, and / or connected to, or reacting with, the other component, but is not limited thereto. For example, one or more intermediate components may exist between the specific component and the other component. Furthermore, in the present invention, "and / or" may include each of one or more of the listed items, or a combination of at least some of one or more of the listed items.

[0036] In the present invention, terms such as "first" and "second" are used to distinguish specific components from other components, and the components described by these terms are not limited. For example, the "first" component may be an element of the same or similar form as the "second" component.

[0037] In the present invention, "child development test" may refer to a method of testing for developmental delays in children by placing the child and parents in the same room, observing the behavior and facial expressions of the child and parents, and recording scores on related indicators.

[0038] In the present invention, the “machine learning model” may include models such as a decision tree, a random forest, a stochastic gradient descent (SDG), and a k nearest neighbors, which are machine learning-based classifiers, or models such as a CNN, LSTM, and VGG16, which are deep learning-based classifiers.

[0039] FIG. 1 is a functional block diagram showing a software configuration of a computing device (100) according to one embodiment of the present invention. According to one embodiment, the computing device (100) is any device for analyzing a child development test, and may include a data collection unit (102), a frame determination unit (104), a spectrogram extraction unit (106), an action detection unit (108), an emotion detection unit (110), etc. In addition, the computing device (100) communicates with an image sensor (120), a microphone sensor (130), etc., and may send and receive any data and / or information for performing a child development test analysis.

[0040] According to one embodiment, the data collection unit (102) can collect video and audio data representing the activities of the child and the parent when the child and the parent perform a child development test. For example, the child and the parent can perform the activity in any location where the image sensor (120) and the microphone sensor (130) are installed, and the data collection unit (102) can collect and generate video data capturing the motions of the child and the parent using the image sensor (120), and collect and generate audio data recording the voices of the child and the parent using the microphone sensor (130).

[0041] According to one embodiment, the frame determination unit (104) can automatically detect or determine the actions currently being performed by the child and the parent using the collected image data. To this end, the frame determination unit (104) can first extract a plurality of key points from the main body points of the child and / or the parent based on the image data. Here, the key points refer to points that match the main body points such as joints, facial features, and fingertips of a person, and can be extracted separately for each person. Then, the frame determination unit (104) can generate and determine a frame by connecting the plurality of extracted key points. Here, the frame can be skeletal information generated by connecting the key points.

[0042] Through the above-described process, a first frame corresponding to the child undergoing the child development test can be determined, and a second frame corresponding to the parent can be determined. In other words, each frame can be determined to correspond to a person. Once the frames corresponding to each person are determined in this way, the frame determination unit (104) can classify each frame into an adult class or a child class based on the relationships between the frames.

[0043] According to one embodiment, the frame determination unit (104) may classify the first frame or the second frame into an adult class or a child class based on the distance between the first keypoint and the second keypoint constituting the first frame and the distance between the third keypoint and the fourth keypoint constituting the second frame. For example, the first keypoint and the second keypoint may be extracted to correspond to the torso part of the child, and the third keypoint and the fourth keypoint may be extracted to correspond to the torso part of the parent. In this case, the second frame having a larger torso length or size may be classified into the adult class, and the first frame having a smaller torso length or size may be classified into the child class. Additionally or alternatively, the frame determination unit (104) may classify the first frame or the second frame into the adult class or the child class by comparing the sizes of the first frame and the second frame.

[0044] According to one embodiment, the spectrogram extraction unit (106) may extract a spectrum using voice data. For example, the spectrogram extraction unit (106) may apply a Fourier transform to the voice data to extract frequency information associated with the voice data, and may generate a spectrum using the extracted frequency information. Specifically, the voice data may include information about the amplitude of the voice over time, and the spectrum may be configured to include information about the amplitude over frequency.

[0045] In one embodiment, the spectrogram extraction unit (106) can generate a spectrogram using the spectrum. For example, the spectrogram extraction unit (106) can extract information about the amplitude according to the frequency at each time by applying a Fourier transform to each time point of the spectrum. Then, the spectrogram extraction unit (106) can convert the amplitude into decibels and apply a log scale to the frequency to generate a spectrogram. The spectrogram generated in this way can be used to predict emotions of the child and / or parent's behavior, or can be used together with the frame to classify the frame into an adult class and / or a child class.

[0046] According to one embodiment, the action detection unit (108) can predict the child's and / or parent's actions based on the form in which the determined frame changes over time. For example, the action detection unit (108) can extract a first detailed frame corresponding to the torso among the frames, and extract a first coordinate value of the extracted first detailed frame over time. Then, the action detection unit (108) can provide the extracted first coordinate value to a first machine learning model trained to analyze actions, thereby generating a result value representing the child's and / or parent's actions. For example, when a change in the first coordinate value over time of the first detailed frame is input, the first machine learning model can generate or output a class such as 'running', 'walking', or 'sitting' as a result value as an action according to the change in the corresponding coordinate value.

[0047] Additionally or alternatively, the emotion detection unit (110) can predict the emotions of the child and / or parent based on the form in which the determined frame changes over time. For example, the emotion detection unit (110) can extract a second detailed frame corresponding to a face among the frames, and extract the second coordinate value of the extracted second detailed frame over time. Then, the emotion detection unit (110) can provide the extracted second coordinate value to a second machine learning model trained to analyze emotions to generate a result value representing the emotions of the child and / or parent. For example, when the change in the second coordinate value of the second detailed frame over time is input, the second machine learning model can generate or output a class such as 'joy', 'sadness', or 'depression' as an emotion according to the change in the corresponding coordinate value.

[0048] Additionally or alternatively, the emotion detection unit (110) may predict the emotions of the child and / or parent based on the voice spectrogram. For example, the emotion detection unit (110) may provide the voice spectrogram to a third machine learning model trained to analyze emotions to generate output values ​​representing the emotions of the child and / or parent. For example, when the voice waveform included in the voice spectrogram is input, the third machine learning model may generate or output classes such as "happy," "sad," and "angry" as output values ​​based on the voice waveform.

[0049] Additionally or alternatively, the emotion detection unit (110) can predict the emotions of the child and / or parent using both the frame and the voice spectrogram. For example, to address the problem of incorrect emotion prediction in special situations, such as when a person's face is covered by a mask or the like, or when the sound is low and therefore unreliable, the emotion detection unit (110) can also predict the emotions of the child and / or parent using both the frame and the voice spectrogram. According to one embodiment, the emotion detection unit (110) can provide the second result value extracted by the second machine learning model and the third result value extracted by the third machine learning model to an ensemble model to generate a fourth result value representing the emotions of the child and / or parent. Here, the ensemble model relates to a method of combining the result values ​​of multiple models and may include models such as bagging, boosting, and voting.

[0050] Although each functional component included in the computing device (100) is separately described in FIG. 1, this is merely to aid understanding of the invention, and a single computing device may perform more than one function. With this configuration, the computing device (100) can automatically analyze and extract the behaviors and emotions of children and parents undergoing child development testing, thereby obtaining objective test analysis results.

[0051] FIG. 2 is a flowchart (200) illustrating an example of detecting a user's actions according to one embodiment of the present invention. As described above, the computing device (100 of FIG. 1) can capture the motions of a first user and a second user using a plurality of image sensors to generate image data (S202). Here, the plurality of image sensors may include at least one fixed camera sensor (e.g., CCTV, etc.) and at least one PTZ (pan tilt zoom) camera sensor. In this case, the plurality of image sensors may be installed in various locations to capture the user's actions.

[0052] According to one embodiment, the computing device may extract a first set of multiple key points from major body points of the first user based on image data (S204). Furthermore, the computing device may connect the extracted first set of multiple key points to determine a first frame corresponding to the first user (S206). In this case, the computing device may extract a first detailed frame corresponding to the torso of the first user from the first frame (S208). Furthermore, the computing device may extract a second detailed frame corresponding to the face of the first user from the first frame (S210).

[0053] According to one embodiment, the computing device can predict the behavior and emotions of the first user based on the extracted first detailed frame and the second detailed frame. For example, the computing device can extract the first coordinate value according to the time of the extracted first detailed frame (S212). Furthermore, the computing device can provide the extracted first coordinate value to a first machine learning model trained to analyze behavior, thereby generating a first result value representing the behavior of the first user (S216).

[0054] In one embodiment, the computing device can extract second coordinate values ​​based on the time of the extracted second detailed frame (S214). Furthermore, the extracted second coordinate values ​​can be provided to a second machine learning model trained to analyze facial expressions to generate a second result value representing the first user's emotions (S218).

[0055] FIG. 3 is a flowchart (300) illustrating an example of detecting a user's emotions according to one embodiment of the present invention. As described above, the computing device (100 of FIG. 1) can capture the motions of a first user and a second user using multiple image sensors to generate image data. In addition, the computing device can record the voices of the first user and the second user using a microphone sensor to generate voice data (S302).

[0056] In one embodiment, the computing device may extract a first voice spectrogram of the first user based on voice data (S304). In this case, the computing device may provide the extracted first voice spectrogram to a third machine learning model trained to analyze emotions to generate a third result value representing the first user's emotions (S306). Additionally or alternatively, the computing device may predict the user's emotions using both image data and voice data. For example, the computing device may provide the second result value and the third result value to an ensemble model to generate a fourth result value representing the first user's emotions (S308).

[0057] FIG. 4 is a block diagram showing the hardware configuration of a computing device (100) according to one embodiment of the present invention. According to one embodiment, the computing device (100) may include a memory (410), a processor (420), a communication module (430), and an input / output interface (440), and as illustrated in FIG. 4, the computing device (100) may be configured to communicate information and / or data via a network using the communication module (430).

[0058] The memory (410) may include any non-transitory computer-readable recording medium. According to one embodiment, the memory (410) may include a non-volatile mass storage device such as a random access memory (RAM), a read only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. As another example, a non-volatile mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be included in the computing device (100) as a separate permanent storage device distinct from the memory. In addition, an operating system and at least one program code may be stored in the memory (410).

[0059] These software components may be loaded from a computer-readable recording medium separate from the memory (410). This separate computer-readable recording medium may include a recording medium directly connectable to the computing device (100), for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. As another example, the software components may be loaded into the memory (410) via a communication module (430) other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (410) based on a computer program that is installed by files provided by developers or a file distribution system that distributes installation files of applications via the communication module (430).

[0060] The processor (420) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to another user terminal (not shown) or another external system via the memory (410) or the communication module (430).

[0061] The communication module (430) may provide a configuration or function for a user terminal (not shown) and a computing device (100) to communicate with each other via a network, and may provide a configuration or function for the computing device (100) to communicate with an external system (e.g., a separate cloud system, etc.). For example, control signals, commands, data, etc. provided under the control of the processor (420) of the computing device (100) may be transmitted to the user terminal and / or the external system via the communication module (430) and the network via the communication module of the user terminal and / or the external system.

[0062] In addition, the input / output interface (440) of the computing device (100) may be a means for interfacing with a device (not shown) for input or output that is connected to the computing device (100) or that the computing device (100) may include. In FIG. 4, the input / output interface (440) is illustrated as an element configured separately from the processor (420), but is not limited thereto, and the input / output interface (440) may be configured to be included in the processor (420). The computing device (100) may include more components than those illustrated in FIG. 4. However, there is no need to explicitly illustrate most of the conventional components.

[0063] The processor (420) of the computing device (100) may be configured to manage, process and / or store information and / or data received from multiple user terminals and / or multiple external systems.

[0064] The above-described methods and / or various embodiments may be implemented by digital electronic circuits, computer hardware, firmware, software, and / or a combination thereof. Various embodiments of the present invention may be implemented as a computer program that is executed by a data processing device, for example, one or more programmable processors and / or one or more computing devices, or stored on a computer-readable recording medium and / or a computer-readable recording medium. The above-described computer program may be written in any form of programming language, including a compiled language or an interpreted language, and may be distributed in any form, such as a standalone program, a module, a subroutine, etc. The computer program may be distributed through a single computing device, multiple computing devices connected through the same network, and / or multiple computing devices distributed to be connected through multiple different networks.

[0065] The above-described methods and / or various embodiments may be performed by one or more processors configured to execute one or more computer programs that process, store, and / or manage any function, function, etc. by operating on the basis of input data or generating output data. For example, the methods and / or various embodiments of the present invention may be performed by special-purpose logic circuits such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits), and an apparatus and / or system for performing the methods and / or embodiments of the present invention may be implemented as special-purpose logic circuits such as FPGAs or ASICs.

[0066] The one or more processors executing the computer program may include a general-purpose or special-purpose microprocessor and / or one or more processors of any type of digital computing device. The processor may receive instructions and / or data from each of read-only memory and random-access memory, or may receive instructions and / or data from the read-only memory and the random-access memory. In the present invention, components of a computing device performing the methods and / or embodiments may include one or more processors for executing instructions, and one or more memory devices for storing instructions and / or data.

[0067] According to one embodiment, the computing device can transmit and receive data to and from one or more mass storage devices for storing data. For example, the computing device can receive and / or transfer data from a magnetic disc or an optical disc. A computer-readable storage medium suitable for storing instructions and / or data associated with a computer program may include, but is not limited to, any form of non-volatile memory, including semiconductor memory devices such as Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), flash memory devices, and the like. For example, the computer-readable storage medium may include a magnetic disk such as an internal hard disk or a removable disk, a magneto-optical disk, a CD-ROM, and a DVD-ROM disk.

[0068] To provide interaction with a user, a computing device may include, but is not limited to, a display device (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD), etc.) for providing or displaying information to the user, and a pointing device (e.g., a keyboard, a mouse, a trackball, etc.) for allowing the user to provide input and / or commands to the computing device. That is, the computing device may further include any other types of devices for providing interaction with the user. For example, the computing device may provide any form of sensory feedback to the user, including visual feedback, auditory feedback, and / or tactile feedback, for interaction with the user. In this regard, the user may provide input to the computing device through various gestures, such as visual, vocal, or motion.

[0069] In the present invention, various embodiments may be implemented in a computing system that includes backend components (e.g., a data server), middleware components (e.g., an application server), and / or front-end components. In this case, the components may be interconnected via any form or medium of digital data communication, such as a communications network. For example, the communications network may include a Local Area Network (LAN), a Wide Area Network (WAN), etc.

[0070] A computing device based on the exemplary embodiments described herein may be implemented using hardware and / or software configured to interact with a user, including a user device, a user interface (UI) device, a user terminal, or a client device. For example, the computing device may include a portable computing device, such as a laptop computer. Additionally or alternatively, the computing device may include, but is not limited to, Personal Digital Assistants (PDAs), tablet PCs, game consoles, wearable devices, Internet of Things (IoT) devices, virtual reality (VR) devices, augmented reality (AR) devices, and the like. The computing device may further include other types of devices configured to interact with a user. Furthermore, the computing device may include a portable communication device (e.g., a mobile phone, a smart phone, a wireless cellular phone, etc.) suitable for wireless communication over a network, such as a mobile communication network. The computing device may be configured to communicate wirelessly with a network server using wireless communication technologies and / or protocols, such as Radio Frequency (RF), Microwave Frequency (MWF), and / or Infrared Ray Frequency (IRF).

[0071] The various embodiments of the present invention, including specific structural and functional details, are exemplary. Therefore, the embodiments of the present invention are not limited to those described above and may be implemented in various other forms. Furthermore, the terminology used herein is intended to describe certain embodiments and is not intended to limit the embodiments. For example, singular terms and the above may be interpreted to include plural forms, unless the context clearly dictates otherwise.

[0072] In the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which they pertain. Furthermore, commonly used terms, such as terms defined in dictionaries, should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology.

[0073] While the present invention has been described in connection with certain embodiments herein, it should be understood that various modifications and variations can be made without departing from the scope of the invention as understood by those skilled in the art. Furthermore, such modifications and variations should be considered to fall within the scope of the claims appended to this specification.

Claims

1. A method for analyzing child development tests performed by at least one processor, A step of generating image data by capturing the motions of a first user and a second user using multiple image sensors; A step of generating voice data by recording the voices of the first user and the second user using a microphone sensor; A step of extracting a first set of multiple key points from major body points of the first user based on the image data; A step of determining a first frame corresponding to the first user by connecting a plurality of key points of the first set extracted above; A step of extracting a first voice spectrogram of the first user based on the voice data; and A step of detecting the behavior and emotion of the first user using the first frame and the first voice spectrogram; A method of analyzing child development tests including:

2. In paragraph 1, A step of extracting a second set of multiple key points from major body points of the second user based on the image data; A step of connecting a plurality of key points of the extracted second set to determine a second frame corresponding to the second user; and A step of classifying the first frame or the second frame into an adult class or a child class based on the distance between the first keypoint and the second keypoint constituting the first frame and the distance between the third keypoint and the fourth keypoint constituting the second frame; A method of analyzing child development tests that further includes:

3. In paragraph 1, A step of extracting a second set of multiple key points from major body points of the second user based on the image data; A step of connecting a plurality of key points of the extracted second set to determine a second frame corresponding to the second user; and A step of comparing the size of the first frame and the size of the second frame to classify the first frame or the second frame into an adult class or a child class; A method of analyzing child development tests that further includes:

4. In paragraph 1, A step of extracting a second voice spectrogram of the second user based on the voice data; and A step of comparing the first voice spectrogram and the second voice spectrogram to classify the first user and the second user into an adult class or a child class; A method of analyzing child development tests that further includes:

5. In paragraph 1, The step of detecting the behavior and emotion of the first user using the first frame and the first voice spectrogram is as follows: A step of extracting a first detailed frame corresponding to the torso of the first user among the first frames; A step of extracting a first coordinate value according to time of the extracted first detailed frame; and A step of providing the extracted first coordinate value to a first machine learning model trained to analyze behavior to generate a first result value representing the behavior of the first user; A method of analyzing child development tests including:

6. In paragraph 1, The step of detecting the behavior and emotion of the first user using the first frame and the first voice spectrogram is as follows: A step of extracting a second detailed frame corresponding to the face of the first user from the first frame; A step of extracting a second coordinate value according to time of the extracted second detailed frame; and A step of providing the extracted second coordinate values ​​to a second machine learning model trained to analyze facial expressions to generate a second result value representing the emotions of the first user; A method of analyzing child development tests including:

7. In paragraph 1, The step of detecting the behavior and emotion of the first user using the first frame and the first voice spectrogram is as follows: A step of providing the extracted first voice spectrogram to a third machine learning model trained to analyze emotions to generate a third result value representing the emotions of the first user; A method of analyzing child development tests including:

8. In paragraph 7, The step of detecting the behavior and emotion of the first user using the first frame and the first voice spectrogram is as follows: A step of extracting a second detailed frame corresponding to the face of the first user from the first frame; A step of extracting a second coordinate value according to time of the extracted second detailed frame; A step of providing the extracted second coordinate values ​​to a second machine learning model trained to analyze facial expressions to generate a second result value representing the emotion of the first user; and A step of providing the generated second result value and the third result value to an ensemble model to generate a fourth result value representing the emotion of the first user; A method of analyzing child development tests including:

9. In paragraph 1, The above plurality of image sensors, A method for analyzing child development testing, comprising at least one fixed camera sensor and at least one PTZ camera sensor.

10. A computer program stored in a computer-readable recording medium for executing the method according to any one of paragraphs 1 to 9 on a computer.

11. As a computing device, Communication module; memory; and At least one processor connected to said memory and configured to execute at least one computer-readable program contained in said memory Including, At least one program above, By using multiple image sensors to capture the motions of the first user and the second user, video data is generated, Recording the voices of the first user and the second user using a microphone sensor to generate voice data, Extracting a first set of multiple key points from the main body points of the first user based on the above image data, Connecting a plurality of key points of the first set extracted above to determine a first frame corresponding to the first user, Extracting a first voice spectrogram of the first user based on the above voice data, A computing device comprising commands for detecting actions and emotions of the first user using the first frame and the first voice spectrogram.

Images