Child developmental disorder diagnosis assistance device and child developmental disorder diagnosis assistance method based on VR evaluation data collection pipeline technology

Abstract

Disclosed are a child developmental disorder diagnosis assistance device and a child developmental disorder diagnosis assistance method based on virtual reality (VR) evaluation data collection pipeline technology. The child developmental disorder diagnosis assistance device according to an embodiment of the present invention is a child developmental disorder diagnosis assistance device for assisting diagnosis of child developmental disorders and comprises: a virtual reality (VR) evaluation data collection unit configured to collect VR evaluation data including a VR image provided to a user by a VR device and user data generated from the user's voice, gaze, face, and hands; and a child developmental disorder assessment unit configured to, on the basis of the VR evaluation data, evaluate the user's cognitive ability, social ability, and language ability to assess child developmental disorders.

Description

Diagnostic aid for child developmental disorders and diagnostic aid method for child developmental disorders based on VR evaluation data collection pipeline technology

[0001] The present invention relates to an assisting device for diagnosing child developmental disorders and a method for diagnosing child developmental disorders, and more specifically, to an assisting device for diagnosing child developmental disorders and a method for diagnosing child developmental disorders based on virtual reality (VR) evaluation data collection pipeline technology.

[0002] This invention was carried out as a result of research conducted under the “Emotional Intelligent Child Care System Convergence Research Center” project (Project No.: 2710001589, Project No.: 00218176) of the Leading Research Center (CRC) program promoted by the National Research Foundation of Korea with support from the Ministry of Science and ICT. Meanwhile, the Government of the Republic of Korea does not hold any ownership rights regarding this invention.

[0003] Developmental disorders, occurring in 1–3% of children under the age of five, are a general term used to describe significant delays in the expected developmental process across major domains, including motor, language, cognitive, and socio-emotional development. Since developmental potential is high during infancy and early childhood, early detection of developmental disorders is crucial for improving a child's developmental process. Developmental disorders need to be addressed through early intervention to improve the developmental process, particularly when problems arise in multiple areas.

[0004] Tools such as the Child Behavior Checklist (CBCL) for children aged 1.5 to 5 years, the Modified Checklist for Autism in Toddlers (M-CHAT), the Social Communication Questionnaire (SCQ), and the Childhood Autism Rating Scale (CARS) are used to evaluate developmental disorders in children. However, these tools have limitations in terms of age sensitivity, efficiency, and accuracy when used for screening Autism Spectrum Disorders (ASD). For example, the CBCL lacks the sensitivity to detect ASD in infancy, while the SCQ and CARS require the adjustment of cutoffs for evaluating developmental disorders in children depending on socio-cultural contexts.

[0005] Tools widely used for ASD diagnosis, such as ADOS-2 (Autism Diagnostic Observation Schedule-2) and ADI-R (Autism Diagnostic Interview-Revised), also demonstrate low performance in terms of the effectiveness of early detection of childhood developmental disorders. This is because symptoms similar to those of other conditions, such as language disorders and Attention-Deficit / Hyperactivity Disorder (ADHD), overlap with the condition.

[0006] Meanwhile, with the rapid advancement of digital technology, the application of metaverse technology in fields such as education and rehabilitation is gaining attention. Virtual reality (VR) provides various behavioral data, such as the user's gaze direction, facial expressions, and hand movements, offering an immersive environment where children can participate naturally. Despite the potential of such metaverse technology, research regarding tools that use VR to screen for and provide early diagnosis of developmental disorders in children remains insufficient.

[0007] VR devices are primarily utilized to analyze game logs or track user activity records for the development of entertainment content. While VR devices focus on entertainment and enhancing user experience rather than on research purposes regarding childhood developmental disorders, it is rare to see them used as assistive tools to assess the cognitive, social, and language abilities of children with developmental disabilities. Furthermore, regarding VR devices, the various plugins and APIs provided by Meta (Oculus, OpenXR) often make it difficult to collect accurate data for diagnosing childhood developmental disorders due to conflict issues.

[0008] One objective of the present invention is to provide an assisting device for diagnosing developmental disorders in children and a method for diagnosing developmental disorders in children based on virtual reality (VR) evaluation data collection pipeline technology.

[0009] In addition, the present invention aims to effectively assist in the diagnosis of developmental disorders in children by reliably acquiring VR evaluation data from users based on VR evaluation data collection pipeline technology.

[0010] In addition, the present invention has the purpose of effectively assisting in the diagnosis of developmental disorders in children by configuring a VR evaluation data collection unit to collect various VR evaluation data without API conflict issues.

[0011] In addition, the present invention has the purpose of accurately assessing the potential for child developmental disorders by evaluating the user's cognitive ability, social ability, and language ability through user interaction with VR video.

[0012] The technical problems to be solved by the embodiments of the present invention are not limited to those described above, and other technical problems can be inferred from the following embodiments.

[0013] A diagnostic assistance device for child developmental disorders according to one embodiment of the present invention comprises: a VR evaluation data collection unit configured to collect VR evaluation data including VR video provided to a user by a virtual reality (VR) device and user data generated from the user's voice, gaze, face, and hands; and a child developmental disorder evaluation unit configured to evaluate the user's cognitive ability, social ability, and language ability for evaluating child developmental disorders based on the VR evaluation data.

[0014] The above VR evaluation data collection unit may include a data management unit that manages data collected from a head-mounted display (HMD) equipped with a camera; and an interaction management unit that manages the interaction between the user and an object within a VR video corresponding to a virtual world.

[0015] The above data management unit may include a tracking space setting unit that manages a coordinate system defining the relationship between the real space and the virtual space where the user moves in a VR environment. The tracking space setting unit may include a left field of view management module that manages the left field of view; a right field of view management module that manages the right field of view; a middle field of view management module that manages the middle field of view; a left hand position management module that manages the position of the left hand; a right hand position management module that manages the position of the right hand; a left eye management module that manages the position of the left eye; a right eye management module that manages the position of the right eye; and an eye tracking module that checks whether the user's gaze is looking at an object within the VR image.

[0016] The tracking space setting unit may further include a blinking correction module that identifies a change in the user's gaze coordinates, classifies the change in the gaze coordinates as blinking when it exceeds a predefined threshold, and corrects the gaze position of the blinking interval to the average of the gaze point before the blinking and the gaze point after the blinking.

[0017] The interaction management unit may include: an HMD data management unit that manages HMD data generated from the HMD; a hand movement management unit that manages hand movement data related to the user's two hands; a controller data management unit that manages controller data operated by the user's two hands; and a synchronization module that visually synchronizes the user's two hands with the two hands of the virtual screen.

[0018] The above VR evaluation data collection unit may include a voice data collection unit that collects the voice of the user; and a video collection unit that collects a VR video input to the VR screen of the VR device and a video acquired regarding a scene in which the user interacts with the VR video.

[0019] The VR video provided to the user through the VR device may include a plurality of animation clips arranged sequentially. The VR evaluation data collection unit may adjust the order of the plurality of animation clips and provide them to the user according to at least one of the user type, user characteristics, and user tendencies related to the child developmental disorder.

[0020] The above-mentioned child developmental disability evaluation unit may include: a cognitive ability evaluation unit that evaluates cognitive abilities including shape recognition, object classification, and numerical understanding through the user's interaction with the VR video; a social ability evaluation unit that evaluates social abilities including social referencing and peer interaction through the user's interaction with the VR video; and a language ability evaluation unit that evaluates language abilities including language expression and language cognition through the user's interaction with the VR video.

[0021] The above child developmental disability evaluation unit can evaluate the user's cognitive ability, social ability, and language ability based on the user's facial expression changes and body movements regarding the VR video.

[0022] The above-mentioned child developmental disability evaluation unit may include: a gaze analysis module that analyzes the user's gaze; a facial expression analysis module that analyzes the user's facial expression; a voice-to-text analysis module that analyzes the user's voice; and an evaluation module that evaluates the user's potential for child developmental disability based on the analysis results of the user's gaze, facial expression, and voice.

[0023] The above-described gaze analysis module classifies the user's gaze movements into a fixed state in which the gaze is fixed at a specific point and saccadic movements corresponding to gaze movements between fixed states; and can evaluate the user's potential for childhood developmental disorders based on the time elapsed from the start of the stimulus provided by the VR video to the user's fixation, the time elapsed to gaze at the object of interest, the time elapsed to gaze at the object of interest, spatiotemporal information of the user's gaze movement, and the number of times the gaze was returned to the object of interest.

[0024] The facial expression analysis module can evaluate the possibility of a child developmental disorder in the user by analyzing facial movements including eye blinking, lip movements, and eyebrow and glabella movements.

[0025] The above-described voice-to-text analysis module can evaluate the possibility of a user's child developmental disorder by converting the user's voice into text and evaluating a language evaluation scale including the user's vocabulary development level, average utterance length, and speaking structure.

[0026] A diagnostic assistance device for child developmental disorders according to an embodiment of the present invention may further include an evaluation result output unit that outputs an evaluation result for said child developmental disorder.

[0027] The above evaluation result output unit may include an evaluation data extraction management unit that extracts and manages evaluation data serving as the basis for evaluating the child developmental disorder of the user based on correlations related to the child developmental disorder among the VR evaluation data obtained for the user.

[0028] The evaluation data extraction management unit above extracts video or audio portions that serve as the basis for evaluation of the child developmental disorder from the VR evaluation data above; and can store information corresponding to the start and end sections of the extracted video or audio portions.

[0029] According to an embodiment of the present invention, a computer-readable non-transient recording medium is provided on which a program is recorded for executing a method to assist in the diagnosis of a child developmental disorder. The method to assist in the diagnosis of a child developmental disorder comprises: a step of collecting VR evaluation data by a VR evaluation data collection unit, the VR video provided to a user by a virtual reality (VR) device, and user data generated from the user's voice, gaze, face, and hands; and a step of evaluating the user's cognitive ability, social ability, and language ability for the evaluation of a child developmental disorder by a child developmental disorder evaluation unit, based on the VR evaluation data.

[0030] According to an embodiment of the present invention, a diagnostic aid for child developmental disorders and a diagnostic aid for child developmental disorders based on a virtual reality (VR) evaluation data collection pipeline technology are provided.

[0031] In addition, according to an embodiment of the present invention, VR evaluation data can be reliably acquired from a user based on VR evaluation data collection pipeline technology, thereby effectively assisting in the diagnosis of developmental disorders in children.

[0032] In addition, according to an embodiment of the present invention, a VR evaluation data collection unit is configured to collect various VR evaluation data without API conflict issues, thereby effectively assisting in the diagnosis of developmental disorders in children.

[0033] In addition, according to an embodiment of the present invention, the possibility of a child's developmental disorder can be accurately assessed by evaluating the user's cognitive ability, social ability, and language ability through user interaction with VR video.

[0034] The effects obtainable through the present invention are not limited to those described above, and other unmentioned technical effects will be clearly understood by a person skilled in the art from the description of the invention below.

[0035] FIG. 1 is a configuration diagram of a diagnostic assistance device for child developmental disorders according to one embodiment of the present invention.

[0036] FIG. 2 is a flowchart of a method to assist in diagnosing child developmental disorders according to one embodiment of the present invention.

[0037] FIG. 3 is a conceptual diagram of a VR evaluation data collection pipeline for collecting VR evaluation data according to an embodiment of the present invention.

[0038] FIG. 4 is an example diagram showing the hierarchical structure of 'OVRCameraRig' corresponding to the data management unit constituting the child developmental disorder diagnosis assistance device according to an embodiment of the present invention.

[0039] FIG. 5 is an example diagram showing the hierarchical structure of 'OVRInteraction' corresponding to the interaction management unit constituting the child developmental disorder diagnosis assistance device according to an embodiment of the present invention.

[0040] FIG. 6 is an exemplary diagram showing a manager UI that constitutes a diagnostic assistance device for child developmental disorders according to an embodiment of the present invention.

[0041] FIGS. 7 and FIGS. 8 are exemplary diagrams of VR content used in a diagnostic aid device and method for child developmental disorders according to an embodiment of the present invention.

[0042] FIG. 9 is a flowchart showing step S200 of FIG. 2 in more detail.

[0043] FIG. 10 is a flowchart illustrating the process of evaluating a child's developmental disorder according to one embodiment of the present invention.

[0044] Hereinafter, embodiments of the present invention are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention. The present invention may be embodied in various different forms and is not limited to the embodiments described herein. It should be noted that the drawings are schematic and not drawn to scale. Identical structures, elements, or parts appearing in two or more drawings are given the same reference numerals to indicate similar features.

[0045] The embodiments of the present invention specifically illustrate ideal embodiments of the present invention. As a result, various variations of the illustrations are expected. Accordingly, the embodiments are not limited to specific forms of the illustrated areas. Unless otherwise defined, all technical and scientific terms used herein have the meaning generally understood by those skilled in the art to which the present invention pertains. All terms used herein are selected for the purpose of further clarifying the present invention and are not selected to limit the scope of the rights according to the present invention.

[0046] Expressions used herein such as 'comprising,' 'comprising,' 'having,' etc., should be understood as open-ended terms implying the possibility of including other embodiments, unless otherwise stated in the phrase or sentence containing such expressions. Singular expressions described herein may include a plural meaning unless otherwise stated, and this applies likewise to singular expressions described in the claims.

[0047] As used in this specification, 'module' and 'part' refer to a unit that processes at least one function or operation, and may refer to hardware components such as software, an FPGA, or one or more processors. In describing embodiments of the present invention, if it is determined that a detailed description of related known functions or known configurations may unnecessarily obscure the essence of the present invention, such detailed description may be omitted.

[0048] A diagnostic aid device and method for child developmental disorders according to an embodiment of the present invention collects VR evaluation data including VR images provided to a user by a virtual reality (VR) device and user data generated from the user's voice, gaze, face, and hands, and evaluates the user's cognitive ability, social ability, and language ability based on the collected VR evaluation data to assist in the diagnosis of child developmental disorders. According to an embodiment of the present invention, various VR evaluation data can be reliably acquired from a user based on VR evaluation data collection pipeline technology to effectively assist in the diagnosis of child developmental disorders.

[0049] FIG. 1 is a configuration diagram of a diagnostic aid for a child developmental disorder according to an embodiment of the present invention. FIG. 2 is a flowchart of a diagnostic aid method for a child developmental disorder according to an embodiment of the present invention. Referring to FIG. 1 and FIG. 2, a diagnostic aid for a child developmental disorder (10) according to an embodiment of the present invention is intended to assist in the diagnosis of a child developmental disorder, and for this purpose, it may include a VR evaluation data collection unit (100), a child developmental disorder evaluation unit (200), and an evaluation result output unit (300).

[0050] Child developmental disorder may refer to a situation in which a child's development, such as cognitive, social, or emotional abilities, is delayed compared to peers or experiences difficulties in specific areas during the developmental process. The diagnostic aid and method for child developmental disorder according to an embodiment of the present invention can be utilized as an auxiliary tool for diagnosing developmental disorders in users who are children. In particular, targeting children aged 3 to 6 years, it can accurately evaluate child developmental disorders based on a pipeline technology that collects and stores all data generated while using VR content, such as VR content that evaluates cognitive, social, and language abilities, and video recordings of the user's voice, gaze, face, hands, and reactions to the VR content.

[0051] A VR evaluation data collection unit (100) can collect VR evaluation data including VR video provided to a user by a VR device and various user data generated from the user's voice, gaze, face, hands, and other body parts movements (step S100 of FIG. 2). The VR device may be provided as, for example, a head-mounted display (HMD) worn on the head of a child corresponding to the user. The HMD may be provided with a VR video output unit (not shown) that provides VR video to the user, a microphone that collects the user's voice, an eye-tracking module that tracks the user's gaze, and one or more video recording devices that record the movements of the user's face, hands, and other body parts. Some of the video recording devices may be provided as external devices separate from the HMD.

[0052] User data may include data acquired based on user interaction related to VR video (VR content) provided to the user, such as user voice data, gaze data, facial expression data, hand movement data, and other body movements. The VR evaluation data collection unit (100) may include a voice data collection unit (e.g., a microphone) that collects the user's voice, a VR video input to the VR screen of a VR device, and a video collection unit that collects images acquired regarding scenes in which the user interacts with the VR video (e.g., a camera mounted on the VR device and / or a camera that captures the body movements of a user wearing the VR device, a video input / output device, etc.). The VR evaluation data collected by the VR evaluation data collection unit (100) may be provided to the child developmental disability evaluation unit (200) for the auxiliary diagnosis of child developmental disabilities.

[0053] The child developmental disability evaluation unit (200) evaluates the user's cognitive ability, social ability, and language ability based on VR evaluation data obtained by the VR evaluation data collection unit (100), and can evaluate the child developmental disability based on the comprehensive evaluation results of the user's cognitive ability, social ability, and language ability (step S200 of FIG. 2). In order for the child developmental disability evaluation unit (200) to accurately evaluate the user's cognitive ability, social ability, and language ability, it is necessary to configure the VR evaluation data collection unit (100) so that user data can be collected accurately and stably without conflict issues between APIs for collecting data by cameras, HMD cameras, microphones, etc.

[0054] FIG. 3 is a conceptual diagram of a VR evaluation data collection pipeline for collecting VR evaluation data according to an embodiment of the present invention. Referring to FIGS. 1 to 3, in order to collect user data accurately and stably without conflict issues between APIs, the VR evaluation data collection unit (100) may be implemented by dividing it into two or more parts, including a part corresponding to a data management unit (110) and a part corresponding to an interaction management unit (120).

[0055] The data management unit (110) can manage data collected from an HMD equipped with a camera. The data management unit (110) may include a tracking space setting unit that manages a coordinate system defining the relationship between the real space and the virtual space where the user moves in a VR environment. The tracking space setting unit may be designed considering the physical characteristics of the user, such as a child. The tracking space setting unit may include a left field of view management module (111) that manages the user's left field of view, a right field of view management module (112) that manages the user's right field of view, a middle field of view management module (113) that manages the user's middle field of view, a left hand position management module (114) that manages the position of the user's left hand, a right hand position management module (115) that manages the position of the user's right hand, a left eye management module (116) that manages the position of the user's left eye, a right eye management module (117) that manages the position of the user's right eye, an eye tracking module (118) that checks whether the user's gaze is looking at an object within the VR video, and a blink correction module (119) that corrects the gaze position according to the user's eye blinking.

[0056] The left field of vision management module (111), right field of vision management module (112), middle field of vision management module (113), left hand position management module (114), right hand position management module (115), left eye management module (116), right eye management module (117), and eye tracking module (118) can set shooting areas and / or areas of interest such as the user's two eyes and two hands, and collect and manage data regarding the movement of the user's two eyes and two hands, etc., for the set shooting areas and / or areas of interest.

[0057] The blink correction module (119) identifies changes in the user's gaze coordinates and can classify the changes in the gaze coordinates as eye blinking when they exceed a predefined threshold. When the user's eye movement is classified as eye blinking, the blink correction module (119) can correct the gaze position of the blinking interval corresponding to the eye blinking to the average of the gaze point before the blinking and the gaze point after the blinking. Accordingly, the discontinuity of the gaze coordinates caused by the user's eye blinking is resolved, and natural gaze movements can be extracted and utilized for analysis to diagnose developmental disorders in children.

[0058] The interaction management unit (120) can manage interactions with objects within the VR video corresponding to the user's virtual world. The interaction management unit (120) may include an HMD data management unit (121) that manages HMD data generated from the HMD, a hand movement management unit (122) that manages hand movement data related to the user's two hands, a controller data management unit (123) that manages controller data operated by the user's two hands, and a synchronization module (124) that visually synchronizes the user's two hands with the two hands of the virtual screen.

[0059] Referring to the example in FIG. 3, a pipeline technology for collecting VR evaluation data according to an embodiment of the present invention will be described. The player shown in FIG. 3 corresponds to a game object (20) representing a user who is the subject of evaluation for a child developmental disability. The VR evaluation data collection unit (100) may be implemented by dividing it into two parts: a data management unit (110), which is a part that manages data collected from an HMD including a camera, and an interaction management unit (120), which manages user interaction with VR content.

[0060] The pipeline technology may be configured to include a data acquisition unit (30) for acquiring VR evaluation data for a game object (20) corresponding to a user, and a data processing unit (40) for generating VR evaluation data (50) by processing the data acquired by the data acquisition unit (60). The data acquisition unit (30) may include an observer camera (31) for capturing the user's body movements, a camera (32) for capturing the user's left / right eyes and an eye-tracking module (EyeRaycaster) (33), an HMD (34) equipped with the camera (32), an expression acquisition module (OVR Face Expression) (35) for acquiring the user's facial expressions, a hand observation module (36) for observing both of the user's hands, and a voice acquisition module (37) for acquiring the user's voice.

[0061] The data processing unit (40) may include a video recorder (41) that generates video data (51) by recording images acquired by an observer camera (31), a data processing module (CSVwrite) (42) that generates user data (Eyegaze, Face, Hand Coordinates) (52) such as eye coordinates, face expression data, and hand coordinates by processing images acquired for the user's face, both hands, both eyes, etc., and a voice recorder (47) that generates voice data (53) by processing the user's voice. The data processing module (42) may include an eye gaze observation module (eyeGazeObserver) (43) that analyzes the user's eye gaze data, a face / hand observation module (HeadObserver, HandObserver) (44, 45) that observes the user's head, face, and face expressions, etc., and a hand observation module (HandObserver) (46) that observes the user's hands.

[0062] FIG. 4 is an exemplary diagram showing the hierarchical structure of 'OVRCameraRig' corresponding to the data management unit constituting the child developmental disorder diagnosis assistance device according to an embodiment of the present invention. With reference to FIGS. 1 to 4, the hierarchical structure of 'OVRCameraRig' corresponding to the data management unit (110) will be described. The Unity of the data management unit (110) can provide an Oculus Plugin and OpenXR that can be used in a Head Mounted Device (HMD). OpenXR is a standard VR / AR API developed by the Khronos Group and can be designed to be compatible with various VR / AR hardware and software. As a hardware-independent API, it enables VR / AR developers to use the same interface on multiple devices, thereby increasing compatibility between various VR / AR platforms and providing a consistent user experience.

[0063] OVRPlugin is a plugin provided by Meta, designed primarily for optimization and integration with Meta’s VR devices, including Oculus headsets. OVRPlugin provides features and performance tailored to the Oculus environment, enabling a high level of optimization and access to additional features on Oculus devices. "OVRCameraRig" sets the "TrackingSpace" corresponding to the tracking space setting section of the data management section (110). This may consist of a module managing the user's left field of view, right field of view, center field of view, and both hand positions, and a module managing the positions of both eyes. "TrackingSpace" is a coordinate system that defines the relationship between the real space where the user moves in a VR environment and the virtual space that reflects it. It tracks the user's position and movements in a VR system such as Oculus to maintain a consistent position and orientation in the virtual environment. This may be the initial reference position of the VR device, and the relative positions of objects within the VR environment can be determined from this position. In other words, "TrackingSpace" is related to the physical range that can read the user's movements and reflect them in virtual reality.

[0064] "TrackingSpace" sets "leftEyeAnchor" and "rightEyeAnchor" based on the user's left and right fields of vision. "leftEyeAnchor" and "rightEyeAnchor" correspond to the user's two eyes, and "leftEyeAnchor" can be positioned to the left and "rightEyeAnchor" to the right. "leftEyeAnchor" and "rightEyeAnchor" can be set, for example, as "leftEyeAnchor.localPosition = new Vector3(-0.1f, 0, 0); rightEyeAnchor.localPosition = new Vector3(0.1f, 0, 0);".

[0065] "leftHandAnchor" and "rightHandAnchor" can be set to the left and right positions, respectively, according to the user's hand position. Since VR content is intended for children, "leftHandAnchor" and "rightHandAnchor" can be positioned as follows: for example, "leftHandAnchor.localPosition = new Vector3(-0.1f, -0.25f, 0.25f); rightHandAnchor.localPosition = new Vector3(0.1f, -0.25f, 0.25f);".

[0066] The "eyeRaycaster" module is used to check whether the user's gaze is directed at a specific object within the VR content. To avoid conflicts with the "eyeRaycaster" module, you can configure it by adding a new object responsible for the user's two fields of view as a child of "OVRCameraRig," and adding the "RayCasters" for the user's two eyes—which correspond to game objects—as components.

[0067] FIG. 5 is an exemplary diagram showing the hierarchical structure of 'OVRInteraction' corresponding to the interaction management unit constituting the child developmental disorder diagnosis assistance device according to an embodiment of the present invention. With reference to FIGS. 1 to 3 and FIG. 5, the hierarchical structure of 'OVRInteraction' corresponding to the interaction management unit (120) will be described. "OVRInteraction" is a module used for a user to interact with an object in virtual reality. It supports the user in performing interactions such as touching and throwing a model of VR content, and for this purpose, it is a module that enables the user to utilize a controller or fingers. The data handled by "OVRInteraction" may include data generated from the HMD, movement data of both hands, and controller data related to controller operation. In particular, "OVRHands" can be utilized by adding "HandGrabInteractor" for physical interactions such as holding and throwing objects. To visually synchronize the user's actual hand with the hand on the virtual screen, a "HandSynthetic" module can be added to make the movement of the actual hand match the virtual hand.

[0068] The "OVRCameraRig" and "OVRInteraction" modules, which are the main modules for data collection, are composed as children of the same object and can manage all data generated by game objects (Players). In particular, data used for the diagnosis of developmental disorders in children is collected through "OVRCameraRig," which enables the collection and analysis of the user's field of vision and gaze data.

[0069] To interact with virtual reality models (objects), the "OVRCameraRig" module, which manages data generated from game objects (Players), and the "OVRInteraction" module, which manages direct interaction, must be configured together. The data collected within the "Player" game object may include data for both fields of view, for example, by including both a bilateral field of view camera and a center field of view camera. For the camera serving as the basis for diagnosing developmental disorders in children, the primary camera may utilize, for instance, a center field of view camera.

[0070] To collect clean data, a "VoiceRecorder" can be configured on a game object. While an "AudioSource" is typically assigned to a "NPC (Non-Player Character)," it is essential to assign it to a game object to collect the user's voice. This module collects user voice data in real time and can analyze children's interaction responses or linguistic expressions based on it.

[0071] "VideoRecorder" can acquire video footage derived from game objects (Players). The footage acquired by "VideoRecorder" can be classified into video entering the VR screen and video viewable in the administrator UI. In particular, to analyze user VR usage alongside the video viewed in the administrator UI, it is necessary to record the screen entering the VR and posture data, including head movements. The footage acquired by "VideoRecorder" can be utilized for the analysis of developmental disorders in children.

[0072] FIG. 6 is an exemplary diagram showing an administrator UI that constitutes a diagnostic assistance device for child developmental disorders according to an embodiment of the present invention. Referring to FIG. 6, the data storage process is described as follows: HMD detection is performed simultaneously with the start of content, and recording begins when the "Recording / Start" button is pressed after entering user information. When the "EXIT" button is pressed, the recorded video is saved. The collected data may be stored in a user-specified folder along with a timestamp. To facilitate data management and retrieval, a unique identifier (ID) may be assigned to the saved file.

[0073] The administrator UI provides a function to monitor children's VR usage in real time. For efficient data management, the "Record / Start" button may not be activated if the child's study number is not entered or if the HMD is not detected. Content related to the assessment of cognitive, language, and social abilities can be performed in batches depending on the child's condition. Each piece of content may consist of more than 10 questions, each composed of approximately 3 to 5 animation clips, and the order of the animation clips may be adjusted based on the judgment of the researcher / administrator or the user's type, characteristics, or tendencies.

[0074] If the child removes the HMD or the content is not recognized while performing the content, the content may be automatically paused. If the HMD is recognized without pressing a separate button, playback resumes from the moment it stopped, and the researcher may also manually choose to pause and play based on their judgment. As an important part of the administrator's UI, since the children performing the content are aged 3 to 6 and their heights vary when seated, a 'Center' button to shift the gaze to the center of the "tracking space," a function to select the camera's up, down, left, and right, and a function to move forward and backward (z-value) may be included.

[0075] VR video provided to a user through a VR device may include multiple animation clips arranged sequentially. The VR evaluation data collection unit (100) may provide the user with the order of multiple animation clips adjusted according to user types related to child developmental disabilities (e.g., age, gender, etc.), user characteristics (e.g., physical, mental, or environmental factors related to the child, etc.), and / or user tendencies (e.g., distracted children who frequently remove the HMD, personal preferences or areas of interest, etc.).

[0076] The child developmental disability evaluation unit (200) may include a cognitive ability evaluation unit (210), a social ability evaluation unit (220), and a language ability evaluation unit (230). The cognitive ability evaluation unit (210) can evaluate cognitive abilities including shape recognition, object classification, and numerical understanding through user interaction with VR video. The social ability evaluation unit (220) can evaluate social abilities including social referencing and peer interaction through user interaction with VR video. The language ability evaluation unit (230) can evaluate language abilities including language expression and language cognition through user interaction with VR video.

[0077] The child developmental disability evaluation unit (200) can evaluate the user's cognitive ability, social ability, and language ability based on the user's facial expression changes and the user's physical movements in relation to VR video. The child developmental disability evaluation unit (200) may include an eye gaze analysis module that analyzes the user's gaze, an expression analysis module that analyzes the user's facial expressions, a voice-text analysis module that analyzes the user's voice, and an evaluation module that evaluates the user's potential for child developmental disability based on the analysis results of the user's gaze, facial expressions, and voice.

[0078] The gaze analysis module can classify a user's gaze movements into a fixed state, where the gaze is fixed on a specific point, and saccadic movements, which correspond to the movement of the gaze between fixed states. Based on the time elapsed from the onset of stimulation provided by the VR video to the user's fixation, the time spent gazing at the object of interest, the duration of gaze on the object of interest, spatiotemporal information regarding the movement of the user's gaze, and the number of times the gaze was returned to the object of interest, the gaze analysis module can assess the user's potential for childhood developmental disorders.

[0079] The facial expression analysis module can assess the user's potential for childhood developmental disorders by analyzing facial movements, including eye blinking, lip movements, and eyebrow-glabella movements. The speech-text analysis module can assess the user's potential for childhood developmental disorders by converting the user's voice into text and evaluating language assessment scales that include the user's vocabulary development level, average utterance length, and speech structure.

[0080] The evaluation result output unit (300) can output evaluation result data related to child developmental disability evaluated by the child developmental disability evaluation unit (200) to a display device or an administrator / guardian terminal, etc. The evaluation result output unit (300) may include an evaluation data extraction management unit (310). The evaluation data extraction management unit (310) can extract and manage evaluation data that serves as the basis for evaluating the child developmental disability of a user based on correlations related to child developmental disability among the VR evaluation data obtained for the user.

[0081] Specific examples of an assistive device and method for diagnosing child developmental disorders according to an embodiment of the present invention are described below. To develop an assistive tool for the early diagnosis of child developmental disorders based on the metaverse, cognitive and social assessment scripts were prepared through a comprehensive literature review of existing developmental disorder assessment tools. Subsequently, the initial draft underwent expert verification, including two rounds of feedback from experts in the field of child development. In the initial stage, eight experts reviewed the script through a written survey, and subsequently, five experts participated in in-depth interviews to finalize the script. The final cognitive script consists of 10 categories and 30 items targeting essential areas of cognitive development in early childhood, as shown below. Examples of items for evaluating cognitive function are as follows.

[0082] - Memory Recall (3 items): Ask the child to recall and match the same object.

[0083] - Object Naming (4 items): Specify familiar object names.

[0084] - Recognition of Similarity (3 items): Add a demonstration to identify similar stimuli and enhance understanding.

[0085] - Classification (6 items): Classifies objects into logical groups and simplifies them to reduce cognitive load.

[0086] - Color Naming and Distinction (4 items each): Identify and name various colors.

[0087] - Calculate (1 item): Calculates individual objects.

[0088] - Basic Comparison (3 items): Understand and express the differences between objects.

[0089] To facilitate social interaction, the script was optimized for simplicity and familiarity. Activities involving interaction with peers and reactions to teacher avatars were streamlined to encourage more natural responses. Based on feedback that group activities could overwhelm young children, tasks involving multiple peers were simplified to include only a single peer. Two additional tasks were incorporated into the assessment to evaluate responses to social cues, such as greetings and mimicking peer behavior. The social script for assessing social functioning consists of 12 items focusing on core social behaviors, such as taking orders, cooperation, and emotional expression; examples are provided below.

[0090] - Greetings

[0091] Say your name.

[0092] - Follow the response: Wave your hand.

[0093] - Reaction to the name

[0094] - Answer to the teacher's question: This is a question about the experience of playing with blocks.

[0095] - Responding after listening to the teacher's explanation: Explanation of block play

[0096] - Expressing your opinion on the question: Suggest playing with blocks.

[0097] - Understanding the Rules of Play: Explanation of the Order of Play

[0098] - Changing order: You can see who is taking their turn in block play.

[0099] - Talking about feelings: Express the feeling of playing with blocks.

[0100] - Empathy: The reaction when a colleague is sad.

[0101] - Help a Friend: A reaction to prevent the top of the block from collapsing.

[0102] A language screening tool for evaluating language function was developed to assess the communication abilities of children aged 3 to 4 years. The language tasks were designed to evaluate three major language domains: comprehension, expression, and cognitive language ability. The items were classified according to the specifications described in the language domain evaluation items for children with developmental disabilities shown in Table 1 below.

[0103] DomainNounsVerbsAdjectives / AdverbsSentencesLanguage CognitionTotal ItemsCognitive LinguisticsObject Function / Role / Memory3Language ComprehensionBird, Fruit, TailTear, FollowHappy, Sharp'to', 'for', 'and' Negative Sentences11Language ExpressionFridge, TomatoEat, Push, FallBig6Total Items5534320

[0104] A preliminary study was conducted on five 2-year-olds and three 3-year-olds to evaluate the age appropriateness and accuracy of the screening items. Following data collection and a refinement process, 20 final items were selected that accurately reflect the language development of the children. The selected cognitive, social, and linguistic assessment items were implemented within a VR environment to provide children with an immersive experience. The VR scenarios were designed to simulate real-world environments, such as homes or playgrounds, through transitions between indoor and outdoor settings to maintain child engagement. The VR interface is structured around tasks children are already familiar with, allowing them to interact with avatars and virtual objects in the following ways: - Environment: Familiar home and outdoor environments are created to ensure the child's comfort.

[0105] - Avatar: The teacher avatar guides the child by providing voice prompts such as "What is this?" or "Please sort the toys by color."

[0106] - Interactive task: Children record their behavior for real-time analysis by physically grasping objects or responding verbally.

[0107] FIGS. 7 and 8 are exemplary diagrams of VR content utilized in the diagnostic aid device and method for child developmental disorders according to an embodiment of the present invention. To verify whether the content was suitable for development, objects and avatars were created using software such as Unity Asset Store, Sketchfab, and MAYA. In particular, motion capture was used to mimic the natural gestures of the avatar interacting with the child. By utilizing motion capture technology, the avatar's movements and object manipulation capabilities were enhanced to facilitate accurate and natural interaction between the child and the virtual environment. To accommodate the motor skills of young children, hand tracking technology was used to allow children to intuitively grasp objects by adjusting their finger angles. Additionally, modifications were made to improve the interaction experience with objects outside the recognition range of the virtual reality device. This prevented objects from falling when the child's hand moved out of the detection area and back in, thereby improving the fluidity and engagement of the task.

[0108] This study aimed to utilize gaze data collected during the execution of VR content in the context of gaze data from typically developing children and children with developmental disabilities. The VR content used in this study was created by experts for the purpose of evaluating developmental disabilities, with the aim of comparing and analyzing the gaze patterns of children with developmental disabilities and children with developmental delays. Through this, the children's visual attention patterns and cognitive processes were analyzed using gaze pattern analysis. The modules required to construct the analysis model are as follows.

[0109] 1. Eye Tracking Module - Eye tracking analysis focuses on understanding the Area of ​​Interest (AOI) where the user pays attention. Since eye movements provide important clues about user engagement, this applies equally to VR environments. The Meta Quest Pro, a VR device, includes an integrated sensor that tracks eye movements by capturing binocular gaze coordinates, and this was utilized to collect data for the assessment of developmental disabilities in children. Using the Meta Quest Pro, which features a 106° (horizontal) and 96° (vertical) field of view, eye movements were tracked at a rate of 90 times per second.

[0110] Blink Correction Module - Voluntary or involuntary eye blinking causes abrupt changes in gaze coordinates, introducing noise into gaze data. Therefore, to maintain the accuracy of gaze tracking data, correction for blink events is required. For blink correction, a process is performed to identify sudden changes in gaze coordinates and classify them as blinks when these changes exceed a predefined threshold. The corrected gaze position is calculated as the average of the previous and next gaze points, ensuring a smoother and more natural gaze trajectory.

[0111] Eye Gaze Analysis Module – Eye movements can be broadly classified into fixed gaze and saccadic gaze. Fixed gaze refers to the moment when the gaze remains fixed on a specific point, whereas saccadic gaze refers to rapid eye movements between fixed gazes when a user searches for visual information. To analyze children's developmental patterns, these eye movements can be classified and various indicators calculated to evaluate children's visual behavior. Based on fixed gaze, saccadic gaze, and Area of ​​Interest (AOI), the Eye Gaze Analysis Module can calculate various metrics, including the following:

[0112] - Time to First Fixation: The period between the onset of stimulation and the child's first fixation on a specific object (AOI; Areas of Interest).

[0113] - Dwell Time: The total time spent taking a specific AOI

[0114] - Rate: Time spent by respondents looking at a specific AOI

[0115] - Fixed sequence: Information about when and where the participant viewed it, based on spatial and temporal information

[0116] - Number of return visits: The number of times the child returned their gaze to the AOI they had previously seen.

[0117] - First fixation period: Information on how long the first fixation lasted.

[0118] - Average duration of fixation: Information on how long the average fixation lasted (information by individual or group)

[0119] In addition to fixed periods and heatmap visualizations, these metrics can provide insights into children's cognitive processing and visual attention patterns. Furthermore, this data can be utilized in additional statistical analysis or machine learning algorithms to compare children's developmental trajectories and distinguish between children who develop normally and those who show developmental delays.

[0120] 2. Facial Expression Analysis Module – Meta Quest Pro’s facial blendshape technology holds potential for a wide range of applications beyond simple Virtual Reality (VR) and Mixed Reality (MR) interactions. In particular, the ability to accurately track facial expressions and muscle movements can serve as an important tool for the early diagnosis and intervention of developmental disorders. Children with developmental disorders often exhibit subtle differences in facial expressions and muscle movements. For example, children with developmental disorders may be unable to display appropriate facial expressions during social interactions or may have limitations in expressing emotions.

[0121] Meta Quest Pro's face blend shape technology is an effective tool in developing diagnostic and intervention content for the early detection of developmental disabilities. For example, it can analyze real-time facial expression changes as children respond to various stimuli in a virtual environment, and evaluate emotional responses and social interaction abilities. This data can help design intervention programs by identifying a child's specific weaknesses and strengths.

[0122] Meta Quest Pro's facial blend shape technology can capture these subtle nuances of facial expressions with high precision, providing real-time data for analysis. Factors such as the frequency of a child's blinking or subtle facial expressions like lip movements can play a crucial role in detecting signs of developmental disorders. Various facial muscle movements, including the space between the eyebrows, eye blinking, and lip movements, can be captured using Meta Quest Pro's blend shape capabilities. Each blend shape can be quantified, for example, on a range from 0 to 1, enabling immediate analysis without preprocessing. This quantitative data is advantageous for monitoring and analyzing facial movements and expressions in real time.

[0123] Blend shape technology can be directly applied to track facial movement patterns in children with developmental delays, aiding in early diagnosis and the development of intervention strategies. Detecting abnormal muscle movements based on blend shape data and analyzing them to create intervention strategies for the facial expressions and emotional responses of children with developmental disabilities is a crucial step in the early diagnosis and intervention process. Furthermore, content developed using this technology can facilitate the improvement of diagnostic support for childhood developmental disorders by providing feedback on the child's facial muscle movements, going beyond simple interaction.

[0124] Through real-time facial expression analysis, it is possible to monitor whether a child is expressing emotions appropriately and apply customized interventions accordingly. This approach significantly contributes to supporting the social and emotional development of children with developmental delays. Therefore, Meta Quest Pro's face blendshape technology can serve as an innovative tool for the early diagnosis and intervention of children with developmental delays. Real-time analysis and feedback capabilities based on precise facial expression data play a pivotal role in enhancing the efficiency of diagnostic and intervention content.

[0125] 3. Speech-to-Text (STT) Module – In early language screening tests, a child's vocabulary development level, mean utterance length (MLU), and speech structure serve as important language development measures for evaluating typical development. During the assessment, eye tracking not only provides important data on engagement and interest but can also help identify the causes of incorrect responses. Below, we outline the development procedure for an early language development screening test utilizing the metaverse and present language evaluation scales for early screening. Additionally, we describe the automatic Speech-to-Text (STT) module and speech analysis system for analyzing children's utterances within the metaverse environment.

[0126] While it is possible to capture a child's voice using Meta Quest Pro, problems arise where it is difficult to accurately evaluate the child's response if background noise or multiple voices are recorded simultaneously. Unless these issues are resolved, it is impossible to fully utilize the STT module. Therefore, in the audio data preprocessing stage, it is necessary to perform Voice Activity Detection (VAD) and Speaker Diarization in advance. VAD is a technique used to distinguish between actual speech segments and non-speech or noise segments within an audio signal. Audio data collected from various environments often contains unnecessary noise and silent parts. VAD improves data processing efficiency and enhances model performance by removing these non-speech segments and extracting only the parts containing speech. After VAD, a speaker diarization process is applied to classify speakers based on speech characteristics. In the embodiment of the present invention, the focus was placed on distinguishing between the voice of an adult female and a child, as well as distinguishing the voice of a guardian, in order to remove noise and analyze the child's response more accurately.

[0127] VR content for the early screening and intervention of children with developmental disabilities is divided into three parts: cognitive, linguistic, and social. The duration of the VR content was designed to be approximately 5 minutes. To avoid providing special stimulation, two avatars were created: one featuring a warm-hearted teacher commonly seen in Korean kindergartens, and another representing an ordinary child without distinctive features such as clothing or appearance. Several conditions were carefully observed during the development and utilization of the VR content for the experiment. During the development process, the VR content was designed to be controller-free, taking into account the children's hand sizes and motor skills. Since some children may struggle to form a pincer grip, object manipulation was implemented using fist gestures instead of pinching motions. Range limits were established to prevent unexpected behaviors, such as throwing objects outside the boundaries of the content. Additionally, to enhance experimental efficiency and ensure the accuracy of results, a module was developed that allows experts to pause the experiment or modify the content based on their judgment.

[0128] During the experiment phase, VR headsets were not worn for more than 10 minutes to prevent dizziness, and the test was stopped immediately if symptoms were reported. Considering the young age of the children, the test was conducted with the participation of an expert and a primary caregiver. The test was stopped if the child showed signs of discomfort or refusal. In the experiment involving approximately 20 children, no children refused to participate, and no serious side effects from the VR content were observed.

[0129] Eye gaze analysis results - The eye tracking module was found to have the ability to successfully capture and correct data missing due to blinking, achieving a blink detection accuracy of 92.7%. In addition, an analysis of fixation and saccadic movement patterns revealed that children with developmental delays have short fixation times and irregular eye sequences, indicating difficulties with sustained attention and visual processing.

[0130] Facial Expression Analysis Results - The facial expression module showed 99.1% accuracy in classifying adult emotional responses through the deployment of a Cubic SVM model.

[0131] Speech Analysis Results - The STT module fine-tuned using Whisper demonstrated a significant improvement in children's speech recognition, exhibiting the lowest character error rate (CER) of 9.56% among the tested models. This improvement in transcription accuracy has made it possible to conduct a more reliable assessment of children's language abilities, including vocabulary, sentence structure, and pronunciation.

[0132] As described above, a diagnostic tool capable of early detection of developmental delays in children was developed using VR technology. In particular, a module for the following was introduced.

[0133] 1. Cognitive Assessment: To assess cognitive abilities, tasks such as memory recall, object naming, and classification were included in the VR content. In the VR environment, interaction with virtual objects is possible, and memory, concept recognition, and classification skills can be evaluated by measuring children's reaction times and accuracy.

[0134] 2. Social Interaction: Scenarios in VR content can evaluate social skills through tasks such as interacting with a virtual teacher or peer, handling objects according to instructions, cooperative play, and recognizing social cues. This may include observing social norms during play, such as taking turns participating in a ball-throwing game.

[0135] 3. Language Development: Language tasks can assess vocabulary, sentence generation, and speaking patterns. By using real-time speech-to-text technology, the assessment can include complex language tasks such as color discrimination and comparison, which can improve the accuracy of language development assessments.

[0136] 4. Attention Tracking: Using eye-tracking technology, it is possible to analyze where a child's gaze is directed within a virtual environment and evaluate attention and cognitive processing through the gaze path and task duration.

[0137] 5. Emotional Response: The facial expression module uses blend shape technology to analyze facial muscle movements and capture subtle emotional responses in real time. This can provide insight into a child's emotional state while interacting within a VR environment.

[0138] The integration of the aforementioned modules can improve the accuracy of the assessment of developmental disabilities in children and help establish targeted intervention strategies for children with developmental disabilities.

[0139] FIG. 9 is a flowchart illustrating step S200 of FIG. 2 in more detail. Referring to FIG. 1 and FIG. 9, the cognitive ability evaluation unit (210) can evaluate cognitive abilities including shape recognition, object classification, and numerical understanding through user interaction with VR video (S210). The social ability evaluation unit (220) can evaluate social abilities including social referencing and peer interaction through user interaction with VR video (S220). The language ability evaluation unit (230) can evaluate language abilities including language expression and language cognition through user interaction with VR video (S230).

[0140] FIG. 10 is a flowchart illustrating a process for evaluating a child developmental disorder according to an embodiment of the present invention. The gaze analysis module can classify the user's gaze movements into a fixed state in which the gaze is fixed on a specific point and saccadic movements corresponding to the gaze movements between the fixed states (S240). The gaze analysis module can evaluate the user's potential for a child developmental disorder based on the time elapsed from the start of the stimulus provided by the VR video to the user's fixation, the time elapsed to gaze at the object of interest, the time elapsed to gaze at the object of interest, the spatiotemporal information of the user's gaze movement, and the number of times the gaze was returned to the object of interest (S250).

[0141] The facial expression analysis module can assess the user's potential for childhood developmental disorders by analyzing facial movements, including eye blinking, lip movements, and eyebrow-glabella movements. The speech-text analysis module can assess the user's potential for childhood developmental disorders by converting the user's voice into text and evaluating language assessment scales that include the user's vocabulary development level, average utterance length, and speech structure.

[0142] The evaluation result output unit (300) can output evaluation result data of the child developmental disability evaluation unit (200). The evaluation result output unit (300) may include a display unit that outputs evaluation results, a communication unit that transmits evaluation results to a terminal such as an administrator or a child guardian, etc. The evaluation result output unit (300) may include an evaluation data extraction management unit (310). The evaluation data extraction management unit (310) can extract and manage evaluation data that serves as the basis for evaluating the child developmental disability of a user based on correlations related to the child developmental disability among the VR evaluation data obtained for the user.

[0143] The evaluation data extraction management unit (310) can extract parts such as video or audio that serve as the basis for the evaluation of a child's developmental disability and store information corresponding to the start and end sections of the extracted parts. When an administrator or guardian selects an evaluation-related data verification item to check the parts that serve as the basis for the evaluation of a child's developmental disability, the corresponding video or audio may be output. Accordingly, the administrator or guardian can check the data that serves as the basis for the user's evaluation of a child's developmental disability to improve their understanding of the evaluation of a child's developmental disability and use it as supplementary material to perform an accurate evaluation of the possibility of a child's developmental disability and derive a plan to improve the child's developmental process.

[0144] The evaluation data extraction management unit (310) may, upon the request of an administrator or guardian, play back data parts (video parts, audio parts) that serve as core grounds for evaluating a child developmental disorder, and at the same time provide additional reasons that serve as grounds for or grounds for not having a child developmental disorder in the relevant video parts and / or audio parts. For example, explanations regarding parts judged to be related to a child developmental disorder, such as scenes where the child's gaze is unstable, or the child's facial expressions, hand movements, eye gaze, or pitch instability while playing audio segments, may be provided to the administrator and / or guardian in the form of voice or text.

[0145] Early diagnosis and intervention for developmental disorders in children aged 3 to 6 using timely and repetitive content is essential for facilitating the overall education and developmental process. According to an embodiment of the present invention as described above, using the SDK of MetaQuest Pro, the system collected and analyzed data on facial expressions, gaze patterns, and voice responses to evaluate cognitive abilities such as shape recognition, object classification, and numerical comprehension, social abilities such as social referencing and peer interaction, and language abilities including both expressive and receptive language.

[0146] By utilizing AI to detect subtle changes in facial expressions and analyzing all reactions related to performance capabilities occurring during content performances, indicators capable of early diagnosis of developmental disorders were identified. This invention not only improves the diagnostic accuracy of childhood developmental disorders but also enhances the ability to monitor the effectiveness of interventions, thereby providing a foundation for an educational and metaverse-based therapeutic environment that supports the educational and developmental needs of children with developmental disorders.

[0147] At least some of the configurations of the embodiments described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an Arithmetic Logic Unit (ALU), a Digital Signal Processor, a microcomputer, a Field Programmable Gate Array (FPGA), a Programmable Logic Unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions.

[0148] The processing unit may execute an operating system and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, the processing unit may be described as being used as a single unit, but a person of ordinary skill in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements.

[0149] For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as a parallel processor, are also possible. Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure the processing unit to operate as desired or instruct the processing unit independently or collectively.

[0150] Software and / or data may be embodied in any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by a processing device or to provide instructions or data to a processing device. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0151] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software.

[0152] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiments, and vice versa.

[0153] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive, and the scope of the present invention is defined by the claims set forth below. Furthermore, all modifications or variations derived from the meaning and scope of the claims and their equivalents should be interpreted as being included within the scope of the present invention.

[0154] [Explanation of the symbol]

[0155] 10: Diagnostic aids for child developmental disorders

[0156] 100: VR Evaluation Data Collection Department

[0157] 110: Data Management Department

[0158] 111: Left field of view management module

[0159] 112: Right field of view management module

[0160] 113: Intermediate View Management Module

[0161] 114: Left hand position management module

[0162] 115: Right hand position management module

[0163] 116: Left eye management module

[0164] 117: Right Eye Management Module

[0165] 118: Eye Tracking Module

[0166] 119: Blinking Correction Module

[0167] 120: Interaction Management Department

[0168] 121: HMD Data Management Department

[0169] 122: Hand Movement Management Department

[0170] 123: Controller Data Management Department

[0171] 124: Synchronization Module

[0172] 200: Child Developmental Disabilities Assessment Department

[0173] 300: Evaluation result output section

[0174] 310: Evaluation Data Extraction Management Department

Claims

1. As an auxiliary diagnostic device for child developmental disorders to assist in the diagnosis of child developmental disorders, A VR evaluation data collection unit configured to collect VR evaluation data including VR video provided to a user by a virtual reality (VR) device and user data generated from the user's voice, gaze, face, and hands; and A child developmental disorder diagnostic aid comprising: a child developmental disorder evaluation unit configured to evaluate the user’s cognitive ability, social ability, and language ability for the evaluation of the child developmental disorder based on the above VR evaluation data.

2. In Claim 1, The above VR evaluation data collection unit A data management unit that manages data collected from a head-mounted display (HMD) equipped with a camera; and An assistive device for diagnosing child developmental disorders, comprising: an interaction management unit that manages the interaction between the user and an object within a VR video corresponding to a virtual world.

3. In Claim 2, The above data management department A tracking space setting unit that manages a coordinate system defining the relationship between the real space and the virtual space where the user moves in a VR environment; and The above tracking space setting unit A left vision management module that manages the left vision of the above user; A right vision management module for managing the right vision of the above user; An intermediate vision management module that manages the intermediate vision of the above user; A left hand position management module that manages the position of the user's left hand; A right hand position management module that manages the position of the user's right hand; A left eye management module that manages the position of the user's left eye; A right eye management module that manages the position of the user's right eye; and An eye tracking module that checks whether the user's gaze is looking at an object within the VR video; A diagnostic aid for child developmental disorders including 4. In Claim 3, The above tracking space setting unit A blink correction module that identifies changes in the user's gaze coordinates, classifies the changes in the gaze coordinates as blinking when they exceed a predefined threshold, and corrects the gaze position of the blinking interval to the average of the gaze point before the blink and the gaze point after the blink; A diagnostic aid for child developmental disorders that further includes 5. In Claim 4, The above interaction management unit HMD data management unit that manages HMD data generated from the above HMD; A hand movement management unit that manages hand movement data related to both hands of the above-mentioned user; A controller data management unit that manages controller data operated by both hands of the user; and A synchronization module that visually synchronizes the user's two hands with the two hands on a virtual screen; A diagnostic aid for child developmental disorders including 6. In Claim 5, The above VR evaluation data collection unit A voice data collection unit for collecting the voice of the above-mentioned user; and A video collection unit that collects a VR video input to the VR screen of the above-mentioned VR device and a video acquired regarding a scene in which the user interacts with the VR video; A diagnostic aid for child developmental disorders including 7. In Claim 1, The VR video provided to the user through the VR device includes a plurality of animation clips arranged sequentially, and The above VR evaluation data collection unit The order of the plurality of animation clips is adjusted according to at least one of the user type, user characteristics, and user tendencies related to the child developmental disorder, and provided to the user. Diagnostic aid for child developmental disorders.

8. In Claim 1, The aforementioned child developmental disability evaluation department A cognitive ability evaluation unit that evaluates cognitive abilities including shape recognition, object classification, and numerical understanding through the user's interaction with the above VR video; A social competency evaluation unit that evaluates social competencies, including social reference and peer interaction, through the user's interaction with the VR video; and A language ability evaluation unit that evaluates language ability, including language expression and language cognition, through the user's interaction with the VR video; A diagnostic aid for child developmental disorders including 9. In Claim 1, The aforementioned child developmental disability evaluation department Evaluating the possibility of child developmental disorders by assessing the user's cognitive ability, social ability, and language ability based on the user's facial expression changes and body movements regarding the VR video. Diagnostic aid for child developmental disorders.

10. In Claim 1, The aforementioned child developmental disability evaluation department Eye gaze analysis module for analyzing the eye gaze of the above-mentioned user; A facial expression analysis module that analyzes the facial expression of the above-mentioned user; A voice-to-text analysis module that analyzes the voice of the above-mentioned user; and An evaluation module that evaluates the possibility of a child developmental disorder in the user based on the analysis results of the user's gaze, facial expressions, and voice; A diagnostic aid for child developmental disorders including 11. In Claim 10, The above gaze analysis module is The above user’s gaze movements are classified into a fixed state in which the gaze is fixed on a specific point, and saccadic movements corresponding to the gaze movements between the fixed states; and Based on the time elapsed from the onset of stimulation provided by the VR video to the user's fixation, the time elapsed for gazing at the object of interest, the time sustained gazing at the object of interest, spatiotemporal information regarding the movement of the user's gaze, and the number of times the gaze was returned to the object of interest, the possibility of the user having a child developmental disorder is evaluated. Diagnostic aid for child developmental disorders.

12. In Claim 10, The above facial expression analysis module is Analyzing facial movements of the user, including eye blinking, lip movements, and eyebrow and glabella movements, to assess the user's potential for childhood developmental disorders Diagnostic aid for child developmental disorders.

13. In Claim 10, The above voice-to-text analysis module is The possibility of a child developmental disorder in the user is assessed by converting the user's voice into text and evaluating a language evaluation scale that includes the user's vocabulary development level, average utterance length, and speaking structure. Diagnostic aid for child developmental disorders.

14. In Claim 1, It further includes an evaluation result output unit that outputs the evaluation results for the above-mentioned child developmental disorder, and The above evaluation result output unit It includes an evaluation data extraction management unit that extracts and manages evaluation data serving as the basis for evaluating a child developmental disorder for the user based on correlations related to the child developmental disorder among the VR evaluation data obtained for the user; The above evaluation data extraction management department Among the above VR evaluation data, extract the video or audio portions that serve as the basis for evaluating the above child developmental disorder; Storing information corresponding to the start and end sections of an extracted video or audio portion Diagnostic aid for child developmental disorders.

15. A computer-readable, non-transient recording medium having a program recorded thereon for executing a diagnostic aid method for a child developmental disorder to aid in the diagnosis of a child developmental disorder, The above method for assisting in the diagnosis of child developmental disorders A step of collecting VR evaluation data by a VR evaluation data collection unit, including VR video provided to a user by a virtual reality (VR) device and user data generated from the user's voice, gaze, face, and hands; and A non-temporary recording medium comprising: a step of evaluating the user’s cognitive ability, social ability, and language ability for the purpose of evaluating the child developmental disability based on the VR evaluation data by the child developmental disability evaluation department.

Images