Digital phenotyping-based ai matching, recommendation, and feedback coaching apparatus and method

Abstract

The present invention relates to a digital phenotyping-based AI matching, recommendation, and feedback coaching apparatus and method. The apparatus includes: a data collection unit for collecting data on behavior patterns of users in a process of using digital content; a data analysis unit for pre-processing and analyzing the data to calculate at least one digital index related to the behavior pattern; a coaching feedback generation unit for generating coaching feedback on the behavior pattern of each user by using an AI coaching model pre-constructed on the basis of a dataset on the digital index; a coaching report generation unit for providing the coaching feedback to a manager terminal to generate a coaching report including a signature of a corresponding manager; and a coaching report provision unit for providing the coaching report to each user through a user interface.

Description

AI matching, recommendation, and feedback coaching device and method based on digital phenotyping

[0001] The present invention relates to AI coaching service technology, and more specifically, to a digital phenotyping-based AI matching, recommendation, and feedback coaching device and method that collects various digital signals while a user uses a smart device, analyzes the user's current emotional state or mental health state from a set of collected digital signals, and provides AI-generated information.

[0002]

[0003] Digital phenotyping can refer to the continuous quantitative evaluation of users' cognitive, emotional, behavioral, physiological, social, and environmental indicators collected moment by moment in their daily lives without hospital visits using personal digital devices such as smartphones. Digital phenotyping is attracting attention as a new mental health measurement tool because it can provide multidimensional and large-scale objective information. In the future, it is expected to be used as a tool for personalized medicine (preventive medicine, predictive medicine, participatory medicine, and precision medicine) that aims to improve the diagnostic accuracy of psychiatry by reflecting individual patient data or establishing a new dimensional diagnostic system for RDoC.

[0004] Recently, there have been attempts to collect, analyze, and utilize users' biometric indicators, mobile phone usage patterns, and behavioral characteristics through mobile devices and wearable devices. However, there are limitations in that the scope of utilization is very limited in the collection and analysis of massive amounts of data.

[0005] In particular, public education settings like schools are highly sensitive to personal information, and data collection can be significantly restricted due to parental opposition. Therefore, digital phenotyping technology is needed to non-invasively collect and utilize information beyond sensitive personal information like face, facial expressions, physical behavior, words or sentences expressed in writing or speech, and social media posts, including touch, strokes, and stylus usage.

[0006]

[0007] [Prior Art Literature]

[0008] [Patent Document]

[0009] Korean Publication No. 10-2021-0076462 (June 24, 2021)

[0010]

[0011] One embodiment of the present invention provides a digital phenotyping-based AI matching, recommendation, and feedback coaching device and method that collects various digital signals while a user uses a smart device, analyzes the user's current attention, learning, emotional state, or mental health state from a set of collected digital signals, and provides AI-generated information.

[0012]

[0013] Among the embodiments, the AI ​​coaching device includes a data collection unit that collects data on behavioral patterns of users in the process of using digital content; a data analysis unit that preprocesses and analyzes the data to produce at least one digital indicator on the behavioral pattern; a coaching feedback generation unit that generates coaching feedback on the behavioral pattern of each user using an AI coaching model that has been built based on a dataset on the digital indicator; a coaching report generation unit that provides the coaching feedback to an administrator terminal to generate a coaching report including the signature of the administrator; and a coaching report provision unit that provides the coaching report to each user through a user interface.

[0014] The above data collection unit collects multimodal data on user behavior through an application executed on the user terminal during a digital learning process on the user terminal, and the multimodal data may include action indicators based on the user behavior on the display screen of the user terminal.

[0015] The above data collection unit can collect keyboard-related actions and stroke actions of each user collected by the on-screen keyboard sensor and touchscreen sensor of the user terminal during the problem-solving process of the digital learning process as the multimodal data.

[0016] The data analysis unit can determine at least one digital indicator from among a plurality of indicators including stroke acceleration, stroke speed, vertical distance, number of blank spaces, and time interval.

[0017] The above coaching feedback generation unit can provide feedback and recommendations reflecting the user's attention, emotions, and mental health based on at least one digital indicator regarding the user's behavioral pattern collected during the process of using digital content.

[0018] The above coaching feedback generation unit can generate the coaching feedback including visualized information regarding the at least one digital indicator and an evaluation text of a behavioral pattern associated with the digital indicator.

[0019] The above coaching report generation unit can calculate a correlation index between the visualized information and the evaluation text for each digital indicator, and independently receive the signature of the corresponding manager for the corresponding evaluation text for a digital indicator whose correlation index is below a preset threshold value on the manager terminal.

[0020] Among the embodiments, the AI ​​coaching method includes a step of collecting data on behavioral patterns of users in the process of using digital content through a data collection unit; a step of preprocessing and analyzing the data through a data analysis unit to produce at least one digital indicator on the behavioral pattern; a step of generating coaching feedback on the behavioral pattern of each user using an AI coaching model built based on a dataset on the digital indicator through a coaching feedback generation unit; a step of providing the coaching feedback to an administrator terminal through a coaching report generation unit to generate a coaching report including the signature of the administrator; and a step of providing the coaching report to each user through a user interface through a coaching report provision unit.

[0021]

[0022] The disclosed technology may have the following effects. However, this does not mean that a particular embodiment must include all or only the following effects, and thus the scope of the disclosed technology should not be construed as being limited thereby.

[0023] A digital phenotyping-based AI matching, recommendation, and feedback coaching device and method according to one embodiment of the present invention can collect various digital signals while a user uses a smart device, and analyze the user's current attention, learning, emotional state, or mental health state from a set of collected digital signals to provide AI-generated information.

[0024] The AI ​​matching, recommendation, and feedback coaching device and method based on digital phenotyping according to one embodiment of the present invention can be expanded to the market for elementary, middle, and high schools, students, parents, and teachers, and from public education institutions to the private education market, and can improve and supplement the inconveniences of the existing system and provide various data to the digital mental health-related market that has been growing since COVID-19.

[0025]

[0026] Figure 1 is a drawing illustrating an AI coaching system according to the present invention.

[0027] Figure 2 is a drawing explaining the functional configuration of the user terminal of Figure 1.

[0028] Fig. 3 is a drawing explaining the system configuration of the AI ​​coaching device of Fig. 1.

[0029] Fig. 4 is a drawing explaining the functional configuration of the AI ​​coaching device of Fig. 1.

[0030] Figure 5 is a flowchart illustrating an AI coaching method based on digital phenotyping according to the present invention.

[0031] Figure 6 is a diagram explaining the correlation and multiple regression results between IGDS and indicators according to the present invention.

[0032] Figure 7 is a drawing illustrating one embodiment of an AI coaching process according to the present invention.

[0033]

[0034] The description of the present invention is merely an example for structural and functional explanation, and therefore, the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, since the embodiments can be modified in various ways and can take various forms, the scope of the present invention should be understood to include equivalents that can realize the technical idea. In addition, the purposes or effects presented in the present invention do not mean that a specific embodiment must include all of them or only such effects, and therefore, the scope of the present invention should not be construed as being limited thereby.

[0035] Meanwhile, the meaning of the terms described in this application should be understood as follows.

[0036] Terms such as "first" and "second" are intended to distinguish one component from another, and the scope of the rights should not be limited by these terms. For example, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

[0037] When a component is said to be "connected" to another component, it should be understood that while it may be directly connected to that other component, there may also be other components intervening. Conversely, when a component is said to be "directly connected" to another component, it should be understood that there are no other intervening components. Similarly, other expressions describing relationships between components, such as "between" and "directly between," or "adjacent to" and "directly adjacent to," should be interpreted similarly.

[0038] Singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as "comprises" or "have" should be understood to specify the presence of a feature, number, step, operation, component, part or combination thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0039] For each step, the identifiers (e.g., a, b, c, etc.) are used for convenience of explanation and do not describe the order of the steps. The steps may occur in a different order than stated unless the context clearly dictates a specific order. That is, the steps may occur in the same order as stated, may be performed substantially simultaneously, or may be performed in the opposite order.

[0040] The present invention can be implemented as computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Furthermore, the computer-readable recording medium can be distributed across network-connected computer systems, so that the computer-readable code can be stored and executed in a distributed manner.

[0041] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted to be consistent with their meaning within the context of the relevant technology, and should not be interpreted as having an idealized or overly formal meaning unless explicitly defined herein.

[0042]

[0043] Figure 1 is a drawing illustrating an AI coaching system according to the present invention.

[0044] Referring to FIG. 1, the AI ​​coaching system (100) may include a user terminal (110), an AI coaching device (130), and a database (150).

[0045] A user terminal (110) may correspond to a terminal device operated by a user. In the embodiment of the present invention, a user may be understood as one or more users, and multiple users may be classified into one or more user groups. Each of the one or more users may correspond to one or more user terminals (110). That is, a first user may correspond to a first user terminal, a second user may correspond to a second user terminal, ..., an n-th user (where n is a natural number) may correspond to an n-th user terminal.

[0046] In addition, the user terminal (110) may be a computing device that can participate in the process of using digital content by linking with the AI ​​coaching device (130) as a device that constitutes the AI ​​coaching system (100). Here, digital content refers to content that exists in the form of digital data, and may correspond to content such as text, voice, sound, image, and video that is produced or processed in a digital format by digitizing existing analog content. For example, digital content may include games, animations, mobile content, digital images, electronic learning, electronic books, and digital audio. In particular, electronic learning (or digital learning) may correspond to a learning process in which an individual participates online in an educational process conducted in an internet space, such as non-face-to-face education or online education.

[0047] In addition, the user terminal (110) may be implemented as a smartphone, laptop, or computer that is connected to and operable with the AI ​​coaching device (130), but is not necessarily limited thereto and may also be implemented as various devices, including tablet PCs, etc. In particular, the user terminal (110) may install and execute a dedicated program or application for linking with the AI ​​coaching device (130).

[0048] For example, a user terminal (110) can participate in a dedicated online learning space provided by an AI coaching device (130) and perform digital learning as one of the digital contents, and can participate in a problem-solving process in which an evaluation is performed by entering the correct answer to a given problem during the digital learning process. In this case, the user terminal (110) may correspond to the terminal of a learner (or student) participating in online learning. As another example, the user terminal (110) may participate in the online learning space as an administrator conducting an online learning process. In this case, the user terminal (110) may correspond to the terminal of a teacher (or instructor) conducting online learning.

[0049] Additionally, the user terminal (110) may provide an interface for utilizing digital content through a dedicated program or application. Specifically, when utilizing digital learning, the user may participate online in educational courses conducted in the Internet space, such as non-face-to-face education or online education, through the interface, and the user terminal (110) may collect pattern data regarding the user's behavior resulting from interaction with the interface. Meanwhile, the user terminal (110) may be connected to the AI ​​coaching device (130) via a network, and multiple user terminals (110) may be simultaneously connected to the AI ​​coaching device (130).

[0050] The AI ​​coaching device (130) may be implemented as a server corresponding to a computer or program that performs the AI ​​coaching method based on digital phenotyping according to the present invention. Furthermore, the AI ​​coaching device (130) may be connected to a user terminal (110) via a wired network or a wireless network such as Bluetooth, WiFi, or LTE, and may transmit and receive data with the user terminal (110) via the network.

[0051] Additionally, the AI ​​coaching device (130) may be implemented to operate in connection with an independent external system (not shown in FIG. 1) to perform the AI ​​coaching method based on digital phenotyping according to the present invention. For example, the AI ​​coaching device (130) may operate in conjunction with a learning system that provides online learning, a learning management system that manages learning history, an artificial intelligence system that builds an AI model, and the like.

[0052] Meanwhile, for the convenience of explanation, the user terminal (110) and the AI ​​coaching device (130) are expressed as independent devices here, but this is not necessarily limited to the above, and it goes without saying that one device may be implemented by being included in the other device.

[0053] The database (150) may correspond to a storage device that stores various information required during the operation of the AI ​​coaching device (130). For example, the database (150) may store information regarding content materials for using digital content or data regarding users' behavioral patterns collected during the use of digital content. However, the database (150) is not necessarily limited thereto, and may store information collected or processed in various forms during the process of the AI ​​coaching device (130) performing the AI ​​coaching method based on digital phenotyping according to the present invention.

[0054] In addition, in FIG. 1, the database (150) is depicted as a device independent of the AI ​​coaching device (130), but it is not necessarily limited thereto, and it can be implemented as a logical storage device included in the AI ​​coaching device (130).

[0055]

[0056] Figure 2 is a drawing explaining the functional configuration of the user terminal of Figure 1.

[0057] Referring to FIG. 2, the user terminal (110) can execute a dedicated program or application for utilizing digital content. For example, the user terminal (110) can execute an online learning program for digital learning or a non-face-to-face meeting application, and perform operations to collect various multimodal data related to user behavior during the digital learning process on a dedicated interface.

[0058] To this end, the user terminal (110) may be implemented to include independent modules for monitoring user behavior and collecting data. Specifically, the user terminal (110) may include a multimodal data collection module (210), a foreground process monitoring module (230), and a web browser monitoring module (250). However, embodiments of the present invention do not necessarily include all of the above-described components simultaneously, and some of the above-described components may be omitted or selectively included in some or all of the above-described components depending on the embodiment.

[0059] The multimodal data collection module (210) can collect various passive sensor data, keyboard input data, and multimodal data. To this end, the multimodal data collection module (210) can operate in conjunction with various sensors included in the user terminal (110). For example, the multimodal data collection module (210) can periodically collect data information from passive sensors such as angular velocity, acceleration, and light sensors. The multimodal data collection module (210) can collect data such as keyboard values ​​and time when a user inputs a keyboard, and can collect data when an event such as a user using a stylus or touching with a finger occurs. The data collected by the multimodal data collection module (210) can be stored and preserved in the internal memory, and can be periodically transmitted to the AI ​​coaching device (130) by the user terminal (110).

[0060] The foreground process monitoring module (230) can perform an operation to detect the execution of a process, and for this purpose, can include a command to check the package name of the foreground process. The foreground process monitoring module (230) can perform process monitoring as a periodic task, and can collect and record information about the foreground process using the display screen of the user terminal (110) at preset intervals.

[0061] The web browser monitoring module (250) can perform an operation to examine information of a web page that a user accesses through a web browser on a user terminal (110). The web browser monitoring module (250) can operate in an event-driven manner in which a processing function is executed whenever an event occurs, and can collect and record text information whenever a user interacts with a text input window (e.g., a search window, an address window, etc.) of a web browser.

[0062]

[0063] Fig. 3 is a drawing explaining the system configuration of the AI ​​coaching device of Fig. 1.

[0064] Referring to FIG. 3, the AI ​​coaching device (130) may include a processor (310), a memory (330), a user input / output unit (350), and a network input / output unit (370).

[0065] The processor (310) can execute an AI coaching procedure based on digital phenotyping according to an embodiment of the present invention, manage a memory (330) that is read or written during this process, and schedule a synchronization time between volatile memory and non-volatile memory in the memory (330). The processor (310) can control the overall operation of the AI ​​coaching device (130), and is electrically connected to the memory (330), the user input / output unit (350), and the network input / output unit (370) to control data flow therebetween. The processor (310) can be implemented as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) of the AI ​​coaching device (130).

[0066] The memory (330) may include an auxiliary memory device implemented as a non-volatile memory such as an SSD (Solid State Disk) or an HDD (Hard Disk Drive) and used to store all data required for the AI ​​coaching device (130), and may include a main memory device implemented as a volatile memory such as a RAM (Random Access Memory). In addition, the memory (330) may store a set of commands that execute the AI ​​coaching method based on digital phenotyping according to the present invention by being executed by an electrically connected processor (310).

[0067] The user input / output unit (350) includes an environment for receiving user input and an environment for outputting specific information to the user, and may include, for example, an input device including an adapter such as a touchpad, a touch screen, a virtual keyboard, or a pointing device, and an output device including an adapter such as a monitor or a touch screen. In one embodiment, the user input / output unit (350) may correspond to a computing device connected via remote access, and in such a case, the AI ​​coaching device (130) may be performed as an independent server.

[0068] The network input / output unit (370) provides a communication environment for connecting to a user terminal (110) via a network, and may include, for example, an adapter for communication such as a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), and a Value Added Network (VAN). In addition, the network input / output unit (370) may be implemented to provide a short-range communication function such as WiFi or Bluetooth, or a wireless communication function of 4G or higher for wireless transmission of data.

[0069]

[0070] Fig. 4 is a drawing explaining the functional configuration of the AI ​​coaching device of Fig. 1.

[0071] Referring to FIG. 4, the AI ​​coaching device (130) can perform the AI ​​coating method based on digital phenotyping according to the present invention. To this end, the AI ​​coaching device (130) can include a data collection unit (410), a data analysis unit (430), a coaching feedback generation unit (450), a coaching report generation unit (470), a coaching report provision unit (490), and a control unit (not shown in FIG. 4).

[0072] At this time, the embodiments of the present invention do not need to include all of the above-described components simultaneously. Depending on the embodiment, some of the above-described components may be omitted, or some or all of the above-described components may be selectively included. Hereinafter, each component will be described in more detail.

[0073] The data collection unit (410) can collect data on users' behavioral patterns while using digital content. To this end, the data collection unit (410) can be implemented to operate in conjunction with a user terminal (110). For example, the data collection unit (410) can receive various data collected through a dedicated program or application running on the user terminal (110) from the dedicated program or application and store them in a database (150).

[0074] In one embodiment, the data collection unit (410) may collect multimodal data regarding user behavior through an application executed on the user terminal (110) during a digital learning process on the user terminal (110). At this time, the multimodal data may include action indicators resulting from user actions on the display screen of the user terminal (110). For example, the multimodal data may include data collected from touch or stroke actions by the user on a touch screen. In particular, the data collection unit (410) may collect various behavioral information generated while the user uses a dedicated application for a specific purpose as multimodal data.

[0075] In one embodiment, the data collection unit (410) may collect, as multimodal data, keyboard-related actions and stroke actions of each user collected by the on-screen keyboard sensor and touchscreen sensor of the user terminal (110) during the problem-solving process in the digital learning process. Here, the problem-solving process is one of the digital learning processes, and may correspond to a process in which a problem is provided through the display screen of the user terminal (110) and the user reads the problem and inputs the correct answer. At this time, the user may input the correct answer by touching the display screen or using the keyboard displayed on the screen, and the data collection unit (410) may collect, as a motion index, multimodal data collected through the display screen among the user actions occurring during the problem-solving process or the correct answer input process.

[0076] In one embodiment, when collecting data on a user's behavioral pattern during a problem-solving process in a digital learning course, the data collection unit (410) may select and provide customized problems and collect corresponding multimodal data. That is, the data collection unit (410) may provide the user with customized problems composed of problems at a level appropriate for the user by considering the user's gender, age, academic ability, etc., and may collect keyboard-related actions and stroke actions of each user collected by the on-screen keyboard sensor and touchscreen sensor of the user terminal (110) while solving the customized problems as multimodal data. The data collection unit (410) may classify the collected multimodal data by difficulty (or level) of the customized problems and store them in the database (150).

[0077] The data analysis unit (430) can preprocess and analyze data to produce at least one digital indicator regarding a behavioral pattern. A user's behavioral pattern may correspond to a regularity in a series of repeated actions during the user's behavioral process. For example, if a user has symptoms such as Internet Gaming Disorder (IGD), a specific pattern may exist in the user's behaviors. The data analysis unit (430) can extract digital indicators from the user's behavioral data and, based on these, extract patterns regarding the user's behavior.

[0078] Additionally, the data analysis unit (430) may perform preprocessing operations on data collected from users to more accurately calculate various digital indicators. For example, the data analysis unit (430) may perform preprocessing operations such as missing data processing and denoising on the collected data.

[0079] In one embodiment, the data analysis unit (430) may determine at least one digital indicator from among a plurality of indicators including stroke acceleration, stroke speed, vertical distance, number of blank spaces, and time interval. Specifically, stroke acceleration may correspond to a rate of change in speed while making a stroke on the touchscreen, stroke speed may correspond to a number of pixels per second or a distance, and vertical distance may correspond to a distance from the top of the screen to a starting position during a stroke. In addition, the number of blank spaces may correspond to the number of times the spacebar is pressed between letters, words, and sentences, and time interval may correspond to the time (in milliseconds) between touching one key and touching another key.

[0080] For example, in the correlation between Internet Game Addiction (IGD) and digital indicators, the IGDS score, which is a measure of game addiction, can show a positive correlation with indicators such as stroke acceleration, stroke speed, and vertical distance, and can show a negative correlation with indicators such as number of blanks and time interval. In this way, the data analysis unit (430) can selectively extract digital indicators that have a high correlation with Internet game addiction from among 83 digital indicators collected during the digital learning process. As another example, the data analysis unit (430) can selectively extract digital indicators that have a high correlation with the user's emotional state or mental health state from among digital indicators collected during the problem-solving process during the digital learning process.

[0081] That is, the data analysis unit (430) can selectively extract digital indicators that have a high correlation with the digital content usage process and user behavior patterns among various digital indicators through data analysis, and the selected digital indicators can then be utilized in the coaching feedback generation stage.

[0082] The coaching feedback generation unit (450) can generate coaching feedback on each user's behavioral pattern using a pre-built AI coaching model based on a dataset of digital indicators. Here, the AI ​​coaching model may correspond to an artificial intelligence model that generates coaching feedback for the user as output based on input data regarding the user's behavioral pattern. In this case, the coaching feedback may correspond to a report generated by analyzing data collected during the user's use of digital content, and may include analysis results regarding the user's behavioral pattern, mental health status, academic achievement, etc. The dataset used for training the AI ​​coaching model may include digital indicators regarding the user's behavioral pattern collected during the content use process, and the user's behavioral pattern, mental health status, academic achievement, etc. that match the digital indicators.

[0083] Additionally, an AI coaching model can be implemented by including one or more sub-models. For example, an AI coaching model may include multiple sub-models that independently generate analysis results based on behavioral patterns, mental health status, and academic achievement based on inputs composed of digital indicators. In other words, the AI ​​coaching model can ultimately generate coaching feedback for the user by integrating the outputs of each sub-model. Furthermore, the AI ​​coaching model can be designed according to various structures, including sequential or parallel connections between one or more sub-models.

[0084] In one embodiment, the coaching feedback generation unit (450) may generate coaching feedback that includes visualized information regarding at least one digital indicator and evaluation text regarding a behavioral pattern associated with the digital indicator. The AI ​​coaching model may analyze correlations between digital indicators received as input and then output the analysis results for the corresponding user as evaluation text. The coaching feedback generation unit (450) may convert digital indicators regarding the user's behavioral pattern into visualized information and integrate them with the evaluation text output by the AI ​​coaching model to generate coaching feedback for the corresponding user.

[0085] In one embodiment, the coaching feedback generation unit (450) may select digital indicators highly correlated with the evaluation text generated by the AI ​​coaching model, and may generate visualized information only for the selected digital indicators and add them to the coaching feedback. For example, when generating evaluation text regarding the degree and presence of a user's Internet gaming disorder in a digital learning process through the AI ​​coaching model, the coaching feedback generation unit (450) may select stroke acceleration or stroke speed as digital indicators highly correlated with Internet gaming disorder and then generate visualized information. That is, a high IGDS (Internet Gaming Disorder Score) score may be highly correlated with an increase in stroke acceleration and a decrease in stroke speed. In addition, a higher IGDS score may be associated with a tendency for strokes to start near the bottom of the screen, and may be associated with rapid typing and the omission of spaces between letters, words, or sentences.

[0086] In one embodiment, the coaching feedback generation unit (450) may generate coaching feedback regarding the user's behavioral patterns using data collected during the use of digital content and at least one digital indicator extracted from the data. In this case, the coaching feedback may be implemented as feedback and recommendations that reflect not only the user's learning achievements but also their attention span, emotions, and mental health.

[0087] For example, the coaching feedback generation unit (450) may provide coaching feedback to a student whose behavior pattern is determined to be impulsive to slow down the problem-solving speed or to read the problem text carefully and solve the problem even if the correct answer has been entered during the problem-solving process.

[0088] As another example, when recommending the next problem to a user, the coaching feedback generation unit (450) may provide coaching feedback recommending an easier problem than the previous one or recommending a problem of a different subject / genre to a student whose behavioral pattern in solving the previous problem showed a bad feeling.

[0089] In addition, the coaching feedback generation unit (450) can predict the learning status and emotional state of a user (e.g., a student) in that data on the user's behavioral pattern collected during the process of using digital content corresponds to multimodal time series sensor data, and if the prediction result is negative, it can selectively provide a path, environment, or problem where the same situation does not occur.

[0090] The coaching report generation unit (470) can provide coaching feedback to an administrator terminal and generate a coaching report including the administrator's signature. Here, the administrator terminal may correspond to a terminal operated by an administrator who manages users using online content. For example, the administrator may correspond to a teacher or instructor who manages learners participating in online learning, and the administrator terminal may correspond to a teacher's or instructor's terminal. If the coaching report generation unit (470) receives a signature from the administrator terminal as the administrator's approval of the coaching feedback, it can generate a coaching report including the final evaluation content. In other words, the coaching report may correspond to a report in which the evaluation content of the coaching feedback generated by the AI ​​coaching model has been finally approved by the administrator. In addition, the coaching reports generated by the coaching report generation unit (470) may be stored in the database (150) in a manner differentiated by user.

[0091] In one embodiment, the coaching report generation unit (470) can calculate a correlation index between visualized information and evaluation text for each digital indicator, and independently receive the administrator's signature for the corresponding evaluation text for digital indicators whose correlation index is below a preset threshold value on the administrator terminal. That is, the coaching report generation unit (470) can operate in conjunction with the administrator terminal and provide coaching feedback on learners participating in an online learning course to the administrator terminal. The learner's coaching feedback can be provided through an interface on the administrator terminal, and the teacher can check the detailed information of the coaching feedback and input a signature through the interface if he or she agrees to the evaluation of the coaching feedback.

[0092] In particular, the coaching report generation unit (470) can calculate an association index between visualized information and evaluation text for each digital indicator while providing coaching feedback to the administrator terminal. The association index can be calculated higher as the similarity between the visualized information and the evaluation text increases. That is, if the association index is below a preset threshold, it can be estimated that the information expressed by the digital indicator is not accurately reflected in the evaluation content of the evaluation text, and the coaching report generation unit (470) can provide a separate interface that requests the administrator's signature input for digital indicators or evaluation texts whose association index is below the threshold while providing coaching feedback through the interface of the administrator terminal.

[0093] At this point, the administrator can edit the wording of the assessment text and update the coaching feedback by entering a signature for the revised assessment. Furthermore, the administrator's signature can be entered independently for each digital indicator or assessment text.

[0094] In one embodiment, the coaching report generation unit (470) may provide coaching feedback to a manager model to generate a coaching report including a signature of the manager model. Instead of providing the coaching feedback to the manager terminal, the coaching report generation unit (470) may provide the coaching feedback as input to a pre-built manager model. That is, the coaching report generation unit (470) may generate a coaching report based on coaching feedback modified by the manager model, and may add a signature generated based on identification information of the manager model to the coaching report. Here, the manager model may correspond to an artificial intelligence model that generates an output of the modified coaching feedback based on input data regarding at least one digital indicator and coaching feedback. In addition, the manager model may output the modified coaching feedback by modifying the evaluation text with the lowest similarity to each digital indicator among the evaluation texts included in the coaching feedback.

[0095] In one embodiment, the coaching report generation unit (470) can generate a coaching report by first modifying coaching feedback through the manager model and then providing the modified coaching feedback to the manager terminal, thereby receiving the manager's signature. In this case, the manager can confirm the coaching feedback modified by the manager model through the manager terminal's interface and then enter a signature to approve the coaching feedback. Upon receiving the manager's signature, the coaching report generation unit (470) can generate a coaching report based on the modified coaching feedback and the manager's signature.

[0096] The coaching report provider (490) can provide each user with a coaching report generated by the coaching report generator (470) via a user interface. Specifically, the coaching report can be displayed on the user terminal (110) using various templates. For example, the coaching report can be generated and provided to users according to various templates, such as a mobile version, a PC version, or a tablet version.

[0097] In one embodiment, the coaching report provider (490) may receive user feedback on a coaching report from a user terminal (110). To this end, an interface for user feedback input may be provided on the user terminal (110). User feedback may be input for each digital indicator or evaluation text of the coaching report. Furthermore, user feedback may be input in the form of evaluation points, comments, ratings, etc. The user feedback information input from the user terminal (110) may be used as data for updating the AI ​​coaching model in a subsequent process.

[0098] The control unit (not shown in FIG. 4) controls the overall operation of the AI ​​coaching device (130) and can manage the control flow or data flow between the data collection unit (410), the data analysis unit (430), the coaching feedback generation unit (450), the coaching report generation unit (470), and the coaching report provision unit (490).

[0099]

[0100] Figure 5 is a flowchart illustrating an AI coaching method based on digital phenotyping according to the present invention.

[0101] Referring to FIG. 5, the AI ​​coaching device (130) can collect data on users' behavioral patterns during the digital content usage process via the data collection unit (410) (step S510). The AI ​​coaching device (130) can preprocess and analyze the data via the data analysis unit (430) to produce at least one digital indicator regarding the behavioral pattern (step S530).

[0102] In addition, the AI ​​coaching device (130) can generate coaching feedback on the behavioral patterns of each user using an AI coaching model built based on a dataset on digital indicators through a coaching feedback generation unit (450) (step S550). The AI ​​coaching device (130) can provide coaching feedback to an administrator terminal through a coaching report generation unit (470) to generate a coaching report including the signature of the administrator (step S570). The AI ​​coaching device (130) can provide a coaching report to each user using a user interface through a coaching report provision unit (490) (step S590).

[0103]

[0104] Figure 6 is a diagram explaining the correlation and multiple regression results between IGDS and indicators according to the present invention.

[0105] Referring to FIG. 6, the AI ​​coaching device (130) can determine digital indicators correlated with the IGDS to detect a user's Internet gaming disorder during a digital learning process, which is one type of digital content. To this end, the AI ​​coaching device (130) can extract meaningful indicators from multimodal data associated with the user's behavioral patterns collected during the digital learning process.

[0106] Figure 6 presents multiple regression results for five digital indicators highly correlated with IGDS. Specifically, F(5, 921) = 18.54, with a significance level of p < 0.001, indicating that IGDS indicators account for 10.0% of the adjusted coefficient of determination.

[0107] Accordingly, the AI ​​coaching device (130) can collect digital indicators of a user's behavior during a digital learning process and generate coaching feedback regarding the user's behavioral patterns from the digital indicators using a pre-trained AI coaching model. Based on the coaching feedback, the AI ​​coaching device (130) can generate a coaching report assessing the user's level of Internet gaming disorder and provide it to the user.

[0108]

[0109] Figure 7 is a drawing illustrating one embodiment of an AI coaching process according to the present invention.

[0110] Referring to FIG. 7, the AI ​​coaching process performed in the AI ​​coaching device (130) may be performed by collecting data on the user's behavioral pattern during the process of using digital content (Data Collection), analyzing the collected data (Data Analysis), and generating information output by the AI ​​coaching model (Recommendation Engine) regarding the user's current emotional state or mental health state as the user's coaching feedback.

[0111] In particular, the AI ​​coaching process can be updated through the manager's review, modification, and signature of coaching feedback, and then provided to the user through the user interface as a coaching report, and the process of optimizing the AI ​​coaching model through reinforcement learning (RLHF) based on the user's feedback can be carried out.

[0112] The AI ​​coaching device (130) can analyze the behavior of a user using digital content and generate and provide feedback on the same. It can also provide various information on the user's emotional state, mental health state, etc. through continuous updates reflecting the user's opinions, so that the user's digital content usage process can be effectively managed.

[0113]

[0114] Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

[0115]

[0116] [National Research and Development Project Supporting This Invention]

[0117] [Project Number] IITP-RS-2021-II212068

[0118] [Assignment number]

[0119] [Ministry Name] Ministry of Science and ICT

[0120] [Name of Project Management (Specialist) Agency] Information and Communications Technology Planning and Evaluation Institute

[0121] [Research Project Name] AI Innovation Hub

[0122] [Research Project Name] Research and Development of an Artificial Intelligence Innovation Hub

[0123] [Contribution rate] 50%

[0124] [Name of the project performing organization] Korea University Industry-Academic Cooperation Foundation

[0125] Research Period: July 1, 2021 - December 31, 2025

[0126]

[0127] [Project ID] 2410005261

[0128] [Assignment Number] 20025696

[0129] Ministry of Trade, Industry and Energy

[0130] [Name of Project Management (Specialist) Agency] Korea Industrial Technology Evaluation and Planning Institute

[0131] [Research Project Name] Industrial Technology Alchemist Project

[0132] [Research Project Title] A Mindful Multiverse Platform Based on Multisensory Digital Phenotyping of Youth Avatars

[0133] [Contribution rate] 50%

[0134] [Name of the project performing organization] Seoul National University Industry-Academic Cooperation Foundation

[0135] [Research Period] April 1, 2023 - December 31, 2024

[0136]

[0137] [Explanation of symbols]

[0138] 100: AI Coaching System

[0139] 110: User terminal 130: AI coaching device

[0140] 150: Database

[0141] 210: Multimodal Data Collection Module

[0142] 230: Foreground Process Monitoring Module

[0143] 250: Web Browser Monitoring Module

[0144] 310: Processor 330: Memory

[0145] 350: User input / output section 370: Network input / output section

[0146] 410: Data Collection Department 430: Data Analysis Department

[0147] 450: Coaching Feedback Generation Unit 470: Coaching Report Generation Unit

[0148] 490: Coaching Report Provider

Claims

1. Data collection unit that collects data on users’ behavioral patterns while using digital content; A data analysis unit that preprocesses and analyzes the above data to produce at least one digital indicator regarding the behavioral pattern; A coaching feedback generation unit that generates coaching feedback on the behavioral patterns of each user using an AI coaching model built based on a dataset on the above digital indicators; A coaching report generation unit that provides the above coaching feedback to the administrator terminal and generates a coaching report including the signature of the administrator; and An AI coaching device including a coaching report providing unit that provides the coaching report to each user through a user interface.

2. In paragraph 1, the data collection unit During the digital learning process on the user terminal, multimodal data on user behavior is collected through the application executed on the user terminal, An AI coaching device, characterized in that the multimodal data includes action indicators based on the user's actions on the display screen of the user terminal.

3. In paragraph 1, the data collection unit An AI coaching device characterized in that it collects keyboard-related movements and stroke movements of each user collected by the on-screen keyboard sensor and touchscreen sensor of the user terminal during the problem-solving process in the digital learning process as the multimodal data.

4. In paragraph 1, the data analysis unit An AI coaching device characterized by determining at least one digital indicator from among a plurality of indicators including stroke acceleration, stroke speed, vertical distance, number of blank spaces, and time interval.

5. In the first paragraph, the coaching feedback generation unit An AI coaching device characterized by providing feedback and recommendations reflecting the user's attention, emotions, and mental health based on at least one digital indicator of the user's behavioral pattern collected during the use of digital content.

6. In the first paragraph, the coaching feedback generation unit An AI coaching device characterized in that it generates the coaching feedback including visualized information about at least one digital indicator and an evaluation text of a behavioral pattern associated with the digital indicator.

7. In paragraph 6, the coaching report generation unit An AI coaching device characterized in that it calculates a correlation index between the visualized information and the evaluation text for each digital indicator and independently receives the signature of the corresponding manager for the corresponding evaluation text for digital indicators whose correlation index is below a preset threshold value on the manager terminal.

8. In an AI coaching method performed in an AI coaching device, A step of collecting data on users' behavioral patterns during the process of using digital content through the data collection department; A step of preprocessing and analyzing the data through a data analysis unit to produce at least one digital indicator regarding the behavioral pattern; A step of generating coaching feedback on the behavioral pattern of each user using an AI coaching model built based on a dataset on the digital indicators through a coaching feedback generation unit; Step of providing the coaching feedback to the manager terminal through the coaching report generation unit to generate a coaching report including the manager's signature; and An AI coaching method comprising: providing the coaching report to each user through a user interface via a coaching report providing unit;

Images