Apparatus and method for determining roi by using eye tracking technology in online, offline, and hybrid meeting and class situations

Abstract

The present invention relates to an apparatus and a method for determining a region of interest (ROI) by using eye tracking technology. The apparatus comprises: a monitoring performing unit for tracking a gaze of a corresponding user through a camera module of each second user terminal according to a monitoring request of a first user terminal; an ROI determination unit for identifying an ROI corresponding to the gaze of the corresponding user in a screen area of the each second user terminal; and a concentration level analysis unit for determining a concentration level of the corresponding user by analyzing a change in the ROI during the monitoring.

Description

Device and method for determining ROI using eye tracking technology in online, offline, and hybrid meeting and class situations.

[0001] The present invention relates to attention management technology, and more particularly, to a device and method for determining ROI using eye tracking technology that can monitor whether users are paying attention, their attention area, attention time, order and pattern, and provide feedback in online, offline face-to-face, and hybrid classes, meetings, group projects, and gatherings in which various users participate.

[0002]

[0003] The recent COVID-19 outbreak has led to a rapid increase in demand globally for remote meetings and classes via video conferencing platforms. The use of personal devices in offline meetings and classes has also increased. In Korea, video conferencing platforms have become increasingly common in elementary, middle, and high schools, universities, and private academies for remote classes, and the use of smart devices in in-person classes has become widespread. Furthermore, hybrid methods, where some participants are present in person while others participate remotely online, have become increasingly common.

[0004] In addition, with the proliferation of various multimedia devices, such as personal computers, tablet PCs, cell phones, smart phones, and other portable learning terminals, learners can now easily learn in various locations, not just at home or at school.

[0005] However, when students take online or offline face-to-face classes using the above multimedia devices, there is a problem that they cannot concentrate on the class because they operate the class device or their own other devices during the class. In addition, there may be a problem that it is difficult for teachers conducting the class to simultaneously instruct and manage a large number of students participating in the class.

[0006] Accordingly, the need for functions that can efficiently manage the attention of multiple users in online remote and offline face-to-face classes, lectures, conferences, and meetings conducted using multimedia devices is increasing.

[0007]

[0008] [Prior Art Literature]

[0009] [Patent Document]

[0010] Korean Publication No. 10-2014-0002389 (January 8, 2014)

[0011]

[0012] One embodiment of the present invention provides a device and method for determining ROI using eye tracking technology that can monitor whether users are paying attention, their attention area, and their attention time, order, and pattern, and provide feedback in online remote, offline face-to-face, and hybrid classes, conferences, and meetings where various users participate using multimedia devices.

[0013]

[0014] Among the embodiments, the ROI determination device using eye tracking technology includes a monitoring unit that tracks the gaze of a user through a camera module of each second user terminal in response to a monitoring request from a first user terminal; an ROI determination unit that identifies a region of interest (ROI) corresponding to the gaze of the user in a screen area of ​​each second user terminal; and an attention method analysis unit that analyzes changes in the region of interest during the monitoring to determine the level of attention concentration of the user.

[0015] The above monitoring execution unit can forcibly switch the screen of each of the second user terminals to the screen of the first user terminal in response to a screen sharing request of the first user terminal.

[0016] The above monitoring unit can track the eye movement of the user through the camera module to calculate the position of the eyes and the direction of gaze.

[0017] The ROI determination unit can calculate a gaze coordinate value corresponding to the gaze of the user in a two-dimensional coordinate system of the screen area and determine an area within a preset radius centered on the gaze coordinate value as the focus area.

[0018] The above screen sharing unit can forcibly switch the screen of each second user terminal to the screen of the first user terminal when the screen sharing is initiated.

[0019] The above monitoring unit can track the eye movement of the user through the camera module to calculate the position of the eyes and the direction of gaze.

[0020] The ROI determination unit can calculate a gaze coordinate value corresponding to the gaze of the user in a two-dimensional coordinate system of the screen area and determine an area within a preset radius centered on the gaze coordinate value as the focus area.

[0021] The above-mentioned attention concentration method analysis unit can identify a content area of ​​content shared through the screen of the first user terminal and calculate the attention concentration level based on the overlap rate between the user's concentration area and the content area.

[0022] The above-mentioned attention focus analysis unit can adaptively update the position and size of the content area according to the user movement when the user movement is detected on the screen of the first user terminal.

[0023] The above attention concentration method analysis unit can provide a visualization of the attention concentration of the corresponding user associated with each of the second user terminals through the screen of the first user terminal.

[0024] Among the embodiments, a method for determining ROI using eye tracking technology includes: a step of tracking the gaze of a user through a camera module of each second user terminal in response to a monitoring request of a first user terminal through a monitoring performing unit; a step of identifying a region of interest (ROI) corresponding to the gaze of the user in a screen area of ​​each second user terminal through an ROI determining unit; and a step of analyzing a change in the region of interest during the monitoring through an attention concentration method analysis unit to determine the attention concentration of the user.

[0025]

[0026] The disclosed technology may have the following effects. However, this does not mean that a particular embodiment must include all or only the following effects, and thus the scope of the disclosed technology should not be construed as being limited thereby.

[0027] An ROI determination device and method using eye tracking technology according to one embodiment of the present invention can monitor the attention of users and provide feedback in online, offline, and hybrid classes, meetings, and gatherings in which various users participate.

[0028] For example, you can determine whether users are paying attention to lecture materials they share with others; if they are taking an exam, are they concentrating on the exam paper or looking elsewhere, which could be a sign of cheating; whether they are looking at key areas within a document or at margins or areas of unnecessary information; whether they are looking at text, formulas, or information in the right order or focusing in some other unusual way; and whether they are focusing excessively on unimportant information.

[0029]

[0030] Figure 1 is a drawing illustrating an ROI determination system according to the present invention.

[0031] Fig. 2 is a drawing explaining the system configuration of the ROI determination device of Fig. 1.

[0032] Fig. 3 is a drawing explaining the functional configuration of the ROI determination device of Fig. 1.

[0033] Figure 4 is a flowchart explaining a method for determining ROI using eye tracking technology according to the present invention.

[0034] FIG. 5 is a drawing illustrating one embodiment of a screen sharing process according to the present invention.

[0035] FIG. 6 is a drawing illustrating one embodiment of an attention monitoring process according to the present invention.

[0036] Figure 7 is a drawing illustrating one embodiment of an eye tracking process according to the present invention.

[0037] FIG. 8 is a diagram illustrating one embodiment of a process for updating a content area adaptive to user actions according to the present invention.

[0038]

[0039] The description of the present invention is merely an example for structural and functional explanation, and therefore, the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, since the embodiments can be modified in various ways and can take various forms, the scope of the present invention should be understood to include equivalents that can realize the technical idea. In addition, the purposes or effects presented in the present invention do not mean that a specific embodiment must include all of them or only such effects, and therefore, the scope of the present invention should not be construed as being limited thereby.

[0040] Meanwhile, the meaning of the terms described in this application should be understood as follows.

[0041] Terms such as "first" and "second" are intended to distinguish one component from another, and the scope of the rights should not be limited by these terms. For example, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

[0042] When a component is said to be "connected" to another component, it should be understood that while it may be directly connected to that other component, there may also be other components intervening. Conversely, when a component is said to be "directly connected" to another component, it should be understood that there are no other intervening components. Similarly, other expressions describing relationships between components, such as "between" and "directly between," or "adjacent to" and "directly adjacent to," should be interpreted similarly.

[0043] Singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as "comprises" or "have" should be understood to specify the presence of a feature, number, step, operation, component, part or combination thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0044] For each step, the identifiers (e.g., a, b, c, etc.) are used for convenience of explanation and do not describe the order of the steps. The steps may occur in a different order than stated unless the context clearly dictates a specific order. That is, the steps may occur in the same order as stated, may be performed substantially simultaneously, or may be performed in the opposite order.

[0045] The present invention can be implemented as computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Furthermore, the computer-readable recording medium can be distributed across network-connected computer systems, so that the computer-readable code can be stored and executed in a distributed manner.

[0046] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted to be consistent with their meaning within the context of the relevant technology, and should not be interpreted as having an idealized or overly formal meaning unless explicitly defined herein.

[0047]

[0048] Figure 1 is a drawing illustrating an ROI determination system according to the present invention.

[0049] Referring to FIG. 1, it may include a user terminal (110), an ROI determination device (130), and a database (150).

[0050] A user terminal (110) may correspond to a terminal device operated by a user. In the embodiment of the present invention, a user may be understood as one or more users, and multiple users may be classified into one or more user groups. Each of the one or more users may correspond to one or more user terminals (110). That is, a first user may correspond to a first user terminal, a second user may correspond to a second user terminal, ..., an n-th user (where n is a natural number) may correspond to an n-th user terminal. For example, a teacher participating in an online class may correspond to a first user, and the remaining students may correspond to at least one second user.

[0051] In addition, the user terminal (110) may be a computing device that can participate in a digital learning process by linking with the ROI determination device (130) as a device that constitutes the ROI determination system (100). Here, digital learning may correspond to a learning process in which the user participates online in an educational process conducted in an Internet space, such as a non-face-to-face class or an online lecture. The user terminal (110) may be implemented as a smartphone, laptop, or computer that is connected to and operable with the ROI determination device (130), but is not necessarily limited thereto, and may also be implemented as various devices, including tablet PCs.

[0052] In particular, the user terminal (110) can install and execute a dedicated program or application for linking with the ROI determination device (130). For example, the user terminal (110) can participate in a dedicated online learning space provided by the ROI determination device (130) to perform digital learning, and can provide multimodal data according to the user's movements during the digital learning process to the ROI determination device (130).

[0053] Additionally, the digital learning process and data collection operations can be performed through an interface provided by a dedicated program or application, and the user's gaze tracking operation during digital learning can be performed in conjunction with a camera module. That is, the user terminal (110) can be implemented to include a camera module capable of capturing the user's face for the ROI determination operation using the eye tracking technology according to the present invention.

[0054] Meanwhile, the user terminal (110) can be connected to the ROI determination device (130) through a network, and a plurality of user terminals (110) can be connected to the ROI determination device (130) simultaneously.

[0055] The ROI determination device (130) may be implemented as a server corresponding to a computer or program that performs the user impulsivity analysis method based on digital phenotyping according to the present invention. Furthermore, the ROI determination device (130) may be connected to a user terminal (110) via a wired network or a wireless network such as Bluetooth, WiFi, or LTE, and may transmit and receive data with the user terminal (110) via the network.

[0056] Additionally, the ROI determination device (130) may be implemented to operate in connection with an independent external system (not shown in FIG. 1) to perform the ROI determination method using the eye-tracking technology according to the present invention. For example, the ROI determination device (130) may operate in conjunction with a learning system that provides online learning, a learning management system that manages learning history, an artificial intelligence system that builds an artificial intelligence model, and the like.

[0057] Meanwhile, for the convenience of explanation, the user terminal (110) and the ROI determination device (130) are expressed here as independent devices, but this is not necessarily limited to the above, and it goes without saying that one device may be implemented by being included in the other device.

[0058] The database (150) may correspond to a storage device that stores various information required during the operation of the ROI determination device (130). For example, the database (150) may store learning content and learning management information for digital learning, or may store multimodal data collected during the digital learning process. However, the database is not necessarily limited thereto, and may store information collected or processed in various forms during the process of the ROI determination device (130) performing the ROI determination method using the eye tracking technology according to the present invention.

[0059] In addition, in FIG. 1, the database (150) is depicted as a device independent of the ROI determination device (130), but it is not necessarily limited thereto, and it can be implemented as a logical storage device included in the ROI determination device (130).

[0060]

[0061] Fig. 2 is a drawing explaining the system configuration of the ROI determination device of Fig. 1.

[0062] Referring to FIG. 2, the ROI determination device (130) may include a processor (210), a memory (230), a user input / output unit (250), and a network input / output unit (270).

[0063] The processor (210) can execute an ROI determination procedure using eye tracking technology according to an embodiment of the present invention, manage the memory (230) that is read or written in this process, and schedule a synchronization time between the volatile memory and the non-volatile memory in the memory (230). The processor (210) can control the overall operation of the ROI determination device (130), and can be electrically connected to the memory (230), the user input / output unit (250), and the network input / output unit (270) to control the data flow therebetween. The processor (210) can be implemented as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) of the ROI determination device (130).

[0064] The memory (230) may include an auxiliary memory device implemented as a non-volatile memory such as an SSD (Solid State Disk) or an HDD (Hard Disk Drive) and used to store all data required for the ROI determination device (130), and may include a main memory device implemented as a volatile memory such as a RAM (Random Access Memory). In addition, the memory (230) may store a set of commands that are executed by an electrically connected processor (210) to execute the ROI determination method using the eye tracking technology according to the present invention.

[0065] The user input / output unit (250) includes an environment for receiving user input and an environment for outputting specific information to the user, and may include, for example, an input device including an adapter such as a touchpad, a touch screen, a virtual keyboard, or a pointing device, and an output device including an adapter such as a monitor or a touch screen. In one embodiment, the user input / output unit (250) may correspond to a computing device connected via a remote connection, and in such a case, the ROI determination device (130) may be performed as an independent server.

[0066] The network input / output unit (270) provides a communication environment for connecting to a user terminal (110) via a network, and may include, for example, an adapter for communication such as a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), and a Value Added Network (VAN). In addition, the network input / output unit (270) may be implemented to provide a short-range communication function such as WiFi or Bluetooth, or a wireless communication function of 4G or higher for wireless transmission of data.

[0067]

[0068] Fig. 3 is a drawing explaining the functional configuration of the ROI determination device of Fig. 1.

[0069] Referring to FIG. 3, the ROI determination device (130) can perform an ROI determination method using eye tracking technology according to the present invention. To this end, the ROI determination device (130) can include a screen sharing unit (310), a monitoring execution unit (330), an ROI determination unit (350), an attention focus analysis unit (370), and a control unit (not shown in FIG. 3).

[0070] At this time, the embodiments of the present invention do not necessarily include all of the above-described components simultaneously. Depending on the embodiment, some of the above-described components may be omitted, or some or all of the above-described components may be selectively included. The operation of each component will be described in detail below.

[0071] The screen sharing unit (310) can share the screen of the first user terminal with at least one second user terminal in response to a screen sharing request from the first user terminal. To this end, a space in which the first user terminal and at least one second user terminal participate can be opened online. For example, the online space may correspond to a learning space where online classes are conducted. That is, a learning space for an online class can be opened in response to a class opening request from the first user terminal, and one or more second user terminals can participate in the learning space as a result of being invited by the first user terminal. In addition, when an online class begins, the first user terminal can request screen sharing through a dedicated interface, and the screen sharing unit (310) can share the screen of the first user terminal with other second user terminals participating in the learning space. In other words, both the first user terminal and the second user terminal can share the same screen.

[0072] In one embodiment, the screen sharing unit (310) can forcibly switch the screen of each second user terminal to the screen of the first user terminal when screen sharing is initiated. The screen sharing unit (310) can receive a screen selected from among multiple screens on the first user terminal along with a screen sharing request, and can transmit the selected screen to multiple second user terminals so that the selected screen is forcibly displayed on each second user terminal. For example, during an online class, all students can only view the class screen shared by the teacher in response to the teacher's screen sharing request, and the individual screen switching function for each student can be blocked during screen sharing.

[0073] Meanwhile, the screen sharing unit (310) can share the screen of the first user terminal with at least one second user terminal by applying various sharing methods. For example, the screen sharing unit (310) can selectively share the screen only with one or more second user terminals selected from the first user terminal. As another example, the screen sharing unit (310) can also share the screen of a specific second user terminal selected by the first user terminal back to the first user terminal.

[0074] The monitoring unit (330) can track the gaze of each user via the camera module of each second user terminal during screen sharing, in response to a monitoring request from the first user terminal. Upon receiving a monitoring request from the first user terminal, the monitoring unit (330) can initiate an eye-tracking operation in conjunction with the camera module of each second user terminal. In other words, the monitoring unit (330) can track the gaze of each user during screen sharing and record it in the database (150).

[0075] In one embodiment, the monitoring unit (330) may forcibly switch the screen of each second user terminal to the screen of the first user terminal in response to a monitoring request from the first user terminal. The monitoring unit (330) may operate in conjunction with the screen sharing unit (310) for screen sharing and switching, and a detailed description thereof will be omitted.

[0076] In one embodiment, the monitoring unit (330) can track the eye movement of the user through the camera module to calculate the position and direction of the eyes. For example, the monitoring unit (330) can generate a gaze vector regarding the eye movement of the user, and the gaze vector can include information regarding the position and direction of the eyes. The monitoring unit (330) can collect a video or image captured by the camera module to track the eye movement, and can identify the user's face area from the video or image and track the movement of the eye pupil. The monitoring unit (330) can calculate the position and direction of the eyes based on changes in the pupil movement. To this end, the monitoring unit (330) can apply eye tracking technology to the video or image of the user's face.

[0077] The ROI determination unit (350) can identify a region of interest (ROI) corresponding to the gaze of the user in the screen area of ​​each second user terminal. The ROI determination unit (350) can estimate the distance between the screen of the second user terminal and the user from a video or image captured by the camera module, and can determine the point where the user's gaze is directed in the screen area of ​​the second user terminal based on the eye position and gaze direction. The ROI determination unit (350) can determine a nearby area based on the point where the user's gaze is directed as a region of interest, i.e., an area where the user is currently focusing and looking.

[0078] In one embodiment, the ROI determination unit (350) may calculate a gaze coordinate value corresponding to the user's gaze in a two-dimensional coordinate system of the screen area and determine an area within a preset radius centered on the gaze coordinate value as a focus area. The ROI determination unit (350) may obtain coordinate information of a point where the user's gaze is directed in the screen area through a series of gaze tracking processes. In addition, the ROI determination unit (350) may determine an area within a predefined distance centered on the gaze coordinate value as a focus area. For example, the ROI determination unit (350) may determine a circular area within a radius of 5 cm centered on the gaze coordinate value as a focus area. Meanwhile, the size and shape of the focus area may be preset and applied.

[0079] The attention pattern analysis unit (370) can determine the attention concentration of a user by analyzing changes in the focus area during monitoring. For example, the attention pattern analysis unit (370) can calculate the user's attention concentration by measuring the time the user's gaze remains on the screen. That is, if a student's gaze moves within the screen area for most of the time in an online class, the student's attention concentration can be estimated to be high. As another example, the attention pattern analysis unit (370) can calculate the user's attention concentration based on the number of times the user's gaze moves outside the screen. The attention pattern analysis unit (370) can determine the attention concentration of each user by applying various methods based on various monitoring information collected during monitoring.

[0080] In one embodiment, the attention method analysis unit (370) can monitor whether the user is concentrating by analyzing changes in the focus area during monitoring, and can monitor the time, order, and pattern of the focus based on changes in the focus level. For example, the attention method analysis unit (370) can determine that the user is not concentrating if the focus level is below a preset threshold. In addition, the attention method analysis unit (370) can monitor the time of focus by measuring the time during which the focus level is within a preset range, and can monitor the order of the user's focus by tracking changes in the location of the focus area. In addition, the attention method analysis unit (370) can monitor the user's attention pattern by tracking changes in the focus area and the focus level.

[0081] In one embodiment, the attention mode analysis unit (370) can identify the content area of ​​content shared through the screen of the first user terminal and calculate the attention level based on the overlap rate between the user's focus area and the content area. Here, the content area may correspond to a partial area where the core content of the content is displayed within the screen area where the content is output, and the content area may also change as the content of the content output in the screen area changes. The attention mode analysis unit (370) can identify the content area based on the content of the content shared through the screen of the first user terminal.

[0082] For example, if content in the form of text is shared, the area where the text is displayed or the area where the main keyword in the text is displayed may be determined as the content area, and if content in the form of an image is shared, the area where the image is displayed or the area where the main object in the image is displayed may be determined as the content area.

[0083] In addition, the attention focus analysis unit (370) can calculate the user's attention focus based on whether the user's focus area overlaps with the content area. That is, the attention focus analysis unit (370) can indirectly estimate whether the user is simply looking at the screen area or looking at the content within the screen area, based on whether the focus area overlaps with the content area. Consequently, the attention focus can correspond to an index indicating the degree to which the user's gaze changes depending on the content displayed on the screen. For example, the higher the user's focus, the higher the attention focus can be calculated.

[0084] In addition, the attention focus analysis unit (370) can calculate the overlap rate based on the overlapping area ratio between the focus area and the content area. Specifically, the attention focus analysis unit (370) can calculate the overlap rate through 'overlapping area / total area'. Here, the overlapping area corresponds to the overlapping area between the focus area and the content area, and the total area corresponds to the sum of the areas of the focus area and the content area.

[0085] In one embodiment, the attention mode analysis unit (370) can adaptively update the position and size of the content area according to the user movement detected on the screen of the first user terminal. For example, a teacher conducting an online class may input text to add explanation to the content displayed on the screen or input an action to emphasize a part of the content, in which case the content area may move toward the teacher's input location or expand to that location. The attention mode analysis unit (370) can update by changing at least one of the position and size of the virtual content area according to the user's interaction with the content.

[0086] In one embodiment, the attention analysis unit (370) can visualize and provide the attention level of each user associated with each second user terminal through the screen of the first user terminal. For example, a teacher conducting an online class can monitor each student's screen through the screen, and in this case, the color of the screen border can be changed and displayed according to each student's attention level. Therefore, the teacher can individually recognize the attention level of each student through the color change of the screen, and indirectly recognize the concentration level of the entire class.

[0087] The control unit (not shown in FIG. 3) controls the overall operation of the ROI determination device (130) and can manage the control flow or data flow between the screen sharing unit (310), the monitoring execution unit (330), the ROI determination unit (350), and the attention focus analysis unit (370).

[0088]

[0089] Figure 4 is a flowchart explaining a method for determining ROI using eye tracking technology according to the present invention.

[0090] Referring to FIG. 4, the ROI determination device (130) can receive a monitoring request from a first user terminal via the monitoring execution unit (330) (step S410). Here, the monitoring request may correspond to a monitoring request regarding attention on at least one second user terminal connected to the first user terminal. The ROI determination device (130) can track the gaze of the corresponding user via the camera module of each second user terminal during screen sharing, in response to the monitoring request of the first user terminal via the monitoring execution unit (330) (step S430).

[0091] In addition, the ROI determination device (130) can identify a region of interest (ROI) corresponding to the gaze of the user in the screen area of ​​each second user terminal through the ROI determination unit (350) (step S450). The ROI determination device (130) can analyze changes in the region of interest during monitoring through the attention concentration method analysis unit (370) to determine the level of attention concentration of the user (step S470).

[0092]

[0093] FIG. 5 is a drawing illustrating one embodiment of a screen sharing process according to the present invention.

[0094] Referring to FIG. 5, a teacher participating in an online class can check the class screen (530) and the screens (550) of students participating in the online class through the screen (510) of the first user terminal. At this time, the ROI determination device (130) can provide a menu for screen sharing during the online class within the screen (510) of the first user terminal, and the teacher can share the content (e.g., AbC...) displayed on the class screen (530) with the students participating in the class through the menu. Accordingly, the same content displayed on the class screen (530) can be shared simultaneously through each student's screen (550).

[0095]

[0096] FIG. 6 is a drawing illustrating one embodiment of an attention monitoring process according to the present invention.

[0097] Referring to FIG. 6, the ROI determination device (130) can share each student's screen (650) via the teacher's screen during an online class. Furthermore, the ROI determination device (130) can provide a menu for screen monitoring during an online class on the screen (610) of the first user terminal used by the teacher. Accordingly, the teacher conducting the online class can request monitoring of students' attention levels through this menu.

[0098] In addition, the ROI determination device (130) can perform an eye-tracking operation to track students' gazes through the camera module of each terminal, and can determine the level of attention of each student by evaluating whether the students' gazes are focused on the screen. When the level of attention of each student is calculated, the ROI determination device (130) can visualize the level of attention and output it through the screen (610) of the first user terminal.

[0099] For example, in the case of FIG. 6, on the screen (610) of the first user terminal, students with low concentration and thus distraction, and students with high concentration and thus concentration, can be visualized separately. In addition, the ROI determination device (130) can calculate the concentration level of the entire online class by integrating the concentration level of each student, and can also visualize and display this on the screen.

[0100]

[0101] Figure 7 is a drawing illustrating one embodiment of an eye tracking process according to the present invention.

[0102] Referring to FIG. 7, the ROI determination device (130) can identify a focus area (730) corresponding to the gaze of the user in the screen area (700) of each second user terminal through the ROI determination unit (350). In the case of FIG. (a), as a result of the user's gaze being focused on the lower area of ​​the screen, a focus area (730) can be formed centered on that location. In the case of FIG. (b), as a result of the user's gaze being focused on text displayed on the upper area of ​​the screen, a focus area (730) can be formed centered on that location.

[0103] In addition, when the screen of the first user terminal is shared with the second user terminal, the ROI determination device (130) can identify the content area (710) of the shared content and calculate the user's attention concentration based on whether there is an overlap between the user's focus area (730) and the content area (710). For example, in the case of Figure (a), if there is no overlap between the focus area (730) and the content area (710), the user's attention concentration can be calculated as a low value, and in the case of Figure (b), if there is an overlap between the focus area (730) and the content area (710), the user's attention concentration can be calculated as a high value.

[0104]

[0105] FIG. 8 is a diagram illustrating one embodiment of a process for updating a content area adaptive to user actions according to the present invention.

[0106] Referring to FIG. 8, the ROI determination device (130) can adaptively update the position and size of the content area (810) according to the user movement when the user movement is detected on the screen (800) of the first user terminal. In FIG. (a), the text '123xyz...' can be input by the user while the text content 'AbC...' is displayed on the screen (800) of the first user terminal.

[0107] That is, the ROI determination device (130) can update the location of the content area (810) by changing it to the location where the text was input by the user in response to the user's text input, as shown in Figure (b). As another example, the ROI determination device (130) can also update the size of the content area (810) by expanding the range of the content area (810) to the location where the text was input by the user in response to the user's text input. Thereafter, the ROI determination device (130) can calculate the user's level of attention based on whether there is overlap between the updated content area (810) and the focus area according to the user's gaze.

[0108]

[0109] Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

[0110]

[0111] [National Research and Development Project Supporting This Invention]

[0112] [Project Number] IITP-RS-2021-II212068

[0113] [Assignment number]

[0114] [Ministry Name] Ministry of Science and ICT

[0115] [Name of Project Management (Specialist) Agency] Information and Communications Technology Planning and Evaluation Institute

[0116] [Research Project Name] AI Innovation Hub

[0117] [Research Project Name] Research and Development of an Artificial Intelligence Innovation Hub

[0118] [Contribution rate] 50%

[0119] [Name of the project performing organization] Korea University Industry-Academic Cooperation Foundation

[0120] Research Period: July 1, 2021 - December 31, 2025

[0121]

[0122] [Project ID] 2410005261

[0123] [Assignment Number] 20025696

[0124] Ministry of Trade, Industry and Energy

[0125] [Name of Project Management (Specialist) Agency] Korea Industrial Technology Evaluation and Planning Institute

[0126] [Research Project Name] Industrial Technology Alchemist Project

[0127] [Research Project Title] A Mindful Multiverse Platform Based on Multisensory Digital Phenotyping of Youth Avatars

[0128] [Contribution rate] 50%

[0129] [Name of the project performing organization] Seoul National University Industry-Academic Cooperation Foundation

[0130] [Research Period] April 1, 2023 - December 31, 2024

[0131]

[0132] [Explanation of symbols]

[0133] 100: ROI Determination System

[0134] 110: User terminal 130: ROI determination device

[0135] 150: Database

[0136] 210: Processor 230: Memory

[0137] 250: User input / output section 270: Network input / output section

[0138] 310: Screen sharing unit 330: Monitoring unit

[0139] 350: ROI Determination Section 370: Attention Focus Analysis Section

Claims

1. A monitoring unit that tracks the gaze of each user through the camera module of each second user terminal according to a monitoring request from the first user terminal; A ROI determination unit that identifies a region of interest (ROI) corresponding to the gaze of the user in the screen area of ​​each of the second user terminals; and An ROI determination device using eye tracking technology, comprising an attention concentration analysis unit that determines the level of attention of the corresponding user by analyzing changes in the focus area during the monitoring.

2. In paragraph 1, the monitoring performing unit An ROI determination device using eye tracking technology, characterized in that the screen of each of the second user terminals is forcibly switched to the screen of the first user terminal in response to a monitoring request of the first user terminal.

3. In paragraph 1, the monitoring performing unit An ROI determination device using eye tracking technology, characterized in that it tracks the eye movement of the user through the camera module and calculates the eye position and gaze direction.

4. In the first paragraph, the ROI determination unit An ROI determination device using eye tracking technology characterized in that it calculates a gaze coordinate value corresponding to the gaze of the user in a two-dimensional coordinate system of the above screen area and determines an area within a preset radius centered on the gaze coordinate value as the focus area.

5. In paragraph 1, the attention concentration method analysis unit An ROI determination device using eye tracking technology, characterized in that it identifies a content area of ​​content shared through the screen of the first user terminal and calculates the degree of attention concentration based on the overlap rate between the focus area of ​​the corresponding user and the content area.

6. In paragraph 5, the attention concentration method analysis unit An ROI determination device using eye tracking technology, characterized in that when user movement is detected on the screen of the first user terminal, the location and size of the content area are adaptively updated according to the user movement.

7. In paragraph 1, the attention concentration method analysis unit An ROI determination device using eye tracking technology, characterized in that it provides a visualization of the attention concentration of a corresponding user associated with each second user terminal through the screen of the first user terminal.

8. In the ROI determination method performed in the ROI determination device, A step of tracking the gaze of a user through the camera module of each second user terminal according to a monitoring request of the first user terminal through the monitoring execution unit; A step of identifying a region of interest (ROI) corresponding to the gaze of the user in the screen area of ​​each second user terminal through the ROI determination unit; and A method for determining ROI using eye tracking technology, comprising: a step of analyzing changes in the focused area during the monitoring through an attention focus analysis unit to determine the level of attention of the corresponding user;

Images