Roi determination device and method using eye tracking technology in online, offline, hybrid meeting and class situations

Abstract

The present invention relates to an ROI determination device and method using eye-tracking technology, wherein the device comprises: a monitoring execution unit that tracks the gaze of a user through a camera module of each second user terminal in response to a monitoring request from a first user terminal; an ROI determination unit that identifies a region of interest (ROI) corresponding to the user's gaze in a screen area of ​​each second user terminal; and an attention method analysis unit that determines the level of attention of the user by analyzing changes in the region of interest during the monitoring.

Description

Technology Field

[0001] The present invention relates to attention management technology, and more specifically, to an ROI determination device and method using eye-tracking technology capable of providing feedback by monitoring whether users are paying attention, the area of ​​attention, the duration of attention, the sequence, and patterns of attention in online, offline face-to-face, and hybrid classes, meetings, group projects, and gatherings in which various users participate. Background Technology

[0003] With the recent outbreak of the COVID-19 pandemic, the global demand for non-face-to-face meetings and classes via video conferencing platforms has surged, and the use of personal devices in offline meetings and classes has also increased. In Korea, too, elementary, middle, and high schools, universities, and private academies have adopted video conferencing platforms for remote learning, and the use of smart devices in in-person classes has become commonplace. Furthermore, hybrid methods have frequently emerged, where some participants are physically present in person while others participate remotely online.

[0004] In addition, with the widespread adoption of various multimedia devices—namely, personal computers, tablet PCs, mobile phones, smartphones, and other portable learning terminals—learners are no longer limited to studying in restricted locations such as home or school, but can easily engage in learning in different places.

[0005] However, when students take online or offline face-to-face classes using such multimedia devices, there is a problem in that they cannot concentrate on the lesson because they operate the class device or their own other devices during the class. Additionally, there may be a problem for the teacher conducting the class in that it is difficult to simultaneously guide and manage a large number of participating students.

[0006] Therefore, there is an increasing need for functions that can efficiently manage the attention of multiple users in online remote and offline face-to-face classes, lectures, meetings, and other activities conducted using multimedia devices. Prior art literature

[0008] Korean Publication No. 10-2014-0002389 (2014.01.08) The problem to be solved

[0009] One embodiment of the present invention aims to provide an ROI determination device and method using eye-tracking technology capable of providing feedback by monitoring whether users are paying attention, the area of ​​attention, the duration of attention, the sequence, and the pattern of attention in online remote, offline face-to-face, and hybrid classes, meetings, and gatherings in which various users participate using multimedia devices. means of solving the problem

[0011] Among the embodiments, an ROI determination device using eye-tracking technology comprises: a monitoring execution unit that tracks the gaze of a user through a camera module of each second user terminal in response to a monitoring request from a first user terminal; an ROI determination unit that identifies a region of interest (ROI) corresponding to the user's gaze in a screen area of ​​each second user terminal; and an attention method analysis unit that determines the level of attention of the user by analyzing changes in the region of interest during the monitoring.

[0012] The above monitoring unit can forcibly switch the screen of each second user terminal to the screen of the first user terminal in response to a screen sharing request from the first user terminal.

[0013] The above monitoring unit can calculate the position of the eyes and the direction of gaze by tracking the eye movement of the corresponding user through the camera module.

[0014] The ROI determination unit can calculate a gaze coordinate value corresponding to the user's gaze in a two-dimensional coordinate system of the screen area and determine the area within a preset radius centered on the gaze coordinate value as the concentration area.

[0015] The above screen sharing unit can forcibly switch the screen of each second user terminal to the screen of the first user terminal when the screen sharing is initiated.

[0016] The above monitoring unit can calculate the position of the eyes and the direction of gaze by tracking the eye movement of the corresponding user through the camera module.

[0017] The ROI determination unit can calculate a gaze coordinate value corresponding to the user's gaze in a two-dimensional coordinate system of the screen area and determine the area within a preset radius centered on the gaze coordinate value as the concentration area.

[0018] The above attention method analysis unit can identify the content area of ​​the content shared through the screen of the first user terminal and calculate the level of attention based on the overlap rate between the user's concentration area and the content area.

[0019] The above attention method analysis unit can adaptively update the position and size of the content area according to the user movement when user movement is detected on the screen of the first user terminal.

[0020] The above attention method analysis unit can visualize and provide the level of attention of the corresponding user associated with each of the second user terminals through the screen of the first user terminal.

[0021] Among the embodiments, the method for determining ROI using eye-tracking technology comprises: a step of tracking the gaze of a user through a camera module of each second user terminal in response to a monitoring request from a first user terminal through a monitoring execution unit; a step of identifying a Region of Interest (ROI) corresponding to the user's gaze in a screen area of ​​each second user terminal through an ROI determination unit; and a step of determining the level of attention of the user by analyzing changes in the region of attention during the monitoring through an attention method analysis unit. Effects of the invention

[0023] The disclosed technology may have the following effects. However, this does not mean that a specific embodiment must include all of the following effects or only the following effects; therefore, the scope of the rights of the disclosed technology should not be understood as being limited by this.

[0024] An ROI determination device and method using eye-tracking technology according to one embodiment of the present invention can monitor the attention of users in online, offline, and hybrid classes, meetings, and gatherings in which various users participate, and provide feedback.

[0025] For example, it is possible to determine whether users are paying attention to shared lecture materials, whether they are focusing on the test paper or looking elsewhere when taking an exam, whether they are looking at key areas within the document or looking at whitespace or unnecessary information areas, whether they are examining text, formulas, or information in the correct order or focusing in other abnormal ways, and whether they are over-focusing on unimportant information. Brief explanation of the drawing

[0027] FIG. 1 is a diagram illustrating an ROI determination system according to the present invention. Figure 2 is a diagram illustrating the system configuration of the ROI determination device of Figure 1. Figure 3 is a diagram illustrating the functional configuration of the ROI determination device of Figure 1. FIG. 4 is a flowchart illustrating a method for determining ROI using eye-tracking technology according to the present invention. FIG. 5 is a diagram illustrating an embodiment of a screen sharing process according to the present invention. FIG. 6 is a diagram illustrating an embodiment of an attention monitoring process according to the present invention. FIG. 7 is a drawing illustrating an embodiment of an eye-tracking process according to the present invention. FIG. 8 is a diagram illustrating an embodiment of a content area update process adaptive to user behavior according to the present invention. Specific details for implementing the invention

[0028] The description of the present invention is merely an example for structural or functional explanation, and therefore the scope of the present invention should not be interpreted as being limited by the examples described in the text. That is, since the examples are subject to various modifications and may take various forms, the scope of the present invention should be understood to include equivalents capable of realizing the technical concept. Furthermore, the objectives or effects presented in the present invention do not imply that a specific example must include all of them or only such effects; therefore, the scope of the present invention should not be understood as being limited by them.

[0029] Meanwhile, the meaning of the terms described in this application should be understood as follows.

[0030] Terms such as "first," "second," etc., are intended to distinguish one component from another, and the scope of rights shall not be limited by these terms. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0031] When it is stated that one component is "connected" to another component, it should be understood that it may be directly connected to that other component, or that there may be other components in between. Conversely, when it is stated that one component is "directly connected" to another component, it should be understood that there are no other components in between. Meanwhile, other expressions describing the relationships between components, such as "between" and "exactly between," or "adjacent to" and "directly adjacent to," should be interpreted in the same way.

[0032] A singular expression should be understood to include a plural expression unless the context clearly indicates otherwise, and terms such as "include" or "have" are intended to specify the existence of the implemented features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood not to preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0033] In each step, identifiers (e.g., a, b, c, etc.) are used for convenience of explanation and do not describe the order of the steps; the steps may occur differently from the specified order unless a specific order is clearly indicated in the context. That is, the steps may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

[0034] The present invention may be implemented as computer-readable code on a computer-readable recording medium, and the computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. Additionally, the computer-readable recording medium may be distributed across networked computer systems, so that computer-readable code can be stored and executed in a distributed manner.

[0035] Unless otherwise defined, all terms used herein have the same meaning as generally understood by those skilled in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having meanings consistent with the context of the relevant technology and should not be interpreted as having an ideal or overly formal meaning unless explicitly defined in this application.

[0037] FIG. 1 is a diagram illustrating an ROI determination system according to the present invention.

[0038] Referring to FIG. 1, it may include a user terminal (110), an ROI determination device (130), and a database (150).

[0039] A user terminal (110) may correspond to a terminal device operated by a user. In an embodiment of the present invention, a user may be understood as one or more users, and a plurality of users may be divided into one or more user groups. Each of the one or more users may correspond to one or more user terminals (110). That is, a first user may correspond to a first user terminal, a second user to a second user terminal, ..., and an nth user (where n is a natural number) may correspond to an nth user terminal. For example, a teacher participating in an online class may correspond to a first user, and the remaining students may correspond to at least one second user.

[0040] Additionally, the user terminal (110) may be a computing device capable of participating in a digital learning process by linking with the ROI identification device (130) as a device constituting the ROI identification system (100). Here, digital learning may correspond to a learning process in which one participates online in an educational process conducted in the internet space, such as non-face-to-face classes or online lectures. The user terminal (110) may be implemented as a smartphone, laptop, or computer capable of operating in connection with the ROI identification device (130), but is not necessarily limited thereto and may be implemented as various devices including tablet PCs.

[0041] In particular, the user terminal (110) may install and run a dedicated program or application to link with the ROI identification device (130). For example, the user terminal (110) may perform digital learning by participating in a dedicated online learning space provided by the ROI identification device (130), and may provide multimodal data based on the user's movements to the ROI identification device (130) during the digital learning process.

[0042] In addition, the digital learning process and data collection operation can be performed through an interface provided via a dedicated program or application, and the user's eye tracking operation during digital learning can be performed in conjunction with a camera module. That is, the user terminal (110) can be implemented to include a camera module capable of capturing the user's face for ROI determination operation using the eye-tracking technology according to the present invention.

[0043] Meanwhile, a user terminal (110) can be connected to an ROI identification device (130) via a network, and multiple user terminals (110) can be connected to the ROI identification device (130) simultaneously.

[0044] The ROI determination device (130) may be implemented as a server corresponding to a computer or program that performs the digital phenotyping-based user impulsivity analysis method according to the present invention. Additionally, the ROI determination device (130) may be connected to a user terminal (110) via a wired network or a wireless network such as Bluetooth, WiFi, LTE, etc., and may transmit and receive data with the user terminal (110) through the network.

[0045] Additionally, the ROI determination device (130) may be implemented to operate in connection with an independent external system (not shown in FIG. 1) to perform the ROI determination method using eye-tracking technology according to the present invention. For example, the ROI determination device (130) may operate in conjunction with a learning system that provides online learning, a learning management system that manages learning history, an artificial intelligence system that builds an artificial intelligence model, etc.

[0046] Meanwhile, for the sake of convenience of explanation, the user terminal (110) and the ROI determination device (130) are described as independent devices, but are not necessarily limited thereto, and it is understood that one device may be included in the other device.

[0047] The database (150) may correspond to a storage device that stores various information required during the operation of the ROI determination device (130). For example, the database (150) may store learning content and learning management information for digital learning or multimodal data collected during the digital learning process, but is not necessarily limited thereto, and may store information collected or processed in various forms during the process in which the ROI determination device (130) performs the ROI determination method using eye-tracking technology according to the present invention.

[0048] In addition, in FIG. 1, the database (150) is shown as a device independent of the ROI determination device (130), but it is not necessarily limited to this and can be implemented as a logical storage device included in the ROI determination device (130).

[0050] Figure 2 is a diagram illustrating the system configuration of the ROI determination device of Figure 1.

[0051] Referring to FIG. 2, the ROI determination device (130) may include a processor (210), memory (230), user input / output unit (250), and network input / output unit (270).

[0052] The processor (210) can execute an ROI determination procedure using eye-tracking technology according to an embodiment of the present invention, manage memory (230) that is read or written during this process, and schedule the synchronization time between volatile memory and non-volatile memory in memory (230). The processor (210) can control the overall operation of the ROI determination device (130) and can control the data flow between the memory (230), user input / output unit (250), and network input / output unit (270) by being electrically connected to them. The processor (210) can be implemented as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) of the ROI determination device (130).

[0053] The memory (230) may include an auxiliary storage device implemented as non-volatile memory such as an SSD (Solid State Disk) or HDD (Hard Disk Drive) and used to store all data required for the ROI determination device (130), and may include a main memory device implemented as volatile memory such as RAM (Random Access Memory). Additionally, the memory (230) may store a set of instructions that execute an ROI determination method using eye-tracking technology according to the present invention by being executed by an electrically connected processor (210).

[0054] The user input / output unit (250) includes an environment for receiving user input and an environment for outputting specific information to the user, and may include an input device including an adapter such as a touch pad, touch screen, virtual keyboard, or pointing device, and an output device including an adapter such as a monitor or touch screen. In one embodiment, the user input / output unit (250) may correspond to a computing device connected via remote access, and in such case, the ROI determination device (130) may be performed as an independent server.

[0055] The network input / output unit (270) provides a communication environment for connecting to a user terminal (110) through a network and may include an adapter for communication such as a LAN (Local Area Network), MAN (Metropolitan Area Network), WAN (Wide Area Network), and VAN (Value Added Network). Additionally, the network input / output unit (270) may be implemented to provide short-range communication functions such as WiFi and Bluetooth, or wireless communication functions of 4G or higher, for wireless transmission of data.

[0057] Figure 3 is a diagram illustrating the functional configuration of the ROI determination device of Figure 1.

[0058] Referring to FIG. 3, the ROI determination device (130) can perform an ROI determination method using eye-tracking technology according to the present invention. To this end, the ROI determination device (130) may include a screen sharing unit (310), a monitoring execution unit (330), an ROI determination unit (350), an attention-focusing method analysis unit (370), and a control unit (not shown in FIG. 3).

[0059] At this time, embodiments of the present invention are not required to include all of the above components simultaneously; depending on each embodiment, some of the components may be omitted, or some or all of the components may be selectively included. The operation of each component will be described in detail below.

[0060] The screen sharing unit (310) can share the screen of the first user terminal with at least one second user terminal in response to a screen sharing request from the first user terminal. To this end, a space in which the first user terminal and at least one second user terminal participate may be established online. For example, the online space may correspond to a learning space for conducting online classes. That is, a learning space for online classes may be established in response to a request to open a class by the first user terminal, and one or more second user terminals may participate in the learning space as a result of being invited by the first user terminal. Additionally, when an online class begins, the first user terminal may request screen sharing through a dedicated interface, and the screen sharing unit (310) can share the screen of the first user terminal with other second user terminals participating in the learning space. That is, the first user terminal and the second user terminals can all share the same screen.

[0061] In one embodiment, the screen sharing unit (310) can forcibly switch the screen of each second user terminal to the screen of the first user terminal when screen sharing is initiated. The screen sharing unit (310) can receive a screen selected from among several screens on the first user terminal along with a screen sharing request, and can transmit the selected screen to a plurality of second user terminals so that the selected screen is forcibly displayed on each second user terminal. For example, during an online class, all students may only view the class screen shared by the teacher in response to the teacher's screen sharing request, and individual screen switching functions for each student may be blocked during screen sharing.

[0062] Meanwhile, the screen sharing unit (310) can share the screen of the first user terminal with at least one second user terminal by applying various sharing methods. For example, the screen sharing unit (310) can selectively share the screen only with one or more second user terminals selected on the first user terminal. As another example, the screen sharing unit (310) may inversely share the screen of a specific second user terminal selected by the first user terminal with the first user terminal.

[0063] The monitoring unit (330) can track the gaze of the user through the camera module of each second user terminal in response to a monitoring request from the first user terminal during screen sharing. When the monitoring unit (330) receives a monitoring request from the first user terminal, it can initiate an eye-tracking operation in conjunction with the camera module of each second user terminal. That is, the monitoring unit (330) can track the gaze of individual users during screen sharing and record it in the database (150).

[0064] In one embodiment, the monitoring execution unit (330) may forcibly switch the screen of each second user terminal to the screen of the first user terminal in response to a monitoring request from the first user terminal. The monitoring execution unit (330) may operate in conjunction with the screen sharing unit (310) for screen sharing and switching, and a detailed description thereof is omitted.

[0065] In one embodiment, the monitoring unit (330) can calculate the position of the eyes and the direction of gaze by tracking the eye movement of the user through a camera module. For example, the monitoring unit (330) can generate a gaze vector regarding the user's eye movement, and the gaze vector may include information regarding the position of the eyes and the direction of gaze. The monitoring unit (330) can collect video or images captured by the camera module for tracking eye movement, identify the user's face region from the video or images, and track the movement of the pupils of the eyes. The monitoring unit (330) can calculate the position of the eyes and the direction of gaze based on changes in pupil movement. To this end, the monitoring unit (330) can apply eye tracking technology to the user's face video or image.

[0066] The ROI determination unit (350) can identify a Region of Interest (ROI) corresponding to the user's gaze in the screen area of ​​each second user terminal. The ROI determination unit (350) can estimate the distance between the screen of the second user terminal and the user from a video or image captured by a camera module, and can determine the point where the user's gaze is directed in the screen area of ​​the second user terminal based on the position of the eyes and the direction of the gaze. The ROI determination unit (350) can determine a nearby area based on the point where the user's gaze is directed as the Region of Interest, that is, the area the user is currently focusing on.

[0067] In one embodiment, the ROI determination unit (350) can calculate a gaze coordinate value corresponding to the user's gaze in a two-dimensional coordinate system of the screen area and determine an area within a preset radius centered on the gaze coordinate value as a concentration area. The ROI determination unit (350) can obtain coordinate information of the point where the user's gaze is directed in the screen area through a series of gaze tracking processes. Additionally, the ROI determination unit (350) can determine an area within a preset distance centered on the gaze coordinate value as a concentration area. For example, the ROI determination unit (350) can determine a circular area within a radius of 5 cm centered on the gaze coordinate value as a concentration area. Meanwhile, the size and shape of the concentration area can be preset and applied.

[0068] The attention method analysis unit (370) can determine the level of attention of the user by analyzing changes in the area of ​​concentration during monitoring. For example, the attention method analysis unit (370) can calculate the level of attention of the user by measuring the time the user's gaze stays within the screen. That is, if a student's gaze moves within the screen area for most of the time during an online class, the student's level of attention can be estimated to be high. As another example, the attention method analysis unit (370) can calculate the level of attention of the user based on the number of times the user's gaze moves out of the screen. The attention method analysis unit (370) can determine the level of attention of each user by applying various methods based on various monitoring information collected during monitoring.

[0069] In one embodiment, the attention method analysis unit (370) can monitor whether the user is paying attention by analyzing changes in the focus area during monitoring, and can monitor the attention time, sequence, and pattern based on changes in the level of attention. For example, the attention method analysis unit (370) can determine that the user is not paying attention if the level of attention is below a preset threshold. Additionally, the attention method analysis unit (370) can monitor the attention time by measuring the time during which the level of attention falls within a preset range, and can monitor the user's attention sequence by tracking changes in the location of the focus area. Additionally, the attention method analysis unit (370) can monitor the user's attention pattern by tracking changes in the focus area and the level of attention.

[0070] In one embodiment, the attention method analysis unit (370) can identify the content area of ​​the content shared through the screen of the first user terminal and calculate the level of attention based on the overlap rate between the user's concentration area and the content area. Here, the content area may correspond to a partial area where the core content of the content is displayed within the screen area where the content is output, and the content area may change as the content output in the screen area changes. The attention method analysis unit (370) can identify the content area according to the content of the content shared through the screen of the first user terminal.

[0071] For example, when text-based content is shared, the area where the text is displayed or the area where the main keywords within the text are displayed may be determined as the content area, and when image-based content is shared, the area where the image is displayed or the area where the main objects within the image are displayed may be determined as the content area.

[0072] Additionally, the attention method analysis unit (370) can calculate the user's attention level based on whether there is an overlap between the concentration area where the user's gaze is directed and the content area. That is, the attention method analysis unit (370) can indirectly estimate whether the user is simply looking at the screen area or looking at the content within the screen area, depending on whether there is an overlap between the concentration area and the content area. Consequently, the attention level may correspond to an indicator representing the degree to which the user's gaze changes according to the content displayed on the screen. For example, the higher the user's level of concentration, the higher the attention level may be calculated.

[0073] Additionally, the attention method analysis unit (370) can calculate the overlap rate based on the ratio of the overlapping area between the attention area and the content area. Specifically, the attention method analysis unit (370) can calculate the overlap rate through 'overlapping area / total area'. Here, the overlapping area corresponds to the overlapping area between the attention area and the content area, and the total area corresponds to the sum of the areas of the attention area and the content area.

[0074] In one embodiment, the attention method analysis unit (370) can adaptively update the position and size of the content area according to the user's movement when user movement is detected on the screen of the first user terminal. For example, a teacher conducting an online class may input text to add explanations to the content displayed on the screen or input an action to highlight a part of the content, and in this case, the content area may move toward the teacher's input location or the content area may be expanded to that location. The attention method analysis unit (370) can change and update at least one of the position and size of the virtual content area according to the user's interaction with the content.

[0075] In one embodiment, the attention method analysis unit (370) can visualize and provide the attention level of the corresponding user associated with each second user terminal through the screen of the first user terminal. For example, a teacher conducting an online class can monitor each student's screen through the screen, and in this case, the color of the screen boundary may be changed and displayed according to each student's attention level. Therefore, the teacher can individually perceive the students' attention levels through the color change of the screen and indirectly perceive the overall attention level of the class.

[0076] The control unit (not shown in FIG. 3) controls the overall operation of the ROI determination device (130) and can manage the control flow or data flow between the screen sharing unit (310), the monitoring execution unit (330), the ROI determination unit (350), and the attention method analysis unit (370).

[0078] FIG. 4 is a flowchart illustrating a method for determining ROI using eye-tracking technology according to the present invention.

[0079] Referring to FIG. 4, the ROI determination device (130) can receive a monitoring request from a first user terminal through a monitoring execution unit (330) (step S410). Here, the monitoring request may correspond to a monitoring request regarding attention on at least one second user terminal connected to the first user terminal. The ROI determination device (130) can track the gaze of the corresponding user through the camera module of each second user terminal in accordance with the monitoring request of the first user terminal during screen sharing through the monitoring execution unit (330) (step S430).

[0080] Additionally, the ROI identification device (130) can identify a region of interest (ROI) corresponding to the user's gaze in the screen area of ​​each second user terminal through the ROI identification unit (350) (step S450). The ROI identification device (130) can determine the level of attention of the user by analyzing changes in the region of interest during monitoring through the attention method analysis unit (370) (step S470).

[0082] FIG. 5 is a diagram illustrating an embodiment of a screen sharing process according to the present invention.

[0083] Referring to FIG. 5, a teacher participating in an online class can view the class screen (530) and the screens (550) of the students participating in the online class through the screen (510) of the first user terminal. At this time, the ROI identification device (130) can provide a menu for screen sharing during the online class within the screen (510) of the first user terminal, and the teacher can share the content displayed on the class screen (530) (e.g., AbC...) with the students participating in the class through the menu. Accordingly, the same content displayed on the class screen (530) can be shared simultaneously through each student's screen (550).

[0085] FIG. 6 is a diagram illustrating an embodiment of an attention monitoring process according to the present invention.

[0086] Referring to FIG. 6, the ROI identification device (130) can share each student's screen (650) through the teacher's screen during an online class. Additionally, the ROI identification device (130) can provide a menu for screen monitoring during the online class within the screen (610) of the first user terminal used by the teacher. Accordingly, the teacher conducting the online class can request monitoring of the students' attention through the menu.

[0087] Additionally, the ROI determination device (130) can perform an eye-tracking operation to track the students' gaze through the camera module of each terminal, and can determine the level of attention for each student by evaluating whether the students' gaze is focused on the screen. When the level of attention for each student is calculated, the ROI determination device (130) can visualize the level of attention and output it through the screen (610) of the first user terminal.

[0088] For example, in the case of FIG. 6, on the screen (610) of the first user terminal, a distracted student with low attention and a focused student with high attention can be distinguished and visualized. In addition, the ROI determination device (130) can calculate the attention of the entire online class by integrating the attention of each student and can also visualize and display it on the screen.

[0090] FIG. 7 is a drawing illustrating an embodiment of an eye-tracking process according to the present invention.

[0091] Referring to FIG. 7, the ROI determination device (130) can identify a concentration area (730) corresponding to the user's gaze in the screen area (700) of each second user terminal through the ROI determination unit (350). In the case of Figure (a), as a result of the user's gaze being concentrated on the bottom area of ​​the screen, a concentration area (730) can be formed centered on that location. In the case of Figure (b), as a result of the user's gaze being concentrated on the text displayed in the top area of ​​the screen, a concentration area (730) can be formed centered on that location.

[0092] Additionally, when the screen of the first user terminal is shared with the second user terminal, the ROI determination device (130) can identify the content area (710) of the shared content and calculate the user's attention level based on whether there is an overlap between the user's concentration area (730) and the content area (710). For example, in the case of figure (a), the user's attention level may be calculated as a low value as a result of no overlap between the concentration area (730) and the content area (710), and in the case of figure (b), the user's attention level may be calculated as a high value as a result of overlap between the concentration area (730) and the content area (710).

[0094] FIG. 8 is a diagram illustrating an embodiment of a content area update process adaptive to user behavior according to the present invention.

[0095] Referring to FIG. 8, the ROI determination device (130) can adaptively update the position and size of the content area (810) according to the user movement when user movement is detected on the screen (800) of the first user terminal. In FIG. (a), text content 'AbC...' in the form of text is displayed on the screen (800) of the first user terminal, and text '123xyz...' can be entered by the user.

[0096] That is, the ROI determination device (130) can update the location of the content area (810) by changing it to the location where the text was entered by the user in response to the user's text input, as shown in Figure (b). As another example, the ROI determination device (130) can update the size of the content area (810) by extending the range of the content area (810) to the location where the text was entered by the user in response to the user's text input. Subsequently, the ROI determination device (130) can calculate the level of attention of the user based on whether there is an overlap between the updated content area (810) and the focus area according to the user's gaze.

[0098] Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the following claims. Explanation of the symbols

[0100] 100: ROI Identification System 110: User terminal 130: ROI identification device 150: Database 210: Processor 230: Memory 250: User I / O Section 270: Network I / O Section 310: Screen Sharing Unit 330: Monitoring Execution Unit 350: ROI Identification Unit 370: Attention Method Analysis Unit

Claims

Claim 1 A monitoring execution unit that, in response to a monitoring request from a first user terminal, tracks the gaze of the user through the camera module of each second user terminal and calculates the position of the eyes and the direction of gaze based on changes in pupil movement to generate a gaze vector regarding the user's eye movement; an ROI determination unit that estimates the distance between the screen of the second user terminal and the user from an image captured by the camera module of the second user terminal, determines the point where the user's gaze is directed within the screen area of ​​the second user terminal based on the position of the eyes and the direction of gaze, and identifies a nearby area as a Region of Interest (ROI) based on the point where the user's gaze is directed; and an attention method analysis unit that determines the level of attention of the user by analyzing (1) the overlap rate between the user's area of ​​attention and the content area calculated through the identification of the content area of ​​the content shared through the screen of the first user terminal during the monitoring, (2) the attention time measured by the time the user's gaze remains within the screen, (3) the attention sequence that tracks the positional change of the area of ​​attention, and (4) the attention pattern that tracks the change in the area of ​​attention and the level of attention during the monitoring.An ROI determination device using eye-tracking technology, comprising: a content area that, when text-type content is shared, may be determined as an area where the text is displayed or an area where major keywords among the text are displayed; when image-type content is shared, may be determined as an area where the image is displayed or an area where major objects among the image are displayed; and when the attention method analysis unit detects user movement resulting from text input regarding the shared content or user interaction highlighting a part of the content at the first user terminal, changes and updates at least one of the position and size of the content area according to the user interaction, and calculates the overlap rate between the updated content area and the concentration area corresponding to the gaze of the user at each second user terminal to determine the level of attention of the user associated with each second user terminal. Claim 2 An ROI determination device using eye-tracking technology according to claim 1, wherein the monitoring performing unit forcibly switches the screen of each second user terminal to the screen of the first user terminal in response to a monitoring request of the first user terminal. Claim 3 An ROI determination device using eye-tracking technology according to claim 1, wherein the monitoring performing unit tracks the eye movement of the corresponding user through the camera module to calculate the position of the eye and the direction of gaze. Claim 4 An ROI determination device using eye-tracking technology according to claim 1, wherein the ROI determination unit calculates gaze coordinate values ​​corresponding to the gaze of the corresponding user in a two-dimensional coordinate system of the screen area and determines an area within a preset radius centered on the gaze coordinate values ​​as the concentration area. Claim 5 An ROI determination device using eye-tracking technology according to claim 1, wherein the attention method analysis unit identifies a content area of ​​content shared through the screen of the first user terminal and calculates the attention level based on the overlap rate between the user's concentration area and the content area. Claim 6 delete Claim 7 A device for determining ROI using eye-tracking technology, characterized in that, in claim 1, the attention method analysis unit visualizes and provides the attention level of the corresponding user associated with each second user terminal through the screen of the first user terminal. Claim 8 delete

Images