Assistance system, assistance method and computer program product

Abstract

An assistance system according to one embodiment includes at least one processor. The at least one processor performs the following processing: acquiring target data representing the movement of a target user's viewpoint; referencing a storage unit storing correspondence data and auxiliary information, wherein the correspondence data is data obtained by statistically processing a plurality of sample data and represents the correspondence between the movement of the user's viewpoint and the user's comprehension of content, wherein the plurality of sample data are data obtained from a plurality of sample users, each sample data representing a pair of the movement of the viewpoint of a sample user who visually recognized sample content and the sample user's comprehension of the sample content; and the auxiliary information is information corresponding to the user's comprehension of the content; estimating the target user's comprehension based on the target data and the correspondence data; and outputting the auxiliary information corresponding to the target user's comprehension.

Description

Technical Field

[0001] One aspect of the present disclosure relates to an assistance system, an assistance method, and a computer program product. Background Art

[0002] Technologies that assist users in visually recognizing content are known. For example, Patent Document 1 describes a learning assistance device for assisting with foreign language reading comprehension. This learning assistance device tracks the learner's gaze movements while reading a foreign language text, calculates the frequency of rereading and the duration of gaze, and presents information related to this frequency and duration to the instructor. Patent Documents 2 to 6 also describe technologies related to user assistance.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2005-338173

[0006] Patent Document 2: Japanese Patent Application Laid-Open No. 2010-039646

[0007] Patent Document 3: Japanese Patent Application Laid-Open No. 2016-114684

[0008] Patent Document 4: Japanese Patent Application Laid-Open No. 2018-097266

[0009] Patent Document 5: Japanese Patent No. 6636670

[0010] Patent Document 6: Japanese Patent Application Laid-Open No. 2014-194637 Summary of the Invention

[0011] Problems to be solved by the invention

[0012] A method that can appropriately assist a user in visually confirming content is desired.

[0013] Means for solving problems

[0014] An auxiliary system according to one aspect of the present disclosure includes at least one processor. The at least one processor performs the following processing: acquiring target data representing the movement of a target user's viewpoint on a screen displaying target content; referencing a storage unit that stores correspondence data and auxiliary information, wherein the correspondence data is data obtained by statistically processing a plurality of sample data and represents a correspondence between the movement of the user's viewpoint and the user's understanding of the content, wherein the plurality of sample data is data obtained from a plurality of sample users who visually confirmed the sample content, each sample data representing a pair of the movement of the viewpoint of the sample user who visually confirmed the sample content and the sample user's understanding of the sample content, and the auxiliary information is information corresponding to the user's understanding of the content; estimating the target user's understanding of the target content based on the target data and the correspondence data; and outputting auxiliary information corresponding to the estimated target user's understanding.

[0015] In this aspect, correspondence data is generated by statistically processing sample data obtained from sample users. The target user's comprehension level is estimated based on this correspondence data and target data representing the movement of the target user's viewpoint relative to the target content. Using the correspondence data obtained through statistical processing, the target user's comprehension level is estimated based on the user's actual tendency to visually review the content. By outputting auxiliary information based on this estimate, it is possible to appropriately assist the target user in visually reviewing the target content.

[0016] Effects of the Invention

[0017] According to the embodiments of the present disclosure, it is possible to appropriately assist a user who visually confirms content. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a diagram showing an example of application of the support system according to the embodiment.

[0019] Figure 2 This is a diagram showing an example of a hardware configuration related to the support system according to the embodiment.

[0020] Figure 3 This is a diagram showing an example of a functional configuration related to the support system according to the embodiment.

[0021] Figure 4 This is a flowchart showing an example of the operation of the support system according to the embodiment.

[0022] Figure 5 This is a flowchart showing an example of the operation of the eye tracking system according to the embodiment.

[0023] Figure 6This is a diagram showing an example of a guide area set for the first content.

[0024] Figure 7 This is a diagram showing an example of a guide area set for the first content.

[0025] Figure 8 This is a flowchart showing an example of the operation of the support system according to the embodiment.

[0026] Figure 9 This is a flowchart showing an example of the operation of the support system according to the embodiment.

[0027] Figure 10 It is a diagram showing an example of auxiliary information. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are denoted by the same reference numerals, and repeated descriptions are omitted.

[0029] [System Overview]

[0030] The auxiliary system of the embodiment is a computer system that assists users in visually confirming content. Content refers to information that is provided by a computer or computer system and can be recognized by humans. Electronic data representing content is called content data. There is no limitation on the form of expression of content. For example, content can also be expressed by documents, images (for example, photos, videos, etc.), or a combination thereof. There is no limitation on the purpose and usage scenario of content. For example, content can be used for various purposes such as education, news, lectures, business transactions, entertainment, medical care, games, and chatting.

[0031] Auxiliary systems provide content to users by sending content data to user terminals. Users are those who wish to receive information from the auxiliary system; in other words, they are content viewers. User terminals can also be referred to as "viewer terminals." Auxiliary systems can provide content data to user terminals based on user requests or based on instructions from a publisher other than the user. A publisher is the person who delivers information to the user (viewer); in other words, the sender of the content.

[0032] The auxiliary system not only provides the user with content as needed, but also provides the user with auxiliary information corresponding to the user's understanding. The user's understanding is an indicator that represents the user's understanding of the content. For example, in the case where the content includes an article, the user's understanding can also be an indicator that represents how much the user understands the article (for example, whether the user understands the meaning of the words contained in the article, whether the user understands the grammar of the article, etc.). Auxiliary information is information used to promote the user's understanding of the content. For example, in the case where the content includes an article, the auxiliary information can also be information that represents the meaning of the words contained in the article, the grammar of the article, etc. In the following description, the user who will be the target of estimating the understanding (in other words, the user who will be the target of providing auxiliary information as needed) is referred to as the target user, and the content that will be visually confirmed by the target user is referred to as the target content.

[0033] To output auxiliary information, the auxiliary system estimates the target user's comprehension based on the movement of the target user's viewpoint on the screen displaying the target content. Specifically, the auxiliary system refers to correspondence data that represents the correspondence between the movement of the user's viewpoint and the user's comprehension. Correspondence data is electronic data generated by statistically processing pre-acquired sample data. Sample data is electronic data that represents the correspondence between the movement of the user's viewpoint who visually recognized the content and the user's comprehension of the content. In the following description, the user who provides the sample data used to generate the correspondence data is referred to as the sample user, and the content visually recognized by the sample user is referred to as the sample content.

[0034] The auxiliary system obtains data representing the movement of the target user's viewpoint from the user terminal of the target user. The data representing the movement of the viewpoint is data indicating how the user's viewpoint moves on the screen of the user terminal, and is also referred to as viewpoint data in this disclosure. Hereinafter, the data representing the movement of the target user's viewpoint (i.e., the viewpoint data of the target user) will be referred to as target data. The auxiliary system uses the correspondence data and the target data to estimate the target user's understanding. Thereafter, the auxiliary system outputs auxiliary information corresponding to the target user's understanding to the user terminal of the target user as needed.

[0035] In this disclosure, when there is no need to distinguish between sample users and target users, they are sometimes collectively referred to as users for explanation.

[0036] Viewpoint data is acquired by an eye tracking system. Based on the movement of the user's eyes, the eye tracking system determines the user's viewpoint coordinates at given time intervals and acquires viewpoint data representing multiple viewpoint coordinates arranged along a time series. Viewpoint coordinates are coordinates that represent the position of the viewpoint on the screen of the user terminal. Viewpoint coordinates can also be represented using a two-dimensional coordinate system. The eye tracking system can be installed on the user terminal or on another computer different from the user terminal. Alternatively, the tracking system can also be implemented by the user terminal collaborating with other computers.

[0037] The eye tracking system performs processing, or correction, to more accurately determine the user's viewpoint coordinates. As an example, the eye tracking system first sets a portion of the content displayed on the screen of the user terminal as a guide area for the user to focus on. Hereinafter, the content with the guide area set is referred to as the first content. Then, based on the user's eye movements, the eye tracking system determines the viewpoint coordinates of the user gazing at the guide area as the first viewpoint coordinates and calculates the difference between the first viewpoint coordinates and the regional coordinates of the guide area. The regional coordinates of the guide area are coordinates that represent the position of the guide area on the screen of the user terminal. Thereafter, when the user visually confirms the content (second content) displayed on the screen of the user terminal, the eye tracking system determines the viewpoint coordinates of the user viewing the second content as the second viewpoint coordinates based on the user's eye movements. The eye tracking system then uses the pre-calculated difference to correct the determined second viewpoint coordinates. The second content is the content the user is viewing when the second viewpoint coordinates are corrected.

[0038] As described above, the purpose and usage scenarios of the content are not limited. In this embodiment, educational content is shown as an example of content, and the auxiliary system assists students in visually confirming the educational content. Therefore, the target content is "target content for education" and the sample content is "sample content for education". Educational content is content used to educate students, and can be, for example, tests such as exercises and test questions, or can be textbooks. Educational content can also include articles, mathematical formulas, charts or graphs, etc. Students refer to people who receive education in academic subjects, skills, etc. Students are an example of users (viewers). As described above, content can also be published to viewers based on the instructions of the publisher. In the case where the content is educational content, the publisher can also be a teacher. A teacher refers to a person who teaches academic subjects, skills, etc. to students. The teacher can be a person with a teacher's certificate or a person without a teacher's certificate. There is no limitation on the age and affiliation of the teacher and the student. Therefore, the purpose and usage scenarios of educational content are also not limited. For example, educational content can be used in various schools, including nurseries, kindergartens, elementary schools, middle schools, high schools, universities, graduate schools, vocational colleges, preparatory schools, and online schools, as well as in locations and settings outside of school. Furthermore, educational content can be used for a variety of purposes, including early childhood education, compulsory education, higher education, and career development. Furthermore, educational content encompasses not only school-based education but also content used in seminars and training sessions within companies and other settings.

[0039] [System Structure]

[0040] Figure 1 This diagram illustrates an example of an application of a support system 1 according to an embodiment. In this embodiment, the support system 1 includes a server 10. The server 10 is communicatively connected to a user terminal 20 and a database 30 via a communication network N. The structure of the communication network N is not limited. For example, the communication network N may include the Internet or a local area network.

[0041] The server 10 is a computer that distributes content to the user terminal 20 and provides auxiliary information to the user terminal 20 as needed. The server 10 may be composed of one or more computers.

[0042] The user terminal 20 is a computer used by the user. In this embodiment, the user is a student who watches educational content. In one example, the user terminal 20 has the function of accessing the auxiliary system 1 to receive content data and auxiliary information and display them, and the function of sending viewpoint data to the auxiliary system 1. The type of user terminal 20 is not limited, and it may be, for example, a high-performance portable phone (smartphone), a tablet terminal, a wearable terminal (for example, a head-mounted display (HMD), smart glasses, etc.), a laptop personal computer, a portable phone, or other portable terminal. Alternatively, the user terminal 20 may be a fixed terminal such as a desktop personal computer. In Figure 1 Although three user terminals 20 are shown, the number of user terminals 20 is not limited. In this embodiment, to distinguish between the terminals of a sample user and the terminals of a target user, the terminal of the sample user is labeled "user terminal 20A" and the terminal of the target user is labeled "user terminal 20B." Users log in to the support system 1 by operating a user terminal 20 and can view content. This embodiment assumes that the user of the support system 1 has already logged in.

[0043] The database 30 is a non-transitory storage device that stores data used by the auxiliary system 1. In this embodiment, the database 30 stores content data, sample data, correspondence data, and auxiliary information. The database 30 can be a single database or a collection of multiple databases.

[0044] Figure 2 This is a diagram showing an example of a hardware configuration related to the support system 1 . Figure 2 A server computer 100 functioning as a server 10 and a terminal computer 200 functioning as a user terminal 20 are shown.

[0045] As an example, the server computer 100 includes a processor 101 , a main storage unit 102 , an auxiliary storage unit 103 , and a communication unit 104 as hardware components.

[0046] The processor 101 is a computing device that executes an operating system and application programs. Examples of processors include a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), but the type of processor 101 is not limited to these.

[0047] The main storage unit 102 stores programs for implementing the server 10, calculation results output from the processor 101, etc. The main storage unit 102 is composed of, for example, at least one of a ROM (Read Only Memory) and a RAM (Random Access Memory).

[0048] The auxiliary storage unit 103 is generally a device capable of storing a larger amount of data than the main storage unit 102. The auxiliary storage unit 103 is comprised of, for example, a non-volatile storage medium such as a hard disk or flash memory. The auxiliary storage unit 103 stores a server program P1 and various data for enabling the server computer 100 to function as the server 10. In this embodiment, an auxiliary program is installed as the server program P1.

[0049] The communication unit 104 is a device that performs data communication with other computers via the communication network N. The communication unit 104 is configured by, for example, a network card or a wireless communication module.

[0050] Each functional element of the server 10 is implemented by having the processor 101 or the main storage unit 102 read the server program P1 and having the processor 101 execute the program. The server program P1 contains code for implementing each functional element of the server 10. The processor 101 operates the communication unit 104 according to the server program P1, and reads and writes data from the main storage unit 102 or the auxiliary storage unit 103. This processing realizes each functional element of the server 10.

[0051] The server 10 may be composed of one or more computers. When a plurality of computers are used, these computers are connected to each other via a communication network N, thereby logically forming one server 10.

[0052] As an example, the terminal computer 200 includes a processor 201 , a main storage unit 202 , an auxiliary storage unit 203 , a communication unit 204 , an input interface 205 , an output interface 206 , and an imaging unit 207 as hardware components.

[0053] The processor 201 is a computing device that executes an operating system and application programs. The processor 201 may be, for example, a CPU or a GPU, but the type of the processor 201 is not limited thereto.

[0054] The main storage unit 202 is a device that stores programs for realizing the user terminal 20, calculation results output from the processor 201, and the like. The main storage unit 202 is composed of, for example, at least one of a ROM and a RAM.

[0055] The auxiliary storage unit 203 is generally capable of storing a larger amount of data than the main storage unit 202. The auxiliary storage unit 203 is comprised of, for example, a nonvolatile storage medium such as a hard disk or flash memory. The auxiliary storage unit 203 stores the client program P2 and various data for enabling the terminal computer 200 to function as the user terminal 20.

[0056] The communication unit 204 is a device that performs data communication with other computers via the communication network N. The communication unit 204 is configured by, for example, a network card or a wireless communication module.

[0057] The input interface 205 is a device that receives data based on user operations or actions. For example, the input interface 205 is composed of at least one of a keyboard, operation buttons, a pointing device, a touch panel, a microphone, a sensor, and a camera.

[0058] The output interface 206 is a device that outputs data processed by the terminal computer 200. For example, the output interface 206 is composed of at least one of a monitor, a touch panel, an HMD, and a speaker.

[0059] The imaging unit 207 is a device that captures images of the real world, specifically a camera. The imaging unit 207 can capture moving images (videos) or still images (photos). The imaging unit 207 can also function as the input interface 205.

[0060] Each functional element of the user terminal 20 is implemented by having the processor 201 or the main storage unit 202 read the client program P2 and having the processor 201 execute this program. The client program P2 contains code for implementing each functional element of the user terminal 20. The processor 201 operates the communication unit 204, input interface 205, output interface 206, or camera unit 207 according to the client program P2, and reads and writes data from the main storage unit 202 or the auxiliary storage unit 203. Through this processing, each functional element of the user terminal 20 is implemented.

[0061] At least one of the server program P1 and the client program P2 may be provided in a non-transitory manner recorded on a tangible recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory. Alternatively, at least one of these programs may be provided as a data signal superimposed on a carrier wave via a communication network N. These programs may be provided individually or together.

[0062] Figure 3This is a diagram showing an example of a functional structure associated with the auxiliary system 1. The server 10 includes a content distribution unit 11, a statistical processing unit 12, an estimation unit 13, and an auxiliary unit 14 as functional elements. The statistical processing unit 12 is a functional element that generates correspondence data. The statistical processing unit 12 generates correspondence data by performing statistical processing on sample data stored in the database 30, and stores the correspondence data in the database 30. The estimation unit 13 is a functional element that estimates the target user's understanding of the target content. The estimation unit 13 obtains target data representing the movement of the target user's viewpoint from the user terminal 20B, and estimates the target user's understanding based on the target data and the correspondence data. The auxiliary unit 14 is a functional element that sends auxiliary information corresponding to the target user's understanding to the user terminal 20B.

[0063] The user terminal 20 includes a setting unit 21, a determination unit 22, a calculation unit 23, a tracking unit 24, and a display control unit 25 as functional elements. The setting unit 21 sets a partial area of ​​the first content displayed on the screen of the user terminal 20 as a guide area. The determination unit 22 identifies the user's first viewpoint coordinates based on the movement of the user's eyes while gazing at the guide area. The calculation unit 23 calculates the difference between the area coordinates of the guide area set by the setting unit 21 and the first viewpoint coordinates determined by the determination unit 22. The tracking unit 24 generates viewpoint data by observing the eye movement of the user viewing the content displayed on the screen of the user terminal 20. The tracking unit 24 uses the calculated difference to correct the second viewpoint coordinates of the user viewing the second content and generates viewpoint data representing the corrected second viewpoint coordinates. The display control unit 25 controls the display of the screen on the user terminal 20. In this embodiment, the eye tracking system includes the setting unit 21, the determination unit 22, the calculation unit 23, the tracking unit 24, and the display control unit 25.

[0064] [System Action]

[0065] Figure 4 This is a flowchart showing the operation of the support system 1 as a processing flow S1. Figure 4 , the overall processing of the auxiliary system 1 is described.

[0066] In step S11 , the statistical processing unit 12 of the server 10 performs statistical processing on a plurality of sample data to generate correspondence relationship data.

[0067] An example of collecting sample data, which is a prerequisite for step S11, will be described. First, the content distribution unit 11 distributes sample content to each of a plurality of user terminals 20A. The timing of distributing sample content to each user terminal 20A is not limited. For example, the content distribution unit 11 may distribute sample content to each user terminal 20A in response to a request from that user terminal 20A, or may distribute the sample content to two or more user terminals 20A simultaneously. In each user terminal 20A, the display control unit 25 receives and displays the sample content. The tracking unit 24 of the user terminal 20A then generates viewpoint data representing the movement of the viewpoint of a sample user visually viewing the sample content. In one example, the sample user inputs their understanding of the sample content into the user terminal 20A in the form of responses to a questionnaire, and the user terminal 20A accepts this input data. Alternatively, the user terminal 20A or the server 10 may estimate the sample user's understanding based on the user's responses to the sample content (e.g., answers to questions). The input or estimated comprehension level indicates, for example, whether the meaning of the words contained in the article can be understood, whether the grammar of the article can be understood, etc. In one example, the user terminal 20A generates sample data representing a pair of the generated viewpoint data and the input or estimated comprehension level, and sends the sample data to the server 10. Alternatively, the user terminal 20A may send the viewpoint data to the server 10, and the server 10 may generate sample data representing a pair of the viewpoint data and the estimated comprehension level. In any case, the server 10 stores the sample data in the database 30. The server 10 stores a plurality of sample data obtained from a plurality of user terminals 20A regarding a particular sample content in the database 30. The server 10 may store a plurality of sample data for each of the plurality of sample contents. The auxiliary system 1 collects sample data through this series of processing.

[0068] The statistical processing unit 12 reads a plurality of sample data from the database 30, performs statistical processing on the plurality of sample data, and generates correspondence data. The statistical processing method performed by the statistical processing unit 12 and the representation format of the generated correspondence data are not limited.

[0069] For example, the statistical processing unit 12 clusters multiple sample data based on the viewpoint movement of the sample users and their comprehension of the sample content, thereby generating correspondence data. The statistical processing unit 12 may also determine the similarity of viewpoint movement based on at least one of the viewpoint movement speed, the number of viewpoint reversals (the number of times the viewpoint's direction of movement changes), or the area of ​​the viewpoint movement region. Alternatively, the statistical processing unit 12 may determine the similarity of content comprehension based on at least one of the comprehension of the meaning of a word or the comprehension of the grammar of the text. Alternatively, the statistical processing unit 12 may vectorize features related to viewpoint movement and features related to comprehension for each sample data item into feature vectors, so that sample data items with common or similar feature vectors belong to the same cluster. Based on the clustering results, the statistical processing unit 12 derives a correspondence between the user's viewpoint movement and the user's comprehension. More specifically, this correspondence represents a pair of user viewpoint movement trends and corresponding comprehension levels. The statistical processing unit 12 generates correspondence data representing this correspondence and stores it in the database 30.

[0070] As another example, the statistical processing unit 12 may generate correspondence data by performing regression analysis. Specifically, the statistical processing unit 12 quantifies the sample user's viewpoint movement and the sample user's comprehension level based on a predetermined rule. The statistical processing unit 12 performs regression analysis on the quantified data to generate a regression equation with the sample user's comprehension level as the target variable and the sample user's viewpoint movement as the explanatory variable. In this case, the statistical processing unit 12 may decompose the sample user's viewpoint movement into multiple factors, such as viewpoint movement speed and viewpoint reversal frequency, and set multiple explanatory variables corresponding to these multiple factors. For example, the statistical processing unit 12 may quantify the viewpoint movement speed and viewpoint reversal frequency as independent explanatory variables and perform a multivariate regression analysis using these multiple explanatory variables. The statistical processing unit 12 stores the regression equation generated by the regression analysis as correspondence data in the database 30. The regression analysis method performed by the statistical processing unit 12 may also be partial least squares regression (PLS) or support vector regression (SVR). In short, this correspondence data also represents a pair of user viewpoint movement trends and the corresponding comprehension levels.

[0071] As another example, the statistical processing unit 12 may also analyze the correspondence between the movement of the viewpoint of the sample user and the degree of understanding of the sample user through machine learning to generate correspondence data. Machine learning may also be deep learning using a neural network. The statistical processing unit 12 uses a machine learning model configured to output data representing the degree of understanding of the user when data representing the movement of the user's viewpoint is input into an input layer, performs supervised learning using the sample data as learning data, and adjusts the weighting parameters within the learning model. The statistical processing unit 12 stores the model (learning-completed model) with the adjusted weighting parameters as correspondence data in the database 30. When machine learning is employed, the statistical processing unit 12 may also pre-process the sample data stored in the database 30 and convert it into data in a form suitable for machine learning.

[0072] The statistical processing unit 12 may also appropriately select sample data used in the statistical processing to generate a variety of correspondence data. For example, the statistical processing unit 12 may also generate correspondence data for each of a plurality of sample contents using sample data obtained from a plurality of sample users who watch the sample contents. In this case, correspondence data is generated for each content. Hereinafter, this correspondence data is referred to as "content-specific correspondence data." Alternatively, the statistical processing unit 12 may also generate correspondence data using sample data of a plurality of sample contents (for example, a plurality of sample contents belonging to the same category). In this case, common correspondence data is generated for a plurality of contents (for example, a plurality of contents belonging to the same category). Hereinafter, this correspondence data is referred to as "generalized correspondence data."

[0073] In step S12, the auxiliary unit 14 provides auxiliary information to the target user who is visually confirming the target content as needed. The auxiliary unit 14 estimates the target user's understanding of the target content and provides auxiliary information corresponding to the understanding as needed. The details of the process of outputting the auxiliary information will be described later. The correspondence between the user's understanding and the auxiliary information is predetermined, and the auxiliary information is pre-stored in the database 30 in such a way that the correspondence can be determined. The user's understanding and the auxiliary information can also be matched in such a way that the auxiliary information supplements the target user's insufficient understanding of the target content. For example, for the user's understanding of a word contained in an article in the content that indicates that the user does not understand the meaning of the word, the meaning of the word can also be matched as auxiliary information.

[0074] Figure 5This flowchart shows the operation of the eye tracking system as process flow S2. The eye tracking system's processing is broadly divided into the steps of calculating the difference used for viewpoint coordinate correction (steps S21 to S23), and correcting the user's viewpoint coordinates using the calculated difference (steps S24 and S25).

[0075] In step S21, the setting unit 21 dynamically sets a partial area of ​​the first content displayed on the screen of the user terminal 20 as a guide area. The first content is any content distributed by the content distribution unit 11 and displayed by the display control unit 25. The first content can be educational content or non-educational content. The guide area is an area for attracting the user's attention and is composed of a plurality of pixels arranged continuously. Dynamically setting the guide area means that in response to the display of first content on the screen without a pre-set area for attracting the user's attention, the guide area is set in the first content. In one example, the guide area is set only while the first content is displayed on the screen. The location of the guide area in the first content displayed on the screen is not restricted. For example, the setting unit 21 can set the guide area in any position such as the center, top, bottom, or corner of the first content. In one example, after the setting unit 21 sets the guide area, the display control unit 25 displays the guide area in the first content based on the setting. The shape and area (number of pixels) of the guide area are also not limited. The guide area is an area that the user focuses on to correct viewpoint coordinates. Therefore, the setting unit 21 typically sets the area of ​​the guide area to be much smaller than the area of ​​the first content displayed on the screen (ie, the area of ​​the display device).

[0076] There are no limitations on the method for dynamically setting the guidance area. In one example, the setting unit 21 may visually distinguish the guidance area from the non-guidance area by displaying the guidance area differently from the area outside the guidance area (hereinafter referred to as the non-guidance area). The method for setting the display method is not limited. As a specific example, the setting unit 21 may distinguish the guidance area from the non-guidance area by reducing the resolution of the non-guidance area while maintaining the resolution of the guidance area, thereby relatively increasing the resolution of the guidance area. As another specific example, the setting unit 21 may distinguish the guidance area from the non-guidance area by blurring the non-guidance area without changing the display method of the guidance area. For example, the setting unit 21 may perform blurring by setting the color of a target pixel in the non-guidance area to the average color of multiple pixels adjacent to the target pixel. The setting unit 21 may perform blurring while maintaining the resolution of the non-guidance area or by reducing the resolution. As another specific example, the setting unit 21 may distinguish the guidance area from the non-guidance area by surrounding the outer edge of the guidance area with a frame of a specific color or type. The setting unit 21 may distinguish the guide area from other areas by combining any two or more methods among resolution adjustment, blurring, and drawing of a frame line.

[0077] Alternatively, in the case where the first content includes a selection object that can be selected by the user, the setting unit 21 may also set the area displaying the selection object as a guide area. That is, the setting unit 21 may also determine the selection object as a partial area and set the selection object as a guide area. Typically, the selection object may also be a selection button or link displayed in the tutorial screen of the application. Alternatively, in the case of question practice or test by the user terminal 20, the selection object may be a button for selecting a question or a button for starting practice or test. The setting unit 21 may reduce the resolution of the non-guide area while maintaining the resolution of the selection object set as the guide area. In addition to or instead of this processing, the setting unit 21 may blur the non-guide area or surround the outer edge of the selection object set as the guide area with a specific color or a specific type of frame line.

[0078] The setting unit 21 sets the area coordinates of the guide area using any method. For example, the setting unit 21 may set the coordinates of the center or center of gravity of the guide area as the area coordinates. Alternatively, the setting unit 21 may set the position of any pixel in the guide area as the area coordinates.

[0079] In step S22, the determination unit 22 determines the viewpoint coordinates of the user who is looking at the guide area as the first viewpoint coordinates. The determination unit 22 determines the viewpoint coordinates based on the movement of the user's eyes. The method for determining the viewpoint coordinates is not limited. As an example, the determination unit 22 may also capture the peripheral image of the user's eyes through the camera unit 207 of the user terminal 20, and determine the viewpoint coordinates based on the position of the iris with the inner corner of the user's eye as the reference point. As another example, the determination unit 22 may also use the corneal reflection method (PCCR) to determine the user's viewpoint coordinates. When the corneal reflection method is adopted, the user terminal 20 may also have an infrared emitting device and an infrared camera as a hardware structure.

[0080] In step S23, the calculation unit 23 calculates the difference between the first viewpoint coordinates determined by the determination unit 22 and the area coordinates of the guide area set by the setting unit 21. For example, if the position on the screen of the user terminal 20 is represented by an XY coordinate system, and the first viewpoint coordinates are (105, 105) and the area coordinates are (100, 100), the difference is (105-100, 105-100) = (5, 5). The calculation unit 23 stores the calculated difference in any storage device, such as the main storage unit 202 or the auxiliary storage unit 203.

[0081] To improve the accuracy of the correction, the user terminal 20 may repeat the process from step S21 to step S23 multiple times while changing the position of the guide area. In this case, the calculation unit 23 may set the statistical value (e.g., the average value) of the calculated multiple differences as the difference used in the subsequent correction process (step S25).

[0082] In step S24, the tracking unit 24 determines the viewpoint coordinates of the user viewing the second content as the second viewpoint coordinates. The second content is any content distributed by the content distribution unit 11 and displayed by the display control unit 25. For example, the second content may be sample content or target content. The tracking unit 24 may determine the second viewpoint coordinates using the same method as the determination unit 22 for determining the first viewpoint coordinates (i.e., the same method as the process in step S22). The second content may be different from or the same as the first content.

[0083] In step S25, the tracking unit 24 uses the difference to correct the second viewpoint coordinates. For example, if the second viewpoint coordinates determined in step S24 are (190, 155) and the difference calculated in step S23 is (5, 5), the tracking unit 24 corrects the second viewpoint coordinates so that (190-5, 155-5) = (185, 150).

[0084] The tracking unit 24 may also repeatedly perform steps S24 and S25 to obtain multiple corrected second viewpoint coordinates arranged in time series and generate viewpoint data representing the movement of the user's viewpoint. Alternatively, the tracking unit 24 may obtain multiple corrected second viewpoint coordinates, and the server 10 may generate viewpoint data based on the multiple second viewpoint coordinates.

[0085] Reference Figure 6 and Figure 7 , an example of setting the boot area is described. Figure 6 and Figure 7 Each of these diagrams shows an example of a guide area set by the setting unit 21 for the first content.

[0086] exist Figure 6 In the example, the setting unit 21 sets the guide area by reducing the resolution of the non-guide area. In this example, the user terminal 20 displays the first content C11 including the child, the lawn, and the ball, and calculates the difference while changing the position of the guide area on the first content C11. As the position of the guide area changes, the display changes in the order of screens D11, D12, and D13. Figure 6 In FIG, the non-guide area is indicated by a dotted line.

[0087] First, the setting unit 21 sets the portion of the child's face as the guide area A11. Screen D11 corresponds to this setting. The setting unit 21 does not change the resolution of the guide area A11, but instead reduces the resolution of the area outside the guide area (the non-guide area). As an example, the setting unit 21 may reduce the resolution of the non-guide area so that the resolution of the guide area A11 is at least twice or at least four times that of the non-guide area. For example, if the resolution of the guide area A11 is 300 ppi, the resolution of the non-guide area may be set to 150 ppi or less, or 75 ppi or less. With this resolution setting, the non-guide area appears blurrier than the guide area A11, so the user's gaze is generally directed toward the clearly displayed guide area A11. This allows the viewpoint coordinates (first viewpoint coordinates) of the user looking at the guide area A11 to be determined. While screen D11 is being displayed, the determination unit 22 obtains the user's first viewpoint coordinates. Next, the calculation unit 23 calculates the difference between the first viewpoint coordinates and the area coordinates of the guide area A11.

[0088] Next, the setting unit 21 sets the portion of the ball as the guide area A12. Screen D12 corresponds to this setting. The setting unit 21 restores the resolution of guide area A12 to its original value and reduces the resolution of the area outside of guide area A12 (the non-guide area). As a result, the user's line of sight is always directed toward guide area A12. While screen D12 is being displayed, the determination unit 22 obtains the coordinates of the user's first viewpoint. Next, the calculation unit 23 calculates the difference between the first viewpoint coordinates and the area coordinates of guide area A12.

[0089] Afterwards, the setting unit 21 sets the lower right portion (the lawn portion) of the first content C11 as the guide area A13. Screen D13 corresponds to this setting. The setting unit 21 restores the resolution of the guide area A13 to its original value and reduces the resolution of the area outside the guide area A13 (non-guide area). As a result, the user's line of sight is usually directed toward the guide area A13. While the screen D13 is being displayed, the determination unit 22 obtains the user's first viewpoint coordinates. Next, the calculation unit 23 calculates the difference between the first viewpoint coordinates and the area coordinates of the guide area A13. The calculation unit 23 obtains a statistical value of the multiple calculated differences. The statistical value is used by the tracking unit 24 to correct the second viewpoint coordinates (step S25).

[0090] exist Figure 7 In the example shown in FIG, the setting unit 21 sets the selected object in the first content C21 as the guidance area. In this example, the first content C21 is a tutorial for an online academic achievement test. As the tutorial progresses, the display changes in the order of screens D11, D12, and D13.

[0091] Screen D21 includes a string such as "Questions about learning to speak a foreign language." and an OK button. The OK button is a selection object. The setting unit 21 sets the area where the OK button is displayed as the guide area A21. Usually, when a user operates a selection object, they look at the selection object. Therefore, it is possible to determine the viewpoint coordinates (first viewpoint coordinates) of the user who is looking at the guide area A21. In one example, when the user selects the OK button, the determination unit 22 obtains the user's first viewpoint coordinates. Next, the calculation unit 23 calculates the difference between the first viewpoint coordinates and the area coordinates of the guide area A21.

[0092] If the user operates the OK button, the display control unit 25 switches the screen D21 to the screen D22. The screen D22 includes a character string such as "Please select the number of questions." and three selection buttons such as "5", "10", and "15". These selection buttons are selection objects. The setting unit 21 sets the areas where the three selection buttons are displayed as the guide area A22, the guide area A23, and the guide area A24, respectively. In one example, when the user selects any one of the three selection buttons, the determination unit 22 determines the user's viewpoint coordinates (first viewpoint coordinates). Then, the calculation unit 23 calculates the difference between the first viewpoint coordinates and the area coordinates of the guide area (one of the guide areas A22 to A24) corresponding to the selected object selected by the user.

[0093] If the user selects a selection button, the display control unit 25 switches the screen D22 to the screen D23. The screen D23 includes a character string such as "Do you want to start the test?" and a start button. The start button is a selection object. The setting unit 21 sets the area where the start button is displayed as the guide area A25. In one example, when the user selects the start button, the determination unit 22 obtains the user's first viewpoint coordinates. Next, the calculation unit 23 calculates the difference between the first viewpoint coordinates and the area coordinates of the guide area A25. The calculation unit 23 calculates the statistical value of the multiple calculated differences. The statistical value is used by the tracking unit 24 to correct the second viewpoint coordinates (step S25).

[0094] Figure 8 This flowchart shows an example of the operation of support system 1 as process flow S3. Process flow S3 illustrates the steps for providing support information to a target user viewing target content. Process flow S3 assumes that the target user is logged into support system 1. Furthermore, it assumes that the eye tracking system has already calculated the difference used for viewpoint coordinate correction.

[0095] In step S31, the display control unit 25 of the user terminal 20B displays the target content on the screen of the user terminal 20B. The display control unit 25 receives content data distributed from the content distribution unit 11 from the server 10, for example, and displays the target content based on the content data.

[0096] In step S32, the tracking unit 24 of the user terminal 20B obtains the viewpoint coordinates (second viewpoint coordinates) of the target user visually viewing the target content. Specifically, the tracking unit 24 determines the viewpoint coordinates (pre-correction viewpoint coordinates) based on the movement of the target user's eyes while viewing the target content, and corrects the determined viewpoint coordinates using the pre-calculated difference. The tracking unit 24 may also obtain the corrected viewpoint coordinates at predetermined time intervals and generate viewpoint data in which the multiple viewpoint coordinates are arranged in a time series (i.e., target data representing the movement of the target user's viewpoint).

[0097] In step S33, the estimation unit 13 obtains target data. For example, the estimation unit 13 may receive the target data from the tracking unit 24 of the user terminal 20B. Alternatively, the tracking unit 24 may sequentially transmit a plurality of corrected viewpoint coordinates to the server 10, and the estimation unit 13 may generate viewpoint data (target data) in which the plurality of viewpoint coordinates are arranged in a time series.

[0098] In step S34, the estimation unit 13 retrieves the correspondence data from the database 30 and estimates the target user's comprehension of the target content based on the target data and the correspondence data. For example, if the correspondence data is generated through clustering, the estimation unit 13 estimates the target user's comprehension as the comprehension level represented by the cluster to which the target data belongs. For another example, if the correspondence data is generated through regression analysis, the estimation unit 13 applies the target data to the regression equation to estimate the target user's comprehension level. For yet another example, if the correspondence data represents a learned model, the estimation unit 13 estimates the target user's comprehension level by inputting the target data into the learned model.

[0099] In step S35, the assistance unit 14 retrieves auxiliary information corresponding to the target user's comprehension level from the database 30 and transmits the auxiliary information to the user terminal 20B. The display control unit 25 of the user terminal 20B displays the auxiliary information on the screen of the user terminal 20B. The timing of outputting the auxiliary information is not limited. For example, the display control unit 25 may output the auxiliary information after a predetermined time (e.g., 15 seconds) has elapsed since the target content was displayed on the screen of the user terminal 20. Alternatively, the display control unit 25 may output the auxiliary information in response to a user request. The display control unit 25 may adjust the display time of the auxiliary information based on the user's comprehension level. Alternatively, the display control unit 25 may display the auxiliary information only for a display time pre-set by the user. Alternatively, the assistance unit 14 may display the auxiliary information until the display of the target content is switched, or until the user inputs information about the target content (e.g., answers to questions). If the estimated comprehension level indicates that the target user has sufficient understanding of the target content, the assistance unit 14 may terminate the process without outputting the auxiliary information. The method of outputting the auxiliary information is not limited. When the auxiliary information includes audio data, the user terminal 20 may output the audio data from a speaker.

[0100] As shown in step S36, the support system 1 repeatedly performs the processing from step S32 to step S35 while the target content is displayed on the user terminal 20B. As an example, the support system 1 repeatedly performs this series of processing while the target content is displayed.

[0101] Figure 9This flowchart shows an example of the operation of support system 1 as process flow S4. Process flow S4 also involves providing support information to a target user viewing target content, but the specific steps differ from process flow S3. Process flow S4 also assumes that the target user has logged into support system 1 and that the eye tracking system has already calculated the difference.

[0102] In step S41, the display control unit 25 of the user terminal 20B displays the target content on the screen of the user terminal 20B. In step S42, the tracking unit 24 of the user terminal 20B obtains the viewpoint coordinates (second viewpoint coordinates) of the target user who is visually viewing the target content. In step S43, the estimation unit 13 obtains target data indicating the movement of the target user's viewpoint. This series of processing is the same as steps S31 to S33.

[0103] In step S44, the estimation unit 13 obtains the generalized correspondence data by referring to the database 30, and estimates the target user's understanding level (first understanding level) of the target content based on the target data and the generalized correspondence data. The specific estimation method is the same as that in step S34.

[0104] In step S45, the assisting unit 14 acquires assisting information corresponding to the target user's first comprehension level from the database 30 and transmits the assisting information to the user terminal 20B. The display control unit 25 of the user terminal 20B outputs the assisting information to the screen of the user terminal 20B.

[0105] In step S46, the assistance unit 14 determines whether to provide additional assistance to the target user, that is, whether to provide additional assistance information to the target user. If the assistance unit 14 determines not to provide additional assistance, the process moves to step S49. If the assistance unit 14 determines to provide additional assistance, the process moves to step S47. As an example, if user input regarding the target content (e.g., an answer to a question) is received within a specified time, the assistance unit 14 may determine not to provide additional assistance, and if no user input is received within the specified time, the assistance unit 14 may determine to provide additional assistance.

[0106] In step S47, the estimation unit 13 retrieves the target content-specific correspondence data from the database 30 and estimates the target user's comprehension level (second comprehension level) of the target content based on the target data and the content-specific correspondence data. This process assumes that the same content is used as the sample content and the target content. The specific estimation method is the same as that of step S34.

[0107] In step S48, the assisting unit 14 acquires additional assistive information corresponding to the target user's second level of understanding from the database 30 and transmits the assistive information to the user terminal 20B. The display control unit 25 of the user terminal 20B outputs the additional assistive information to the screen of the user terminal 20B.

[0108] As shown in step S49, the support system 1 repeatedly performs the processing from step S42 to step S48 while the target content is being displayed on the user terminal 20B. As an example, the support system 1 repeatedly performs this series of processing while the target content is being displayed.

[0109] Figure 10 This figure shows an example of auxiliary information. In this example, the target content Q11 is part of an English question, and the target user is a Japanese student. In this example, the auxiliary system 1 refers to correspondence data containing information related to comprehension level Ra indicating "lack of vocabulary," comprehension level Rb indicating "lack of grammatical proficiency," and comprehension level Rc indicating "lack of understanding of the context of the text." For example, if the estimation unit 13 estimates, based on the target data and its correspondence data, that the target user's vocabulary level is insufficient, the auxiliary unit 14 outputs auxiliary information B11 corresponding to that comprehension level. If the estimation unit 13 estimates that the target user's grammatical proficiency is insufficient, the auxiliary unit 14 outputs auxiliary information B12 corresponding to that comprehension level. If the estimation unit 13 estimates that the target user does not understand the context of the text, the auxiliary unit 14 outputs auxiliary information B13 corresponding to that comprehension level. The display control unit 25 of the user terminal 20B displays the output auxiliary information. The target user can refer to this auxiliary information to solve the problem.

[0110] [Effect]

[0111] As described above, an auxiliary system according to one aspect of the present disclosure includes at least one processor. The at least one processor performs the following processing: acquiring target data representing the movement of a target user's viewpoint on a screen displaying target content; referring to a storage unit that stores correspondence data and auxiliary information, wherein the correspondence data is data obtained by statistically processing a plurality of sample data and represents the correspondence between the movement of the user's viewpoint and the user's understanding of the content, wherein the plurality of sample data is data obtained from a plurality of sample users who have visually confirmed the sample content, each sample data representing a correspondence between the movement of the viewpoint of a sample user who has visually confirmed the sample content and the sample user's understanding of the sample content, and the auxiliary information is information corresponding to the user's understanding of the content; estimating the target user's understanding of the target content based on the target data and the correspondence data; and outputting auxiliary information corresponding to the estimated target user's understanding.

[0112] An assistance method according to one aspect of the present disclosure is executed by an assistance system having at least one processor. The assistance method includes the following steps: acquiring target data representing the movement of a target user's viewpoint on a screen displaying target content; referring to a storage unit, wherein the storage unit stores correspondence data and auxiliary information, wherein the correspondence data is data obtained by statistically processing a plurality of sample data and represents the correspondence between the movement of the user's viewpoint and the user's understanding of the content, wherein the plurality of sample data is data obtained from a plurality of sample users who have visually confirmed the sample content, each sample data represents a correspondence between the movement of the viewpoint of the sample user who has visually confirmed the sample content and the sample user's understanding of the sample content, and the auxiliary information is information corresponding to the user's understanding of the content; estimating the target user's understanding of the target content based on the target data and the correspondence data; and outputting auxiliary information corresponding to the estimated target user's understanding.

[0113] According to one aspect of the present disclosure, an auxiliary program enables a computer to perform the following steps: obtaining target data representing the movement of a target user's viewpoint on a screen displaying target content; referring to a storage unit, the storage unit stores correspondence data and auxiliary information, the correspondence data being data obtained by statistically processing a plurality of sample data, and representing the correspondence between the movement of the user's viewpoint and the user's understanding of the content, wherein the plurality of sample data are data obtained from a plurality of sample users who have visually confirmed the sample content, each sample data representing a correspondence between the movement of the viewpoint of the sample user who has visually confirmed the sample content and the sample user's understanding of the sample content, and the auxiliary information being information corresponding to the user's understanding of the content; estimating the target user's understanding of the target content based on the target data and the correspondence data; and outputting auxiliary information corresponding to the estimated target user's understanding.

[0114] In such an aspect, correspondence data is generated by performing statistical processing on sample data obtained from sample users, and the target user's understanding is estimated based on the correspondence data and target data representing the movement of the target user's viewpoint relative to the target content. By using the correspondence data obtained through statistical processing, the target user's understanding is estimated based on the actual tendency of the user who visually confirms the content. By outputting auxiliary information based on this estimate, the target user who visually confirms the target content can be appropriately assisted. Since the correspondence between the movement of the user's viewpoint and the user's understanding is derived through statistical processing, there is no need to pre-set an assumption for the correspondence. In addition, the correspondence can be obtained with high precision through statistical processing (it is very difficult to set an assumption with high precision). Therefore, the target user can be appropriately assisted according to the actual situation.

[0115] In the assistance system according to another aspect, the statistical processing may include clustering the plurality of sample data based on the movement of the sample user's viewpoint and the sample user's understanding. In this case, the corresponding relationship between the movement of the user's viewpoint and the user's understanding can be appropriately derived through clustering.

[0116] In other aspects of the support system, the statistical processing may include performing regression analysis on a plurality of sample data. In this case, the corresponding relationship between the movement of the user's viewpoint and the user's understanding can be appropriately derived through the regression analysis.

[0117] In another aspect of the assistance system, at least one processor may output assistance information after a predetermined time has passed since the target content was displayed on the screen. In this case, the target user can be given more time to consider the target content without using the assistance information, and, for example, the target user's freedom of learning using the target content can be increased.

[0118] In another aspect of the assistance system, the correspondence data may also include: generalized correspondence data obtained by statistically processing a plurality of first sample data, the plurality of first sample data including sample data obtained from sample users who visually viewed sample content different from the target content; and content-specific correspondence data obtained by statistically processing a plurality of second sample data, the plurality of second sample data obtained from a plurality of sample users who visually viewed the target content as the sample content. At least one processor may estimate a first comprehension level of the target user for the target content based on the target data and the generalized correspondence data, output auxiliary information corresponding to the estimated first comprehension level of the target user, and estimate a second comprehension level of the target content based on the target data and the content-specific correspondence data, and output auxiliary information corresponding to the estimated second comprehension level of the target user. In this case, the user can be effectively assisted by two types of auxiliary information: auxiliary information based on the generalized correspondence data (general auxiliary information not limited to the target content) and auxiliary information based on the content-specific correspondence data (auxiliary information specific to the target content).

[0119] In other aspects of the assistance system, at least one processor may output auxiliary information corresponding to a second level of comprehension for the target user after outputting auxiliary information corresponding to a first level of comprehension for the target user. In this case, for a target user whose understanding of the target content is insufficient using only auxiliary information based on generalized correspondence data, auxiliary information based on content-specific correspondence data (i.e., more specific auxiliary information) can be used to effectively assist the user.

[0120] [Modification]

[0121] The above is a detailed description based on the embodiments of the present disclosure. However, the present disclosure is not limited to the above embodiments. Various modifications can be made to the present disclosure without departing from the scope of the present disclosure.

[0122] In the above embodiment, the auxiliary system 1 is constructed using the server 10, but the auxiliary system 1 can also be constructed without using the server 10. In this case, the various functional elements of the server 10 can be installed in any one of the user terminals 20, for example, they can be installed in any one of the terminals used by the publisher of the content and the terminals used by the viewers of the content. Alternatively, the various functional elements of the server 10 can be installed separately in multiple user terminals 20, for example, they can be installed separately in the terminal used by the publisher and the terminal used by the viewers. In connection with this, the auxiliary program can also be implemented as a client program. By having the functions of the server 10 in the user terminal 20, the load on the server 10 can be reduced. In addition, information related to the viewers of the content, such as students (for example, data indicating the movement of the viewpoint) is not sent to the outside of the user terminal 20, so that the privacy of the viewers can be protected more reliably.

[0123] In the above embodiment, the eye tracking system is composed only of the user terminal 20, but the system can also be constructed using the server 10. In this case, some functional elements of the user terminal 20 can also be installed in the server 10. For example, the functional element corresponding to the calculation unit 23 can also be installed in the server 10.

[0124] In the above embodiment, auxiliary information is displayed separately from the target content. However, the auxiliary information can also be displayed as part of the target content. For example, if the target content includes an article, the auxiliary unit 14 can also highlight a portion of the article (e.g., a portion important for understanding the article) as auxiliary information. In other words, the auxiliary information can also be a visual effect added to the target content. In this case, the auxiliary unit 14 can also highlight the portion of the article that is the subject of the auxiliary information by making it appear in a different color or font than other portions.

[0125] In the above embodiment, the support system 1 outputs support information corresponding to the target user's comprehension level. However, the support system 1 may output support information without using the comprehension level. This modification will be described below.

[0126] The server 10 obtains viewpoint data representing the movement of the viewpoint of a sample user who has visually confirmed the sample content and sample data representing the auxiliary information presented to the sample user from each user terminal 20A, and stores the sample data in the database 30. In one example, the auxiliary information presented to the sample user (i.e., the auxiliary information corresponding to the sample user) is determined by manual experiments or surveys, questionnaires for the sample users, etc., and input into the user terminal 20A. The statistical processing unit 12 performs statistical processing on the sample data in the database 30, generates correspondence data representing the correspondence between the movement of the user's viewpoint and the auxiliary information of the content, and stores the correspondence data in the database 30. As in the above embodiment, the statistical processing method and the expression form of the generated correspondence data are not limited. Therefore, the statistical processing unit 12 can generate the correspondence data by various methods such as clustering, regression analysis, and machine learning.

[0127] The server 10 outputs auxiliary information corresponding to the target data based on the target data and its corresponding relationship data received from the user terminal 20B. In one example, the estimation unit 13 obtains the corresponding relationship data with reference to the database 30 and determines the auxiliary information corresponding to the target data. In the case where the corresponding relationship data is generated by clustering, the estimation unit 13 determines the auxiliary information represented by the class to which the target data belongs. As another example, in the case where the corresponding relationship data is generated by regression analysis, the estimation unit 13 applies the target data to the regression equation as auxiliary information. As yet another example, in the case where the corresponding relationship data is a learned model, the estimation unit 13 determines the auxiliary information by inputting the target data into the learned model. The auxiliary unit 14 obtains the determined auxiliary information from the database 30 and sends the auxiliary information to the user terminal 20B.

[0128] That is, an auxiliary system according to one aspect of the present disclosure includes at least one processor. The at least one processor performs the following processing: acquiring target data representing the movement of a target user's viewpoint on a screen displaying target content; referencing a storage unit storing correspondence data obtained by statistically processing a plurality of sample data and representing the correspondence between the movement of the user's viewpoint and auxiliary information of the content, wherein the plurality of sample data is obtained from a plurality of sample users who visually viewed the sample content, each sample data representing the movement of the viewpoint of the sample user who visually viewed the sample content and the auxiliary information corresponding to the sample user; and outputting auxiliary information corresponding to the target data based on the target data and the correspondence data.

[0129] In this aspect, statistical processing is performed on sample data obtained from sample users to generate correspondence data. Auxiliary information is then output based on this correspondence data and target data representing the movement of the target user's viewpoint relative to the target content. By using the statistically processed correspondence data, auxiliary information can be output based on the actual tendencies of the user visually confirming the content. This allows for appropriate assistance to the target user in visually confirming the target content.

[0130] In this disclosure, the expression "at least one processor executes a first process, executes a second process, ... executes an nth process" or a corresponding expression encompasses the case where the execution subject (i.e., the processor) of n processes from the first process to the nth process changes midway. In other words, this expression encompasses both the case where all n processes are executed by the same processor and the case where the processor changes arbitrarily within the n processes.

[0131] The processing order of the method executed by at least one processor is not limited to the examples in the above embodiment. For example, part of the above steps (processing) may be omitted, or the steps may be performed in another order. In addition, any two or more of the above steps may be combined, or part of the steps may be modified or deleted. Alternatively, other steps may be performed based on the above steps.

[0132] Explanation of symbols

[0133] 1... auxiliary system, 10... server, 11... content distribution unit, 12... statistical processing unit, 13... estimation unit, 14... auxiliary unit, 20... 20A, 20B... user terminal, 21... setting unit, 22... determination unit, 23... calculation unit, 24... tracking unit, 25... display control unit, 30... database, 100... server computer, 101... processor, 102... main storage unit, 103... auxiliary storage unit, 104... communication unit, 200... terminal computer, 201…processor, 202…main storage unit, 203…auxiliary storage unit, 204…communication unit, 205…input interface, 206…output interface, 207…camera unit, A11, A12, A13, A21, A22, A23, A24, A25…boot area, C11, C21…first content, D11, D12, D13, D21, D22, D23…screen, N…communication network, P1…server program, P2…client program

Claims

1. An auxiliary system comprising at least one processor, wherein: The at least one processor performs the following processing: acquiring, from a user terminal of a target user, target data representing movement of a viewpoint of the target user on a screen of the user terminal displaying target content; Reference is made to a storage unit, the storage unit storing correspondence data and auxiliary information, the correspondence data being data obtained by statistically processing a plurality of sample data and representing a correspondence between a user's viewpoint movement and the user's understanding of content, the plurality of sample data being data obtained from a plurality of sample users who visually confirmed sample content, each sample data representing viewpoint data and the sample user's understanding of the sample content, the viewpoint data being data representing viewpoint movement of the sample user who visually confirmed the sample content, and the auxiliary information being information corresponding to the user's understanding of the content; estimating the target user's understanding of the target content based on the target data and the corresponding relationship data; outputting the auxiliary information corresponding to the estimated understanding degree of the target user to the user terminal, the auxiliary information being information for promoting the target user's understanding of the target content; The corresponding relationship data includes: generalized correspondence relationship data obtained by performing the statistical processing on a plurality of first sample data, the plurality of first sample data including the sample data obtained from the sample user who visually confirmed a sample content different from the target content; as well as content-specific correspondence data obtained by performing the statistical processing on a plurality of second sample data obtained from a plurality of sample users who visually confirmed the target content as sample content, The at least one processor performs the following processing: estimating a first understanding degree of the target user to the target content based on the target data and the generalized correspondence relationship data; outputting auxiliary information corresponding to the estimated first understanding level of the target user; estimating a second degree of understanding of the target content by the target user based on the target data and the content-specific correspondence data; Auxiliary information corresponding to the estimated second understanding level of the target user is output.

2. The assistance system according to claim 1, wherein: The statistical processing includes a process of clustering the plurality of sample data based on the movement of the viewpoint of the sample user and the degree of understanding of the sample user.

3. The assistance system according to claim 1, wherein: The statistical processing includes performing regression analysis on the plurality of sample data.

4. The assistance system according to any one of claims 1 to 3, wherein: The at least one processor outputs the auxiliary information acquired from the storage unit to the user terminal after a predetermined time has passed since the target content was displayed on the screen and until a user inputs the target content.

5. The assistance system according to claim 1, wherein: The at least one processor outputs auxiliary information corresponding to the second understanding level of the target user after outputting the auxiliary information corresponding to the first understanding level of the target user.

6. The assistance system according to claim 5, wherein: The statistical processing includes a process of clustering the plurality of sample data based on the movement of the viewpoint of the sample user and the degree of understanding of the sample user.

7. The assistance system according to claim 5, wherein: The statistical processing includes performing regression analysis on the plurality of sample data.

8. An assistance method, which is an assistance method performed by an assistance system having at least one processor, in, The following steps are involved: acquiring, from a user terminal of a target user, target data representing movement of a viewpoint of the target user on a screen of the user terminal displaying target content; Referring to a storage unit, the storage unit stores correspondence data and auxiliary information, the correspondence data being data obtained by statistically processing a plurality of sample data and representing a correspondence between a user's viewpoint movement and the user's understanding of content, wherein the plurality of sample data is data obtained from a plurality of sample users who visually confirmed sample content, each sample data representing viewpoint data and the sample user's understanding of the sample content, the viewpoint data being data representing viewpoint movement of the sample user who visually confirmed the sample content, and the auxiliary information being information corresponding to the user's understanding of the content; estimating the target user's understanding of the target content based on the target data and the corresponding relationship data; outputting the auxiliary information corresponding to the estimated understanding degree of the target user to the user terminal, the auxiliary information being information for promoting the target user's understanding of the target content; The corresponding relationship data includes: generalized correspondence relationship data obtained by performing the statistical processing on a plurality of first sample data, the plurality of first sample data including the sample data obtained from the sample user who visually confirmed sample content different from the target content; and content-specific correspondence data obtained by performing the statistical processing on a plurality of second sample data obtained from a plurality of sample users who visually confirmed the target content as sample content, estimating a first understanding degree of the target user to the target content based on the target data and the generalized correspondence relationship data; outputting auxiliary information corresponding to the estimated first understanding level of the target user; estimating a second degree of understanding of the target content by the target user based on the target data and the content-specific correspondence data; Auxiliary information corresponding to the estimated second understanding level of the target user is output.

9. A computer program product comprising an auxiliary program, wherein: The auxiliary program enables the computer to execute the following steps: acquiring, from a user terminal of a target user, target data representing movement of a viewpoint of the target user on a screen of the user terminal displaying target content; Referring to a storage unit, the storage unit stores correspondence data and auxiliary information, the correspondence data being data obtained by statistically processing a plurality of sample data and representing a correspondence between a user's viewpoint movement and the user's understanding of content, wherein the plurality of sample data is data obtained from a plurality of sample users who visually confirmed sample content, each sample data representing viewpoint data and the sample user's understanding of the sample content, the viewpoint data being data representing viewpoint movement of the sample user who visually confirmed the sample content, and the auxiliary information being information corresponding to the user's understanding of the content; estimating the target user's understanding of the target content based on the target data and the corresponding relationship data; outputting the auxiliary information corresponding to the estimated understanding degree of the target user to the user terminal, the auxiliary information being information for promoting the target user's understanding of the target content; The corresponding relationship data includes: generalized correspondence relationship data obtained by performing the statistical processing on a plurality of first sample data, the plurality of first sample data including the sample data obtained from the sample user who visually confirmed sample content different from the target content; and content-specific correspondence data obtained by performing the statistical processing on a plurality of second sample data obtained from a plurality of sample users who visually confirmed the target content as sample content, The auxiliary program enables the computer to execute the following steps: estimating a first understanding degree of the target user to the target content based on the target data and the generalized correspondence relationship data; outputting auxiliary information corresponding to the estimated first understanding level of the target user; estimating a second degree of understanding of the target content by the target user based on the target data and the content-specific correspondence data; Auxiliary information corresponding to the estimated second understanding level of the target user is output.

10. An assistance system comprising at least one processor, wherein: The at least one processor performs the following processing: acquiring, from a user terminal of a target user, target data representing movement of a viewpoint of the target user on a screen of the user terminal displaying target content; referring to a storage unit, the storage unit storing correspondence data, the correspondence data being data obtained by statistically processing a plurality of sample data and indicating a correspondence between a user's viewpoint movement and auxiliary information of content, wherein the plurality of sample data are data obtained from a plurality of sample users who visually confirmed sample content, each sample data indicating viewpoint data and auxiliary information corresponding to the sample user, the viewpoint data being data indicating a viewpoint movement of the sample user who visually confirmed the sample content; Based on the target data and the corresponding relationship data, the auxiliary information corresponding to the target data is output to the user terminal, wherein the auxiliary information is information for promoting the target user's understanding of the target content. The corresponding relationship data includes: generalized correspondence relationship data obtained by performing the statistical processing on a plurality of first sample data, the plurality of first sample data including the sample data obtained from the sample user who visually confirmed sample content different from the target content; and content-specific correspondence data obtained by performing the statistical processing on a plurality of second sample data obtained from a plurality of sample users who visually confirmed the target content as sample content, The at least one processor performs the following processing: estimating a first understanding degree of the target user to the target content based on the target data and the generalized correspondence relationship data; outputting auxiliary information corresponding to the estimated first understanding level of the target user; estimating a second degree of understanding of the target content by the target user based on the target data and the content-specific correspondence data; Auxiliary information corresponding to the estimated second understanding level of the target user is output.

Images