Information display terminal, information transmission method, and recording medium

Abstract

If the information about what URL page and what position the reader who is browsing the web is currently browsing is collected, it is very useful for the advertiser of the banner advertisement or the poster of the homepage, but since such browsing information is personal information, it cannot be easily collected. The following information display terminal, information transmission method, and computer program are configured: a storage unit is set to prompt the content of payment corresponding to the collection of browsing information and to store the promised electronic message as evidence when the reader agrees, and then collect the browsing information. If the reader's attribute information (age, gender, occupation, industry, family composition, etc.) is collected under the promise and stored in the registrant information storage unit when the promise is obtained, the additional value of the browsing information is increased.

Description

Technical Field

[0001] The present disclosure relates to an information display terminal, an information transmission method, and a computer program suitable for a system and the like for collecting WEB browsing information of users. Background Art

[0002] As prior art, the following structures have been proposed: based on images captured around the eyes of a user who is viewing an information terminal such as a smartphone (PDA: personal digital assistant) or a personal computer, the direction in which the user's eyes are currently directed is detected and matched with information on content currently displayed on the information terminal, thereby obtaining information on what the current user is specifically looking at in the display content of the information terminal.

[0003] For example, Patent Document 1 discloses a structure that uses the line of sight information of viewers who visually recognize the display screen of a broadcast service for mobile devices such as mobile phones and car navigation systems to measure viewing quality such as "how the program is viewed" which is unknown using only existing viewing rates.

[0004] In addition, the following patent document 2 discloses a system that determines the temporal focus position of content (dynamic image) of a first user, outputs focus time information indicating the determined temporal focus position in correspondence with the content information and the focus position information, and when a display unit visually recognized by a second user reproduces and displays the content (dynamic image), the display unit is enabled to identifiably display the temporal focus position based on the focus time information output together with the content information and the spatial focus position based on the focus position information through a common indicator, thereby enabling the second user to efficiently grasp the temporal focus position and spatial focus position of the first user within the content.

[0005] Furthermore, regarding gaze detection technology, which is a basic technology for obtaining gaze information of these users, the following non-patent document 1 discloses an improvement in the calibration technology for each person when photographing the eyeballs and calculating the gaze direction. Similarly, the following patent document 3 discloses a technology for using an image template under a genetic algorithm to improve the correct detection rate when photographing the eyeballs and wanting to detect the gaze direction.

[0006] In addition, the following non-patent document 2 shows that: first, more than 2.5 megaframes of eye tracking image data (called "GazeCapture") are collected from more than 1,450 subjects, and the "iTracker" as a deep neural network for eye tracking is learned using the GazeCapture, so that the accuracy of eye tracking based on mobile information terminals such as smartphones that are not equipped with special sensors can be reduced to less than 1.71 cm.

[0007] Prior art literature

[0008] Patent Literature

[0009] Patent Document 1: Japanese Patent Application Publication No. 2009-5094

[0010] Patent Document 2: Japanese Patent Application Publication No. 2017-228197

[0011] Patent Document 3: Japanese Patent Application Publication No. 2013-45151

[0012] Non-patent literature

[0013] Non-patent document 1: Takehiko Ohno, Naoki Takekawa, Atsushi Yoshikawa; "Measurement of Eyeball Shape Eyeball Shape Eye Sight Measurement" (Eighth Image Gallery, pp307-31 2, 2002) (Ohno Takehiko, Takekawa Naoki, Yoshikawa Atsushi; "Gaze measurement method based on eyeball shape model" (8th Image Sensing Symposium, pp307-312, 2002))

[0014] Non-patent literature 2: Kyle Krafka, et al.; "Eye Tracking for Everyone", CVPR 2016, http: / / gazecapture.csail.mit.edu / Summary of the invention

[0015] Problems to be solved by the invention

[0016] As the Internet becomes more popular, various businesses are conducted via the Web, and sales of goods and services are conducted via the Web, the importance of Web page browsing records as basic information for understanding the interests and preferences of viewers, i.e., purchasers, is expected to increase in the future. Therefore, it is expected that companies that use the Web for business in some form, or individuals who conduct personal transactions via the Web, will increasingly pursue more detailed Web browsing information with higher added value.

[0017] Furthermore, if it were possible to input this WEB browsing information on a multi-browser basis, it would be expected to be utilized at a higher level using data mining.

[0018] However, since detailed browsing information such as which page a person is currently viewing on a web page is personal information, it is difficult to collect and use it without the consent of the viewer. On the other hand, as described above, it is important to obtain information about the number of people, so it is cumbersome and unrealistic for the information collector to obtain permission from the viewer (user) every time the viewer browses the web page, which is personal information.

[0019] Furthermore, from the perspective of the readers (users) who provide information, providing their reading information to a third party is actually a loss, and since no benefit can be seen, it is difficult to collect information from this perspective.

[0020] Furthermore, if the attributes of the viewers (gender, age, occupation, etc.) are included in the web browsing information, the possibility of utilization by companies and the like that want to utilize the information commercially will be expanded.

[0021] The present invention has been completed in view of the above-mentioned situation, and its purpose is to provide the following information display terminal, information sending method, and computer program: regardless of whether it is personal information or not, it is possible to smoothly and massively collect information related to where specifically the viewers of the display objects such as WEB reading devices that display published information whose display content changes over time (eye tracking information) are looking at in the displayed content, and it is also possible to collect information related to the attributes of the readers.

[0022] Means for solving problems

[0023] In order to solve the above-mentioned problems, the present invention provides an information display terminal, an information transmission method, and a computer program described in the following 1) to 10).

[0024] 1) The information display terminal is characterized by having:

[0025] (A) Display screen;

[0026] (B) a viewpoint detection unit that detects a viewpoint position of an operator operating the information display terminal on the display screen;

[0027] (C) an output unit that outputs information that specifies display content of the display screen and the detection result of the viewpoint position to a predetermined transmission destination in association with each other,

[0028] The information display terminal also includes an inquiry signal receiving and responding unit, which receives an inquiry signal and sends a reply to the inquiry signal according to an operation of an operator, wherein the inquiry signal requests the operator to grant permission for the information display terminal to execute the function of the output unit described in (C) on the condition that the operator of the information display terminal is paid.

[0029] 2) The information display terminal according to 1), further comprising a second transmitting unit configured to transmit information on attributes of an operator who has permitted processing of the inquiry signal.

[0030] 3) The information transmission method is characterized by comprising:

[0031] (A) a step of displaying content on a display screen of an information display terminal;

[0032] (B) a step of detecting, by a viewpoint detection unit of the information display terminal, a viewpoint position of an operator operating the information display terminal on the display screen;

[0033] (C) the output unit of the information display terminal outputs information that specifies display content of the display screen and the detection result of the viewpoint position to a predetermined transmission destination in association with each other,

[0034] The information sending method also includes: an inquiry signal receiving and response unit of the information display terminal receives an inquiry signal and sends a reply to the inquiry signal according to an operation of an operator, wherein the inquiry signal requests the operator to grant permission for the information display terminal to execute the function of the output unit described in (C) on the condition that the operator of the information display terminal is paid.

[0035] 4) The information transmission method as described in 3), wherein the information transmission method further comprises a step in which the second transmission unit of the information display terminal transmits information related to the attribute of the operator who has permitted the processing of the inquiry signal.

[0036] 5) The computer program is used to execute each step of the information transmission method described in 3) or 4).

[0037] 6) The information display terminal is characterized by having:

[0038] (A) Display screen;

[0039] (B) a viewpoint detection unit that detects a viewpoint position of an operator operating the information display terminal on the display screen;

[0040] (C) an output unit that outputs information that specifies display content of the display screen and the detection result of the viewpoint position to a predetermined transmission destination in association with each other,

[0041] The information display terminal further includes a permitted operation processing unit that records and stores the fact that the operator has permitted the output to the predetermined destination to be performed by the output unit in advance, or transmits it to the outside of the terminal.

[0042] 7) The information display terminal according to 6), wherein the output to the predetermined transmission destination by the output unit is performed including information related to the attribute of the operator.

[0043] 8) The information transmission method is characterized by comprising:

[0044] (A) a step of displaying content on a display screen of an information display terminal;

[0045] (B) a step of detecting, by a viewpoint detection unit of the information display terminal, a viewpoint position of an operator operating the information display terminal on the display screen; and

[0046] (C) the output unit of the information display terminal outputs information that specifies display content of the display screen and the detection result of the viewpoint position to a predetermined transmission destination in association with each other,

[0047] The information transmission method further includes a step in which the permitted action processing unit of the information display terminal records and stores the fact that the operator has permitted the output unit to execute the output to the specified transmission destination in advance, or transmits it to the outside of the terminal.

[0048] 9) The information transmission method according to 8), wherein the output to the predetermined transmission destination by the output unit is performed including information related to the attribute of the operator.

[0049] 10) The computer program is used to execute each step of the information sending method described in 8) or 9).

[0050] Effects of the Invention

[0051] By having the above-mentioned structure, the present invention can particularly provide the following information display terminal, information sending method, and computer program: regardless of whether it is personal information or not, it is possible to smoothly and massively collect information related to where specifically a viewer of a display object such as a WEB reading device looks at when viewing published information whose display content changes over time, and to collect information related to the attributes of the viewer. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a diagram schematically showing a display screen of a smartphone as a mode in which the browsing viewpoint information collecting system according to the present invention can be appropriately utilized.

[0053] Figure 2 This is a diagram schematically showing a display screen of a smartphone as a mode in which the browsing viewpoint information collecting system according to the present invention can be appropriately utilized.

[0054] Figure 3 This is a diagram showing the configuration of a browsing viewpoint information collection system according to one embodiment of the present invention.

[0055] Figure 4 yes Figure 3 A structural diagram of an information collection server included in the viewing viewpoint information collection system.

[0056] Figure 5 yes Figure 3An example of a record format used by the viewing viewpoint information collection system.

[0057] Figure 6 yes Figure 3 A flowchart of information provider registration performed by the viewing viewpoint information collection system.

[0058] Figure 7 yes Figure 3 A flowchart of information provider registration performed by the viewing viewpoint information collection system.

[0059] Figure 8 yes Figure 3 A flowchart of sending viewing point information executed by the viewing point information collection system.

[0060] Fig. 9 It is a schematic diagram of a virtual three-dimensional space for explaining another embodiment of the present invention.

[0061] Fig.10 Observe from various viewpoints Fig. 9 Schematic diagram of an image of a virtual three-dimensional space. DETAILED DESCRIPTION

[0062] [One of the aspects in which each embodiment of the present invention can be appropriately utilized]

[0063] Before describing each embodiment of the present invention, several aspects in which each embodiment can be appropriately utilized will be described.

[0064] Figure 1 This is a diagram schematically showing a display screen of a smartphone as a mode in which the browsing viewpoint information collecting system according to the present invention can be appropriately utilized.

[0065] Figure 1 This is a diagram schematically showing a situation where a personal diary page is displayed on a smartphone.

[0066] The smartphone 1 includes a display panel 1a, a front camera 1b capable of capturing the field of view of a user (web browser) viewing the display panel 1a, a speaker 1c, and a main button 1d.

[0067] First, the front camera 1b is used for capturing video conversations (video conferencing, video calls) or selfie images and is installed in most smartphones or mobile phones, in contrast to the rear camera (out camera) not shown (a camera for capturing scenery facing the opposite direction from the operator).

[0068] In the display panel 1a, the home page 1-1 of the blog "Observations from Oku-Nikko" created by the blog contributor 1-3 is currently displayed. In the home page 1-1, a hotel 1-2 for lodging, the blog contributor 1-3, a passenger car 1-4, a first banner advertisement 1-5 containing the record "A quiet lodging hotel ○○", and a second banner advertisement 1-6 containing the record "An off-road SUV is □□" are displayed as images.

[0069] A viewer who has accessed the home page 1-1 can view other pages arranged in the lower layer of the home page 1-1, such as a page for posting photos or a page for recording diaries, by clicking the displayed menu button 1-7 and selecting them.

[0070] As described later, by using the embodiments of the present invention, it is possible to know which position of the display panel 1a the viewer's eye 100 is currently viewing and the movement trajectory 100-2 of the viewpoint on the display panel 1a in front of the line of sight 100-1.

[0071] In the prior art, although the blog contributor 1-3 can identify who has visited the homepage 1-1 by reading, it is not known which part of the homepage 1-1 the reader has focused on. However, in the embodiments of the present invention, since the moving track 100-2 of the viewpoint can be known, it is possible to know which part of the homepage 1-1 designed by oneself attracts the attention of others, and what is being focused on, so as to be rewarded for the labor of creating the blog. At the same time, if a part that one expects to have a high reading rate is unexpectedly not focused on, the part can be moved to the homepage, or the evaluation of the part can be recorded in the blog, so as to guide "please look here".

[0072] In addition, the motives for each reader who reads and visits the homepage 1-1 of the blog are also varied. A certain reader visits the blog because he or she has a personal acquaintance with the blog contributor 1-3. Although other readers do not know the blog, they may have knowledge and interest in a travel destination such as "Oku-Nikko". Such readers also notice the photos of the scenery posted on the homepage 1-1 and the photos of the hotel 1-2 where they stay, and may stare at them for a long time. Alternatively, although other readers do not know the blog contributor 1-3 and pay little attention to places such as "Oku-Nikko", they are interested in the passenger car 1-4 used by the blog contributor 1-3 during the trip, and may also stare at the photos for a long time. Regarding the passenger car 1-4, they may not fully read the recorded position of the diary written by the blog contributor. By obtaining the sightline trajectory 100-2 of each reader through the blog contributor 1-3, it can be used as a reference for the blog creation from next time, so that it can be improved in a way with a higher number of readings.

[0073] On the other hand, the blog publisher 1-3 makes an agreement (contract) with an Internet advertiser or an Internet provider to post two banner advertisements 1-5 and 1-6 on his homepage 1-1. In the advertisement posting contract, a pay-per-view method is often adopted in which the advertiser pays the blog contributor 1-3 a posting fee according to the number of visits to the homepage 1-1 by a third party reader.

[0074] From the standpoint of advertisers, in the prior art, it is not clear how much attention the company's advertisement can receive from the readers, but according to the embodiments of the present invention, the sight track 100-2 of the readers can be obtained first, so it is clear whether the company's banner advertisement actually receives attention, and as a result, the effect and advantage of posting the company's banner advertisement on the homepage 1-1 can be known. Moreover, as described later, according to each embodiment of the present invention, the sight track 100-2 with the attributes of the readers can be obtained, so for example, for a manufacturer of passenger cars, although the company's banner advertisement "off-road SUV is □□" is indeed paid attention to by a number of readers, when observing the attributes of these readers, it may not be said that they are the top purchasers of passenger cars, so it can be used as a material for judging whether to continue to post the company's banner advertisement on the homepage 1-1.

[0075] [Second aspect in which each embodiment of the present invention can be appropriately utilized]

[0076] Figure 2 This is a diagram schematically showing a display screen of a smartphone as a mode in which the browsing viewpoint information collecting system according to the present invention can be appropriately utilized.

[0077] Figure 2 and Figure 1 Similarly, the smartphone 1 displays the status of the web page, and schematically indicates that it is not Figure 1 Instead, such a personal blog shows the status of the portal homepage 2-1 of a company such as "△△ Electronics Co., Ltd.".

[0078] The home page 2-1 includes several display boxes, including a "CEO message" box 2-2, a "product line up" box 2-3, a "latest news" box 2-4, an inquiry information box 2-5 for the company, and a home page search box 2-6.

[0079] It is preferred that the company that posted the homepage 2-1 has as many readers as possible read it. For example, if most readers first pay attention to the CEO's message box 2-2, it can be determined that the CEO has carried out various activities in the industry (lectures, enlightenment, publications, etc.), which is related to the reading, so that these activities can be further strengthened. If there is a lot of attention to the latest news box, it can be known what kind of news will attract attention as long as it is sent, or the content of past news can be analyzed.

[0080] Furthermore, if the viewer who has visited and browsed the home page 2-1 focuses his / her eyes on the product line upper frame 2-3, and fails to press the "product introduction" button in the title frame 2-1 above to browse the lower page, and finally searches for a component name such as "thyristor" in the in-page search frame 2-6, the viewer will find the thyristor of the electronic component manufacturer, and fail to achieve the purpose of listing the thyristor on the product line upper frame 2-3 of the home page 2-1, and even the lower product introduction page cannot be found. Furthermore, if the viewer's attributes are observed, it can be seen that in the case of "actively engaged in products for home appliance manufacturers", there is a significant opportunity loss, and it is clear that the listing method of the product line on the home page 2-1 should be improved quickly, which will greatly contribute to the competition of enterprises.

[0081] [First Embodiment: Push-Type Information Collection]

[0082] Next, use Figures 3 to 7 A browsing viewpoint information collection system 3 as a first embodiment of the present invention will be described. This system 3 uses a dedicated application software (hereinafter referred to as a "dedicated application") installed in the smartphone 1 to collect viewpoint information when the smartphone 1 user browses the WEB.

[0083] Figure 3 This is a diagram showing the configuration of a browsing viewpoint information collection system according to one embodiment of the present invention.

[0084] like Figure 3 As shown in the structure diagram of , the viewing viewpoint information collection system 3 includes a smartphone 1, an information collection server 20, and an application download server 40, which are interconnected in a data exchangeable manner via an Internet communication network 30. In addition, in the figure, there is only one smartphone 1, but in actual operation, multiple smartphones are included.

[0085] The smartphone 1 is a general-purpose smartphone that has the following components after a dedicated application is downloaded from an application download server 40 and installed.

[0086] That is to say, first Figure 1 , Figure 2As described in the previous section, the display panel 1a of the smartphone 1 displays the contents of each page on the WEB. As described above, the front camera 1b can also capture the user of the smartphone 1, i.e., the WEB viewer, and in particular, can capture the image of the periphery of the eyeball to detect the viewer's line of sight. The speaker 1c can also use a dedicated application to register the viewer, or can emit a voice for the operation instruction information when the smartphone 1 sends the viewing viewpoint information to the server 20 after registration.

[0087] The viewpoint detection unit 1e uses the image of the viewer's eyeball periphery captured by the built-in camera 1b to detect the viewer's line of sight, and generates the position of the viewpoint on the display panel 1a located in front of the line of sight as a coordinate value. The specific line of sight detection method can be used based on the non-patent document 1 described above and other known information. The interface unit 1f is a sending and receiving interface inside and outside the smartphone 1.

[0088] In addition, the viewpoint detection unit 1e can also be configured to detect the viewer's line of sight by using a mobile information terminal such as a smartphone that is not equipped with a special sensor as previously described in non-patent document 2, instead of using an image around the eyeball to detect the viewer's line of sight, by using methods such as machine learning and neural networks based on stored data related to line of sight tracking, thereby detecting the line of sight of the smartphone viewer or the position of a viewpoint located in front of the line of sight.

[0089] The storage unit 1g is used to readablely store various application programs including operating system programs and reading (browsing) programs for various actions performed by the smartphone 1, and also stores a dedicated application for collecting reading viewpoint information. The CPU (central processing unit) 1h is a central processing unit that uniformly controls various actions performed by the smartphone 1. Specifically, the operation unit 1i is a touch sensor of the main button 1d and the display panel 1a, which can use the voice of the reader picked up by a microphone not shown as voice recognition, and is a user interface function unit that uses them. These components are connected to each other through a bus 1j in a manner that allows data exchange.

[0090] The application download server 40 is a processor that stores various applications that can be used by the smart phone 1, not limited to special applications for viewpoint information collection, and responds to the download of these applications based on the request from the smart phone 1. The application download server 40 has: an interface unit 40a for unified control of data transmission and reception inside and outside the server 40, a storage unit 40b for storing various applications, and a central processing unit, i.e., CPU 40c, for unified control of various actions of the server 40. These components are similarly connected to each other via a bus line 40d in a manner that can exchange data.

[0091] (Structure of the Information Collecting Server 20)

[0092] Next, use Figure 4 The structure diagram, Figure 5 The configurations of the information collecting server 20 included in the present system 3 will be described with reference to the recording format explanatory diagram (an example of the browsing viewpoint information associated recording format).

[0093] Figure 4 yes Figure 3 A structural diagram of an information collection server included in the reading viewpoint information collection system.

[0094] Figure 5 yes Figure 3 An example of a record format used by the viewing viewpoint information collection system.

[0095] The information collecting server 20 includes a viewer registration management unit 20a, a viewing viewpoint information storage unit 20b, a viewing remuneration recording unit 20c, a viewing viewpoint information distribution destination management unit 20e, an interface unit 20f, a viewing viewpoint information distribution unit 20g, a CPU (central processing unit) 20h, a storage unit 201, a viewer remuneration calculation unit 20j, and a bus 20k.

[0096] The viewer registration management unit 20a is a unit for managing the information of the registrants who have agreed to provide the viewpoint information by the WEB viewers using the smartphone 1. It stores the email records, etc., of the registrants' basic information, such as the letter of commitment that has agreed to the collection of the viewing viewpoint information as personal information by the system 3. Figure 5 As shown in the example, the registered account ( Figure 5 In the example, it is "123456"), the registrant's year of birth and gender (also "1979 male"), the registrant's occupation and industry (also "pharmaceutical manufacturing and sales, business"), family composition (also "spouse, 2 males, 1 female"), interests (also "fishing, camping"), etc. In addition, these items are not limited to Figure 5 These are examples and can be selected appropriately.

[0097] In addition, when collecting the viewing viewpoint information, the present system 3 adopts, as an example, personal authentication based on face authentication. The reason is that, as described later, the registrant (viewer) receives compensation from the operator of the system 3 as a price for providing the viewing viewpoint information, and the purchaser of the information uses the viewer's attribute information, so it is necessary to have the viewing viewpoint information that the registrant has viewed. In addition, as personal authentication, in addition to face authentication, various biometric authentication technologies such as iris authentication, fingerprint authentication, or other authentication technologies can be used.

[0098] Therefore, the viewer registration management unit 20a stores data such as facial images used for personal authentication.

[0099] The browsing viewpoint information storage unit 20b is a unit that stores the browsing viewpoint information that is accepted and collected from the registrants (WEB viewers) for each registrant. Figure 5 As shown in the example, the registered account of the reader is stored ( Figure 5 In the example, it is "123456"), the date of browsing (also "August 1, 2018"), the time of browsing (also "every 5 seconds from 7:01:00"), the URL (Uniform Resource Locator) information as the browsing address (also "https: / / ○○○, https: / / ○○○ / △△"), the coordinate value of the browsing viewpoint on the display panel (the value of the pixel unit with the lower left of the display panel as the coordinate (0,0)) (also "(200, 450), (50, 320), (100, 31)", etc. In addition, when implementing, it is not necessary to be limited to these items. In Figure 5 In the example shown in the figure, the viewer starts browsing the home page at 7:01:00, then gazes at the same part of the home page every 10 seconds for 1 minute and 15 seconds, and then moves to the lower page.

[0100] The reading fee recording unit 20c stores, for each registrant, the equivalent price (remuneration) when the registrant is provided with equivalent reading information according to the contract. The recorded remuneration amount can be read if the registrant wishes, and according to the regulations of the system, for example, the remuneration amount can be transferred to the bank account of the registrant at the end of each month.

[0101] The viewing viewpoint information publishing destination management unit 20e stores and saves information such as the company name, URL (Uniform Resource Locator) information of the company portal homepage, conditions for the viewing information desired by the company (for example, information on viewers who visit the company's homepage, or all viewing information of registrants with attributes suitable for being a purchasing tier for the company's products, etc.), and the timing of sending information, as information that is not related to the registrant who provides the viewing information but to the user who purchases the viewing information from the system operating company.

[0102] The interface unit 20 f controls the transmission and reception of information inside and outside the server 20 .

[0103] The browsing viewpoint information distribution unit 20g generates browsing information determined by conditions in the information purchase destination pre-stored in the browsing viewpoint information distribution destination management unit 20e regularly or appropriately and distributes it. It can also be operated to distribute it at any time when there is browsing in the pre-specified URL.

[0104] The CPU (Central Processing Unit) 20 h performs overall control of various operations of the server 20 .

[0105] The storage unit 20i stores various data used for the operation of the system 3 and also stores execution programs.

[0106] The reader remuneration calculation unit 20j is a unit that calculates remuneration based on the actual situation of browsing information collected from registered readers, and calculates the remuneration amount by a time-based system according to the time when browsing information is transmitted, for example.

[0107] The bus 20k connects the components in the server 20 to each other so that data can be exchanged.

[0108] (Registration of information providers)

[0109] Next, use Figure 6 , Figure 7 , illustrating the steps by which a person interested in the business is initially registered in the system 3 as an information provider.

[0110] Figure 6 yes Figure 3 A flowchart of information provider registration performed by the viewing viewpoint information collection system. Figure 6 , for example, a dedicated application download flow chart (part 1) for registering a browsing viewpoint information provider is shown.

[0111] Figure 7 yes Figure 3 A flowchart of information provider registration performed by the viewing viewpoint information collection system. Figure 7 , for example, a dedicated application download flow chart (part 2) for viewing the registration of viewpoint information providers is shown.

[0112] In this embodiment, registration is performed using a dedicated application for a smartphone, and a person who desires information provision who becomes interested in the business through an advertisement or finds a URL accesses a designated URL to download the dedicated application (steps S101 and S102).

[0113] Then, when the dedicated application is started, a menu for registering as a registrant is displayed on the smartphone 1, and the person desiring registration selects the menu (steps S103, S201).

[0114] If the menu is selected, a query (question sentence, question signal, request sentence or request signal) with the main message "Since the browsing information provider wishes to register, the necessary input form is desired to be sent" is sent from the smartphone 1 to the server 20. Hereinafter, "question" is used in the same sense. ) is sent (step S202).

[0115] Accordingly, the server 20 first inquires the smartphone 1, "The browsing location information you provide in the future involves personal information. Do you agree to provide it?" If the user agrees, for example, by clicking a button such as "Agree" displayed on the screen of the smartphone 1, a message record serving as evidence of the agreement is sent from the smartphone 1 to the server 20 and stored in the server 20 (steps S203 and S204). The CPU (central processing unit) 1h performs control of receiving the inquiry signal from the server 20 and sending the server 20 any content of permission, etc. That is, the CPU 1h is configured to receive and respond to the inquiry signal from the server 20.

[0116] If the user does not agree, the dedicated application is forcibly terminated and the operation ends (steps S204 and S207).

[0117] If the applicant agrees, then Figure 5 The request for personal registration information of the applicant as illustrated in the example is sent from the server 20 to the smartphone 1, the requester enters the information and sends it to the server 20 for storage, and the dedicated application is forcibly terminated (steps S205, S206, S207).

[0118] (Collection of reading viewpoint information)

[0119] Next, use Figure 8 The flowchart (flow chart of sending browsing viewpoint information of dedicated application) shows a series of actions for a registered viewer to send browsing viewpoint information when browsing the web and respond to the collection of the server 20.

[0120] Figure 8 yes Figure 3 A flowchart of sending viewing point information executed by the viewing point information collection system.

[0121] In this embodiment, the browsing viewpoint information is collected when the registrant starts the dedicated application. In addition, the present invention is not limited to this. The browsing viewpoint information may be sent to the server 20 at all times when the registrant browses the web using the smartphone 1, or the browsing viewpoint information may be sent by other operations without starting the dedicated application. In any case, the collection is of course performed with the consent of the registrant.

[0122] First, the registrant who wants to send the browsing information starts the dedicated application (step S301), and then the front camera 1b of the smartphone 1 takes a picture of the registrant's face (step S302). The reason for performing face authentication is that, as mentioned above, the registrant must be the one to perform the browsing. In addition, it is also possible to confirm that it is the registrant himself by a method other than face authentication. The face authentication image data is sent from the smartphone 1 to the server 20 together with the information transmission start query (step S302).

[0123] Although the server 20 has already registered as an information provider, if the evidence material of the information collection commitment of the destination (electronic message record with commitment as the main purpose) is confirmed every time information is collected, the intention of the registrant can be confirmed more reliably (steps S303, S304).

[0124] If the registrant's consent is confirmed, the smartphone 1 starts to send the viewing viewpoint information ( Figure 5 For example, the browsing viewpoint information is continuously transmitted at intervals of, for example, 5 seconds until the registrant performs an operation to stop the dedicated application (steps S305, S306, S307, and S308).

[0125] As a result, the browsing viewpoint information associated with the registrant is recorded and stored in the server 20, and the above-described information can be utilized by a company or the like that purchases the information.

[0126] [Second embodiment: Pull-type information collection]

[0127] In the first embodiment of the present invention described above, a viewer (registrant) who wishes to provide viewing viewpoint information collects information of a so-called push type in which the viewer voluntarily transmits information at a desired time.

[0128] On the other hand, on the information purchase side, those who want to use the browsing information attach importance to the attributes of the registrants who browse. For example, there may be a demand to narrow down the browsing information to women of a certain age or to narrow down the browsing information to a certain profession, such as only medical personnel.

[0129] On the other hand, when receiving a request to "collect browsing information by medical personnel", the server 20 may select only medical personnel from the information of registrants already registered in the server 20, and ask only the medical personnel, i.e., registrants, "where can I send browsing information?" In this case, if the remuneration is set high, it also becomes an incentive for the registrants to send the gaze tracking information.

[0130] [Third Embodiment - Implementation in Personal Computer, etc.]

[0131] In the previous embodiment, the smartphone 1 is used as an example of a terminal used by a user who wants to provide gaze tracking information as information on gaze trajectories that the user has browsed. However, the present invention is not limited to smartphones and can be implemented in all PDAs (personal digital assistants) and personal computers.

[0132] When using these personal computers, in order to capture images of the registrant's eye periphery and perform line of sight detection, or to perform line of sight detection from the results of capturing the registrant's image through neural network-based learning, for example, if the personal computer has a built-in camera, a front camera can be used, and a stand-alone digital camera, a camera with USB connection specifications, etc. can be used.

[0133] [Fourth embodiment - embodiment not using application]

[0134] In each of the above embodiments, a dedicated application (in the case of a smartphone, a dedicated application for a smartphone) is installed in advance on the smartphone, personal computer or PDA to be used, and the browsing viewpoint information is transmitted.

[0135] It is not necessarily necessary to use a dedicated application program when practicing the present invention.

[0136] For example, a configuration may be adopted in which, while browsing a homepage posted on a certain web, a program such as an advertisement script is executed on the information terminal of the viewer to guide provision of information, register as a browsing information provider, and transmit browsing viewpoint information. This configuration is also included in the present invention.

[0137] [Fifth Embodiment - Acquisition of Browsing Viewpoint Information for Three-Dimensional Space]

[0138] In the above description, the browsing viewpoint information (information on the trajectory of the viewpoint) is information based on coordinate information on a two-dimensional plane such as the display screen of an information terminal for browsing.

[0139] In recent years, the following structure has been realized: virtual objects, such as items and avatars (proxy objects for characters), are arranged in a virtual three-dimensional space, and these objects can be included in the virtual three-dimensional space and sent as content. Audiences who watch or listen to these contents on multiple information terminals can generate their own avatars and invest them into the three-dimensional space to virtually participate in the content.

[0140] For objects or coordinate positions existing in such a virtual or real three-dimensional space, it is also possible to obtain information on the viewing viewpoint.

[0141] Fig. 9It is a schematic diagram of a virtual three-dimensional space for explaining another embodiment of the present invention. Fig. 9 2 is a schematic diagram showing an example of such a virtual three-dimensional space, which is an example of a three-dimensional virtual space including virtual objects. In the three-dimensional virtual space 200, a background 200-1, a tree object 200-2, a house object 200-3, a first avatar 200-4, and a second avatar 200-5 are arranged. In addition, the virtual three-dimensional space has three coordinate axes: an X axis, a Y axis, and a Z axis.

[0142] Each object in the virtual space 200 has data related to appearance and data related to position or posture (the former is sometimes referred to as coordinate data, and the latter is sometimes referred to as skeleton data or index data), and these data are stored in a personal computer used by the host who generates and transmits the virtual space 200 or in a server that publishes virtual space data. The host transmits the appearance data and position or posture data of each object to the information terminal (personal computer, smart phone, etc.) used by the audience who view and listen to the virtual space via the server, either directly or from the server. In each information terminal used by the audience, based on information related to these objects, a depiction (drawing) is performed for visually identifying the virtual space 200 through a display attached to the information terminal. In addition, drawing can be performed not only in the terminal for viewing and listening, but also in the terminal used by the sender or the server for transmission.

[0143] The rendering is a depiction observed from a certain viewpoint inside or outside the virtual space, and the viewpoint is, for example, the first avatar 200-4, the second avatar 200-5, and the virtual camera 200-6, and the image is depicted from each viewpoint. The audience can select a desired viewpoint from these available viewpoints to perform rendering, or if it is possible on the system, a new virtual camera (not shown) as a separate viewpoint can be newly set inside or outside the space 200, or the objects as these viewpoints can be moved.

[0144] Fig.10 Observe from various viewpoints Fig. 9 Schematic diagram of an image of a virtual three-dimensional space.

[0145] Fig.10 This is an example of a rendered image of a three-dimensional virtual space viewed from each viewpoint. Fig.10 This is a diagram schematically showing a state in which the virtual three-dimensional space 200 is rendered as images observed from respective viewpoints such as the first avatar 200 - 4 , the second avatar 200 - 5 , and the virtual camera 200 - 6 .

[0146] The image observed by the first avatar 200-4 ( Fig.10(A)) has a coordinate system represented by values ​​on two axes x1 and y1 with the lower left corner of the display as the origin. Similarly, the image observed from the second avatar 200-5 ( Fig.10 (C)) has a coordinate system represented by values ​​on two axes x3 and y3, and the image obtained with the virtual camera 200-6 as the viewpoint ( Fig.10 (B)) has a coordinate system represented by numerical values ​​on two axes x2 and y2.

[0147] Then, if the viewer sees a point of the object 200-3 as a house on the display of the information terminal used by the viewer, Fig. 9 As shown in the figure, point A is similar to the configuration of the embodiments described above. As the viewing viewpoint information, the system uses the image ( Fig.10 In the case of (A), its coordinate value (x', y') can be obtained as point A', and in the image ( Fig.10 In the case of (C)), its coordinate value (x', y') can be obtained as point A', and the image observed from the virtual camera 200-6 ( Fig.10 In the case of (B), its coordinate value (x”, y”) can be obtained as point A”.

[0148] Using the coordinate values ​​on the display that displays these drawn images, Fig. 9 In the virtual three-dimensional space 200 shown, the coordinates of point A, which is the corresponding point of the object 200-3 of the house, can be obtained as values ​​of coordinates in the coordinate system represented by the numerical values ​​on the X-axis, Y-axis, and Z-axis described above, as values ​​such as (X1, Y1, Z1). Although there are several methods, since the positions of each viewpoint such as the first avatar 200-4, the second avatar 200-5, and the virtual camera 200-6, the direction of the line of sight, and the virtual angle of view (or virtual focal length) are given in advance, it is possible to obtain information on the vector of the line of sight from the first avatar 200-4 to point A in the three-dimensional virtual space 200 through geometric calculations. Similarly, it is also possible to obtain information on the vector of the line of sight from the second avatar 200-5 to point A, and the intersection of these two line of sight vectors is point A.

[0149] Alternatively, since the appearance data and position data of the house object 200-3 are prepared in advance, for example, the intersection point where the line of sight from the first avatar 200-4 intersects with the position of the house object may be set as point A.

[0150] In the above description, the determination of three-dimensional coordinate values ​​in a three-dimensional virtual space has been described, but three-dimensional coordinate values ​​can also be determined in a three-dimensional actual space (real space) according to the same procedure.

[0151] In addition, nowadays, almost all homepages on the WEB are produced and posted as two-dimensional information. In contrast, if a WEB homepage is produced as three-dimensional information, the viewing viewpoint information of the viewers in the three-dimensional space can be generated according to the above procedure. As a result, the same effect as the two-dimensional viewing viewpoint information can be obtained.

[0152] [Fifth embodiment - embodiment in which it is not necessary to send and receive an inquiry signal when collecting gaze tracking information]

[0153] In each of the embodiments described previously, it has been explained that before the operator sends his or her line of sight tracking information to the server, an inquiry signal (also called an "inquiry") such as "Do you agree to send and collect your line of sight tracking information from this terminal to the server?" is sent from the server to the operator's user terminal. If the operator sees this and takes an action indicating permission (commitment), the fact that permission (commitment) has been given is sent from the user terminal to the server as another signal.

[0154] However, when implementing the present invention, it is not necessary to be limited to such a mode, and it is also possible to implement it as other structures different from the above examples.

[0155] That is, when a smartphone is used as an operator's terminal, smartphone application software (commonly referred to as "application") for collecting gaze tracking information is downloaded to the smartphone in advance for use, but permission as shown above may be received from the operator before downloading.

[0156] Alternatively, an operation of requesting such permission from a dedicated application installed in the smartphone may be performed as a required menu during or after downloading the application.

[0157] In either case, the fact that the operator has made an agreement, such as the fact that a "YES" (or "Agree," "Commit," "Accept," etc.) button displayed on the smartphone screen is pressed by the operator, can be recorded in a recording component inside the smartphone, and the fact that the operator has taken the action of making an agreement can be sent at an appropriate timing to a server for downloading applications, or to other receiving components, or to a server for collecting eye tracking information. Alternatively, the fact may not be sent but instead be recorded and retained inside the smartphone used by the operator.

[0158] With this configuration, the administrator of the server operating the system and the operator providing the eye tracking information do not need to perform repeated redundant and complicated operations, thereby achieving efficient operation.

[0159] In addition, the points constructed in this way are not limited to the case of using a smartphone application, but are also applicable to the case of sending and implementing data or programs to which a scripting language (JavaScript) for sending gaze tracking information is attached to a smartphone or a personal computer, or the case of implementing the sending of gaze tracking information in other ways.

[0160] (Description of Effects of the Invention)

[0161] According to the present invention, the following information display terminal, information sending method and computer program can be provided: regardless of whether it is personal information or not, it is possible to smoothly and massively collect information related to where specifically a viewer of a display object such as a WEB browsing device is gazing at in the displayed content, and to collect information related to the attributes of the viewer in addition.

[0162] The present invention can be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the scope of the invention. These embodiments and their variations are included in the scope and purpose of the invention, and are also included in the invention described in the protection scope of the present invention and its equivalent scope.

[0163] Explanation of symbols

[0164] 1 smart phone; 1b front camera; 3 reading viewpoint information collection system; 20 information collection server; 40 application download server.

Claims

1. An information display terminal, characterized in that: have: (A) Displaying a WEB screen as a display screen of the content; (B) a photographing unit that captures a facial image of the operator for facial authentication and sends it to the server to confirm that the operator is the registered information provider; (C) a viewpoint detection unit that detects the viewpoint position on the display screen by generating a value as a coordinate of the viewpoint position on the display screen that is located in front of the line of sight of an operator operating the information display terminal; (D) an output unit that outputs information that determines the display content of the display screen and the detection result of the viewpoint position to the server as information of a movement trajectory of the viewpoint in the content displayed on the display screen, wherein the information output by the output unit includes: the date when the operator browses the WEB screen, the browsing time, the address information of the browsed WEB screen, and the coordinate information of the browsing viewpoint; and An inquiry signal receiving and responding unit receives an inquiry signal and sends a reply to the inquiry signal according to the operation of the operator, wherein the inquiry signal requests the operator to grant permission for the information display terminal to execute the function of the output unit described in (D) on the condition that the operator of the information display terminal pays a reward.

2. The information display terminal according to claim 1, characterized in that: The information display terminal further includes a second transmitting unit configured to transmit information on the attribute of the operator who has permitted the inquiry signal.

3. A method for sending information, characterized in that: include: (A) a step of displaying a WEB screen as content on a display screen of an information display terminal; (B) a step of collecting a facial image of an operator for facial authentication and sending the image to a server to confirm that the operator is the registered information provider; (C) a step of detecting the viewpoint position on the display screen by a viewpoint detection unit of the information display terminal generating a value as a coordinate of the viewpoint position on the display screen that is located in front of the line of sight of an operator operating the information display terminal; (D) the output unit of the information display terminal outputs to the server information of the movement trajectory of the viewpoint in the content displayed on the display screen in correspondence with the detection result of the viewpoint position, wherein the information output by the output unit includes: the date when the operator browsed the WEB screen, the browsing time, the address information of the browsed WEB screen, and the coordinate information of the browsing viewpoint; and The inquiry signal receiving and responding unit of the information display terminal receives an inquiry signal and sends a reply to the inquiry signal according to the operation of the operator, wherein the inquiry signal requests the operator to grant permission for the information display terminal to execute the function of the output unit described in (D) on the condition that the operator of the information display terminal pays a reward.

4. The information transmission method according to claim 3, characterized in that: The information transmission method further includes a step in which the second transmission unit of the information display terminal transmits information related to the attribute of the operator who has permitted the inquiry signal.

5. A computer-readable recording medium storing a computer program for executing each step of the information transmission method according to claim 3 or 4.

Images