Web page viewing analysis system, method for analyzing viewing of web page, and web page viewing analysis program

Abstract

To provide a detailed insight on a web page from the attitude of a viewer of a web page when the viewer is watching a web page based on biological information of the viewer.SOLUTION: A web page viewing analysis system for analyzing the insight of a viewer of a web page acquires plural pieces of noted point data corresponding to plural viewers, generates a noted point visualization map by visualizing noted points of the viewers corresponding to the noted point data when the viewer watched the web page on the basis of at least some of the plural pieces of noted point data, and displays and superposes the noted point visualization map on the web page.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present disclosure relates to a web page browsing analysis system, a web page browsing analysis method, and a web page browsing analysis program. [Background technology]

[0002] Due to the impact of the COVID-19 pandemic, most aspects of social life are now moving online (in other words, digital transformation (DX)) at an accelerating pace. In particular, in the field of marketing, digital marketing has become more mainstream than traditional marketing using paper media, etc. In the field of digital marketing, marketing using websites has become mainstream.

[0003] Patent Document 1 discloses an advertisement evaluation system for evaluating the effectiveness of digital marketing such as advertisement videos. The advertisement evaluation system includes a user terminal, an e-mail server, a video distribution server, and a management server. These elements are communicatively connected via a communication network. The advertisement evaluation system determines whether or not a user has purchased a product introduced through digital marketing such as an advertisement video based on the user's transaction data, and evaluates the effectiveness of the digital marketing according to the determination result. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7057606 Summary of the Invention [Problem to be solved by the invention]

[0005] In the advertisement evaluation system disclosed in Patent Document 1, the effectiveness of digital marketing is evaluated according to the sales record of whether or not the user actually purchased the product introduced through the digital marketing. For this reason, the advertisement evaluation system cannot evaluate digital marketing at a stage where there is no actual sales record of the advertised product. On the other hand, there is a need for companies that conduct digital marketing for products to grasp detailed insights about the webpage for advertising and selling the product before the launch of the webpage. Here, "insight" means a deep understanding of the customer's potential needs, emotions, motivations, values, behavioral patterns and tendencies, which are presented in a refined form by analyzing customer data and feedback. In particular, there is a need for companies to grasp insights about the webpage based on the biometric information (e.g., gaze, facial expression, etc.) of the viewer who views the webpage before the launch of the webpage. Thus, there is room for consideration of an advertisement evaluation system that can provide the above needs.

[0006] The present disclosure aims to provide a web page browsing analysis system, a web page browsing analysis method, and a web page browsing analysis program that are capable of providing detailed insights into a web page from the viewer's behavior when viewing a web page based on their biometric information. [Means for solving the problem]

[0007] A webpage browsing analysis system according to one aspect of the present disclosure is a webpage browsing analysis system that analyzes viewer insights into a webpage, obtains a plurality of gaze point data corresponding to each of a plurality of viewers, generates a gaze point visualization map based on at least some of the plurality of gaze point data, which visualizes the gaze points when the viewer corresponding to the gaze point data viewed the webpage, and displays the gaze point visualization map superimposed on the webpage.

[0008] According to the above configuration, since the gaze point visualization map is presented to the company terminal, the company can grasp the browsing behavior of the viewers on the web page in detail by checking the gaze point visualization map. For example, the company can grasp the attention area on the web page by the viewers in detail. In this way, it is possible to provide detailed insights on the web page to the company that desires to analyze the web page.

[0009] A webpage browsing analysis system according to one embodiment of the present disclosure is a webpage browsing analysis system that analyzes viewer insights into a webpage, obtains multiple attribute data corresponding to multiple viewers, extracts recommended viewers to be interviewed based on at least some of the multiple attribute data, and displays information about the recommended viewers.

[0010] According to the above configuration, the company can obtain information about viewers who are recommended as interviewees. In this way, it is possible to provide detailed insights about web pages to companies that wish to analyze their web pages.

[0011] A webpage browsing analysis system according to one aspect of the present disclosure is a webpage browsing analysis system that analyzes viewer insights into a webpage, obtains a plurality of facial expression data corresponding to each of a plurality of viewers, generates comprehensive facial expression information indicating the overall facial expression of the viewer corresponding to the facial expression data when viewing the webpage based on at least some of the plurality of facial expression data, and displays the comprehensive facial expression information.

[0012] According to the above configuration, the comprehensive facial expression information is presented on the company's terminal, and the company can grasp the browsing behavior of the viewers of the webpage in detail by checking the comprehensive facial expression information. For example, the company can grasp the facial expressions (e.g., happy, sad, etc.) of the viewers of the webpage in detail. In this way, it is possible to provide detailed insights about the webpage to the company that desires to analyze the webpage.

[0013] A webpage browsing analysis system according to one embodiment of the present disclosure is a webpage browsing analysis system that analyzes viewer insights into a webpage, acquires multiple facial expression data corresponding to each of multiple viewers, generates overall interest level information indicating the overall interest level of the viewer corresponding to the facial expression data when viewing the webpage based on at least some of the multiple facial expression data, and displays the overall interest level information.

[0014] According to the above configuration, since the overall interest level information is presented to the company's terminal, the company can grasp the browsing behavior of the viewer on the web page in detail by checking the overall interest level information. For example, the company can grasp the viewer's interest in the web page in detail. In this way, it is possible to provide detailed insights about the web page to the company that desires to analyze the web page.

[0015] The web page browsing analysis system may input extraction conditions for the plurality of viewers, and extract at least some of the plurality of gaze point data based on the extraction conditions.

[0016] According to the above configuration, the company can grasp in detail the browsing behavior of viewers who meet the extraction conditions, and therefore it is possible to provide the company with more detailed insights regarding web pages.

[0017] In addition, the extraction conditions may be associated with at least one of age information of the multiple viewers, gender information of the multiple viewers, address information of the multiple viewers, occupation information of the multiple viewers, household size information of the multiple viewers, marital information of the multiple viewers, and educational background information of the multiple viewers.

[0018] According to the above configuration, it is possible to obtain a detailed understanding of the browsing behavior of multiple viewers who meet extraction conditions associated with at least one of the viewers' age information, gender information, address information, occupation information, number of people in household, marital information, and educational background information, thereby making it possible to provide companies with more detailed insights into web pages based on the viewers' attributes.

[0019] The gaze point visualization map may be a gaze point heat map.

[0020] According to the above, the company can grasp the browsing behavior of the viewer on the web page in detail by checking the gaze point heat map. In particular, the gaze point heat map makes it possible to more clearly grasp the viewer's attention area on the web page.

[0021] The gaze point visualization map may be a gaze point heat map.

[0022] According to the above, the company can grasp the browsing behavior of the viewer on the web page in detail by checking the gaze point heat map. In particular, the gaze point heat map makes it possible to more clearly grasp the viewer's attention area on the web page.

[0023] The web page browsing analysis system may also display information about the recommended viewers in a sorted or refined format.

[0024] The information about the recommended viewer may include one or more of the viewer's name, sex, age, time spent viewing the web page, whether or not the goal was achieved, and the time required to achieve the goal.

[0025] The comprehensive facial expression information may indicate a probability throughout the entire web page for each of a plurality of facial expression elements.

[0026] The comprehensive facial expression information may indicate, for each of a plurality of facial expression elements, a change in probability corresponding to a time series of the times the web page was viewed.

[0027] The overall interest level information may indicate a change in the interest level corresponding to a time series of when the web page was viewed.

[0028] A webpage browsing analysis method according to one aspect of the present disclosure is a webpage browsing analysis method for analyzing a viewer's insight into a webpage, the method including the steps of acquiring a plurality of gaze point data corresponding to each of a plurality of viewers, generating a gaze point visualization map based on at least some of the plurality of gaze point data, the gaze point visualization map visualizing the gaze points when the viewer corresponding to the gaze point data viewed the webpage, and displaying the gaze point visualization map by superimposing it on the webpage, and is executed by a computer.

[0029] A webpage browsing analysis method according to one aspect of the present disclosure is a webpage browsing analysis method for analyzing viewer insights into a webpage, and includes the steps of acquiring multiple attribute data corresponding to each of multiple viewers, extracting recommended viewers to be interviewed based on at least some of the multiple attribute data, and displaying information about the recommended viewers, and is executed by a computer.

[0030] A webpage browsing analysis method according to one aspect of the present disclosure is a webpage browsing analysis method for analyzing viewer insights into a webpage, and includes the steps of acquiring a plurality of facial expression data corresponding to each of a plurality of viewers, generating comprehensive facial expression information indicating the comprehensive expression of the viewer corresponding to the facial expression data when viewing the webpage based on at least some of the plurality of facial expression data, and displaying the comprehensive facial expression information, and is executed by a computer.

[0031] A webpage browsing analysis method according to one aspect of the present disclosure is a webpage browsing analysis method for analyzing viewer insights into a webpage, and includes the steps of acquiring a plurality of facial expression data corresponding to each of a plurality of viewers, generating overall interest level information indicating the overall interest level of the viewer corresponding to the facial expression data when viewing the webpage based on at least some of the plurality of facial expression data, and displaying the overall interest level information, and is executed by a computer.

[0032] Also, a web page browsing analysis program may be provided that causes a computer to execute the browsing analysis method. Effect of the Invention

[0033] According to the present disclosure, it is possible to provide a web page browsing analysis system, a web page browsing analysis method, and a web page browsing analysis program that are capable of providing detailed insights into a web page from the viewer's behavior when viewing a web page based on their biometric information. [Brief description of the drawings]

[0034] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a webpage browsing analysis system according to an embodiment of the present disclosure (hereinafter, the present embodiment). [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a viewer terminal. [Diagram 3] FIG. 2 illustrates an example of a hardware configuration of a server. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a company terminal. [Diagram 5] FIG. 2 is a sequence diagram for explaining a series of processes executed between a browser terminal and a server in the browsing analysis system according to the present embodiment. [Figure 6] FIG. 2 is a diagram for explaining a viewer's gaze point on a display area of ​​a web page. [Figure 7] FIG. 2 is a diagram for explaining facial feature points of a viewer. [Figure 8] FIG. 2 is a sequence diagram for explaining a series of processes executed between a server and a company terminal in the web page browsing analysis system according to the embodiment. [Figure 9] FIG. 13 is a diagram illustrating an example of a web page browsing analysis screen. [Figure 10] 1A is a diagram showing an example of a webpage, FIG. 1B is a diagram showing an example of a gaze point heatmap, and FIG. 1C is a diagram showing an example of a composite webpage in which the gaze point heatmap is superimposed on the webpage. [Figure 11] FIG. 13 is a diagram showing an example of an extraction condition display screen. [Figure 12] FIG. 13 is a diagram showing an example of a web page browsing analysis screen after extraction conditions are specified. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] Hereinafter, a webpage browsing analysis system 1 according to this embodiment (hereinafter, simply referred to as "browsing analysis system 1") will be described with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of the browsing analysis system 1 according to this embodiment. As shown in FIG. 1, the browsing analysis system 1 includes browser terminals 2a and 2b, a server 3, and a company terminal 4. These are connected to a communication network 8. Each of the browser terminals 2a and 2b is communicatively connected to the server 3 via the communication network 8. The company terminal 4 is communicatively connected to the server 3 via the communication network 8. The communication network 8 is composed of at least one of a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, and a wireless core network.

[0036] The viewer terminal 2a is a terminal associated with the viewer Va and is operated by the viewer Va. The viewer terminal 2b is a terminal associated with the viewer Vb and is operated by the viewer Vb. In this embodiment, for convenience of explanation, the viewer terminals 2a and 2b may be collectively referred to as the viewer terminal 2. Similarly, the viewers Va and Vb may be collectively referred to as the viewer V. In this embodiment, a large number of viewer terminals 2 associated with a large number of viewers are provided in the viewing analysis system 1, but for convenience of explanation, only two of the large number of viewer terminals, the viewer terminals 2a and 2b, are illustrated in FIG. 1. The type of the viewer terminal 2 is not particularly limited, and the viewer terminal 2 may be, for example, a smartphone, a personal computer, a tablet, or a wearable device (for example, a head-mounted display or an AR display), etc.

[0037] Next, the hardware configuration of the viewer terminal 2 will be described below with reference to Fig. 2. Fig. 2 is a diagram showing an example of the hardware configuration of the viewer terminal 2. As shown in Fig. 2, the viewer terminal 2 includes a control unit 20, a storage device 21, an imaging unit 22, a communication unit 23, an input operation unit 24, a display unit 25, a speaker 26, and an RTC (Real Time Clock) 28. These elements constituting the viewer terminal 2 are connected to a communication bus 29.

[0038] The control unit 20 includes a memory and a processor. The memory is configured to store computer-readable instructions (programs). For example, the memory is configured from a ROM (Read Only Memory) in which various programs and the like are stored, and a RAM (Random Access Memory) having multiple work areas in which various programs and the like executed by the processor are stored. The processor is configured from at least one of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit). The CPU may be configured from multiple CPU cores. The GPU may be configured from multiple GPU cores. The processor may be configured to load a program designated from various programs incorporated in the storage device 21 or the ROM onto the RAM and execute various processes in cooperation with the RAM.

[0039] The storage device 21 is, for example, a storage device (storage) such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory, and is configured to store programs and various data. The imaging unit 22 is configured to acquire video data showing the surrounding environment of the viewer terminal 2 through shooting. In particular, the imaging unit 22 is a camera configured to generate image data or video data showing the surrounding environment of the viewer terminal 2 through shooting, and includes an image sensor (for example, a CCD sensor or a CMOS sensor) and an image sensor drive processing circuit. In this embodiment, the control unit 20 functions as a gaze tracking unit that detects a change in the gaze of the viewer V based on the video data acquired by the imaging unit 22. Furthermore, the control unit 20 functions as an expression tracking unit that detects a change in the expression of the viewer V based on the video data.

[0040] The communication unit 23 includes a wireless communication module and / or a wired communication module for communicating with an external device connected to the communication network 8. The wireless communication module is configured to wirelessly communicate with an external device such as a base station or a wireless LAN router, and includes a transmitting / receiving antenna and a wireless transmitting / receiving circuit. The wireless communication module may be a wireless communication module compatible with a short-range wireless communication standard such as Wi-Fi (registered trademark) or Bluetooth (registered trademark), or may be a wireless communication module compatible with an X-generation mobile communication system (for example, a fourth-generation mobile communication system such as LTE) using a SIM (Subscriber Identity Module).

[0041] The input operation unit 24 is, for example, a touch panel, a mouse, and / or a keyboard arranged over the video display of the display unit 25, and is configured to receive an input operation of the viewer V and generate an operation signal according to the input operation. The display unit 25 is, for example, configured with a video display and a video display circuit that drives and controls the video display. The display unit 25 has a display screen 125 on which a web page is displayed (see FIG. 6). The speaker 26 is configured to output sound to the outside based on the audio data. The RTC 28 is configured to obtain information indicating the current time.

[0042] In this embodiment, the viewer terminal 2 has a WEB browser configured to display a web page transmitted from the server 3. The web page is composed of an HTML file, a CSS file, an image or video file, a program file, and the like.

[0043] Returning to FIG. 1, the server 3 is communicatively connected to the viewer terminal 2 and the enterprise terminal 4 via the communication network 8. The server 3 transmits a web page (e.g., a landing page of a predetermined website) to each of the multiple viewer terminals 2 via the communication network 8, and transmits a composite web page (described later) in which a gaze point heat map is superimposed on the web page to the enterprise terminal 4. The server 3 may be composed of multiple servers. The server 3 functions as a WEB server configured to provide a web page browsing analysis application as a WEB application. In this regard, the server 3 is configured to transmit data (e.g., HTML files, CSS files, image and video files, program files, etc.) for displaying a web page browsing analysis screen on the WEB browser of the enterprise terminal 4. In this way, the server 3 functions as a server for providing SaaS (System as a Service). The server 3 may be constructed on-premise or may be a cloud server. The server 3 also functions as a data management server for managing multiple web pages, gaze point data of each viewer V, and facial expression data of each viewer V.

[0044] The hardware configuration of the server 3 will be described below with reference to Fig. 3. Fig. 3 is a diagram showing an example of the hardware configuration of the server 3. As shown in Fig. 3, the server 3 includes a control unit 30, a storage device 31, an input / output interface 32, a communication unit 33, an input operation unit 34, and a display unit 35. These elements constituting the server 3 are connected to a communication bus 36.

[0045] The control unit 30 includes a memory and a processor. The memory is configured to store computer-readable instructions. In particular, the memory may store a web page browsing analysis program for causing the processor to execute a series of processes (web page browsing analysis method) executed by the server 3 in Figs. 5 and 8. The memory is configured with a ROM and a RAM. The processor is configured with at least one of a CPU, an MPU, and a GPU.

[0046] The storage device 31 is, for example, a storage device (storage) such as an HDD, SSD, or flash memory, and is configured to store programs and various data. The storage device 31 stores a plurality of web pages, web event data (event data such as clicks and scrolls) of each viewer V, gaze point data of each viewer V, and facial expression data of each viewer V. The storage device 31 also stores a viewer information table related to information of each viewer V and a user information table related to each user U who uses a web page browsing analysis application. The viewer information table includes attribute information of each viewer V. For example, the viewer information table may include at least one of identification information, gender information, age information, household number information, address information, and occupation information of each viewer V. Furthermore, the viewer information table may further include information such as the duration of stay when viewing a web page, whether or not a goal has been achieved, and the time to achieve the goal. The user information table may include identification information, attribute information, login information, and the like of each user U.

[0047] The input / output interface 32 is an interface that enables a connection between an external device and the server 3, and includes an interface conforming to a predetermined communication standard such as the USB standard or the HDMI (registered trademark) standard. The communication unit 33 may include various wired communication modules for communicating with an external terminal on the communication network 8. The input operation unit 34 is, for example, a touch panel, a mouse, and / or a keyboard, and is configured to receive an input operation by an operator and generate an operation signal according to the input operation by the operator. The display unit 35 is, for example, configured by an image display and an image display circuit.

[0048] The company terminal 4 is a terminal operated by a user U who uses a web page browsing analysis application provided by the server 3. In this embodiment, a plurality of company terminals 4 are provided in the browsing analysis system 1 (in other words, in this embodiment, a plurality of users U use the web page browsing analysis application), but for convenience of explanation, only one company terminal 4 is illustrated in FIG.

[0049] The hardware configuration of the company terminal 4 will be described below with reference to FIG. 4. FIG. 4 is a diagram showing an example of the hardware configuration of the company terminal 4. As shown in FIG. 4, the company terminal 4 may be, for example, a personal computer, a smartphone, a tablet, or a wearable device attached to a user U. The company terminal 4 has a web browser. The web page browsing analysis application runs on the web browser of the company terminal 4. The company terminal 4 includes a control unit 40, a storage device 41, an input / output interface 42, a communication unit 43, an input operation unit 44, and a display unit 45. These elements are connected to a communication bus 46.

[0050] The control unit 40 includes a memory and a processor. The memory is configured to store computer-readable instructions (programs). For example, the memory is configured with a ROM and a RAM. For example, the processor is configured with at least one of a CPU, an MPU, and a GPU.

[0051] The storage device 41 is, for example, a storage device such as an HDD, SSD, or flash memory, and is configured to store programs and various data. The input / output interface 42 is an interface (for example, USB, HDMI, etc.) that enables a connection between an external device and the company terminal 4. The communication unit 43 is configured to connect the company terminal 4 to the communication network 8, and includes a wireless communication module and / or a wired communication module. The input operation unit 44 is, for example, a touch panel, a mouse, and / or a keyboard, and is configured to accept an input operation of the user U and generate an operation signal according to the input operation of the user U. The display unit 45 is, for example, configured by an image display and an image display circuit. In this embodiment, a webpage browsing analysis screen 70 (see FIG. 9) is displayed on the display unit 45.

[0052] Next, a series of processes executed between each of the plurality of viewer terminals 2 and the server 3 in the browsing analysis system 1 according to this embodiment will be described below with reference to FIG. 5. FIG. 5 is a sequence diagram for explaining a series of processes executed between the viewer terminal 2 and the server 3 in the browsing analysis system 1. For convenience of explanation, a series of processes executed between a single viewer terminal 2 and the server 3 is illustrated in FIG. 5, but in this embodiment, each of the plurality of viewer terminals 2 executes the series of processes shown in FIG. 5. In addition, in the series of processes shown in FIG. 5, the viewer V provides the server 3 with gaze point data and facial expression data of the viewer V on a predetermined webpage through browsing of the predetermined webpage provided by the server 3. In this regard, the operating company that operates the server 3 and provides the webpage browsing analysis application may provide the viewer V with an incentive for browsing the webpage. For example, the company may grant each viewer V a predetermined number of points or electronic money for each browsing of a webpage for a predetermined time (for example, one minute).

[0053] Furthermore, the operating company may give each viewer V a mission to purchase a specific product posted on the webpage, in addition to viewing the webpage for a specified period of time. In this case, the operating company can obtain detailed insights into the viewing behavior (gaze and facial expressions) of each viewer V in the process of purchasing a specific product on the webpage. The operating company can freely set the viewing conditions of the webpage to be given to the viewer depending on the nature and type of viewer insight to be obtained.

[0054] 5, in step S1, the viewer terminal 2 transmits a webpage viewing request for viewing a webpage to the server 3 in response to an input operation on the input operation unit 24 (e.g., a touch panel) by the viewer V. Next, the server 3 receives the webpage viewing request from the viewer terminal 2 and transmits the webpage (HTML file, CSS file, image or video file, program file, etc.) stored in the storage device 31 to the viewer terminal 2 (step S2).

[0055] Next, the viewer terminal 2 displays the web page on the WEB browser (step S3). As shown in FIG. 6, the viewer terminal 2 displays the web page on the display screen 125 of the display unit 25. The viewer terminal 2 also starts acquiring gaze point data and facial expression data of the viewer V at the same time as displaying the web page (step S4). As shown in step S5, the viewer terminal 2 continues acquiring gaze point data and facial expression data of the viewer V until the display of the web page ends. On the other hand, the viewer terminal 2 ends acquiring the gaze point data and facial expression data of the viewer V when the display of the web page ends, and then transmits the gaze point data, facial expression data, and attribute data (information such as the viewer's name, sex, age, stay time when viewing the web page, whether or not a goal was achieved, and time to achieve the goal, which have been collected in advance) to the server 3 (step S6). In this regard, the display time of the web page displayed on the WEB browser may be set in advance to a predetermined time (for example, one minute). The web page may also be displayed on the WEB browser of the viewer terminal 2 until the viewer V achieves a specific mission (goal) on the web page. The achievement of a mission (goal) is, for example, the completion of a purchase action of a specific product, etc. In addition, attribute information such as name, sex, age, etc., among the attribute data of the viewer V, is acquired in response to an input operation on a predetermined display screen displayed on the viewer terminal 2 of the viewer V.

[0056] <Method of acquiring gaze point data> The viewer terminal 2 acquires video data showing the viewer V through shooting using the imaging unit 22, and then acquires gaze point data of the viewer V based on the video data. In this regard, the control unit 20 of the viewer terminal 2 functions as an eye tracking unit that detects a change in the gaze of the viewer V based on the video data acquired by the imaging unit 22. As an example of a method of acquiring gaze point data, the control unit 20 identifies the viewer V and the viewer V's eyes shown in a predetermined frame (still image) of the video data, and then acquires position information indicating the position of the viewer V's eyes relative to the imaging unit 22. Next, the control unit 20 identifies the rotation angle of the pupil of the viewer V's eyeball based on the predetermined frame, and then acquires gaze information indicating the direction of the viewer V's gaze L (see FIG. 6). Furthermore, the control unit 20 acquires face direction information indicating the direction of the viewer V's face based on the predetermined frame. The control unit 20 may acquire gaze information based on a three-dimensional eyeball model (passive method), or may acquire gaze information based on a corneal reflex method (active method). When the gaze information is acquired based on the corneal reflex method, the viewer terminal 2 may further include an infrared light emitting element (for example, an infrared LED) configured to emit infrared light, and an infrared camera (infrared image sensor).

[0057] Next, the control unit 20 acquires the position (X, Y) (XY coordinates) of the gaze point P, which is the point where the gaze L of the viewer V intersects with the display screen 125 of the viewer terminal 2, based on the position information, the gaze information, and the face direction information. Here, the position of the gaze point P where the gaze L intersects with the display screen 125 may be acquired as a position on the XY coordinate space set on the display screen 125. In this way, the control unit 20 can acquire the position of the gaze point P where the gaze L intersects with the display screen 125 from a predetermined frame of the video data. The control unit 20 may acquire the position of the gaze point P at an acquisition rate corresponding to the frame rate of the video data, or may acquire the position of the gaze point P at an acquisition rate different from the frame rate. For example, when the frame rate of the video data is 30 fps, the acquisition rate of the gaze point P may be 30 Hz, or may be an acquisition rate lower than 30 Hz (for example, 15 Hz).

[0058] Next, the control unit 20 converts the position of the gaze point P of the viewer V from the coordinate position in the XY coordinate space set on the display screen 125 to the coordinate position in the XY coordinate space set on the display area S of the web page. In this regard, as shown in FIG. 6, when the viewer V performs a flick operation on the display screen 125 in the +Y axis direction, the display area S of the web page displayed on the display screen 125 (see FIG. 6 and FIG. 10(a)) slides in the +Y axis direction. Similarly, when the viewer V performs a flick operation on the display screen 125 in the -Y axis direction, the display area S of the web page displayed on the display screen 125 slides in the -Y axis direction. When the viewer V performs a flick operation on the display screen 125 in the +X axis direction, the display area S of the web page displayed on the display screen 125 slides in the +X axis direction. Similarly, when the viewer V performs a flick operation on the display screen 125 in the -X axis direction, the display area S of the web page displayed on the display screen 125 slides in the -X axis direction.

[0059] In this way, since the display area S of the web page displayed on the display screen 125 changes in response to the flick operation of the viewer V, it is necessary to convert the position of the gaze point P of the viewer V from the XY coordinate position of the display screen 125 to the XY coordinate position of the display area S. Specifically, when the display area S of the web page slides in response to the flick operation of the viewer V, the control unit 20 updates the correspondence between the XY coordinates of the display area S and the XY coordinates of the display screen 125. After that, when the control unit 20 acquires the XY coordinate position of the gaze point P on the display screen 125, it converts the XY coordinate position of the display screen 125 to the XY coordinate position of the display area S based on the correspondence between the XY coordinates of the display area S and the XY coordinates of the display screen 125.

[0060] Furthermore, the control unit 20 acquires the position of the gaze point P on the display area S and the viewing time of the web page when the position of the gaze point P is acquired. Here, the viewing time of the web page refers to the time elapsed since the start of viewing the web page. For example, the viewing time when 10 seconds have elapsed since the start of viewing the web page is 10 seconds. If the viewing time of the web page when the position (X1, Y1) of the gaze point P1 is acquired is t1, the position (X1, Y1) of the gaze point P1 on the display area S and the viewing time t1 are stored in a state in which the position (X1, Y1) of the gaze point P1 on the display area S and the viewing time t1 are associated with each other. In this way, the control unit 20 can acquire gaze point data indicating a temporal change in the position of the gaze point P on the display area S. For example, if the acquisition rate of the gaze point is 30 Hz and the viewing time of the web page is 30 seconds, the number of gaze points P included in the gaze point data is 900.

[0061] <Method of acquiring facial expression data> The viewer terminal 2 acquires video data showing the viewer V through shooting using the imaging unit 22, and then acquires facial expression data of the viewer V based on the video data. In this regard, the control unit 20 of the viewer terminal 2 functions as an expression tracking unit that detects changes in the expression of the viewer V based on the video data acquired by the imaging unit 22. Here, the facial expression data is data indicating temporal changes in the facial expression information of the viewer V. More specifically, the facial expression data is data indicating temporal changes in probability information of the facial expression elements of the viewer V (in this example, five types of facial expression elements: happy, sad, neutral (no expression), anger, and surprise).

[0062] As an example of a method for acquiring facial expression data, the control unit 20 identifies the face of the viewer V shown in a predetermined frame (still image) of the video data M, and then acquires a plurality of facial feature points K included in the face of the viewer V (see FIG. 7). As shown in FIG. 7, in this example, 24 facial feature points K related to the eyebrows, eyes, nose, and mouth are acquired. Next, the control unit 20 calculates a plurality of facial feature amounts based on the acquired plurality of facial feature points K, and acquires probability information of each facial expression element of the viewer V based on the calculated plurality of facial feature amounts and a facial expression recognition model (learning model). In the facial expression recognition model, the value of each unit of the input layer indicates a corresponding one of the plurality of facial feature amounts, and the value of each unit of the output layer indicates probability information of a corresponding one of the plurality of facial expression elements. The probability information of each facial expression element may be expressed as a percentage %. In particular, the probability of the facial expression element of "happy", the probability of the facial expression element of "sad", the probability of the facial expression element of "neutral", the probability of the facial expression element of "anger", and the probability of the facial expression element of "surprise" are each expressed as a percentage %. In this way, the control unit 20 can obtain probability information of each facial expression element of the viewer V from a predetermined frame.

[0063] The control unit 20 may acquire facial expression information of the viewer V at an acquisition rate corresponding to the frame rate of the video data, or may acquire facial expression information of the viewer V at a rate different from the frame rate. For example, when the frame rate of the video data is 30 fps, the acquisition rate of the facial expression information may be 30 Hz, or may be a rate lower than 30 Hz (for example, 15 Hz).

[0064] In addition, the control unit 20 acquires the viewing time of the web page when the facial expression information is acquired together with the facial expression information. For example, if the viewing time of the web page when the probability information of each facial expression element is acquired is t1, the probability information of each facial expression element and the viewing time t1 are stored in a state in which the probability information and the viewing time t1 are associated with each other. In this way, the control unit 20 can acquire facial expression data indicating the temporal change in the facial expression information of the viewer V.

[0065] Returning to FIG. 5, in step S7, the server 3 receives the gaze point data, facial expression data, and attribute data of the viewer V related to the web page from each viewer terminal 2, and stores the gaze point data, facial expression data, and attribute data of each viewer V in the storage device 31 in a state in which the gaze point data, facial expression data, and attribute data of each viewer V are associated with the web page that was the subject of browsing. In the example shown in FIG. 5, for convenience of explanation, one type of web page is transmitted to each viewer terminal 2, but multiple types of web pages may be transmitted to each viewer terminal 2. In this case, the gaze point data and facial expression data may be associated with the identification information of the viewer and the identification information of the web page (e.g., the URL of the web page). For example, the gaze point data and facial expression data of the viewer Va transmitted from the viewer terminal 2a may be associated with the identification information of the viewer Va and the identification information of the web page that was the subject of browsing. Similarly, the gaze point data and facial expression data of the viewer Vb transmitted from the viewer terminal 2b may be associated with the identification information of the viewer Vb and the identification information of the web page that was the subject of browsing. In this manner, through the series of processes shown in FIG. 5, it becomes possible to accumulate gaze point data and facial expression data of each viewer V in the server 3.

[0066] Next, a series of processes executed between the server 3 and the company terminal 4 in the browsing analysis system 1 according to this embodiment will be described below with reference to FIG. 8. FIG. 8 is a sequence diagram for explaining a series of processes executed between the server 3 and the company terminal 4 in the browsing analysis system 1. In this example, a user U who operates the company terminal 4 is engaged in digital marketing work related to web pages, and wishes to grasp detailed insights of the web pages based on biometric information (gazing points and facial expressions) of multiple viewers V who viewed the web pages. The web pages viewed by each viewer V in the series of processes shown in FIG. 5 are web pages related to the user U's digital marketing work.

[0067] As shown in Fig. 8, in step S10, the company terminal 4 transmits the login information of the user U (e.g., a login ID and a login password) to the server 3 through an input operation by the user U. In step S11, the server 3 authenticates the user U based on the login information transmitted from the company terminal 4. Note that the authentication method for the user U is not limited to login authentication. After authenticating the user U, the server 3 identifies the web page associated with the user U by referring to the user information table stored in the storage device 31.

[0068] In step S12, the server 3 transmits data for displaying the webpage browsing analysis screen 70 shown in Fig. 9 (hereinafter, webpage browsing analysis screen data) to the company terminal 4 via the communication network 8. The webpage browsing analysis screen data may be a file (e.g., an HTML file, a CSS file, a program file, an image or video file, etc.) for displaying the webpage browsing analysis screen 70 on the web browser of the company terminal 4. Here, the program file may be a program file executable on the web browser, such as JavaScript.

[0069] As shown in FIG. 9, the webpage browsing analysis screen 70 has a webpage display area 71, a viewer display area 76, a facial expression information display area 72, and a facial expression / interest level information display area 73. Furthermore, the webpage browsing analysis screen 70 has a viewer selection button 74, a webpage selection button 75, and a website selection button 78. The webpage display area 71 displays a composite webpage 63, which will be described later. The viewer display area 76 displays information on viewers of the webpage who are recommended as interviewees. The facial expression information display area 72 displays comprehensive facial expression information in a radar chart format. The facial expression / interest level information display area 73 displays comprehensive facial expression information in a time series graph and comprehensive interest level information in a time series graph.

[0070] In step S13, the company terminal 4 selects a selected web page to be displayed on the web page browsing analysis screen 70 and an extraction condition for the viewer V in response to an input operation of the user U on the web page browsing analysis screen 70. Specifically, the company terminal 4 selects one of a plurality of websites in response to an input operation of the user U on the website selection button 78. A plurality of website candidates may be displayed on the web page browsing analysis screen 70 in response to an input operation on the website selection button 78. One or more websites selectable through the website selection button 78 are websites associated with the user U. Then, the company terminal 4 selects a selected web page to be displayed in the web page display area 71 from a plurality of web pages related to the selected website in response to an input operation of the user U on the web page selection button 75. A plurality of web page candidates associated with the selected website may be displayed in the web page display area 71 in the form of a URL in response to an input operation on the web page selection button 75. Next, an extraction condition for the viewer V is selected in response to an input operation of the user on the viewer selection button 74. More specifically, as shown in FIG. 11, the company terminal 4 displays an extraction condition display screen 80 for specifying extraction conditions for viewer V on the webpage browsing analysis screen 70 in response to an input operation by user U on the viewer selection button 74.

[0071] In the extraction condition display screen 80 shown in FIG. 11, the extraction condition of the viewer V is associated with the gender information of the viewer V, the age information of the viewer V, and the number of people in the household of the viewer V. For example, when the viewer V in his / her 40s is selected as the extraction condition, in step S15, the server 3 acquires the gaze point data of the viewer V in his / her 40s. In addition, the user U can specify an extraction condition associated with at least one of the gender information, age information, and household information through the extraction condition display screen 80. For example, as shown in FIG. 12, when the user U specifies an extraction condition (item name: target) related to a woman in her 20s living in a one-person household, in step S15, the server 3 acquires the gaze point data of the viewer V that matches the woman in her 20s living in a one-person household.

[0072] 12, three items, namely, gender information, age information, and household number information, are exemplified as the extraction conditions for viewer V, but this embodiment should not be limited to these. In this respect, address information, occupation information, marital information, and educational background information of viewer V may be further associated with the extraction conditions for viewer V. In this manner, the extraction conditions for viewer V may be associated with at least one of age information, gender information, address information, occupation information, household number information, marital information, and educational background information of viewer V.

[0073] Furthermore, the extraction conditions for the viewer V may be further associated with at least one of the viewer V's questionnaire information, survey information, and past behavior history information (eg, frequency of going to movie theaters, etc.).

[0074] Next, in step S14, the company terminal 4 requests the server 3 to transmit the composite web page in response to the input operation of the user U. At this point, in response to the input operation of the user U, the company terminal 4 transmits to the server 3 a transmission request for the composite web page associated with the selected web page selected through the web page selection button 75. Furthermore, the company terminal 4 transmits information regarding the extraction conditions of the viewer V together with the transmission request for the composite web page to the server 3.

[0075] In step S15, the server 3 receives the request to send the composite web page and information on the extraction conditions of the viewer V from the company terminal 4, and then extracts the gaze point data of the viewer V that matches the extraction conditions from the storage device 31. In this regard, the server 3 extracts the identification information of the viewer V that matches the extraction conditions by referring to the viewer information table related to the information of each viewer V. Then, the server 3 extracts the gaze point data associated with both the identification information of the selected web page and the identification information of the extracted viewer V. For example, as shown in FIG. 12, when the extraction conditions of the viewer V are age: 20s, gender: female, and number of people in the household: one-person household, the server 3 identifies the identification information of the viewer V that satisfies all the conditions of age: 20s, gender: female, and number of people in the household: one-person household, and then extracts the gaze point data of the viewer V that is associated with both the identification information of the identified viewer V and the identification information of the selected web page. On the other hand, as shown in FIG. 9, when the extraction conditions of the viewer V are all viewers, the gaze point data of the viewer V that is associated with the identification information of the selected web page is extracted.

[0076] In step S16, the server 3 generates a gaze point heat map (one example of a gaze point visualization map) that visualizes the temporal change of the position of the gaze point P of multiple viewers V that meet the extraction condition based on the extracted gaze point data. In the gaze point heat map, the temporal change of the aggregate of the gaze points P of multiple viewers V is visualized. The frame rate of the gaze point heat map may be the same as the acquisition rate of the gaze point data. For example, when the gaze points of viewers V1 to Vn (n is an integer of 2 or more) at the browsing time t1 of the web page are P1 to Pn, the gaze points P1 to Pn are visualized in the gaze point heat map at the browsing time t1. In the gaze point heat map, when the number of gaze points overlapping a predetermined area is large (i.e., when the predetermined area is an area gazed at by multiple viewers V), the predetermined area may be colored in a warm color such as red. On the other hand, when the number of gaze points overlapping a predetermined area is small (i.e., when the predetermined area is an area not gazed at much by multiple viewers V), the predetermined area may be colored in a cool color such as blue. In this way, the color of the predetermined region may gradually change from a cool color to a warm color as the number of gaze points overlapping the predetermined region increases. Furthermore, if there are no gaze points included in the predetermined region (i.e., if the predetermined region is an area that is not gazed upon by any of the viewers V), the predetermined region may not be colored. In this regard, a region that includes only gaze points that are treated as noise (outliers) may not be colored at all.

[0077] Furthermore, in this embodiment, when the extraction condition for the viewer V is all viewers, a gaze point heat map is generated that visualizes the temporal change in the positions of the gaze points P of all viewers V. In this way, the gaze point heat map generated changes according to the extraction condition for the viewer V (in other words, the attributes of the viewer V).

[0078] In step S17, the server 3 generates a composite web page 63 in which the gaze point heat map 62 is superimposed on the web page 60 (see FIG. 10). In particular, the server 3 generates the composite web page 63 in which the gaze point heat map 62 is superimposed on the web page 60. In step S18, the server 3 transmits the composite web page 63 to the company terminal 4. After receiving the composite web page 63 from the server 3, the company terminal 4 displays the composite web page 63 in the web page display area 71 of the web page browsing analysis screen 70. In this manner, the composite web page 63 is played back in the company terminal 4 (step S19).

[0079] In this embodiment, a gaze point heat map is described as an example of a gaze point visualization map that visualizes the temporal change of the gaze points of multiple viewers V, but the gaze point visualization map is not limited to the gaze point heat map. In this respect, a gaze line flow map may be used as another example of a gaze point visualization map. In the gaze line flow map, the flow lines showing the movement trajectories of the gaze points P of multiple viewers V are visualized. The gaze line flow map may be displayed in a still image format or a video format.

[0080] In step S20, the server 3 transmits information about each viewer V to the company terminal 4 based on the attribute data of each viewer V recorded in the viewer information table. The company terminal 4 displays the information about each viewer V in the viewer display area 76 on the web page analysis screen 70 (step S21). In the viewer display area 76, information about the time each viewer V stayed on the web page, information about whether each viewer V achieved their goal (mission), and information about the time each viewer V took to achieve their goal (mission) are displayed along with the name, sex, age, etc. of each viewer V. These pieces of information may be displayed in a sorted or narrowed down format according to information of an item selected by a user instruction, for example. Furthermore, a play button 176 for viewing the gaze point heat map 62 of each viewer V and a contact button 177 for contacting each viewer V are displayed. The contact button 177 may be associated with the email address or mobile phone number of the viewer V. The user U may operate the play button 176 associated with a predetermined viewer V, whereby a composite web page 63 in which a gaze point heat map 62 associated with the predetermined viewer V is superimposed on the web page 60 is displayed in the web page display area 71. More specifically, the company terminal 4 transmits information about the predetermined viewer V to the server 3 in response to an input operation of the play button 176 by the user U. The server 3 then generates a gaze point heat map 62 that visualizes a temporal change in the position of the gaze point P of the predetermined viewer V based on the received gaze point data of the predetermined viewer V. The server 3 then transmits to the company terminal 4 the composite web page 63 in which the generated gaze point heat map 62 is superimposed on the web page 60. In this manner, the updated composite web page 63 is displayed in the web page display area 71.

[0081] Next, in step S22, the server 3 extracts facial expression data of the viewer V that matches the extraction condition from the storage device 31. In this regard, the server 3 extracts the identification information of the viewer V that matches the extraction condition by referring to a viewer information table related to the information of each viewer V. Thereafter, the server 3 extracts facial expression data associated with both the identification information of the selected web page and the identification information of the extracted viewer V. For example, as shown in FIG. 12, when the extraction conditions of the viewer V are age: 20s, gender: female, and number of people in the household: one-person household, the server 3 identifies the identification information of the viewer V that satisfies all the conditions of age: 20s, gender: female, and number of people in the household: one-person household, and then extracts facial expression data of the viewer V that is associated with both the identification information of the identified viewer V and the identification information of the selected web page. On the other hand, as shown in FIG. 9, when the extraction conditions of the viewer V are all viewers, the server 3 extracts facial expression data of the viewer V that is associated with the identification information of the selected web page.

[0082] In step S23, the server 3 generates comprehensive facial expression information and comprehensive interest level information based on the facial expression data of the viewer V extracted in step S22. Thereafter, the server 3 transmits the generated comprehensive facial expression information and comprehensive interest level information to the company terminal 4. As for the timing of transmitting these pieces of information, these pieces of information may be transmitted after the reproduction of the composite web page 63. As shown in FIG. 9, the company terminal 4 displays the comprehensive facial expression information displayed in a radar chart format in the facial expression information display area 72 of the web page browsing analysis screen 70, and also displays the comprehensive facial expression information displayed in a time series graph format in the facial expression and interest level information display area 73 of the web page browsing analysis screen 70 (step S24). Furthermore, the company terminal 4 displays the comprehensive interest level information displayed in a time series graph format in the facial expression and interest level information display area 73.

[0083] The comprehensive facial expression information indicates comprehensive facial expression information of the viewer V that matches the extraction condition, and includes comprehensive facial expression information in a radar chart format and comprehensive facial expression information in a time series graph format. The comprehensive facial expression information in the radar chart format indicates average probability information (percentage %) of each comprehensive facial expression element (happy, sad, neutral, anger, surprise) of the viewer V that matches the extraction condition. For example, the comprehensive facial expression information in the radar chart format may include the average probability of the facial expression element "happy" of multiple viewers V that match the extraction condition, the average probability of the facial expression element "sad" of the multiple viewers V, the average probability of the facial expression element "neutral" of the multiple viewers V, the average probability of the facial expression element "anger" of the multiple viewers V, and the average probability of the facial expression element "surprise" of the multiple viewers V.

[0084] For example, if the extracted viewer V includes viewers Va, Vb, and Vc, the server 3 determines the average probability of each facial expression element of the viewer Va based on the facial expression data indicating the temporal change of each facial expression element of the viewer Va. Similarly, the server 3 determines the average probability of each facial expression element of the viewer Vb based on the facial expression data indicating the temporal change of each facial expression element of the viewer Vb, and determines the average probability of each facial expression element of the viewer Vc based on the facial expression data indicating the temporal change of each facial expression element of the viewer Vc. After that, the server 3 determines the average probability of each overall facial expression element of the viewers Va to Vc based on the average probability of each facial expression element of the viewers Va to Vc. At this point, if the average probability of the facial expression element "happy" of the viewers Va, Vb, and Vc is Ra1, Rb1, and Rc1, the average probability Rave1 of the overall facial expression element "happy" of the viewers Va to Vc is Rave1=(Ra1+Rb1+Rc1) / 3. Similarly, if the average probabilities of the facial expression element “sad” for viewers Va, Vb, and Vc are Ra2, Rb2, and Rc2, the average probability Rave2 of the overall facial expression element “sad” for viewers Va to Vc is Rave2 = (Ra2 + Rb2 + Rc2) / 3.

[0085] The server 3 may also calculate the positivity and negativity of the extracted viewer V with respect to the web page 60 as a percentage based on the average probability of each of the extracted overall facial expression elements of the viewer V. Furthermore, the server 3 may transmit information indicating the positivity and negativity to the company terminal 4. As shown in FIG. 9, the information indicating the positivity and negativity may be displayed in the facial expression information display area 72. For example, when the average probability of the facial expression element "happy" is high, the value of the positivity may be large. On the other hand, when the average probability of the facial expression element "anger" is high, the value of the negativity may be large. The server 3 may calculate the positivity and negativity from the overall average probability of each facial expression element by using a relational expression or a learning model in which the overall average probability of each facial expression element is set as an input value and the values ​​of the positivity and negativity are set as output values.

[0086] The comprehensive facial expression information in the form of a time series graph indicates the time change of each comprehensive facial expression element (happy, sad, neutral, angry, surprised) of the viewer V who matches the extraction condition. For example, the comprehensive facial expression information in the form of a time series graph may include information indicating the time change of the average probability of the facial expression element "happy" of multiple viewers V who match the extraction condition, information indicating the time change of the average probability of the facial expression element "sad" of the multiple viewers V, information indicating the time change of the average probability of the facial expression element "neutral" of the multiple viewers V, information indicating the time change of the average probability of the facial expression element "anger" of the multiple viewers V, and information indicating the time change of the average probability of the facial expression element "surprise" of the multiple viewers V.

[0087] For example, if the extracted viewer V includes viewers Va, Vb, and Vc, the server 3 calculates information indicating the change over time of the average probability of the facial expression element "happy" for viewers Va to Vc, information indicating the change over time of the average probability of the facial expression element "sad" for viewers Va to Vc, information indicating the change over time of the average probability of the facial expression element "neutral" for viewers Va to Vc, information indicating the change over time of the average probability of the facial expression element "anger" for viewers Va to Vc, and information indicating the change over time of the average probability of the facial expression element "surprise" for viewers Va to Vc, and transmits this information to the company terminal 4 as comprehensive facial expression information in the form of a time series graph.

[0088] The server 3 may also calculate overall interest level information in the form of a time series graph based on the overall facial expression information in the form of a time series graph. The overall interest level information indicates a change over time in the overall interest level (percentage %) of the viewer V who meets the extraction condition in the web page 60. For example, if the average probability of each overall facial expression element at the viewing time t1 is R1 to R5, the overall interest level I at the viewing time t1 may be calculated based on the average probability R1 to R5. In this regard, the server 3 may calculate the overall interest level I from the average probability R1 to R5 of each overall facial expression element by using a relational expression or a learning model in which the average probability R1 to R5 of each overall facial expression element is set as an input value and the overall interest level I is set as an output value.

[0089] Furthermore, in this embodiment, when the extraction conditions for viewers V are all viewers V, the comprehensive facial expression information indicates the comprehensive facial expression information of all viewers V, and the comprehensive interest level information indicates the change over time in the comprehensive interest level of all viewers V in the web page 60. In this way, the comprehensive facial expression information and the comprehensive interest level information change according to the extraction conditions for viewers V (in other words, the attributes of the viewers V to be extracted).

[0090] In addition, in this embodiment, the overall interest level information of multiple viewers V is calculated based on the overall facial expression information (particularly, the temporal change in the average probability of each facial expression element of the viewer V), but the overall interest level information may be calculated based on the gaze point data of each viewer V. Specifically, the overall interest level information is calculated based on the interest level data of each viewer V, and the interest level data of the viewer V may be calculated based on the gaze point data of the viewer V, information on the blinking of the viewer V, and drowsiness information of the viewer V. The information on the blinking of the viewer V may be information indicating the number of times the viewer V blinks or information indicating whether or not the viewer V blinks. The information on the blinking of the viewer V may be specified based on each frame of the video data of the viewer V. The drowsiness information of the viewer V may be information indicating whether or not the viewer V is drowsy. The drowsiness information of the viewer V may be specified based on each frame of the video data of the viewer V. For example, the interest level data of the viewer V may be calculated based on information on whether the position of the gaze point P of the viewer V is within the display area S of the video, information on whether the viewer V is sleepy, and information on whether the viewer V is blinking. Based on the interest level data of each viewer V, it is possible to calculate overall interest level information of multiple viewers V.

[0091] In addition, in this embodiment, information indicating the positivity and negativity of multiple viewers V is calculated based on the facial expression data of each viewer V, but this information may also be calculated based on the gaze point data of each viewer V.

[0092] In this embodiment, the order of the processes shown in Fig. 8 is not particularly limited. In addition, in the examples shown in Fig. 9 and Fig. 12, a composite web page 63 in which a gaze point heat map 62 is superimposed on a web page 60 is displayed in the web page display area 71, but the web page 60 may be displayed in the web page display area 71 instead of the composite web page 63. Even in this case, the overall facial expression information in a radar chart format may be displayed in the facial expression information display area 72, and the overall facial expression information and the overall interest level information in a time series graph format may be displayed in the facial expression / interest level information display area 73.

[0093] In this example, two pieces of information, the overall facial expression information and the overall interest level information, are displayed on the webpage browsing analysis screen 70, but only one of the two pieces of information, the overall facial expression information and the overall interest level information, may be displayed on the webpage browsing analysis screen 70. In this example, five types of facial expression elements, happy, sad, neutral, angry, and surprised, are used as the facial expression information, but the types of facial expression elements are not limited to these. For example, fear, happiness, enjoyment, disgust, etc. may be further used as other facial expression elements.

[0094] Also, according to this embodiment, the user U may select a predetermined area of ​​the gaze point heat map 62 included in the composite web page 63 through the company terminal 4, and obtain information on the viewer V linked to the gaze points P included in the selected predetermined area from the server 3. Specifically, after the company terminal 4 transmits information on the selected predetermined area to the server 3, the server 3 identifies a set of gaze points P included in the predetermined area based on the information on the predetermined area. After that, the server 3 identifies attribute information on the viewer V linked to the identified set of gaze points P, and transmits the attribute information on the viewer V to the company terminal 4. In this way, it is possible to utilize attribute information on the viewer V who is interested in a predetermined area (e.g., a specific object) of the gaze point heat map 62 for web marketing.

[0095] According to this embodiment, the composite webpage 63 in which the gaze point heat map 62 is superimposed on the webpage 60 is presented to the company terminal 4, so that the user U can grasp the browsing behavior of the multiple viewers V on the webpage 60 in detail by checking the composite webpage 63 displayed on the webpage browsing analysis screen 70. In particular, the user U can grasp the attention objects (attention areas) in the webpage 60 by the multiple viewers V in detail. In this way, it is possible to provide the company that desires the webpage analysis with detailed insights on the webpage 60 (more specifically, detailed insights on the digital marketing of the webpage). For example, the user U can grasp whether the inquiry button or purchase button arranged on the webpage 60 is being paid attention to by each viewer V. Furthermore, the user U can take measures to improve the CVR (conversion rate) of the webpage (for example, optimal arrangement of the inquiry button or purchase button) through the insights on the webpage.

[0096] In particular, in this embodiment, a gaze point heat map is generated that visualizes the temporal change in the position of the gaze point of the viewer V who satisfies the attribute conditions based on gaze point data of the viewer V that matches the extraction conditions associated with the age information, gender information, household number information, etc. of the multiple viewers V. In this way, it is possible to provide the company with more detailed insights about the web page 60 according to the viewer's attributes.

[0097] Furthermore, according to this embodiment, the overall facial expression information and the overall interest level information in the form of a radar chart and a time series graph are presented on the company terminal 4, so that the user U can check the information displayed on the web page browsing analysis screen 70 to grasp in detail the browsing behavior of the multiple viewers V with respect to the web page 60. In particular, the user U can grasp in detail the facial expressions (happy, sad, neutral, angry, surprised, etc.) and interest levels of the viewers V with respect to the web page 60 for each playback time. In this way, it is possible to provide detailed insights regarding the web page (more specifically, detailed insights regarding the digital marketing of the web page) to a company that desires web page analysis.

[0098] In particular, in this embodiment, based on the facial expression data of viewers V who match the extraction conditions associated with the age information, gender information, household size information, etc. of multiple viewers V, comprehensive facial expression information is generated that indicates the probability information of each comprehensive facial expression element of the viewers V who satisfy these attribute conditions, and comprehensive interest level information is generated that indicates the comprehensive interest level of the viewers V who match the extraction conditions in the web page 60. In this way, through this information, the browsing behavior of the viewers V who match the extraction conditions can be grasped in detail, making it possible to provide the company with more detailed insights into the web page 60.

[0099] Although the embodiment of the present invention has been described above, the technical scope of the present invention should not be interpreted as being limited by the description of the embodiment. The embodiment is an example, and it is understood by those skilled in the art that various modifications of the embodiment are possible within the scope of the invention described in the claims. The technical scope of the present invention should be determined based on the scope of the invention described in the claims and its equivalents. [Explanation of symbols]

[0100] 1: Web page browsing analysis system, 2, 2a, 2b: Viewer terminal, 3: Server, 4: Company terminal, 8: Communication network, 20: Control unit, 21: Storage device, 22: Imaging unit, 23: Communication unit, 24: Input operation unit, 25: Display unit, 26: Speaker, RTC: 28, 30: Control unit, 31: Storage device, 32: Input / output interface, 33: Communication unit, 34: Input operation unit, 35: Display unit, 40: Control unit, 41: Storage device, 42: Input / output interface, 43: Communication unit, 44: Input operation unit, 45: Display unit, 60: Web page, 62: gaze point heat map, 63: synthetic web page, 70: web page browsing analysis screen, 71: web page display area, 72: facial expression information display area, 73: interest level information display area, 74: viewer selection button, 75: web page selection button, 76: viewer display area, 78: website selection button, 80: extraction condition display screen, 125: display screen, 176: play button, 177: contact button, K: facial feature points, L: gaze, M: video data, P: gaze point, S: display area, U: user, V, Va, Vb: viewer

Claims

1. A web page browsing analysis system that analyzes a viewer's insight into a web page, comprising: Acquire a plurality of gaze point data corresponding to each of a plurality of viewers, generating a gaze point visualization map based on at least some of the plurality of gaze point data, the gaze point visualization map visualizing a gaze point when a viewer corresponding to the gaze point data viewed the web page; and displaying the gaze point visualization map by superimposing it on the web page. Web page browsing analysis system.

2. A web page browsing analysis system that analyzes a viewer's insight into a web page, comprising: Acquire multiple attribute data corresponding to each of multiple viewers, Extracting recommended viewers as interviewees based on at least some of the plurality of attribute data; Displaying information about viewers of the recommendation; Web page browsing analysis system.

3. A web page browsing analysis system that analyzes a viewer's insight into a web page, comprising: Acquire a plurality of pieces of facial expression data corresponding to a plurality of viewers, generating comprehensive facial expression information indicating a comprehensive facial expression of a viewer corresponding to the facial expression data when viewing the web page, based on at least some of the plurality of facial expression data; Displaying the comprehensive facial expression information. Web page browsing analysis system.

4. A web page browsing analysis system that analyzes a viewer's insight into a web page, comprising: Acquire a plurality of pieces of facial expression data corresponding to a plurality of viewers, generating overall interest level information indicating an overall interest level of a viewer corresponding to the facial expression data when viewing the web page, based on at least some of the plurality of facial expression data; Displaying the overall interest level information. Web page browsing analysis system.

5. Inputting extraction conditions for the plurality of viewers, extracting at least some of the plurality of gaze point data based on the extraction condition; The web page browsing analysis system according to claim 1 .

6. Inputting extraction conditions for the plurality of viewers, extracting at least some of the plurality of facial expression data based on the extraction conditions; The web page browsing analysis system according to claim 3 or 4.

7. The extraction conditions are: Age information of the plurality of viewers; Gender information of the plurality of viewers; Address information of the plurality of viewers; Occupational information of the plurality of viewers; Information on the number of people in a household of the plurality of viewers; Marriage information of the plurality of viewers; Educational background information of the plurality of viewers; is associated with at least one of The web page browsing analysis system according to claim 5 .

8. The extraction conditions are: Age information of the plurality of viewers; Gender information of the plurality of viewers; address information of the plurality of viewers; Occupational information of the plurality of viewers; Information on the number of viewers in a household; Marriage information of the plurality of viewers; Educational background information of the plurality of viewers; is associated with at least one of The web page browsing analysis system according to claim 6 .

9. The gaze point visualization map is a heat map. The web page browsing analysis system according to claim 1 .

10. Displaying information about the recommended viewers in a sorted or filtered format; The web page browsing analysis system according to claim 2 .

11. The information about the recommended viewer includes at least one of the viewer's name, gender, age, time spent viewing the web page, whether or not the goal was achieved, and time to achieve the goal. The web page browsing analysis system according to claim 2 .

12. the comprehensive facial expression information indicates a probability of each of a plurality of facial expression elements throughout the entire web page; The web page browsing analysis system according to claim 3 .

13. the comprehensive facial expression information indicates a change in probability of each of a plurality of facial expression elements corresponding to a time series of browsing of the web page; The web page browsing analysis system according to claim 3.

14. the overall interest level information indicates a transition of the interest level corresponding to a time series of browsing of the web page; The web page browsing analysis system according to claim 4 .

15. A web page browsing analysis method for analyzing a viewer's insight into a web page, comprising: acquiring a plurality of gaze point data corresponding to a plurality of viewers; generating a gaze point visualization map based on at least some of the plurality of gaze point data, the gaze point being visualized when a viewer corresponding to the gaze point data viewed the web page; A step of displaying the gaze point visualization map by superimposing it on the web page; A method for analyzing web page browsing carried out by a computer, comprising:

16. A web page browsing analysis method for analyzing a viewer's insight into a web page, comprising: acquiring a plurality of attribute data corresponding to each of a plurality of viewers; A step of extracting viewers recommended as interviewees based on at least some of the plurality of attribute data; displaying information about the recommended viewers; A method for analyzing web page browsing carried out by a computer, comprising:

17. A web page browsing analysis method for analyzing a viewer's insight into a web page, comprising: acquiring a plurality of pieces of facial expression data corresponding to a plurality of viewers; generating comprehensive facial expression information indicating a comprehensive facial expression of a viewer corresponding to the facial expression data when viewing the web page, based on at least some of the plurality of facial expression data; displaying the integrated facial expression information; A method for analyzing web page browsing carried out by a computer, comprising:

18. A web page browsing analysis method for analyzing a viewer's insight into a web page, comprising: acquiring a plurality of pieces of facial expression data corresponding to a plurality of viewers; generating overall interest level information indicating an overall interest level of a viewer corresponding to the facial expression data when viewing the web page, based on at least some of the plurality of facial expression data; displaying the overall interest level information; The web page browsing analysis method includes:

19. A web page browsing analysis program that causes a computer to execute the web page browsing analysis method according to any one of claims 15 to 18.

Images