Information processing system, information processing method, program

Abstract

To provide an information processing system, an information processing method, and a program capable of providing information applicable to generate personalized information.SOLUTION: An information processing system includes first acquisition means 31 for acquiring information on a Web page displayed in a Web browser executed on a terminal device, second acquisition means 32 for acquiring information on a state of a user of the terminal device measured by a prescribed measurement device when the Web page is displayed in the Web browser, estimation means 33 for estimating action of the user when the Web page is displayed in the Web browser on the basis of the acquired information on the state of the user, and storage means 34 for storing the information on the Web page and the action of the user in association with each other in a prescribed storage device.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] In recent years, with the spread of terminal devices such as mobile terminals, a function of collecting information of users of terminal devices and providing personalized information to each user (for example, a search engine function in which the results change depending on the user's search history and location, a recommendation function for displaying recommended products according to the browsing history and purchase history, etc.) has been put into practical use.

[0003] For example, in the technology described in Patent Document 1, items related to an item are recommended to a user who has taken an action on the item.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In such an information providing service, information such as the user's search history, location, or browsing / purchase history of web pages is analyzed, and personalized information is generated according to the analysis result. On the other hand, in order to generate more personalized information, it is necessary to perform analysis using more appropriate information.

[0006] The present invention has been made in view of the above problems, and an object thereof is to provide an information processing system, an information processing method, and a program capable of providing information applicable to generating personalized information.

Means for Solving the Problems

[0007] In order to solve the above problems, first, the present invention provides an information processing system including: a first acquisition means for acquiring information regarding a web page displayed on a web browser running on a terminal device; a second acquisition means for acquiring information regarding the state of a user of the terminal device measured by a predetermined measuring device when the web page is displayed on the web browser; an estimation means for estimating the behavior of the user when the web page is displayed on the web browser based on the acquired information regarding the state of the user; and a storage means for storing, in a predetermined storage device, the information regarding the web page and the behavior of the user in an associated state (Invention 1).

[0008] According to such an invention (Invention 1), since the information regarding the web page displayed on the web browser running on the terminal device and the behavior of the user when the web page is displayed on the web browser are stored in an associated state, it becomes possible to grasp how the user browses the web page while performing what kind of behavior based on the stored information. Further, by analyzing the stored information or the like, it becomes possible to generate personalized information according to the behavior of the user, so that it is possible to provide information applicable to generating personalized information.

[0009] In the above invention (Invention 1), a control means for controlling the operation of the terminal device according to the estimated behavior of the user may be provided (Invention 2).

[0010] According to such an invention (Invention 2), for example, when the behavior of the user such as holding the terminal device up is estimated, the terminal device can be appropriately operated according to the behavior of the user when the web page is displayed on the web browser by controlling the terminal device such as activating the camera function of the terminal device.

[0011] In the above inventions (Inventions 1 to 2), a providing means for providing information according to the presumed behavior of the user may be provided (Invention 3).

[0012] According to such an invention (Invention 3), for example, when the behavior of the user such as walking is presumed, appropriate information can be provided according to the behavior of the user when the web page is displayed on the web browser, such as providing advertisement information to the user via the terminal device.

[0013] In the above inventions (Inventions 1 to 3), a changing means for changing the display mode of the web page displayed on the web browser according to the presumed behavior of the user may be provided (Invention 4).

[0014] According to such an invention (Invention 4), for example, when the behavior of the user changes from stationary to walking, the design, layout, etc. of the web page can be changed. Therefore, it becomes possible to dynamically change the display mode of the web page according to the behavior of the user when the web page is displayed on the web browser.

[0015] In the above inventions (Inventions 1 to 4), the estimating means may estimate the behavior of the user based on the information on the state of the user acquired and a learned model based on machine learning using the information on the state of the user as learning data (Invention 5).

[0016] According to such an invention (Invention 5), by using the learned model, the behavior of the user when the web page is displayed on the web browser can be easily estimated.

[0017] In the above inventions (Inventions 1 to 5), the information on the web page may include information on the display state of the web page on the terminal device (Invention 6).

[0018] Here, information regarding the display state of a web page may include, for example, a state where the web page is displayed on the screen of the terminal device (foreground), a state where the web page is not displayed on the screen of the terminal device (background), a state where the web page is pre-rendered (pre-rendering), and the like.

[0019] According to such an invention (Invention 6), it becomes possible to grasp both what actions the user is performing while browsing the web page and what state the web page is being displayed in. Therefore, by analyzing the stored information and the like, it is possible to generate more personalized information according to the user's actions.

[0020] In the above inventions (Inventions 1 to 6), the information regarding the web page may include information regarding whether the web page is being displayed online (Invention 7).

[0021] According to such an invention (Invention 7), it becomes possible to grasp both what actions the user is performing while browsing the web page and whether the web page is being displayed online. Therefore, by analyzing the stored information and the like, it is possible to generate more personalized information according to the user's actions.

[0022] In the above inventions (Inventions 1 to 7), the information regarding the state of the user may include at least one of the user's acceleration and the user's angular velocity (Invention 8).

[0023] According to such an invention (Invention 8), based on at least one of the user's acceleration and angular velocity when the web page is displayed on the web browser, it is possible to estimate the user's actions when the web page is displayed on the web browser.

[0024] In the above inventions (Inventions 1 to 8), the predetermined measuring device may be provided in the terminal device (Invention 9).

[0025] According to such an invention (Invention 9), it becomes possible to acquire information regarding the state of a user by using a measuring device provided in a terminal device.

[0026] Second, the present invention provides an information processing method (Invention 10) that executes steps including: a step in which a computer acquires information regarding a web page displayed on a web browser being executed on a terminal device; a step in which the computer acquires information regarding the state of a user of the terminal device measured by a predetermined measuring device while the web page is being displayed on the web browser; a step in which the computer estimates the behavior of the user while the web page is being displayed on the web browser based on the acquired information regarding the state of the user; and a step in which the computer stores, in a predetermined storage device, the information regarding the web page and the behavior of the user in an associated state.

[0027] Third, the present invention provides a program (Invention 11) for causing a computer to realize functions including: a function of acquiring information regarding a web page displayed on a web browser being executed on a terminal device; a function of acquiring information regarding the state of a user of the terminal device measured by a predetermined measuring device while the web page is being displayed on the web browser; a function of estimating the behavior of the user while the web page is being displayed on the web browser based on the acquired information regarding the state of the user; and a function of storing, in a predetermined storage device, the information regarding the web page and the behavior of the user in an associated state.

Effects of the Invention

[0028] According to the information processing system, information processing method, and program of the present invention, it is possible to provide information applicable to generating personalized information.

Brief Description of the Drawings

[0029]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Mode for Carrying Out the Invention

[0030] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, this embodiment is an example, and the present invention is not limited thereto.

[0031] (1) Basic configuration of the information processing system FIG. 1 is a diagram schematically showing the basic configuration of an information processing system according to an embodiment of the present invention. As shown in FIG. 1, in the information processing system according to the present embodiment, when a user is executing a web browser (for example, Google Chrome (registered trademark), etc.) on the terminal device 10, information regarding a web page displayed on the web browser and information regarding the state of the user measured by a predetermined measuring device 18 (shown in FIG. 2) when the web page is being displayed on the web browser are acquired by the information processing device 20. The terminal device 10 and the information processing device 20 are connected to a communication network NW (network) such as the Internet or a LAN (Local Area Network).

[0032] The terminal device 10 may be, for example, a device wearable by the user (for example, a wearable device), or a portable device that the user can carry. Further, the terminal device 10 may be a device operated by an individual user, such as a mobile terminal, a smartphone, a PDA (Personal Digital Assistant), a personal computer, a television receiver having a two-way communication function (including a so-called multifunctional smart TV).

[0033] The information processing device 20 is a device for providing information applicable to generating personalized information. The information processing device 20 estimates the behavior of the user when the web page is being displayed on the web browser based on the information regarding the state of the user acquired from the terminal device 10. Further, the information processing device 20 stores in a predetermined storage device in a state where the information regarding the web page acquired from the terminal device 10 and the estimated behavior of the user are associated with each other. The information processing device 20 may be a device operated by an individual user, similar to the terminal device 10.

[0034] (2) Configuration of the terminal device The configuration of the terminal device 10 will be described with reference to FIG. 2. FIG. 2 is a block diagram showing the internal configuration of the terminal device 10. As shown in FIG. 2, the terminal device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage device 14, a display processing unit 15, a display unit 16, an input unit 17, a measurement device 18, and a communication interface unit 19, and a bus 10a is provided for transmitting control signals or data signals between the respective units.

[0035] When the power is turned on to the terminal device 10, the CPU 11 loads various programs stored in the ROM 12 or the storage device 14 into the RAM 13 and executes them.

[0036] The storage device 14 may be, for example, a non-volatile storage device such as a flash memory, an SSD (Solid State Drive), a magnetic storage device (e.g., an HDD (Hard Disk Drive), a floppy disk (registered trademark), a magnetic tape, etc.), an optical disk, or the like, or a volatile storage device such as a RAM, and stores programs executed by the CPU 11 and data referred to by the CPU 11.

[0037] The display processing unit 15 displays display data given from the CPU 11 on the display unit 16. The display unit 16 is, for example, an LCD (Liquid Crystal Display) monitor including thin film transistors arranged in pixel units in a matrix form, and drives the thin film transistors based on the display data to display the displayed data on the display screen.

[0038] When the terminal device 10 is a communication device using the button input method, the input unit 17 includes a group of buttons including a plurality of instruction input buttons such as a direction instruction button and a determination button for receiving a user's operation input, and a group of buttons including a plurality of instruction input buttons such as a numeric keypad, and includes an interface circuit for recognizing a press (operation) input of each button and outputting it to the CPU 11.

[0039] When the terminal device 10 is a communication device with a touch panel input method, the input unit 17 mainly accepts touch panel input by touching the display screen with a fingertip or a pen. The touch panel input method may be a known method such as a capacitance method.

[0040] Also, when the terminal device 10 is a device capable of voice input, the input unit 17 may be configured to include a microphone for voice input, or may be provided with an interface circuit for outputting voice data input via an external microphone to the CPU 11. Further, when the terminal device 10 is a communication device capable of inputting moving images and / or still images, the input unit 17 may be configured to include a digital camera or a digital video camera for image input, or may be provided with an interface circuit for receiving image data captured by an external digital camera or digital video camera and outputting it to the CPU 11.

[0041] The measurement device 18 is a device that continuously or intermittently (for example, every predetermined interval (such as 300 milliseconds or 1 second, etc.)) measures the state of the user of the terminal device 10. Here, the information regarding the user's state may be, for example, a value representing the user's state, or a value obtained by substituting a value representing the user's state into a predetermined calculation formula, or information representing the degree of the user's state.

[0042] The measurement device 18 may be, for example, a device (such as an acceleration sensor, a gyro sensor, a motion sensor, a heart rate monitor, a blood pressure monitor, a thermometer, a sweating meter, etc.) that measures the state of the user (such as acceleration in the three-axis or two-axis direction, angular velocity in the three-axis or two-axis direction, heart rate (pulse), blood pressure, body temperature, sweating amount, number of steps, walking speed, posture, exercise intensity (for example, heart rate ÷ maximum heart rate) or calorie consumption, etc.).

[0043] The communication interface unit 19 includes an interface circuit for communicating with other devices (such as the information processing device 20) via the communication network NW.

[0044] (3) Configuration of the information processing device Referring to FIG. 3, the configuration of the information processing apparatus 20 will be described. FIG. 3 is a block diagram showing the internal configuration of the information processing apparatus 20. As shown in FIG. 3, the information processing apparatus 20 includes a CPU 21, a ROM 22, a RAM 23, a storage device 24, a display processing unit 25, a display unit 26, an input unit 27, and a communication interface unit 28, and a bus 20a is provided for transmitting control signals or data signals between the respective units.

[0045] When the power is turned on to the information processing apparatus 20, the CPU 21 loads various programs stored in the ROM 22 or the storage device 24 into the RAM 23 and executes them. In the present embodiment, the CPU 21 reads and executes the programs stored in the ROM 22 or the storage device 24, thereby realizing the functions of the first acquisition means 31, the second acquisition means 32, the estimation means 33, the storage means 34, the control means 35, the provision means 36, and the change means 37 (shown in FIG. 4) described later.

[0046] The storage device 24 may be, for example, a non-volatile storage device such as a flash memory, an SSD, a magnetic storage device (e.g., HDD, floppy disk (registered trademark), magnetic tape, etc.), an optical disk, or a volatile storage device such as a RAM, and stores programs executed by the CPU 21 and data referred to by the CPU 21. Further, the storage device 24 stores the first acquisition data (shown in FIG. 5), the second acquisition data (shown in FIG. 6), the learning data (shown in FIG. 7), the control data (shown in FIG. 8), the first provision data (shown in FIG. 9), and the second provision data (shown in FIG. 10) described later.

[0047] The input unit 27 may be, for example, an information input device such as a mouse or a keyboard, may be configured to include a microphone for voice input, may be configured to include a digital camera or a digital video camera for image input, or may include an interface circuit for receiving image data captured by an external digital camera or digital video camera and outputting it to the CPU 21.

[0048] The communication interface unit 28 includes an interface circuit for performing communication via the communication network NW. Details of other units within the information processing apparatus 20 may be the same as those of the terminal device 10.

[0049] (4) Outline of each function in the information processing system Functions realized by the information processing system of the present embodiment will be described with reference to FIG. 4. FIG. 4 is a functional block diagram for explaining functions that play a major role in the information processing system of the present embodiment. In the functional block diagram of FIG. 4, the first acquisition means 31, the second acquisition means 32, the estimation means 33, and the storage means 34 correspond to the main configurations of the information processing system of the present invention. Other means (control means 35, provision means 36, and change means 37) are not necessarily essential configurations, but are components for further improving the present invention.

[0050] The first acquisition means 31 has a function of acquiring information regarding a web page displayed on a web browser executed on the terminal device 10.

[0051] Here, the information regarding the web page may include information regarding the display state of the web page on the terminal device 10. Further, the information regarding the display state of the web page may include, for example, a state (foreground) in which the web page is displayed on the screen (display unit 16) of the terminal device 10, a state (background) in which the web page is not displayed on the screen (display unit 16) of the terminal device 10, a state (prerendering) in which the web page is being prerendered, and the like. Thereby, it becomes possible to grasp both what actions the user is performing while browsing the web page and what state the web page is being displayed in. Therefore, by analyzing the stored information and the like, it is possible to generate more personalized information according to the user's actions.

[0052] Furthermore, the information about the web page may include information on whether the web page is being displayed online. This makes it possible to grasp both what actions the user is performing while browsing the web page and whether the web page is being displayed online. Therefore, by analyzing the stored information, etc., it is possible to generate more personalized information according to the user's actions.

[0053] The function of the first acquisition means 31 is realized, for example, as follows. First, when the CPU 11 of the terminal device 10 executes a web browser by a user operation using the input unit 17, for example, every time a predetermined time (for example, 10 seconds, etc.) elapses, it acquires information about the web page displayed on the web browser. Here, it is assumed that the information about the web page includes the identification information of the web page (for example, URL (Uniform Resource Locator), etc.), the display state of the web page (for example, foreground, background, prerendering, etc.), and whether the web page is being displayed online. Also, the CPU 11 may acquire information about the web page by executing an application configured using, for example, JavaScript (registered trademark), etc. Then, every time the CPU 11 acquires information about the web page, it transmits the acquired information to the information processing device 20 via the communication interface unit 19 and the communication network NW. Here, the information transmitted to the information processing device 20 may include the acquisition date and time of the information about the web page and the identification information of the terminal device 10 (for example, the serial number of the terminal device 10, the MAC (Media Access Control) address, etc.).

[0054] On the other hand, every time the CPU 21 of the information processing apparatus 20 receives (acquires) information regarding a web page via the communication interface unit 28, the information regarding the web page is stored in the first acquisition data shown in FIG. 5, for example. The first acquisition data is data in which information regarding a web page (in the example of the figure, the URL of the web page, the acquisition date and time of the information regarding the web page, the display state of the web page, and the connection state of the terminal device 10 (whether the web page is being displayed online)) is associated for each identification information of the terminal device 10 (in the example of the figure, the terminal device ID). Here, as shown in FIG. 5, the acquisition date and time of the information regarding the web page, the display state of the web page, and the connection state of the terminal device 10 may be associated for each URL of the web page.

[0055] The second acquisition means 32 has a function of acquiring information regarding the state of the user of the terminal device 10 measured by a predetermined measuring device 18 when the web page is being displayed on the web browser.

[0056] Here, the information regarding the state of the user may include at least one of the acceleration of the user and the angular velocity of the user. Thereby, based on at least one of the acceleration and the angular velocity of the user when the web page is being displayed on the web browser, the behavior of the user when the web page is being displayed on the web browser can be estimated.

[0057] The function of the second acquisition means 32 is realized as follows, for example. First, when the CPU 11 of the terminal device 10 executes a web browser by a user operation using the input unit 17, for example, every time a predetermined time (for example, 300 milliseconds, 1 second, etc.) elapses, the user's state (for example, at least one of the user's acceleration in the three-axis or two-axis direction and the user's angular velocity in the three-axis or two-axis direction) is acquired. Here, the CPU 11 may acquire information regarding the user's state by operating the measuring device 18 by executing an application configured using, for example, JavaScript (registered trademark). Then, every time the CPU 11 measures information regarding the user's state, the measured information regarding the user's state is transmitted to the information processing device 20 via the communication interface unit 19 and the communication network NW. Here, the information transmitted to the information processing device 20 may include the acquisition (measurement) date and time of the information regarding the user's state and the identification information of the terminal device 10 described above.

[0058] On the other hand, every time the CPU 21 of the information processing device 20 receives (acquires) information regarding the user's state via the communication interface unit 28, the information regarding the user's state is stored, for example, in the second acquisition data shown in FIG. 6. The second acquisition data is data described in a state where the acquisition (measurement) date and time of the information regarding the user's state and the information regarding the user's state are associated with each other for each identification information of the terminal device 10 (in the example of the figure, the terminal device ID).

[0059] The estimation means 33 has a function of estimating the user's behavior when a web page is displayed on the web browser based on the acquired information regarding the user's state.

[0060] Further, the estimation means 33 may estimate the user's behavior based on the acquired information regarding the user's state and a learned model based on machine learning using the information regarding the user's state as learning data. In this case, by using the learned model, the user's behavior when a web page is displayed on the web browser can be easily estimated.

[0061] The function of the estimation means 33 is realized as follows, for example. The CPU 21 of the information processing apparatus 20 extracts, for example, information regarding the state of the user stored in the second acquisition data within a predetermined period (information regarding the transition of the information regarding the state of the user within the predetermined period) from the second acquisition data every time a predetermined period (for example, 3 seconds or the like) elapses starting from a predetermined timing (for example, the timing when information regarding the state of the first user is received (acquired) from the terminal device 10). Then, the CPU 21 estimates the behavior of the user within the predetermined period by inputting the extracted information regarding the state of the user (information regarding the transition of the information regarding the state of the user within the predetermined period) into a learned model based on machine learning using the information regarding the state of the user as learning data.

[0062] An example of the learning data is shown in FIG. 7. The learning data shown in FIG. 7 describes the time transition within a predetermined period of information regarding the state of the user (for example, at least one of the acceleration in the three-axis directions of the user and the angular velocity in the three-axis directions) when the user is performing a predetermined action (in the example of the figure, any one of "stationary", "walking", "ascending / descending stairs", "lying down", etc.) in a state associated with the action (action label). As a result of machine learning, a learned model showing the relationship between the time transition within a predetermined period of information regarding the state of the user performing a predetermined action and the action is configured.

[0063] Note that the data described in the learning data may include data when the same user performs different actions, or may include data when different users perform the same action.

[0064] Further, the CPU 21 may learn a model used to estimate the behavior of the user based on the information regarding the state of the user by machine learning using the information regarding the state of the user as learning data.

[0065] In this case, when a predetermined model learning instruction is input using the input unit 27, for example, the CPU 21 may perform model learning using the learning data shown in FIG. 7. The CPU 21 may learn, for example, using a time series corresponding neural network model. Here, as the time series corresponding neural network, for example, an RNN (Recurrent Neural Network), an LSTM (Long Short-Term Memory) which is an advanced type of RNN, etc. can be applied. Further, the CPU 21 may learn using any one of a plurality of models such as a convolutional neural network (CNN) model, a support vector machine (SVM) model, a fully connected neural network (FNN) model, a gradient boosting (HGB) model, a WaveNet (WN) model, etc.

[0066] In this way, the CPU 21 can estimate the user's behavior (here, any one of "stationary", "walking", "ascending / descending stairs", "lying down", etc.) within the predetermined period by inputting information regarding the transition of information regarding the user's state within the predetermined period into the learned model.

[0067] In the present embodiment, the case where the CPU 21 learns a model provided in the information processing apparatus 20 and estimates the user's behavior using this learned model has been described as an example, but the present invention is not limited to this case. For example, the model used to estimate the user's behavior may be provided in another apparatus other than the information processing apparatus 20. In this case, the CPU 21 may input information regarding the transition of information regarding the user's state within the predetermined period into a learned model provided in another apparatus and receive (acquire) information regarding the behavior estimated by the learned model.

[0068] The storage means 34 has a function of storing in the storage device 24 (a predetermined storage device) in a state where information regarding the Web page and the user's behavior are associated with each other.

[0069] The function of the memory means 34 is realized as follows, for example. When the CPU 21 of the information processing apparatus 20 estimates the actions of the user within a predetermined period based on the function of the estimation means 33, among the information regarding the Web pages corresponding to the user (terminal device ID), the information regarding the Web pages whose acquisition date and time is included within the predetermined period is extracted from the first acquisition data. Then, the CPU 21 stores the information regarding the extracted Web pages and the actions of the user in the first provided data shown in FIG. 8, for example, in an associated state. The first provided data is data described in an associated state of the information regarding the Web pages and the actions of the user for each identification information of the terminal device 10 (in the example of the figure, the terminal device ID). Here, the access period in the first provided data shown in FIG. 8 indicates the period during which the corresponding Web page is continuously displayed on the Web browser executed on the user's terminal device 10. Also, in the example shown in FIG. 8, one display state, connection state, and action are associated with one access period each. However, since at least one of the display state, connection state, and action may change during one access period, a plurality of display states, connection states, and actions may be associated with one access period.

[0070] In this way, based on the first provided data, it becomes possible to grasp how the user is browsing the Web pages while performing what actions. Also, based on the first provided data, it becomes possible to grasp both how the user is browsing the Web pages while performing what actions and in what state the Web pages are being displayed. Furthermore, based on the first provided data, it becomes possible to grasp both how the user is browsing the Web pages while performing what actions and whether the Web pages are being displayed online or not.

[0071] The control means 35 has a function of controlling the operation of the terminal device 10 according to the estimated user behavior. Thereby, for example, when the user behavior of holding the terminal device 10 is estimated, the terminal device 10 is controlled such as activating the camera function of the terminal device 10, so that the terminal device 10 can be appropriately operated according to the user behavior when the web page is displayed on the web browser.

[0072] The function of the control means 35 is realized as follows, for example. When the CPU 21 of the information processing device 20 estimates the user behavior when the web page is displayed on the web browser based on the function of the estimation means 33, it accesses the control data shown in FIG. 8 and extracts the control content corresponding to the estimated behavior. The control data is data in which, for each identification information of the web page (for example, the URL of the web page), the estimated behavior (in the example of the figure, "stationary", "holding the terminal device", etc.) and the control content of the terminal device 10 (in the example of the figure, "starting the map information application", "starting the camera function", etc.) are associated. For example, when the estimated behavior when a predetermined web page is displayed on the web browser of the terminal device 10 is "holding the terminal device", the CPU 21 extracts the control content corresponding to the behavior of "holding the terminal device" (here, "starting the camera function") from the control data. Then, the CPU 21 transmits instruction information (command) for operating the terminal device 10 with the extracted control content (here, "starting the camera function") to the terminal device 10 via the communication interface unit 28 and the communication network NW.

[0073] On the other hand, when the CPU 11 of the terminal device 10 receives the instruction information from the information processing device 20, it operates based on the received instruction information (here, starts the camera function). In this way, the terminal device 10 can be appropriately operated according to the user behavior when the web page is displayed on the web browser.

[0074] The providing means 36 may have a function of providing in a state where information regarding a web page stored in the storage device 24 (a predetermined storage device) is associated with the user's actions.

[0075] The function of the providing means 36 is realized as follows, for example. When a predetermined information providing instruction is input using the input unit 27, the CPU 21 of the information processing apparatus 20 may transmit (provide) the first provided data stored in the storage device 24 to an external device (for example, a computer (such as a server) that generates and provides personalized information, etc.) via the communication interface unit 28 and the communication network NW. Further, when a predetermined information providing instruction is input using the input unit 27, the CPU 21 may display (provide) the first provided data stored in the storage device 24 on the display unit 26. Furthermore, the CPU 21 may convert the first provided data into audio data and output it from an audio output device such as a speaker.

[0076] Also, the providing means 36 may have a function of providing information according to the estimated actions of the user. Thereby, for example, when the action of the user, such as walking, is estimated, appropriate information can be provided according to the actions of the user when the web page is being displayed on the web browser, such as providing advertisement information to the user via the terminal device 10.

[0077] In this case, the function of the providing means 36 is realized as follows, for example. When the CPU 21 of the information processing apparatus 20 estimates the user's behavior based on the function of the estimating means 33 while the web page is being displayed on the web browser, it accesses the second providing data shown in FIG. 10 and extracts the providing information corresponding to the estimated behavior. The second providing data is data in which the estimated behavior (in the example of the figure, "stationary", "walking", etc.) and the providing information (in the example of the figure, "map information", "advertising information", etc.) are associated for each identification information of the web page (for example, the URL of the web page). For example, when the estimated behavior when a predetermined web page is being displayed on the web browser of the terminal device 10 is "walking", the CPU 21 extracts the providing information (here, "advertising information") corresponding to the behavior of "walking" from the second providing data. Then, the CPU 21 transmits the extracted providing information to the terminal device 10 via the communication interface unit 28 and the communication network NW.

[0078] On the other hand, when the CPU 11 of the terminal device 10 receives the providing information, it may display the received providing information on, for example, the display unit 16. Further, when the providing information is composed of audio data, the CPU 11 may output the providing information from an audio output device such as a speaker. In this way, appropriate information can be provided according to the user's behavior when the web page is being displayed on the web browser.

[0079] The changing means 37 has a function of changing the display mode of the web page displayed on the web browser according to the estimated user behavior. Thereby, for example, when the user's behavior changes from stationary to walking, the design, layout, etc. of the web page can be changed, so that it becomes possible to dynamically change the display mode of the web page according to the user's behavior when the web page is being displayed on the web browser.

[0080] The function of the changing means 37 is realized as follows, for example. When the CPU 21 of the information processing apparatus 20 estimates the user's actions based on the function of the estimating means 33 while a web page is being displayed on the web browser, it may acquire the web page being displayed on the web browser on the terminal device 10 from, for example, a web server (not shown), and change the acquired web page according to the estimated actions, such as by changing the design, layout, etc. of the web page. Here, information regarding the design, layout, etc. of the web page corresponding to the estimated user actions may be stored in, for example, the storage device 24. Then, the CPU 21 transmits the changed web page to the terminal device 10 via the communication interface unit 28 and the communication network NW.

[0081] On the other hand, when the CPU 11 of the terminal device 10 receives the changed web page from the information processing apparatus 20, it may cause the display unit 16 to display the changed web page. FIG. 11 shows an example of the display mode of the web page displayed on the terminal device 10. FIG. 11(a) is a diagram showing an example of the layout of the web page displayed on the web browser when the estimated user action is "walking", and FIG. 11(b) is a diagram showing an example of the layout of the web page displayed on the web browser when the estimated user action is "lying down horizontally". For example, when the user's action is estimated to be "walking", as shown in FIG. 11(a), a web page with areas A, B, C, and D where text, videos, etc. are displayed is displayed. Here, area A is displayed at the upper part of the screen, area B is displayed at the lower left part of the screen, and areas C and D are laid out so as to be displayed at the lower right part of the screen. On the other hand, when the user's action is estimated to be "lying down horizontally", as shown in FIG. 11(b), a web page with areas D, C, and A (the information displayed in each area may be the same as that displayed in FIG. 11(a)) is displayed. Here, area D is displayed at the upper part of the screen, area C is displayed at the central part of the screen, and area A is laid out so as to be displayed at the lower part of the screen. In this way, the display mode of the web page can be dynamically changed according to the user's actions when the web page is being displayed on the web browser.

[0082] (5) Flow of Main Processing of Information Processing System of This Embodiment Next, an example of the flow of main processing performed by the information processing system of this embodiment will be described with reference to the flowchart of FIG. 12.

[0083] The CPU 21 of the information processing apparatus 20 acquires information regarding the web page displayed on the web browser executed on the terminal apparatus 10 (step S100). Specifically, first, when the CPU 11 of the terminal apparatus 10 executes the web browser by a user operation using the input unit 17, for example, every time a predetermined time (for example, 10 seconds or the like) elapses, information regarding the web page displayed on the web browser is acquired. Then, every time the CPU 11 acquires information regarding the web page, the acquired information is transmitted to the information processing apparatus 20 via the communication interface unit 19 and the communication network NW.

[0084] On the other hand, every time the CPU 21 of the information processing apparatus 20 receives (acquires) information regarding the web page via the communication interface unit 28, the information regarding the web page is stored in the first acquired data.

[0085] Further, the CPU 21 of the information processing apparatus 20 acquires information regarding the state of the user of the terminal apparatus 10 measured by the measuring apparatus 18 when the web page is displayed on the web browser (step S102). Specifically, first, when the CPU 11 of the terminal apparatus 10 executes the web browser by a user operation using the input unit 17, for example, every time a predetermined time (for example, 300 milliseconds, 1 second, or the like) elapses, the state of the user (for example, at least one of the acceleration of the user in the three-axis or two-axis direction and the angular velocity of the user in the three-axis or two-axis direction) is acquired. Then, every time the CPU 11 measures information regarding the state of the user, the measured information regarding the state of the user is transmitted to the information processing apparatus 20 via the communication interface unit 19 and the communication network NW.

[0086] On the other hand, every time the CPU 21 of the information processing apparatus 20 receives (acquires) information regarding the user's state via the communication interface unit 28, it stores the information regarding the user's state in the second acquired data.

[0087] Next, the CPU 21 of the information processing apparatus 20 estimates the user's behavior when the web page is displayed on the web browser based on the acquired information regarding the user's state (step S104). Specifically, for example, the CPU 21 of the information processing apparatus 20 extracts the information regarding the user's state (information regarding the transition of the information regarding the user's state within a predetermined period) stored in the second acquired data within the predetermined period every time a predetermined period (for example, 3 seconds, etc.) elapses starting from a predetermined timing (for example, the timing when the information regarding the first user's state is received (acquired) from the terminal device 10, etc.). Then, the CPU 21 estimates the user's behavior within the predetermined period by inputting the extracted information regarding the user's state (information regarding the transition of the information regarding the user's state within the predetermined period) into a learned model based on machine learning using the information regarding the user's state as learning data.

[0088] Next, the CPU 21 of the information processing apparatus 20 stores in the storage device 24 the information regarding the web page and the user's behavior in an associated state (step S106). Specifically, when the CPU 21 of the information processing apparatus 20 estimates the user's behavior within a predetermined period in the process of step S104, it extracts from the first acquired data the information regarding the web page whose acquisition date and time is included within the predetermined period among the information regarding the web page corresponding to the user (terminal device ID). Then, the CPU 21 stores in the first provided data the extracted information regarding the web page and the user's behavior in an associated state.

[0089] Note that the CPU 21 of the information processing apparatus 20 may transmit (provide) the first provided data stored in the storage device 24 to an external device (for example, a computer (for example, a server, etc.) that generates and provides personalized information) via the communication interface unit 28 and the communication network NW.

[0090] On the other hand, on the service provider side that generates and provides personalized information, based on the first provided data transmitted from the information processing apparatus 20, it becomes possible to grasp what actions the user is performing while browsing the web page. Further, by analyzing the first provided data and the like, it becomes possible to generate personalized information according to the user's actions.

[0091] As described above, according to the information processing system, information processing method, and program of the present embodiment, since the information related to the web page displayed on the web browser executed on the terminal device 10 and the user's actions when the web page is displayed on the web browser are stored in an associated state, based on the stored information, it becomes possible to grasp what actions the user is performing while browsing the web page. Further, by analyzing the stored information and the like, it becomes possible to generate personalized information according to the user's actions, so that it is possible to provide information applicable to generating personalized information.

[0092] Further, according to the information processing system, information processing method, and program of the present embodiment, since the measuring device 18 is provided in the terminal device 10, it becomes possible to acquire information regarding the state of the user by using the measuring device 18 provided in the terminal device 10.

[0093] Hereinafter, a modification of the above-described embodiment will be described. (Modification example) In the above-described embodiment, the estimation means 33 estimates the user's behavior based on information regarding the user's state and a learned model based on machine learning using information regarding the user's state as learning data. This was described as an example, but the present invention is not limited to this case. For example, the estimation means 33 may estimate that the user is performing a predetermined action when the transition of information regarding the user's state within a predetermined period satisfies a predetermined condition corresponding to the predetermined action. Here, the predetermined condition may be, for example, that the transition of information regarding the user's state within a predetermined period is included in the range of information regarding the user's state corresponding to the predetermined action.

[0094] In this case, when the CPU 21 of the information processing apparatus 20 extracts information regarding the user's state (information regarding the transition of information regarding the user's state within a predetermined period) stored in the second acquisition data from the second acquisition data within a predetermined period, the CPU 21 may estimate the user's behavior with reference to the behavior data shown in FIG. 13. The behavior data describes, for each of a plurality of behaviors (in the example of the figure, "stationary", "walking", "ascending / descending stairs", "lying down", etc.), the time transition of the range of information regarding the user's state (in the example of the figure, the range of three-axis acceleration) when the corresponding behavior is estimated to be being performed, in an associated state. The behavior data may be stored, for example, in the storage device 24.

[0095] For example, when the time transition of the information regarding the user's state extracted from the second acquisition data is included in the range of information regarding the user's state corresponding to any one of the behaviors in the behavior data, the CPU 21 may estimate that the user is performing any one of the behaviors (for example, "stationary").

[0096] As described above, according to the information processing system, information processing method, and program according to this modification example, it is possible to exhibit the same operational effects as those of the above-described embodiment.

[0097] Incidentally, the program of the present invention may be stored in a computer-readable storage medium. The storage medium storing this program may be the ROM 12, RAM 13, or storage device 14 of the terminal device 10 shown in FIG. 2, or may be the ROM 22, RAM 23, or storage device 24 of the information processing device 20 shown in FIG. 3. Further, the storage medium may be a CD-ROM or the like that can be read by being inserted into a program reading device such as a CD-ROM drive. Furthermore, the storage medium may be a magnetic tape, cassette tape, flexible disk, MO / MD / DVD, etc., or may be a semiconductor memory.

[0098] The embodiments and modifications described above are described to facilitate the understanding of the present invention, and are not described to limit the present invention. Therefore, each element disclosed in the above embodiments and modifications is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

[0099] For example, in the above-described embodiment, the case where the measurement device 18 is provided in the terminal device 10 has been described as an example, but the present invention is not limited to this case. The measurement device 18 may be provided, for example, in a state of being worn by the user, or may be provided at a position different from the user and the terminal device 10. Further, when the measurement device 18 is provided in a state of being worn by the user, the acceleration, angular velocity, etc. of at least one body part (for example, arm, leg, etc.) of the user may be acquired as information regarding the state of the user.

[0100] Furthermore, in the above-described embodiment, the case where the behavior of one user is estimated has been described as an example, but the behavior of each of a plurality of users may be estimated. The behavior of each of the plurality of users and information regarding the web page referred to by each of the plurality of users may be stored in the first provided data in an associated state.

[0101] Furthermore, in the above-described embodiments, the case where any one of the actions of "standing still", "walking", "ascending / descending stairs", "lying down", and "holding the terminal above the head" is estimated as the user's action has been described as an example, but the content of the action is not limited to these.

[0102] Also, in the above-described embodiments, the case where one information processing apparatus 20 is provided has been described as an example, but the present invention is not limited to this case. For example, a plurality of information processing apparatuses 20 may be provided. In this case, the operation content and processing results, etc. on any one of the information processing apparatuses 20 may be presented in real time on another information processing apparatus 20, or the processing results, etc. on any one of the information processing apparatuses 20 may be shared among the plurality of information processing apparatuses 20.

[0103] Furthermore, in the above-described embodiments, the configuration is such that the information processing apparatus 20 realizes the functions of the first acquisition means 31, the second acquisition means 32, the estimation means 33, the storage means 34, the control means 35, the provision means 36, and the change means 37, but the present invention is not limited to this configuration. For example, a computer or the like (e.g., a general-purpose personal computer or a server computer) communicably connected to the information processing apparatus 20 via a communication network NW such as the Internet or a LAN may be configured to realize the function of at least one of the above means 31 to 37. Also, as shown in FIGS. 14(a) and (b), each function of the functional block diagram shown in FIG. 4 may be arbitrarily shared between the terminal device 10 and the information processing apparatus 20.

[0104] For example, the CPU 11 of the terminal device 10 may estimate the user's behavior when a web page is displayed on the web browser based on information regarding the user's state. Further, the CPU 11 may learn a model used to estimate the user's behavior based on information regarding the user's state by machine learning using the information regarding the user's state as learning data. In this case, the CPU 11 may perform machine learning using, for example, TensorFlow (registered trademark), which is an open-source machine learning library. Thereby, in the terminal device 10, it becomes possible to perform real-time user behavior estimation.

Industrial Applicability

[0105] The information processing system, information processing method, and program of the present invention as described above can be suitably used for an information providing service or the like that provides personalized information to each user, and thus its industrial applicability is extremely large.

Explanation of Signs

[0106] 10... Terminal device 18... Measuring device 20... Information processing device 31... First acquisition means 32... Second acquisition means 33... Estimation means 34... Storage means 35... Control means 36... Provision means 37... Modification means

Claims

1. A first acquisition means for acquiring information regarding a web page displayed on a web browser being executed on a terminal device; A second acquisition means for acquiring information regarding the state of a user of the terminal device measured by a predetermined measuring device when the web page is being displayed on the web browser; An estimation means for estimating the behavior of the user when the web page is being displayed on the web browser based on the acquired information regarding the state of the user; A storage means for storing, in a predetermined storage device, in an associated state, the information regarding the web page, an access period indicating a period during which the web page is continuously displayed on a web browser being executed on the terminal device, and the behavior of the user; Comprising: The information regarding the web page includes identification information of the web page, an information processing system.

2. The information processing system according to claim 1, further comprising a control means for controlling the operation of the terminal device according to the estimated behavior of the user.

3. The information processing system according to claim 1 or 2, further comprising a providing means for providing information according to the estimated behavior of the user.

4. The information processing system according to any one of claims 1 to 3, further comprising a changing means for changing a display mode of a web page displayed on the web browser according to the estimated behavior of the user.

5. The estimation means estimates the behavior of the user based on the acquired information regarding the state of the user and a learned model based on machine learning using the information regarding the state of the user as learning data. The information processing system according to any one of claims 1 to 4.

6. A first acquisition means for acquiring information regarding a web page displayed on a web browser being executed on a terminal device; A second acquisition means for acquiring information regarding the state of a user of the terminal device measured by a predetermined measuring device when the web page is being displayed on the web browser; An estimation means for estimating the behavior of the user when the web page is being displayed on the web browser based on the acquired information regarding the state of the user; A storage means for storing, in a predetermined storage device, in an associated state, the information regarding the web page and the behavior of the user; Comprising: An information processing system, wherein the information regarding the Web page includes information regarding the display state of the Web page on the terminal device. **Claim 7** First acquisition means for acquiring information regarding a Web page displayed on a Web browser being executed on a terminal device; Second acquisition means for acquiring information regarding the state of a user of the terminal device measured by a predetermined measuring device when the Web page is being displayed on the Web browser; Estimation means for estimating the behavior of the user when the Web page is being displayed on the Web browser based on the acquired information regarding the state of the user; Storage means for storing, in a predetermined storage device, the information regarding the Web page and the behavior of the user in an associated state; comprising An information processing system, wherein the information regarding the Web page includes information regarding whether the Web page is being displayed online. **Claim 8** The information processing system according to any one of claims 1 to 7, wherein the information regarding the state of the user includes at least one of the acceleration of the user and the angular velocity of the user. **Claim 9** The information processing system according to any one of claims 1 to 8, wherein the predetermined measuring device is provided in the terminal device. **Claim 10** A computer performs the steps of: acquiring information regarding a Web page displayed on a Web browser being executed on a terminal device; acquiring information regarding the state of a user of the terminal device measured by a predetermined measuring device when the Web page is being displayed on the Web browser; estimating the behavior of the user when the Web page is being displayed on the Web browser based on the acquired information regarding the state of the user; storing, in a predetermined storage device, the information regarding the Web page, an access period indicating a period during which the Web page is continuously displayed on the Web browser being executed on the terminal device, and the behavior of the user in an associated state; performing each of the steps of An information processing method, wherein the information regarding the Web page includes identification information of the Web page. **Claim 11** A computer is caused to have a function of acquiring information regarding a Web page displayed on a Web browser being executed on a terminal device; A function of acquiring information regarding the state of the user of the terminal device measured by a predetermined measuring device when the Web page is being displayed on the Web browser; A function of estimating the behavior of the user when the Web page is being displayed on the Web browser based on the acquired information regarding the state of the user; A function of storing, in a predetermined storage device, in an associated state, information regarding the Web page, an access period indicating a period during which the Web page is continuously displayed on the Web browser running on the terminal device, and the behavior of the user; A program for realizing the above; The information regarding the Web page includes identification information of the Web page. Program.

Images