Information processing systems, information processing methods, programs

Abstract

To provide an information processing system, an information processing method, and a program which can acquire information applicable to generate personalized information.SOLUTION: An information processing system is configured to estimate a user's operation state for a terminal device. The information processing system comprises: state information acquisition means 32 which acquires information about the user's state measured by a measurement device; and operation state estimation means 35 which estimates the user's operation state for the terminal device in a prescribed period, on the basis of transition of the information about the user's state in the prescribed period.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] In recent years, with the spread of terminal devices such as mobile terminals, functions for collecting information of users of terminal devices and providing personalized information to each user (for example, a search engine function in which the results change depending on the user's search history and location, a recommendation function for displaying recommended products according to browsing history and purchase history, etc.) have been put into practical use.

[0003] For example, in the technology described in Patent Document 1, items related to an item are recommended to a user who has taken an action on the item.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In such information providing services, information such as the user's search history, location, or browsing / purchase history of web pages is analyzed, and personalized information is generated according to the analysis results. On the other hand, in order to generate more personalized information, it is necessary to perform analysis using more appropriate information.

[0006] The present invention has been made in view of the above problems, and an object thereof is to provide an information processing system, an information processing method, and a program capable of obtaining information applicable to generating personalized information.

Means for Solving the Problems

[0007] To solve the above problems, firstly, the present invention provides an information processing system for estimating the operating state of a user with respect to a terminal device, comprising: a state information acquisition means for acquiring information about the user's state measured by a measuring device; and an operating state estimation means for estimating the user's operating state with respect to the terminal device during a predetermined period based on the changes in the information about the user's state during the predetermined period (Invention 1).

[0008] According to this invention (Invention 1), the user's operating state with respect to the terminal device during a predetermined period (for example, how often the terminal device is operated or is operable) is estimated based on information about the user's state during a predetermined period. Therefore, by analyzing the estimated user's operating state, for example, it becomes possible to generate personalized information according to the user's operating state. This makes it possible to obtain information (user's operating state) that can be applied to generating personalized information.

[0009] In the above invention (Invention 1), the operation state estimation means may estimate the user's operation state to the terminal device during the predetermined period based on the changes in information regarding the user's state during the predetermined period and a trained model based on machine learning that uses the changes in information regarding the user's state as training data (Invention 2).

[0010] According to this invention (Invention 2), by using a trained model, the user's operating state on the terminal device within a predetermined period can be easily estimated.

[0011] In the above inventions (Inventions 1-2), the system includes behavior estimation means for estimating the user's behavior pattern during a predetermined period based on the changes in information regarding the user's state during that period, and the operation state estimation means may estimate the user's operation state with respect to the terminal device during that predetermined period based on the user's behavior pattern during that period (Invention 3).

[0012] According to this invention (Invention 3), it becomes possible to estimate the user's operating state on the terminal device during a predetermined period of time using the user's behavior pattern estimated based on information about the user's state during that period of time.

[0013] In the above invention (Invention 3), the behavior estimation means may estimate the user's behavior pattern during the predetermined period based on the changes in information regarding the user's state during the predetermined period and a trained model based on machine learning that uses the changes in information regarding the user's state as training data (Invention 4).

[0014] According to this invention (Invention 4), by using a trained model, it is possible to easily estimate the user's behavioral patterns within a predetermined period.

[0015] In the above inventions (inventions 3 to 4), a detection means is provided for detecting a state change point that indicates the timing at which the user's operating state to the terminal device changes, based on information about the user's state, and the behavior estimation means may estimate the user's behavior pattern during the period from a first state change point to a second state change point, which is the state change point following the first state change point (invention 5).

[0016] According to this invention (Invention 5), the user behavior pattern estimation process is started when a state change point is detected, and the user behavior pattern estimation process is not started at any other time. Therefore, compared to, for example, a case where the user behavior pattern estimation process is always started, it is possible to reduce the load on the user behavior pattern estimation process.

[0017] In the above inventions (inventions 1 to 5), a detection means is provided for detecting a state change point that indicates the timing at which the user's operating state to the terminal device changes, based on information regarding the user's state, and the operating state estimation means may estimate the user's operating state to the terminal device during the period from a first state change point to a second state change point, which is the next state change point after the first state change point (invention 6).

[0018] According to this invention (Invention 6), the user operation state estimation process is started when a state change point is detected, and the user operation state estimation process is not started at any other time. Therefore, compared to a case where the user operation state estimation process is always started, for example, it is possible to reduce the load on the user operation state estimation process.

[0019] In the above inventions (inventions 5-6), the detection means may detect the state change point by determining whether the predetermined timing is the state change point when the information regarding the user's state at the predetermined timing satisfies predetermined conditions (invention 7).

[0020] According to this invention (Invention 7), since the state change point detection process is performed when information regarding the user's state satisfies predetermined conditions, it becomes possible to reduce the load on the state change point detection process compared to, for example, when the state change point detection process is performed continuously.

[0021] In the above invention (Invention 7), the predetermined condition may include the fact that the derivative of the curve showing the change in information regarding the user's state is 0 (Invention 8).

[0022] According to this invention (Invention 8), when the derivative of the curve showing the transition of information about the user's state is 0, it becomes possible to perform a state change point detection process.

[0023] In the above inventions (Inventions 7 to 8), the detection means may determine that the predetermined timing is the state change point when the change point score indicating the magnitude of the change in the information regarding the state of the user at the predetermined timing is equal to or greater than a predetermined value (Invention 9).

[0024] According to such an invention (Invention 9), by using the change point score, it becomes possible to easily detect the state change point.

[0025] In the above inventions (Inventions 1 to 9), control means for controlling the operation of the terminal device according to the estimated operation state of the user on the terminal device within the predetermined period may be provided (Invention 10).

[0026] According to such an invention (Invention 10), for example, by controlling the terminal device, such as reducing the brightness of the display unit to suppress the power consumption of the terminal device when the user does not operate the terminal device very much, it becomes possible to appropriately operate the terminal device according to the operation state of the user.

[0027] In the above inventions (Inventions 1 to 10), providing means for providing information according to the estimated operation state of the user on the terminal device within the predetermined period may be provided (Invention 11).

[0028] According to such an invention (Invention 11), for example, by providing appropriate information according to the operation state of the user, such as providing advertisement information to the user via the terminal device when the user operates the terminal device frequently, it becomes possible.

[0029] In the above inventions (inventions 1 to 11), the device is provided with page information acquisition means for acquiring information about a web page displayed in a web browser running on the terminal device, the state information acquisition means for acquiring information about the user's state measured by the measuring device when the web page is displayed in the web browser, and the operation state estimation means for estimating the user's operation state to the terminal device during the predetermined period when the web page is displayed in the web browser (invention 12).

[0030] According to this invention (Invention 12), it becomes possible to estimate the user's operating state when a predetermined web page is displayed in a web browser running on a terminal device.

[0031] In the above inventions (inventions 1 to 12), a means for changing the display mode of the web page displayed in the web browser may be provided according to the user's operation state on the terminal device during the estimated predetermined period (invention 13).

[0032] According to this invention (Invention 13), for example, it becomes possible to dynamically change the design, layout, etc., of a web page in accordance with the user's operation state when the web page is displayed in a web browser.

[0033] In the above inventions (inventions 1 to 13), the information relating to the user's state may include at least one of the user's acceleration and the user's angular velocity (invention 14).

[0034] According to this invention (Invention 14), the user's operating state can be estimated based on at least one of the user's acceleration and angular velocity.

[0035] In the above inventions (inventions 1 to 14), the user's operating state may include at least one of the user's touch state, scroll state, swipe state, flick state, pinch state, and drag state on the terminal device (invention 15).

[0036] According to this invention (Invention 15), it becomes possible to estimate at least one of the following based on information about the user's state: touch state (e.g., the number of touch operations and the duration of touch operations), scroll state (e.g., the number of scroll operations and the duration of scroll operations), swipe state (e.g., the number of swipe operations and the duration of swipe operations), flick state (e.g., the number of flick operations and the duration of flick operations), pinch state (e.g., the number of pinch-in and / or pinch-out operations and the duration of pinch-in and / or pinch-out operations), and drag state (e.g., the number of drag operations and the duration of drag operations).

[0037] In the above inventions (inventions 1 to 15), the measuring device may be provided in the terminal device (invention 16).

[0038] According to this invention (Invention 16), it becomes possible to acquire information about the user's state using a measuring device provided in the terminal device.

[0039] Secondly, the present invention provides an information processing method for estimating the user's operating state with respect to a terminal device using a computer, wherein the computer performs the following steps: acquiring information about the user's state measured by a measuring device; and estimating the user's operating state with respect to the terminal device during a predetermined period based on the changes in the information about the user's state during the predetermined period (Invention 17).

[0040] Thirdly, the present invention provides a program for a computer to estimate the user's operating state with respect to a terminal device, the program to enable the computer to perform the functions of acquiring information about the user's state measured by a measuring device and estimating the user's operating state with respect to the terminal device during a predetermined period based on the changes in the information about the user's state during the predetermined period (Invention 18). [Effects of the Invention]

[0041] According to the information processing system, information processing method, and program of the present invention, it is possible to obtain information applicable to generating personalized information. [Brief explanation of the drawing]

[0042] [Figure 1] This diagram schematically shows the basic configuration of an information processing system according to one embodiment of the present invention. [Figure 2] This is a block diagram showing the configuration of a terminal device. [Figure 3] This is a block diagram showing the configuration of an information processing device. [Figure 4] This is a functional block diagram used to explain the functions that play a major role in an information processing system. [Figure 5] This figure shows an example of the structure of the first acquired data. [Figure 6] This figure shows an example of the structure of the second set of acquired data. [Figure 7] This figure shows an example of how a user's acceleration and change point score change over time. [Figure 8] This figure shows an example of the structure of the first training data. [Figure 9] This figure shows an example of the structure of the second training data. [Figure 10] This figure shows an example of how estimated data can be structured. [Figure 11] This figure shows an example of the control data structure. [Figure 12] This figure shows an example of the structure of the provided data. [Figure 13] (a) and (b) are diagrams showing examples of web page layouts displayed in a web browser according to the estimated user state. [Figure 14] A flowchart showing an example of the main processing steps of an information processing system according to one embodiment of the present invention. [Figure 15] This figure shows an example of the configuration of operation status data. [Figure 16] This figure shows an example of the structure of the third training data set. [Figure 17] This diagram shows an example of the division of responsibilities between terminal devices and information processing devices for each function of an information processing system. [Modes for carrying out the invention]

[0043] One embodiment of the present invention will be described in detail below with reference to the accompanying drawings. However, this embodiment is illustrative and the present invention is not limited thereto.

[0044] (1) Basic configuration of the information processing system Figure 1 is a schematic diagram showing the basic configuration of an information processing system according to one embodiment of the present invention. As shown in Figure 1, the information processing system according to this embodiment is configured to estimate the user's operating state to the terminal device 10. The information processing device 20 acquires information about the user's state measured by a predetermined measuring device 18 (shown in Figure 2), and estimates the user's operating state to the terminal device 10 within a predetermined period based on the changes in the information about the user's state within a predetermined period. The terminal device 10 and the information processing device 20 are connected to a communication network NW (network), such as the Internet or a LAN (Local Area Network).

[0045] The terminal device 10 may be, for example, a device that can be worn by a user (e.g., a wearable device) or a portable device that a user can carry. Furthermore, the terminal device 10 may be a device operated by an individual user, such as a mobile terminal, smartphone, PDA (Personal Digital Assistant), personal computer, or television receiver with two-way communication capabilities (including so-called multi-functional smart TVs). Additionally, the terminal device 10 may be a shared device that can be operated by multiple users.

[0046] The information processing device 20 is a device for obtaining information applicable to generating personalized information. The information processing device 20 may be a device operated by individual users, similar to the terminal device 10.

[0047] (2) Configuration of terminal device The configuration of the terminal device 10 will be described with reference to Figure 2. Figure 2 is a block diagram showing the internal configuration of the terminal device 10. As shown in Figure 2, the terminal device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage device 14, a display processing unit 15, a display unit 16, an input unit 17, a measuring device 18, and a communication interface unit 19, and is provided with a bus 10a for transmitting control signals or data signals between each unit.

[0048] When power is supplied to the terminal device 10, the CPU 11 loads various programs stored in the ROM 12 or storage device 14 into the RAM 13 and executes them.

[0049] The storage device 14 may be a non-volatile storage device such as flash memory, SSD (Solid State Drive), magnetic storage device (e.g., HDD (Hard Disk Drive), floppy disk (registered trademark), magnetic tape, etc.), or optical disk, or it may be a volatile storage device such as RAM, and it stores programs executed by the CPU 11 and data referenced by the CPU 11.

[0050] The display processing unit 15 displays the display data provided by the CPU 11 on the display unit 16. The display unit 16 is, for example, an LCD (Liquid Crystal Display) monitor containing thin-film transistors arranged in a matrix on a pixel-by-pixel basis, and displays the data to be displayed on the display screen by driving the thin-film transistors based on the display data.

[0051] If the terminal device 10 is a button-input type communication device, the input unit 17 includes a group of buttons including a plurality of instruction input buttons such as directional buttons and select buttons for receiving user operation input, and a group of buttons including a plurality of instruction input buttons such as a numeric keypad, and includes an interface circuit for recognizing the press (operation) input of each button and outputting it to the CPU 11.

[0052] If the terminal device 10 is a communication device with a touch panel input method, the input unit 17 primarily accepts input via a touch panel method, such as touching the display screen with a fingertip or pen. The touch panel input method may be a known method such as a capacitive touch method.

[0053] Furthermore, if the terminal device 10 is a device capable of voice input, the input unit 17 may be configured to include a microphone for voice input, or it may include an interface circuit for outputting voice data input via an external microphone to the CPU 11. In addition, if the terminal device 10 is a communication device capable of inputting moving images and / or still images, the input unit 17 may be configured to include a digital camera or digital video camera for image input, or it may include an interface circuit for receiving image data captured by an external digital camera or digital video camera and outputting it to the CPU 11.

[0054] The measuring device 18 is a device that measures the user status of the terminal device 10 continuously or intermittently (for example, at predetermined intervals (for example, every 200 milliseconds)). Here, the information regarding the user status may be, for example, a value representing the user status, a value obtained by substituting a value representing the user status into a predetermined calculation formula, or information representing the degree of the user status.

[0055] The measuring device 18 may be, for example, a device that measures the user's state (e.g., acceleration in three or two axes, angular velocity in three or two axes, heart rate (pulse), blood pressure, body temperature, amount of sweat, number of steps, walking speed, posture, exercise intensity (e.g., heart rate ÷ maximum heart rate), or calories burned, etc.) (e.g., an acceleration sensor, gyro sensor, motion sensor, heart rate monitor, blood pressure monitor, thermometer, sweat meter, etc.).

[0056] The communication interface unit 19 includes an interface circuit for communicating with other devices (e.g., an information processing device 20) via a communication network NW.

[0057] (3) Configuration of the information processing device The configuration of the information processing device 20 will be described with reference to Figure 3. Figure 3 is a block diagram showing the internal configuration of the information processing device 20. As shown in Figure 3, the information processing device 20 comprises a CPU 21, a ROM 22, a RAM 23, a storage device 24, a display processing unit 25, a display unit 26, an input unit 27, and a communication interface unit 28, and is provided with a bus 20a for transmitting control signals or data signals between each unit.

[0058] When power is supplied to the information processing device 20, the CPU 21 loads various programs stored in the ROM 22 or storage device 24 into the RAM 23 and executes them. In this embodiment, the CPU 21 reads and executes programs stored in the ROM 22 or storage device 24 to realize the functions of the page information acquisition means 31, state information acquisition means 32, detection means 33, action estimation means 34, operation state estimation means 35, control means 36, provision means 37, and modification means 38 (shown in Figure 4), which will be described later.

[0059] The storage device 24 may be a non-volatile storage device such as flash memory, SSD, magnetic storage device (e.g., HDD, floppy disk (registered trademark), magnetic tape, etc.), or optical disk, or it may be a volatile storage device such as RAM, and it stores programs executed by the CPU 21 and data referenced by the CPU 21. The storage device 24 also stores the first acquired data (shown in Figure 5), second acquired data (shown in Figure 6), first learning data (shown in Figure 8), second learning data (shown in Figure 9), estimation data (shown in Figure 10), control data (shown in Figure 11), and provided data (shown in Figure 12), which will be described later.

[0060] The input unit 27 may be an information input device such as a mouse or keyboard, or it may be configured to include a microphone for voice input, or it may be configured to include a digital camera or digital video camera for image input, or it may include an interface circuit for receiving image data captured by an external digital camera or digital video camera and outputting it to the CPU 21.

[0061] The communication interface unit 28 includes an interface circuit for performing communication via a communication network NW. Details of other parts within the information processing device 20 may be the same as those of the terminal device 10.

[0062] (4) Overview of each function in the information processing system The functions realized in the information processing system of this embodiment will be described with reference to Figure 4. Figure 4 is a functional block diagram illustrating the functions that play a major role in the information processing system of this embodiment. In the functional block diagram of Figure 4, the state information acquisition means 32 and the operation state estimation means 35 correspond to the main components of the information processing system of the present invention. Other means (page information acquisition means 31, detection means 33, action estimation means 34, control means 36, provision means 37, and modification means 38) are not necessarily essential components, but they are elements that make the present invention even more preferable.

[0063] The page information acquisition means 31 has the function of acquiring information about a web page displayed in a web browser running on the terminal device 10.

[0064] Here, information about the web page may include web page identification information (e.g., URL (Uniform Resource Locator)) or information about the display state of the web page on the terminal device 10. Furthermore, information about the display state of the web page may include, for example, the state in which the web page is displayed on the screen (display unit 16) of the terminal device 10 (foreground), the state in which the web page is not displayed on the screen (display unit 16) of the terminal device 10 (background), the state in which the web page is pre-rendered (pre-rendering), etc. This makes it possible to understand not only the user's operating state when viewing (displaying) the web page, but also the state in which the web page is displayed. By analyzing the estimated user's operating state, it is possible to generate more personalized information according to the user's operating state.

[0065] Furthermore, information about a web page may include information about whether or not the web page is displayed online. This makes it possible to understand not only the user's browsing state but also whether or not the web page is displayed online. By analyzing the estimated user state, it becomes possible to generate more personalized information according to the user's state.

[0066] The function of the page information acquisition means 31 is implemented as follows, for example. First, the CPU 11 of the terminal device 10 acquires information about the web page displayed in the web browser at predetermined intervals (for example, 10 seconds) when the user is operating a web browser (for example, Google Chrome®) using the input unit 17. Here, it is assumed that the information about the web page includes web page identification information (for example, URL), web page display status (for example, foreground, background, pre-rendered, etc.), and whether or not the web page is displayed online. The CPU 11 may also acquire information about the web page by executing an application configured using, for example, JavaScript®. Each time the CPU 11 acquires information about the web page, it transmits the acquired information to the information processing device 20 via the communication interface unit 19 and the communication network NW. Here, the information transmitted to the information processing device 20 may include the date and time the web page information was acquired and the identification information of the terminal device 10 (for example, the serial number or MAC (Media Access Control) address of the terminal device 10).

[0067] On the other hand, the CPU 21 of the information processing device 20 stores the information about the web page in the first acquired data shown in Figure 5, for example, each time it receives (acquires) information about a web page via the communication interface unit 28. The first acquired data is data that associates information about a web page (in the example shown in the figure, the URL of the web page, the date and time the information about the web page was acquired, the display status of the web page, and the connection status of the terminal device 10 (whether or not the web page is displayed online)) with the identification information of the terminal device 10 (in the example shown in the figure, the terminal device ID). Here, as shown in Figure 5, the date and time the information about the web page was acquired, the display status of the web page, and the connection status of the terminal device 10 may be associated with each web page URL.

[0068] The status information acquisition means 32 has the function of acquiring information regarding the user status of the terminal device 10 as measured by the measuring device 18.

[0069] Furthermore, the status information acquisition means 32 may acquire information regarding the user's status measured by the measuring device 18 when the web page is displayed in the web browser.

[0070] Here, information regarding the user's state may include at least one of the user's acceleration and the user's angular velocity. This allows the user's operating state to be estimated based on at least one of the user's acceleration and angular velocity.

[0071] The function of the state information acquisition means 32 is implemented as follows, for example. Here, we will explain as an example the case in which the state information acquisition means 32 acquires information about the user's state measured by the measuring device 18 when a web page is displayed in a web browser. First, the CPU 11 of the terminal device 10 acquires the user's state (for example, at least one of the user's acceleration in the three-axis or two-axis direction and the user's angular velocity in the three-axis or two-axis direction) at predetermined intervals (for example, 200 milliseconds) when the web browser is being executed by user operation using the input unit 17. Here, the CPU 11 may acquire information about the user's state by running an application configured using, for example, JavaScript (registered trademark), to operate the measuring device 18. Then, each time the CPU 11 measures information about the user's state, it transmits the measured information about the user's state to the information processing device 20 via the communication interface unit 19 and the communication network NW. Here, the information transmitted to the information processing device 20 may include the date and time of acquisition (measurement) of the information about the user's state and the identification information of the terminal device 10 described above.

[0072] Meanwhile, the CPU 21 of the information processing device 20 stores the user status information in the second acquired data shown in Figure 6, for example, each time it receives (acquires) information about the user status via the communication interface unit 28. The second acquired data is data that associates the date and time of acquisition (measurement) of the user status information with the user status information for each identification information of the terminal device 10 (in the example shown in the figure, the terminal device ID).

[0073] The detection means 33 has a function to detect state change points that indicate the timing at which the user's operating state with respect to the terminal device 10 changes, based on information about the user's state.

[0074] Furthermore, the detection means 33 may detect a state change point by determining whether a predetermined timing is a state change point when the information regarding the user's state at that timing satisfies predetermined conditions. As a result, since the state change point detection process is performed at the timing when the information regarding the user's state satisfies predetermined conditions, it is possible to reduce the load on the state change point detection process compared to, for example, when the state change point detection process is performed continuously.

[0075] Here, the predetermined condition may include the fact that the derivative of the curve showing the transition of information about the user's state is 0. This makes it possible to perform state change point detection processing when the derivative of the curve showing the transition of information about the user's state is 0. The predetermined condition may also include, for example, the fact that the absolute value of the average change in information about the user's state (e.g., one or more of acceleration and angular velocity) within a certain period including a predetermined timing is less than a predetermined value.

[0076] Furthermore, the detection means 33 may determine that a predetermined timing is a state change point if the change point score, which indicates the magnitude of the change in information regarding the user's state at that predetermined timing, is greater than or equal to a predetermined value. This makes it possible to easily detect state change points by using the change point score.

[0077] The function of the detection means 33 is realized, for example, as follows. Here, as an example, we will explain the case in which the predetermined timing is detected as a state change point when the derivative of the curve showing the transition of information about the user's state at a predetermined timing is 0, and the change point score of the information about the user's state at that predetermined timing is greater than or equal to a predetermined value.

[0078] The CPU 21 of the information processing device 20 extracts information about the user's state stored in the second acquired data (information about the changes in the user's state information within the predetermined period) from the second acquired data every predetermined period (e.g., 3 seconds) starting from a predetermined timing (e.g., the timing when the initial user status information is received (acquired) from the terminal device 10). Next, the CPU 21 uses the extracted user status information within the predetermined period to calculate a quadratic curve representing the change in the user's state information over time within the predetermined period. Then, the CPU 21 extracts the time within the predetermined period (the date and time when the user status information is acquired (measured)) when the derivative value of the calculated quadratic curve becomes 0 (this is an example of a "predetermined timing" in the present invention).

[0079] Next, the CPU 21 of the information processing device 20 determines whether each extracted time point is a state change point. Here, the CPU 21 may determine the state change point using a well-known detection method for state change points. Here, as an example of a detection method, the case using Change Finder will be described. Change Finder has features such as shorter computation time, the ability to handle non-stationary data, and suitability for online processing compared to conventional detection methods. For each extracted time point, the CPU 21 calculates a change point score using Change Finder, and detects the time when this change point score changes from below a predetermined threshold Th to above the threshold Th as a state change point. Figure 7 shows an example of the change over time of acceleration, which is an example of information about the user's state, and the change point score. In the example shown in Figure 7, the times t1 and t2 when the change point score changes from below the threshold Th (20 in the example) to above the threshold Th are detected as state change points.

[0080] The behavior estimation means 34 has the function of estimating the user's behavior pattern during a predetermined period based on the changes in information regarding the user's state during that period.

[0081] Furthermore, the behavior estimation means 34 may estimate the user's behavior pattern within a predetermined period based on the changes in information regarding the user's state within that period and a trained model based on machine learning that uses the changes in information regarding the user's state as training data. In this case, the user's behavior pattern within the predetermined period can be easily estimated by using the trained model.

[0082] Furthermore, the behavior estimation means 34 may estimate the user's behavior pattern during the period from the first state change point to the second state change point, which is the next state change point after the first state change point. In this case, the user behavior pattern estimation process is started when a state change point is detected, and not started at any other time. This reduces the load on the user behavior pattern estimation process compared to, for example, a case where the user behavior pattern estimation process is always running.

[0083] The function of the behavior estimation means 34 is implemented as follows, for example. Here, we will explain as an example the case in which the behavior estimation means 34 estimates the user's behavior pattern during the period from a first state change point (for example, time t1 shown in Figure 7) to a second state change point (for example, time t2 shown in Figure 7), which is the next state change point after the first state change point, using a trained model. The CPU 21 of the information processing device 20 accesses the second acquired data, for example, and extracts information about the user's state acquired during the period from the time detected as the first state change point (the date and time of acquisition (measurement) of information about the user's state) to the time detected as the second state change point. The CPU 21 then inputs the extracted information about the user's state (information about the transition of information about the user's state during a predetermined period) into a first trained model based on machine learning that uses information about the user's state as first training data, thereby estimating the user's behavior pattern during that period.

[0084] An example of the first training data is shown in Figure 8. The first training data shown in Figure 8 is data that describes the time course of information about the user's state (for example, at least one of the user's acceleration in the three axes and angular velocity in the three axes) when the user is performing a predetermined action pattern (in the example in the figure, any of the following: "lie down" → "lie down" → "stay" → "lie down" → "stay", "stay" → "stay" → "walk" → "stay" → "stay", "walk" → "climb stairs" → "walk" → "stay" → "walk" → "walk" etc.), and associates this information with the action pattern (action pattern label). As a result of machine learning, a trained model is constructed that shows the time course of information about the user's state when performing a predetermined action pattern and the relationship with that action pattern. Note that in the example shown in Figure 8, the action pattern is configured to include multiple actions, but the action pattern may be configured to include only one action (for example, "walk").

[0085] The data described in the first training data may include data from cases where the same user performs different behavioral patterns, or data from cases where different users perform the same behavioral patterns.

[0086] Alternatively, the CPU21 may learn a model (first trained model) used to estimate the user's behavior patterns based on information about the user's state, by machine learning using information about the user's state as the first training data.

[0087] In this case, the CPU 21 may, for example, train a model using the first training data shown in Figure 8 when a predetermined model training instruction is input via the input unit 27. The CPU 21 may, for example, train using a time-series-responsive neural network model. Here, as the time-series-responsive neural network, for example, an RNN (Recurrent Neural Network) or an advanced version of RNN such as LSTM (Long Short-Term Memory) can be applied. The CPU 21 may also train using any of several models, such as a graph neural network (GNN) model, a convolutional neural network (CNN) model, a support vector machine (SVM) model, a fully connected neural network (FNN) model, a gradient boosting (HGB) model, or a wavenet (WN) model. Furthermore, the CPU 21 may train using any of the derivatives of GNN, such as a graph convolutional neural network (GCN) model, a graph attention network (GAT) model, or a graph convolutional LSTM (GC-LSTM) model.

[0088] In this way, the CPU 21 can estimate the user's behavior pattern during a given period by inputting information about the transition of information regarding the user's state during the period from the first state change point to the second state change point into the first trained model. Furthermore, in this embodiment, for example, each time information regarding the user's state is newly stored in the second acquired data, it is not necessary to estimate the user's behavior pattern for a predetermined period from the time the information is newly stored, thus reducing the load on the user behavior pattern estimation process.

[0089] In this embodiment, the CPU 21 learns a model (first trained model) provided in the information processing device 20 and uses this first trained model to estimate the user's behavior pattern as an example, but the present invention is not limited to this case. For example, the first trained model used to estimate the user's behavior pattern may be provided in a device other than the information processing device 20. In this case, the CPU 21 may input information regarding the transition of information about the user's state during the period from the first state change point to the second state change point to the first trained model provided in the other device, and receive (acquire) the behavior pattern estimated by the first trained model.

[0090] The operation state estimation means 35 has a function to estimate the user's operation state with respect to the terminal device 10 during a predetermined period, based on the changes in information regarding the user's state during that period.

[0091] Furthermore, the operation state estimation means 35 may estimate the user's operation state of the terminal device 10 during a predetermined period based on the changes in information regarding the user's state during that period and a trained model based on machine learning that uses the changes in information regarding the user's state as training data. In this case, by using a trained model, the user's operation state of the terminal device 10 during the predetermined period can be easily estimated.

[0092] Furthermore, the operation state estimation means 35 may estimate the user's operation state of the terminal device 10 during a predetermined period based on the user's behavior pattern during that period. This makes it possible to estimate the user's operation state of the terminal device 10 during a predetermined period using the user's behavior pattern estimated based on information about the user's state during that period.

[0093] Furthermore, the operation state estimation means 35 may also estimate the user's operation state with respect to the terminal device 10 during a predetermined period while the web page is displayed in the web browser. This makes it possible to estimate the user's operation state when a predetermined web page is displayed in the web browser running on the terminal device 10.

[0094] Furthermore, the operation state estimation means 35 may estimate the user's operation state with respect to the terminal device 10 during the period from the first state change point to the second state change point, which is the next state change point after the first state change point. In this case, the user operation state estimation process is started when a state change point is detected, and is not started at any other time. This reduces the load on the user operation state estimation process compared to, for example, a case where the user operation state estimation process is always started.

[0095] Furthermore, the user's operating state may include at least one of the user's touch state, scroll state, swipe state, flick state, pinch state, and drag state on the terminal device 10. This makes it possible to estimate at least one of the user's touch state (e.g., the number of touch operations and the duration of touch operations), scroll state (e.g., the number of scroll operations and the duration of scroll operations), swipe state (e.g., the number of swipe operations and the duration of swipe operations), flick state (e.g., the number of flick operations and the duration of flick operations), pinch state (e.g., the number of pinch-in and / or pinch-out operations and the duration of pinch-in and / or pinch-out operations), and drag state (e.g., the number of drag operations and the duration of drag operations) based on information about the user's state.

[0096] The function of the operation state estimation means 35 is implemented as follows, for example. Here, we will explain as an example the case in which the operation state estimation means 35 estimates the user's operation state during the period from a first state change point (for example, time t1 shown in Figure 7) to a second state change point (for example, time t2 shown in Figure 7), which is the next state change point after the first state change point, using a trained model. The CPU 21 of the information processing device 20 estimates the user's behavior pattern during the period from the first state change point to the second state change point, based on the function of the behavior estimation means 34 described above, and then estimates the user's operation state during that period by inputting the estimated behavior pattern into a second trained model based on machine learning that uses the user's behavior pattern as second training data.

[0097] An example of the second learning data is shown in FIG. 9. The second learning data shown in FIG. 9 is data that describes a predetermined operation state (operation state label) of the user with respect to the terminal device 10 (in the example of the figure, "the number of touches is X0 times (X0>0) or more", "the number of touches is X1 times (0<X1<X0) or more and less than X0 times", "the number of touches is X2 times (0<X2<X1) or more and less than X1 times", "the number of touches is X3 times (0<X3<X2) or more and less than X2 times", etc.) for the display unit 16 of the terminal device 10) in a state associated with the user's behavior pattern in that operation state (in the example of the figure, "lying down" → "lying down" → "stationary" → "lying down" → "stationary", "stationary" → "stationary" → "walking" → "stationary" → "stationary", "walking" → "climbing stairs" → "walking" → "stationary", "walking" → "walking" → "stationary" → "walking" → "walking", etc.). As a result of machine learning, a learned model showing the relationship between the user's behavior pattern in a predetermined operation state and the operation state is configured.

[0098] In the example shown in FIG. 9, the operation state is indicated by the number of touches on the display unit 16 of the terminal device 10, but the operation state may also be indicated by the touch time on the display unit 16. Further, the operation state may be indicated using at least one of the touch state (for example, the number of touch operations or the touch operation time, etc.), the scroll state (for example, the number of scroll operations or the scroll operation time, etc.), the swipe state (for example, the number of swipe operations or the swipe operation time, etc.), the flick state (for example, the number of flick operations or the flick operation time, etc.), the pinch state (for example, the number of pinch-in and / or pinch-out operations or the pinch-in and / or pinch-out operation time, etc.), and the drag state (for example, the number of drag operations or the drag operation time, etc.). Furthermore, the operation state may be indicated by information representing the degree of operation on the terminal device 10 (for example, "operated frequently (the number of operations such as touches is very large or the operation time is very long)", "operated often (the number of operations such as touches is large or the operation time is long)", "not operated much (the number of operations such as touches is small or the operation time is short)", "hardly operated (the number of operations such as touches is very small or the operation time is very short)", etc.).

[0099] Furthermore, the data described in the second training data may include operational states when the same user performs different behavioral patterns, or it may include operational states when different users perform the same behavioral patterns.

[0100] Alternatively, the CPU 21 may learn a model (second pre-trained model) used to estimate the user's operational state based on behavioral patterns, by machine learning using behavioral patterns as second training data.

[0101] In this case, the CPU 21 may, for example, train a model using the second training data shown in Figure 9 when a predetermined model training instruction is input via the input unit 27. The CPU 21 may, for example, train using any of several models, such as a time-series neural network model, GNN model, CNN model, SVM model, FNN model, HGB model, WN model, GCN model, GAT model, or convolutional LSTM (GC-LSTM) model, similar to the first trained model described above.

[0102] In this way, the CPU 21 can estimate the user's operating state during the period from the first state change point to the second state change point by inputting the user's behavior pattern during that period into the second trained model. Furthermore, in this embodiment, for example, each time information about the user's state is newly stored in the second acquired data, it is not necessary to estimate the user's operating state for a predetermined period from the time the information was newly stored, thus reducing the load on the user's operating state estimation process.

[0103] In this embodiment, the CPU 21 learns a model (second trained model) provided in the information processing device 20 and uses this second trained model to estimate the user's behavior pattern as an example, but the present invention is not limited to this case. For example, the second trained model used to estimate the user's operating state may be provided in a device other than the information processing device 20. In this case, the CPU 21 may input the user's behavior pattern during the period from the first state change point to the second state change point to the second trained model provided in the other device, and receive (acquire) the user's operating state estimated by the second trained model.

[0104] Furthermore, the CPU 21 may estimate the user's operating state within a predetermined period and store the information about the web page and the user's operating state in association. In this case, when the CPU 21 estimates the user's behavior within a predetermined period, it extracts information about web pages whose acquisition date and time fall within that predetermined period from the first acquired data, among the information about web pages corresponding to the user (terminal device ID). The CPU 21 then stores the extracted information about the web page and the user's operating state in association in estimated data, for example, as shown in Figure 10. The estimated data is data in which information about the web page and the user's operating state are associated for each identification information of the terminal device 10 (in the example shown in the figure, the terminal device ID). Here, the access period in the estimated data shown in Figure 10 indicates the period during which the corresponding web page is continuously displayed in the web browser running on the user's terminal device 10. Furthermore, in the example shown in Figure 10, one display state, connection state, and operation state are associated with each access period. However, since at least one of the display state, connection state, and operation state may change during a single access period, multiple display states, connection states, and operation states may be associated with each access period.

[0105] In this case, based on the estimated data, it becomes possible to understand the user's operating state when a predetermined web page is displayed in the web browser running on the terminal device 10. Furthermore, based on the estimated data, it becomes possible to understand not only the operating state in which the user is viewing the web page, but also the state in which the web page is displayed. In addition, based on the estimated data, it becomes possible to understand not only the operating state in which the user is viewing the web page, but also whether or not the web page is displayed online.

[0106] The control means 36 has a function to control the operation of the terminal device 10 according to the user's operation status of the terminal device 10 within an estimated predetermined period. This makes it possible to operate the terminal device 10 appropriately according to the user's operation status by controlling the terminal device 10, for example, by reducing the brightness of the display unit 16 to reduce the power consumption of the terminal device 10 when the user is not operating the terminal device 10 much.

[0107] The function of the control means 36 is implemented as follows, for example. When the CPU 21 of the information processing device 20 estimates the user's operating state based on the function of the operating state estimation means 35, it accesses the control data shown in Figure 11 and extracts the control content corresponding to the estimated operating state. The control data is data that associates the estimated operating state (in the example shown in the figure, "number of touches is X0 or more", "number of touches is X1 or more but less than X0", etc.) with the control content of the terminal device 10 (in the example shown in the figure, "launch a predetermined application (e.g., a map application)", "reduce the brightness of the display unit 16", etc.) for each web page identification information (e.g., the URL of the web page). For example, if the estimated operating state when a predetermined web page is displayed in the web browser of the terminal device 10 is "number of touches is X0 or more", the CPU 21 extracts the control content corresponding to the action of "number of touches is X0 or more" (in this case, "launch a predetermined application") from the control data. The CPU 21 then transmits instruction information (commands) to the terminal device 10 via the communication interface unit 28 and the communication network NW to operate the terminal device 10 with the extracted control content (in this case, "start a predetermined application").

[0108] On the other hand, when the CPU 11 of the terminal device 10 receives instruction information from the information processing device 20, it operates based on the received instruction information (in this case, it launches a predetermined application). In this way, the terminal device 10 can be operated appropriately according to the user's operating state.

[0109] In this explanation, we have described an example where the control data is described in a way that associates the estimated operating state with the control content of the terminal device 10 for each web page identification information. However, the control data may also be described in a way that associates the control content of the terminal device 10 with each estimated operating state (i.e., the web page identification information does not need to be associated).

[0110] The providing means 37 has a function to provide information corresponding to the user's operation status with respect to the terminal device 10 within an estimated predetermined period. This makes it possible to provide appropriate information according to the user's operation status, for example, by providing advertising information to the user via the terminal device 10 when the user is frequently operating the terminal device 10.

[0111] The function of the provisioning means 37 is implemented, for example, as follows. When the CPU 21 of the information processing device 20 estimates the user's operation state based on the function of the operation state estimation means 35, it accesses the provisioning data shown in Figure 12 and extracts provisioning information corresponding to the estimated operation state. The provisioning data is data that describes the estimated operation state (in the example in the figure, "number of touches is X0 or more", "number of touches is X1 or more but less than X0", etc.) and provisioning information (in the example in the figure, "survey information", "advertising information", etc.) associated for each web page identification information (for example, the URL of the web page). For example, if the estimated operation state when a predetermined web page is displayed in the web browser of the terminal device 10 is "survey information", the CPU 21 extracts the provisioning information (in this case, "survey information") corresponding to the action of "survey information" from the provisioning data. The CPU 21 then transmits the extracted provisioning information to the terminal device 10 via the communication interface unit 28 and the communication network NW.

[0112] On the other hand, when the CPU 11 of the terminal device 10 receives the information to be provided from the information processing device 20, it may display the received information on, for example, the display unit 16. Also, if the information to be provided consists of audio data, the CPU 11 may output the information from an audio output device such as a speaker. In this way, appropriate information can be provided according to the user's operating state.

[0113] In this explanation, we have used the example that the provided data is described in a way that the estimated operation state is associated with the provided data for each web page identifier. However, the provided data may also be described in a way that the provided data is associated with each estimated operation state (i.e., the web page identifier is not necessarily associated).

[0114] The modification means 38 includes a function to change the display manner of the web page displayed in the web browser according to the user's operation state on the terminal device 10 within an estimated predetermined period. This makes it possible to dynamically change the design, layout, etc., of the web page according to the user's operation state when the web page is displayed in the web browser.

[0115] The function of the modification means 38 is implemented, for example, as follows: When the CPU 21 of the information processing device 20 estimates the user's operating state based on the function of the operating state estimation means 35, it may obtain the web page displayed on the web browser on the terminal device 10 from, for example, a web server (not shown), and modify the web page by changing the design, layout, etc. of the obtained web page according to the estimated operating state. Here, information regarding the design, layout, etc. of the web page corresponding to the estimated user's operating state may be stored in, for example, the storage device 24. The CPU 21 then transmits the modified web page to the terminal device 10 via the communication interface unit 28 and the communication network NW.

[0116] On the other hand, when the CPU 11 of the terminal device 10 receives the modified web page from the information processing device 20, it may display the modified web page on the display unit 16. Figure 13 shows an example of how a web page is displayed on the terminal device 10. Figure 13(a) shows an example of the layout of a web page displayed in the web browser when the estimated user operation state is "X0 or more touches", and Figure 13(b) shows an example of the layout of a web page displayed in the web browser when the estimated user operation state is "X1 or more touches but less than X0". ​​For example, when the user operation state is estimated to be "X0 or more touches", a web page is displayed with areas A, B, C, and D where text, videos, etc., are displayed, as shown in Figure 13(a). Here, area A is displayed at the top of the screen, area B is displayed at the bottom left of the screen, and areas C and D are displayed at the bottom right of the screen. On the other hand, if the user's operation state is estimated to be "X1 or more touches but less than X0 touches," a web page with areas D, C, and A (the information displayed in each area may be the same as the content displayed in Figure 13(a)) is displayed, as shown in Figure 13(b). Here, area D is displayed at the top of the screen, area C is displayed in the center of the screen, and area A is displayed at the bottom of the screen. In this way, the display mode of the web page can be dynamically changed according to the user's operation state when the web page is displayed in the web browser.

[0117] (5) Flow of the main processing of the information processing system of this embodiment Next, an example of the main processing flow performed by the information processing system of this embodiment will be explained with reference to the flowchart in Figure 14.

[0118] The CPU 21 of the information processing device 20 acquires information regarding the user status of the terminal device 10 measured by the measuring device 18, based on the function of the status information acquisition means 32 (step S100). Here, the CPU 21 may acquire information regarding the user status measured by the measuring device 18 when the web page is displayed in the web browser. The information regarding the user status may also include at least one of the user's acceleration and the user's angular velocity.

[0119] Next, the CPU 21 of the information processing device 20 estimates the user's operating state to the terminal device 10 during a predetermined period based on the changes in information regarding the user's state during that period, based on the function of the operation state estimation means 35 (step S102). Here, the CPU 21 may estimate the user's operating state to the terminal device 10 during a predetermined period based on the changes in information regarding the user's state during that period and a trained model based on machine learning that uses the changes in information regarding the user's state as training data. The CPU 21 may also estimate the user's operating state to the terminal device 10 during a predetermined period based on the user's behavior patterns during that period. Furthermore, the CPU 21 may estimate the user's operating state to the terminal device 10 during a predetermined period when a web page is displayed in the web browser. Moreover, the CPU 21 may estimate the user's operating state to the terminal device 10 during the period from a first state change point to a second state change point, which is the next state change point after the first state change point.

[0120] The CPU 21 of the information processing device 20 may also transmit (provide) data including the estimated user's operating state (estimated data shown in Figure 10) to an external device (for example, a computer that generates and provides personalized information (e.g., a server)) via the communication interface unit 28 and the communication network NW.

[0121] On the other hand, service providers that generate and provide personalized information can understand, based on the estimated data transmitted from the information processing device 20, what kind of user behavior the user is experiencing while browsing the web page. Furthermore, by analyzing the estimated data, it becomes possible to generate personalized information according to the user's behavior.

[0122] As described above, according to the information processing system, information processing method, and program of this embodiment, the user's operating state with respect to the terminal device 10 during a predetermined period (for example, how often the terminal device 10 is operated or is operable) is estimated based on information about the user's state during that predetermined period. Therefore, by analyzing the estimated user's operating state, for example, it becomes possible to generate personalized information according to the user's operating state. This makes it possible to obtain information (user's operating state) that can be applied to generating personalized information.

[0123] Furthermore, according to the information processing system, information processing method, and program of this embodiment, since the measuring device 18 is provided on the terminal device 10, it becomes possible to acquire information regarding the user's status using the measuring device 18 provided on the terminal device 10.

[0124] The following describes some variations of the embodiments described above. (modified version) In the above embodiment, the case in which the operation state estimation means 35 estimates the user's operation state based on the behavior pattern estimated by the behavior estimation means 34 and a second trained model based on machine learning using the behavior pattern as second training data was described as an example, but the present invention is not limited to this case. For example, the operation state estimation means 35 may estimate a predetermined operation state as the user's operation state to the terminal device 10 when the changes in information regarding the user's state within a predetermined period satisfy predetermined conditions corresponding to a predetermined operation state. Here, the predetermined conditions may be, for example, that the changes in information regarding the user's state within a predetermined period fall within the range of information regarding the user's state corresponding to a predetermined operation state.

[0125] In this case, the CPU 21 of the information processing device 20 may extract information about the user's state stored in the second acquired data within a predetermined period (information about the transition of information about the user's state within a predetermined period) from the second acquired data, and then estimate the user's operating state by referring to the operating state data shown in Figure 15. The operating state data is data that describes the time transition of the range of information about the user's state (in the example shown in the figure, the range of 3-axis acceleration) associated with each of several operating states (in the example shown in the figure, "number of touches is X0 or more", "number of touches is X1 or more but less than X0", "number of touches is X2 or more but less than X1", "number of touches is X3 or more but less than X2", etc.). The operating state data may be stored in a storage device 24, for example.

[0126] For example, the CPU 21 may estimate one of the operation states (e.g., "number of touches is X0 or more") as the user's operation state if the time progression of the user state information extracted from the second acquired data falls within the range of user state information corresponding to any of the operation states in the operation state data.

[0127] Furthermore, the behavior estimation means 34 may use data (not shown) configured in the same way as the operation state data shown in Figure 15 to estimate, similar to the operation state estimation means 35, that a user is performing a predetermined behavior pattern if the changes in information regarding the user's state within a predetermined period satisfy predetermined conditions corresponding to a predetermined behavior pattern (for example, the changes in information regarding the user's state within a predetermined period fall within the range of information regarding the user's state corresponding to a predetermined behavior pattern).

[0128] Thus, the information processing system, information processing method, and program according to this modified example can achieve the same effects and advantages as the embodiments described above.

[0129] The program of the present invention may be stored on a computer-readable storage medium. The storage medium on which this program is recorded may be the ROM 12, RAM 13, or storage device 14 of the terminal device 10 shown in Figure 2, or the ROM 22, RAM 23, or storage device 24 of the information processing device 20 shown in Figure 3. The storage medium may also be a CD-ROM or the like, which can be read by being inserted into a program reading device such as a CD-ROM drive. Furthermore, the storage medium may be magnetic tape, cassette tape, flexible disk, MO / MD / DVD, or semiconductor memory.

[0130] The embodiments and modifications described above are provided to facilitate understanding of the present invention and are not intended to limit it. Accordingly, each element disclosed in the above embodiments and modifications is intended to include all design changes and equivalents that fall within the technical scope of the present invention.

[0131] For example, in the embodiment described above, the case in which the measuring device 18 is provided on the terminal device 10 was explained as an example, but the present invention is not limited to this case. The measuring device 18 may be provided while being worn by the user, or it may be provided at a location separate from the user and the terminal device 10. Furthermore, if the measuring device 18 is provided while being worn by the user, the acceleration and angular velocity of at least one body part of the user (e.g., arm, leg, etc.) may be acquired as information about the user's state.

[0132] Furthermore, in the above-described embodiment, the CPU 21 of the information processing device 20 was described as an example in which the operation state estimation means 35 estimates the user's operation state to the terminal device 10 within a predetermined period based on the user's behavior pattern within that period. However, the present invention is not limited to this case. For example, the CPU 21 may estimate the user's operation state within a predetermined period by inputting the changes in information regarding the user's state within that period into a third trained model based on machine learning that uses the changes in information regarding the user's state as third training data.

[0133] An example of the third training data is shown in Figure 16. The third training data shown in Figure 16 is data that describes the transition of information about the user's state and the user's operating state (operating state label) in a corresponding manner. As a result of machine learning, a third trained model is constructed that shows the relationship between the transition of information about the user's state and the user's operating state. Alternatively, the CPU 21 may learn a model (third trained model) used to estimate the user's operating state based on information about the user's state by machine learning using the transition of information about the user's state as the third training data.

[0134] Furthermore, although the above-described embodiment explained as an example where the operating state of the terminal device 10 by one user is estimated, the operating state of each user's terminal device 10 may also be estimated.

[0135] Furthermore, in the embodiments described above, an example was given in which an estimated behavioral pattern includes one or more of the following: "stillness," "walking," "lying down," and "climbing stairs." However, the content of the behaviors included in the behavioral pattern is not limited to these.

[0136] Furthermore, although the above-described embodiment explained the case in which one information processing device 20 is provided as an example, the present invention is not limited to this case. For example, multiple information processing devices 20 may be provided, in which case the operation content and processing results on any of the information processing devices 20 may be displayed in real time on other information processing devices 20, or the processing results on any of the information processing devices 20 may be shared among the multiple information processing devices 20.

[0137] Furthermore, in the above-described embodiment, the information processing device 20 is configured to realize the functions of page information acquisition means 31, state information acquisition means 32, detection means 33, action estimation means 34, operation state estimation means 35, control means 36, provision means 37, and modification means 38. However, the present invention is not limited to this configuration. For example, a computer (e.g., a general-purpose personal computer or server computer) that is communicably connected to the information processing device 20 via a communication network NW such as the Internet or LAN may realize the function of at least one of the above means 31 to 37. Also, each function in the functional block diagram shown in Figure 4 may be arbitrarily divided between the terminal device 10 and the information processing device 20, as shown in Figures 17(a) and (b).

[0138] For example, the CPU 11 of the terminal device 10 may estimate the user's behavior pattern based on information about the user's state, or estimate the user's operating state based on the estimated behavior pattern. In this case, the CPU 11 may train the first trained model and / or the second trained model in the embodiment described above. In this case, the CPU 11 may perform machine learning using, for example, TensorFlow®, an open-source machine learning library. This makes it possible to estimate the user's operating state in real time in the terminal device 10. Furthermore, since the user's operating state estimation process can be performed within the terminal device 10, it is possible to suppress the transmission of information about the user's state and information about the user's behavior pattern to the outside of the terminal device 10. Therefore, it becomes possible to estimate the user's operating state while protecting the user's privacy. [Industrial applicability]

[0139] The information processing system, information processing method, and program of the present invention, as described above, can be suitably used in information provision services that provide personalized information to each user, and therefore have extremely high industrial applicability. [Explanation of symbols]

[0140] 10…Terminal device 18... Measuring device 20…Information Processing Devices 31…Method for obtaining page information 32...Means for acquiring status information 33...Detection means 34…Behavior estimation means 35...Operation information estimation means 36... Control means 37…Providing means 38…Method of change

Claims

1. An information processing system that estimates the user's operating state on a terminal device, A state information acquisition means for acquiring information about the user's state measured by a measuring device, An operation state estimation means for estimating the user's operation state with respect to the terminal device during a predetermined period, based on the changes in information regarding the user's state during that predetermined period, A behavior estimation means for estimating the user's behavior pattern during the predetermined period based on the changes in information regarding the user's state during the predetermined period, The system includes a detection means for detecting a state change point that indicates the timing at which the user's operating state on the terminal device changes, based on the information relating to the user's state. The operation state estimation means estimates the user's operation state to the terminal device during the predetermined period based on the user's behavior pattern during the predetermined period. The behavior estimation means estimates the user's behavior pattern during the period from a first state change point to a second state change point, which is the next state change point after the first state change point. Information processing system.

2. The information processing system according to claim 1, wherein the operation state estimation means estimates the user's operation state to the terminal device during the predetermined period based on the changes in information regarding the user's state during the predetermined period and a trained model based on machine learning that uses the changes in information regarding the user's state as training data.

3. The information processing system according to claim 1, wherein the behavior estimation means estimates the user's behavior pattern during the predetermined period based on the changes in information regarding the user's state during the predetermined period and a trained model based on machine learning that uses the changes in information regarding the user's state as training data.

4. The information processing system according to any one of claims 1 to 3, wherein the operation state estimation means estimates the user's operation state with respect to the terminal device during the period from a first state change point to a second state change point which is the next state change point after the first state change point.

5. The information processing system according to claim 1, wherein the detection means detects the state change point by determining whether the predetermined timing is the state change point when the information regarding the user's state at the predetermined timing satisfies predetermined conditions.

6. The information processing system according to claim 5, wherein the predetermined condition includes the derivative of a curve showing the progression of information regarding the user's state being zero.

7. The information processing system according to claim 5 or 6, wherein the detection means determines that the predetermined timing is the state change point when the change point score, which indicates the magnitude of the change in information regarding the user's state at the predetermined timing, is greater than or equal to a predetermined value.

8. The information processing system according to any one of claims 1 to 7, further comprising control means for controlling the operation of the terminal device in accordance with the user's operation status to the terminal device within the estimated predetermined period.

9. The information processing system according to any one of claims 1 to 8, further comprising a means for providing information corresponding to the user's operation status with respect to the terminal device during the estimated predetermined period.

10. The system includes page information acquisition means for acquiring information about a web page displayed in a web browser running on the terminal device, The state information acquisition means acquires information regarding the user's state measured by the measuring device when the web page is displayed in the web browser. The information processing system according to any one of claims 1 to 9, wherein the operation state estimation means estimates the user's operation state to the terminal device during the predetermined period while the web page is displayed in the web browser.

11. The information processing system according to claim 10, further comprising a means for changing the display mode of a web page displayed in the web browser according to the user's operation status to the terminal device within the estimated predetermined period.

12. The information processing system according to any one of claims 1 to 11, wherein the information relating to the user's state includes at least one of the user's acceleration and the user's angular velocity.

13. The information processing system according to any one of claims 1 to 12, wherein the user's operation state includes at least one of the user's touch state, scroll state, swipe state, flick state, pinch state and drag state on the terminal device.

14. The information processing system according to any one of claims 1 to 13, wherein the measuring device is provided in the terminal device.

15. An information processing method that uses a computer to estimate the user's operating state on a terminal device, The aforementioned computer, The steps include: acquiring information regarding the user's state measured by a measuring device; A step of estimating the user's operating state with respect to the terminal device during a predetermined period based on the changes in information regarding the user's state during that predetermined period, A step of estimating the user's behavior pattern during the predetermined period based on the changes in information regarding the user's state during the predetermined period, Based on the information regarding the user's state, the step of detecting a state change point that indicates the timing at which the user's operating state on the terminal device changes; Perform each step, In the step of estimating the user's operating state, the user's operating state with respect to the terminal device during the predetermined period is estimated based on the user's behavior pattern during the predetermined period. In the step of estimating the user's behavior pattern, the user's behavior pattern during the period from a first state change point to a second state change point, which is the next state change point after the first state change point, is estimated. Information processing methods.

16. A program for a computer to estimate the user's operating state on a terminal device, To the aforementioned computer, A function to acquire information regarding the user's state measured by a measuring device, A function that estimates the user's operating state on the terminal device during a predetermined period based on the changes in information regarding the user's state during that predetermined period, A function to estimate the user's behavioral pattern during the predetermined period based on the changes in information regarding the user's state during the predetermined period, A function to detect a state change point that indicates the timing at which the user's operating state on the terminal device changes, based on the information regarding the user's state. To make it happen, The function for estimating the user's operating state estimates the user's operating state with respect to the terminal device during the predetermined period based on the user's behavior pattern during the predetermined period. A program for estimating the user's behavioral patterns, comprising a function for estimating the user's behavioral patterns during the period from a first state change point to a second state change point, which is the state change point following the first state change point.

Images