Information retrieval support system, information retrieval support method, and information display system

Abstract

Provided are an information search support system, an information search support method, and an information display system, which display information by switching to recommendation information that is easy to arouse the interest of a user without impairing the convenience of the user. The present invention is provided with: a user interestingness calculation unit that estimates, in real time, an interestingness indicating the degree of interest of a user with respect to each piece of recommendation information on the basis of user information acquired over a predetermined time; a user gaze degree calculation unit that estimates, in real time, a gaze degree indicating the degree to which the user currently gazes each piece of recommendation information on the basis of the currently acquired user information; a recommendation fitness calculation unit that calculates a fitness for each recommendation information on the basis of the interestingness and the gaze degree; a recommendation information acquisition unit that acquires new recommendation information from the information recommendation system on the basis of the user information and the interestingness; and a display information control unit which, on the basis of the fitness, the interestingness, and the new recommendation information, determines the timing for switching to the appropriate recommendation information, and requests the information display terminal to switch the recommendation information being displayed.

Description

Technical Field

[0001] This invention is applicable, for example, to information retrieval support systems, information retrieval support methods, and information display systems related to techniques for appropriately retrieving user-suitable recommendation information using user action information. Background Technology

[0002] In recent years, due to the widespread adoption of e-commerce following the COVID-19 pandemic, the number of visitors to physical retail facilities has gradually decreased. To improve the user experience in physical stores or to acquire new revenue streams such as retail media advertising, commercial facility operators or leasing operators are promoting digital applications such as digital signage or virtual reality stores. Through these digital technologies, users can gain a deeper understanding of leasing and merchandise. On the other hand, for example, in physical stores, searching through a large selection of products remains a significant task for users, leading to attempts to innovate the user experience using technologies that recommend information.

[0003] In numerous e-commerce platforms, product recommendation systems have been implemented that leverage user operation history, browsing history, and purchase history to recommend products based on user actions. Patent Document 1 discloses a product recommendation system that recommends products to users based on their operational history, such as purchases, browsing, and adding items to their shopping carts. This system displays information that aligns with the user's interests and preferences based on their actions. The technology disclosed in this patent document can infer the user's interests in real time based on their operation history and recommend suitable information. This is expected to reduce the workload of searching through a large number of products in a store to find recommendations suitable for the user.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: U.S. Patent No. 7,113,917 Summary of the Invention

[0007] However, in the technology described in Patent Document 1, information is switched based on the user's explicit operation (touch, scroll, move, etc.), and the recommended information on the display cannot be switched in real time without the user's operation.

[0008] This invention was made with the above considerations in mind, and aims to provide an information retrieval support system, information retrieval support method, and information display system that can switch to and display recommended information that does not compromise user convenience but easily arouses user interest.

[0009] To address the aforementioned issues, this invention provides an information retrieval support system that collaborates with an information recommendation system that provides recommended information to an information display terminal for displaying recommended information to a specific user. The information retrieval support system comprises: a user information acquisition unit that acquires user information including gaze information representing the user's gaze towards the information display terminal; a user interest calculation unit that, based on user information acquired over a predetermined time, infers in real-time an interest level representing the degree of interest the user has towards each recommended piece of information; a user gaze level calculation unit that, based on currently acquired user information, infers in real-time a gaze level representing the degree to which the user is currently gazing at each recommended piece of information; a recommendation suitability calculation unit that calculates suitability for each recommended piece of information based on the interest level and the gaze level; a recommendation information acquisition unit that, based on the user information and the interest level, acquires new recommended information from the information recommendation system; and a display information control unit that, based on the suitability, the interest level, and the new recommended information, determines the timing for switching to suitable recommended information and requests the information display terminal to switch the displayed recommended information.

[0010] Furthermore, this invention provides an information retrieval support method, executed by an information retrieval support system. This system collaborates with an information recommendation system that provides recommended information to an information display terminal for displaying recommended information to a specific user. The information retrieval support method includes: a user information acquisition step, in which a user information acquisition unit acquires user information including gaze information representing the user's gaze towards the information display terminal; a user interest calculation step, in which a user interest calculation unit, based on user information acquired over a predetermined time, infers in real-time the degree of interest representing the user's interest in each recommended piece of information; and a user gaze calculation step, in which the user's gaze is... The gaze calculation unit estimates the gaze degree, representing the degree to which the user is currently looking at each recommended piece of information, based on the currently acquired user information. The recommendation suitability calculation unit calculates suitability for each recommended piece of information based on the interest level and the gaze degree. The recommendation information acquisition unit acquires new recommended information from the information recommendation system based on the user information and the interest level. Finally, the display information control unit determines the appropriate timing for switching to suitable recommended information based on the suitability, the interest level, and the new recommended information, and requests the information display terminal to switch the displayed recommended information.

[0011] Furthermore, this invention provides an information retrieval support system that collaborates with an information recommendation system that provides recommended information to an information display terminal for displaying recommended information to a specific user. The information retrieval support system comprises: a user information acquisition unit that acquires user information including gaze information representing the user's gaze towards the information display terminal; a user interest calculation unit that, based on user information acquired over a predetermined time, infers in real-time an interest level representing the degree of interest the user has towards each recommended piece of information; a user gaze level calculation unit that, based on currently acquired user information, infers in real-time a gaze level representing the degree to which the user is currently gazing at each recommended piece of information; a recommendation suitability calculation unit that calculates suitability for each recommended piece of information based on the interest level and the gaze level; a recommendation information acquisition unit that, based on the user information and the interest level, acquires new recommended information from the information recommendation system; a display information control unit that, based on the suitability, the interest level, and the new recommended information, determines the timing for switching to suitable recommended information and requests the information display terminal to switch the displayed recommended information; and an information display terminal that displays the recommended information.

[0012] According to the present invention, it is possible to switch to and display recommended information that does not compromise user convenience but easily arouses user interest. Attached Figure Description

[0013] Figure 1 This is a diagram showing the overall outline of the information display system of this embodiment.

[0014] Figure 2 This is a diagram showing the structure of the user information control server.

[0015] Figure 3 This is a diagram showing the structure of the recommendation information control server.

[0016] Figure 4 This is a sequence diagram illustrating the operation of the information retrieval and display system.

[0017] Figure 5 It is detailed in Figure 4 The flowchart shows the process performed by the user interest calculation unit of the user information control server in step S212.

[0018] Figure 6 It is detailed in Figure 4 The flowchart shows the process performed by the user gaze calculation unit of the user information control server in step S213.

[0019] Figure 7 It is detailed in Figure 4 The flowchart shows the process performed by the recommendation suitability calculation unit of the user information control server in step S216.

[0020] Figure 8 This is detailed in the first embodiment. Figure 4 The flowchart shows the processing performed by the display information control unit of the recommendation information control server in steps S218 and S220.

[0021] Figure 9 It is an internal diagram that shows the information data.

[0022] Figure 10A It is shown in Figure 4 The diagram shows the situation where recommended information is displayed on the information display terminal in step S222.

[0023] Figure 10B It is shown that... Figure 10A The example shown is a diagram of an example of the fit calculated for each display ID based on interest and gaze.

[0024] Figure 11A It is shown in Figure 4 The diagram shows the situation where recommended information is displayed on the information display terminal in step S222.

[0025] Figure 11B It is shown that... Figure 11A The example shown is a diagram of an example of the fit calculated for each display ID based on interest and gaze.

[0026] Figure 12 This is detailed in the second embodiment. Figure 4 The flowchart shows the processing performed by the display information control unit of the recommendation information control server in steps S218 and S220. Detailed Implementation

[0027] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0028] (1) First implementation method

[0029] Figure 1 This is a diagram illustrating an example of the overall structure of the information display system S, including the information retrieval support system 1 of the first embodiment.

[0030] like Figure 1As shown, the information display system S includes at least: an information retrieval support system 1 that controls information display based on user information; and information display terminals 5a and 5b that display the recommended results selected by the information retrieval support system 1. Preferably, it also includes: an information recommendation system 7 that provides recommendation information to the information retrieval support system 1; a user information database 8 that stores information related to the user; a content database 9 that stores content provided to the user; and an external input / output device 10 that acquires information related to the user. Furthermore, when there is no need to specifically distinguish between the information display terminal 5a, which is a fixed signage device, and the information display terminal 5b, which is a user-wearable virtual space display device, they are referred to as information display terminal 5.

[0031] The information retrieval support system 1, information display terminals 5a and 5b, information recommendation system 7, user information database 8, and external input / output device 10 are connected via network 6. Furthermore, the recommended information can also be retained by the information retrieval support system 1 itself, instead of being provided through the information recommendation system 7.

[0032] Information retrieval support system 1 collaborates with information recommendation system 7, which provides recommendation information to information display terminal 5 for displaying recommendation information to a specific user. Information retrieval support system 1 includes user information control server 2, recommendation information control server 3, and communication control unit 4. User information control server 2 performs various calculations related to user attributes and actions, as well as calculations related to user state inferences such as interest level, attention level, and suitability.

[0033] The recommendation information control server 3 performs various calculations related to obtaining recommendation information and switching content and timing of recommendation information switching based on user status. The communication control unit 4 transmits and receives information via network 6 with information display terminals 5a and 5b, the information recommendation system 7, the user information database 8, the content database 9, and the external input / output device 10.

[0034] Information display terminals 5a and 5b are, for example, terminal devices that display a summary of recommended information to user 15a as a facility visitor and user 15b as a remote visitor. In the illustrated example, two information display terminals 5a and 5b are shown, but the number of them is not limited to two and can be any number.

[0035] Information display terminal 5 is a device installed at a predetermined location inside or outside the facility and providing recommended information. Specifically, it is, for example, a computer device equipped with a display device such as a general-purpose personal computer (PC), a portable computer, a smartphone, a tablet, or VR (Virtual Reality) goggles. Here, a fixed signage device is designated as 5a, and a wearable virtual space display device is designated as 5b.

[0036] The information recommendation system 7 selects recommended information for the user based on the user information data obtained by the information retrieval support system 1. Here, the recommended information may also include recommendation level, information category, content corresponding to the recommended information, and relevance to other recommended information.

[0037] User information database 8 stores user information such as attributes, interests, purchase history, and browsing history. User information can also be retrieved by identifying the user's user ID.

[0038] Content database 9 stores content display data, which is the data used to display content in the recommendation information. Here, content display data includes at least one of the following: text data, image data, video data, sound data, and 3D model data.

[0039] The external input / output device 10 is not particularly limited as long as it is a device for acquiring information from the user viewing the information display terminal or for providing prompts to the user viewing the information display terminal. In the first embodiment described later, the external input / output device 10 may be, for example, a device for providing information to the user using voice. In the first embodiment described later, the external input / output device 10 may be, for example, a device for measuring the user's heart rate. The external input / output device 10 may be a computer device such as a PC or a portable computer.

[0040] The information retrieval support system 1, information display terminal 5, and external input / output device 10 include, for example, a processor such as a CPU (Central Processing Unit) that controls each part by executing a program, and a display device such as a monitor that displays images and other information. Additionally, the information retrieval support system 1, information display terminal 5, and external input / output device 10 include, for example, an input device such as a touch screen that allows users to input touch operations, swipe operations, etc.

[0041] Furthermore, the information retrieval support system 1, the information display terminal 5, and the external input / output device 10 may have, for example, a communication interface (hereinafter referred to as "communication I / F") used for communicating with external devices, and RAM (Random Access Memory), storage devices, and other memory used for storing system data and internal data.

[0042] Network 6 is a communication unit used for information communication between the information retrieval support system 1, information display terminal 5, information recommendation system 7, user information database 8, content database 9, and external input / output device 10. Examples include the Internet, LAN (Local Area Network), and WAN (Wide Area Network). The communication lines used for information communication can be wired, wireless, or a combination of both. Additionally, relay devices such as gateways and routers can be used to connect multiple networks and communication lines.

[0043] First, the information retrieval support system 1 will be explained. In this embodiment, the user is near an information display terminal 5a, such as a fixed signage device, installed in commercial facilities and shops. When the user stands in front of the device, the information display terminal 5a displays recommended information configured in its display area according to the user's attributes and movement information. At this time, the information retrieval support system 1 and the information display terminal 5a select recommended information that the user is not interested in based on the user's gaze information, and switch to the selected recommended information when the user's gaze is not directed towards it.

[0044] Figure 2 This diagram illustrates the structure of the user information control server 2. As shown, the user information control server 2 includes communication I / Fs 101a and 101b, a CPU 102, a memory 103, and a storage device 104. Communication I / Fs 101a and 101b exchange information with the recommendation information control server 3 and the communication control unit 4.

[0045] CPU 102 performs calculations for executing the processing performed by user information control server 2. CPU 102 functions as a display information data acquisition unit for obtaining display information data 109 of the recommendation information currently displayed on the information display terminal.

[0046] The memory 103 stores programs for implementing the functions of the user information acquisition unit 105, the user interest calculation unit 106, the user gaze calculation unit 107, and the recommendation suitability calculation unit 108, and the CPU 102 executes the programs.

[0047] The user information acquisition unit 105 acquires user information including gaze information indicating the user's gaze towards the display area of ​​the information display terminal 5. In addition to gaze information, the user information acquisition unit 105 also acquires user action information and user attribute information as user information.

[0048] The aforementioned gaze information is obtained by at least one of the gaze tracking device built into the information display terminal 5 and the device set in the external environment, and is projected as the gaze center point onto the gaze detection surface which adopts at least one of two-dimensional and three-dimensional shapes.

[0049] The aforementioned action information includes at least one of the user's posture information, operation information, heart rate, facial expression, and voice information, which is obtained through at least one of the information display terminals 5a and 5b, a device set in the external environment, and a device worn by the user.

[0050] The aforementioned attribute information includes at least one of the user's age, gender, hobbies, Internet purchase history, and browsing history, and is obtained from at least one device owned by the user or an external server device that has registered the user's attribute information.

[0051] As described in detail later, the user information acquisition unit 105 acquires user information from the information display terminal 5 and calculates attribute data 110 related to the user's attributes, gaze data 111 related to the user's gaze, and action data 112 related to the user's actions. Alternatively, a portion of the user information may be acquired from the user information database 8 connected via the network 6.

[0052] As will be described in detail later, the user interest calculation unit 106 calculates interest data 113, which represents the degree of user interest in each recommended message, based on user information over a certain period of time. That is, the user interest calculation unit 106 infers, for example in real time, the degree of user interest in each recommended message based on user information obtained over a predetermined period of time.

[0053] The user interest calculation unit 106 uses gaze information, for example, when the gaze center point is contained in the display area of ​​the recommendation information, to perform an addition operation on the values ​​for each recommendation information, and uses the cumulative value over a certain period of time as the interest level in each recommendation information.

[0054] The user interest calculation unit 106 uses gaze information, for example, to perform a subtraction operation on the interest level for each recommendation when the gaze center point is not included in the display area of ​​the recommendation information.

[0055] The user interest calculation unit 106 uses gaze information, action information, and gaze intensity to adjust the magnitude of the addition and subtraction operation values, and changes them as needed.

[0056] As will be described in detail later, the user gaze calculation unit 107 calculates gaze data 114, which is information about the degree to which the user gazes at each recommended piece of information at a certain point in time, based on user information at a certain point in time. That is, the user gaze calculation unit 107 infers the gaze degree, for example, in real time, representing the degree to which the user is currently gazing at each recommended piece of information, based on the currently acquired user information.

[0057] The user gaze calculation unit 107 uses the value calculated based on the distance between the current gaze center point obtained from the gaze information and the display area of ​​the recommendation information as the gaze degree in each recommendation information.

[0058] As described in detail later, the recommendation suitability calculation unit 108 calculates the suitability data 115, which is the degree to which the user seems to think a certain recommendation is the best when it is displayed to the user. That is, the recommendation suitability calculation unit 108 calculates the suitability for each recommendation based on interest and attention.

[0059] Storage device 104 stores data used by the program executed by CPU 102. Storage device 104 stores various data including display information data 109, attribute data 110, gaze data 111, action data 112, interest data 113, gaze data 114, and fitness data 115.

[0060] As will be described in detail later, the display information data 109 is information related to the recommended information displayed on the information display terminal 5 obtained from the recommended information control server 3, including the position and size of the information display area, as well as the content related to the recommended information.

[0061] As will be described in detail later, attribute data 110 includes information related to user attributes, such as age, gender, hobbies, advertising history, purchase history, and browsing history, obtained from information display terminal 5, user information database 8, and external input / output device 10.

[0062] As will be described in detail later, gaze data 111 includes information related to the user's gaze obtained from the information display terminal 5 and the external input / output device 10.

[0063] As will be described in detail later, action data 112 includes information related to the user's actions obtained from information display terminal 5 and external input / output device 10.

[0064] As described above, the interest data 113 is information calculated by the user interest calculation unit 106, indicating the degree of interest the user has in each of the displayed recommendations. The interest data 113 corresponding to each recommendation can be a single data point or can be processed as data from multiple regions.

[0065] As described above, gaze intensity data 114 represents the degree to which a user gazes at each piece of recommended information displayed at a given point in time. The gaze intensity data 114 corresponding to each piece of recommended information can be either a single data point or processed as data from multiple regions.

[0066] As described above, the fitness data 115 represents information about the degree to which users seem to consider the displayed recommendations to be the best. Furthermore, the fitness data 115 corresponding to each recommendation can be either a single data point or processed as data from multiple regions.

[0067] Figure 3 This diagram illustrates the structure of the recommendation information control server 3. As shown, the recommendation information control server 3 includes communication I / Fs 121a and 121b, a CPU 122, a memory 123, and a storage device 124. Communication I / Fs 121a and 121b exchange information with the user information control server 2 and the communication control unit 4.

[0068] CPU 122 performs calculations for executing the processing performed by the recommendation information control server 3. CPU 122 functions as a display information data acquisition unit, obtaining display information data 127 (which retrieves recommendation information currently displayed on the information display terminal) from the user information control server 2, and as display information data 109. Furthermore, CPU 122 functions as a content data acquisition unit, retrieving content data associated with the recommendation information obtained by the recommendation information acquisition unit 125 from the content database 9 via the network 6.

[0069] The memory 123 stores a program for implementing the functions of the recommendation information acquisition unit 125 and the display information control unit 126, and the CPU 122 executes the program.

[0070] As described in detail later, the recommendation information acquisition unit 125 requests recommendation information from the information recommendation system 7 connected via the network 6 based on user association data 128, including user attribute information and action information obtained from the user information control server 2, obtains recommendation information from the information recommendation system 7, and calculates it as new recommendation information data 129. That is, the recommendation information acquisition unit 125 obtains new recommendation information from the information recommendation system 7 based on user information and interest levels.

[0071] The display information control unit 126 determines the timing for switching to suitable recommendation information based on suitability, interest, and new recommendation information, and requests the information display terminal 5 to switch the displayed recommendation information.

[0072] The display information control unit 126 selects recommended information with high interest as suitable information, calculates the information similarity between suitable information and recommended information with a suitability below a threshold, and switches the display of recommended information with information similarity below the threshold to the display of new recommended information selected from a new group of recommended information obtained from the recommended information acquisition unit 125 based on the suitable information at a time point when the attention level is below the threshold.

[0073] The display information control unit 126 selects recommended information with a suitability of 1 or higher as suitable information, calculates the information similarity between the suitable information and the new recommended information obtained from the recommended information acquisition unit 125, and switches the display of the suitable information to the display of the new recommended information selected from the new recommended information group obtained from the recommended information acquisition unit 125 based on the suitable information at the time point when the gaze level is 1 or higher.

[0074] The display information control unit 126 selects recommended information with a suitability of 1 or higher as suitable information. At the time point when the gaze level is 1 or higher, the display of recommended information with a similarity of less than 1 to the suitable information among the recommended information arranged within a certain distance of the suitable information is switched to the display of new recommended information selected from a new group of recommended information obtained from the recommended information acquisition unit 125 based on the suitable information.

[0075] Storage device 124 stores data used by the program executed by CPU 122. Storage device 124 stores various data including display information data 127, user association data 128, and new recommendation information data 129.

[0076] As described in detail later, display information data 127 is information related to the recommended information displayed on the information display terminal 5, obtained from the recommended information control server 3, including its position and size related to the information display area, as well as the inherent content related to the recommended information. Display information data 127 is data in the same format as display information data 109.

[0077] User-related data 128 is user-related information obtained using the user information control server 2, including attribute data 110, gaze data 111, action data 112, interest data 113, gaze data 114, and fitness data 115 stored in the storage device 104 of the user information control server 2.

[0078] The new recommendation information data 129 includes the recommendation information that can be displayed next on the information display terminal 5, obtained from the recommendation information control server 3.

[0079] The information display terminal 5 is, for example, at least one of an information display terminal 5a as a fixed signage device and an information display terminal 5b as a user-wearable virtual space display device. The recommended information configured in the display areas of the information display terminals 5a and 5b is, for example, at least one of a two-dimensional surface and a three-dimensional shape.

[0080] Figure 4 This is a sequence diagram illustrating the actions of Information Retrieval Support System 1.

[0081] Here, the operations of the information display terminal 5, the user information control server 2, and the recommendation information control server 3 are shown. Furthermore, more detailed operations will be described later regarding a portion of the steps illustrated.

[0082] First, taking advantage of user actions on the information display terminal 5, such as a user standing in front of a fixed information display terminal 5 or a user wearing a wearable information display terminal 5, the screen display APP is activated and information is displayed on the display screen (step S201).

[0083] The information display terminal 5 uses its attached input device to obtain user information (step S202). At this time, it can also obtain the information in cooperation with the external input / output device 10. Next, it obtains information related to the information display area for displaying recommended information.

[0084] The obtained user information and information display area are sent to the user information control server 2, where they are received (step S203). Here, the information display area is processed as static data, but it can also be processed as dynamic data and sent to the user information control server 2 periodically.

[0085] User information control server 2 stores the received user information as attribute data 110 and stores the received information display area as part of display information data 109. Next, based on information included in the user information, such as the user ID, user information acquisition unit 105 requests attribute information from user information database 8 (step S204). The requested attribute information is sent to user information control server 2 and received by user information control server 2. In user information control server 2, when the necessary information is received, attribute data 110 is updated. Then, user information control server 2 sends data to recommendation information control server 3 (step S205).

[0086] When the recommendation information control server 3 receives the attribute data 110, it stores it as part of the user association data 128.

[0087] Next, in the recommendation information control server 3, the recommendation information acquisition unit 125 requests recommendation information from the information recommendation system 7 based on the user association data 128 (step S206). The requested recommendation information is sent to the recommendation information control server 3, which accepts it as new recommendation information data 129.

[0088] Furthermore, as will be described in detail later, in the recommendation information control server 3, the display information control unit 126 calculates the display information data 127 based on the new recommendation information data 129 (step S207). Next, the display information data 127 is sent to the information display terminal 5, and a request to switch the display screen is sent simultaneously (step S208).

[0089] The information display terminal 5 receives the display information data 127 and updates the display of the display device based on the received display screen switching request (step S209).

[0090] The information display terminal 5 uses its attached input device and external input / output device 10 to obtain user information (step S210). This user information includes the user's gaze information and movement information. The obtained user information is sent to the user information control server 2 and received by the user information control server 2 (step S211).

[0091] User information control server 2 calculates gaze data 111 and action data 112 based on the user's gaze information and action information contained in the received user information.

[0092] Next, as will be described in detail later, in the user information control server 2, the user interest calculation unit 106 calculates the user's interest in each recommended information displayed based on the gaze data 111 and the action data 112, and stores it as interest data 113 (step S212).

[0093] As will be described in detail later, in the user information control server 2, the user gaze calculation unit 107 calculates the user's gaze at each piece of recommended information displayed based on the gaze data 111 and the action data 112, and stores it as gaze data 114 (step S213). In addition, the interest data 113 is sent to the recommendation information control server 3, and is stored by the recommendation information control server 3 as part of the user association data 128 (step S214).

[0094] Furthermore, in the recommendation information control server 3, the recommendation information acquisition unit 125 requests recommendation information from the information recommendation system 7 based on the user association data 128 (step S215). The requested recommendation information is sent to the recommendation information control server 3, which accepts it as new recommendation information data 129.

[0095] Next, as will be described in detail later, in the user information control server 2, the recommendation suitability calculation unit 108 calculates the degree of suitability that the user seems to think is appropriate when each recommendation is displayed, based on the interest data 113 and the gaze data 114, and stores it as suitability data 115 (step S216).

[0096] The suitability data 115 is sent to the recommendation information control server 3, and stored by the recommendation information control server 3 as part of the user association data 128 (step S217).

[0097] As will be described in detail later, in the recommendation information control server 3, the display information control unit 126 calculates display information data 127 based on user association data 128 and new recommendation information data 129 (step S218). The display information data 127 is then sent to the information display terminal 5 and received by the information display terminal 5 (step S219).

[0098] In the recommendation information control server 3, the display information control unit 126 determines the timing of switching recommendation information based on the gaze data contained in the user association data 128 (step S220), and sends a screen switching request to the information display terminal 5, which is accepted by the information display terminal 5 (step S221).

[0099] Upon receiving a request to switch the display screen, the information display terminal 5 updates the display of the display device according to the display information data 127 (step S222). Furthermore, during the display update, the sound output device installed on the information display terminal 5 and the external input / output device 10 can also be used to provide sound prompts to the user.

[0100] In addition, the subsequent steps S210 to S222 can be repeated.

[0101] Figure 5 It is detailed in Figure 4 The flowchart shows the process performed by the user interest calculation unit 106 of the user information control server 2 in step S212.

[0102] First, the user interest calculation unit 106 reads the display information data 109 (step S301). As described above, the display information data 109 consists of recommendation information obtained from the information recommendation system 7 and an information display area including the display position and size of the recommendation information.

[0103] Next, the user interest calculation unit 106 reads in gaze data 111, action data 112, and interest data 113 (step S302). Gaze data 111, as described above, is information related to the user's gaze, such as the direction of the user's gaze and the duration of gaze. Action data 112, as described above, is information related to the user's actions, such as the user's body posture, operation information, heart rate, facial expression, and voice information. Interest data 113, as described above, is information about the degree of interest the user has in each of the displayed recommendations.

[0104] Next, the user interest calculation unit 106 sequentially selects recommended information from the display information data 109 (step S303). Next, the user interest calculation unit 106 reads the information display area and interest level corresponding to the selected recommended information based on the display information data 109 and interest data 113 (step S304). Next, it obtains the gaze center point from the gaze data 111 and determines whether it is within the read-in information display area (step S305).

[0105] As a result, when the center of the gaze is in the corresponding information display area ("Yes" in step S305), the user interest calculation unit 106 calculates an addition value for the interest based on the gaze data 111 and the action data 112 (step S306). Specifically, for example, the addition value can be calculated based on the user's heart rate included in the action data 112.

[0106] Next, an additive value is added to the corresponding interest score (step S307). Additionally, if the center of the gaze is not in the corresponding information display area ("No" in step S305), a certain value is subtracted from the corresponding interest score (step S308).

[0107] After steps S307 and S308, the user interest calculation unit 106 determines whether the processing is complete based on all the recommended information of the displayed information data (step S309).

[0108] As a result, if the processing is not completed (No in step S309), the process returns to step S303.

[0109] In contrast, upon completion of the processing ("Yes" in step S309), the user interest calculation unit 106 updates the interest data 113 according to the corresponding interest level (step S310). The user interest calculation unit 106 then ends the user interest calculation process. Figure 4 Step S212).

[0110] Figure 6 It is detailed in Figure 4The flowchart shows the process performed by the user gaze calculation unit 107 of the user information control server 2 in step S215.

[0111] First, the user gaze calculation unit 107 reads the display information data 109 (step S321). Next, the user gaze calculation unit 107 reads the gaze data 111, the action data 112, and the gaze data 114 (step S322). As described above, the gaze data 114 is information about the degree to which the user gazes at each piece of recommended information in the display at a certain point in time.

[0112] Next, the user gaze calculation unit 107 reads the gaze center point based on the gaze data 111 (step S323).

[0113] The user gaze calculation unit 107 sequentially selects recommended information from the display information data 109 (step S324). The user gaze calculation unit 107 reads the information display area corresponding to the selected recommended information based on the display information data 109 and the interest data 113 (step S325).

[0114] Next, the user gaze calculation unit 107 sequentially selects pixels contained in the display area (step S326). It calculates the distance between the display position of each pixel and the center of the gaze point (step S327). Then, based on the calculated distance and the action data 112, it calculates the sum of the values ​​added to the gaze (step S328).

[0115] Specifically, for example, a weighted value can be calculated using the cosine similarity between the vector representing the user's face orientation and the vector representing the gaze data 111, based on the user's body posture information contained in the motion data 112, thereby calculating an additive value. Alternatively, the gaze degree can be calculated using only the distance from the center of the display area.

[0116] Next, the corresponding gaze score is added (step S329). The user gaze score calculation unit 107 determines whether the processing is complete for all pixels in the information display area (step S330).

[0117] As a result, if the user gaze calculation unit 107 has not completed the calculation process for all pixels in the display area ("No" in step S330), it returns to step S326.

[0118] If the user gaze calculation unit 107 has completed the processing ("Yes" in step S330), it determines whether the processing is complete (step S331) based on all the recommended information about the displayed information data.

[0119] As a result, if the user gaze calculation unit 107 has not completed the calculation of all recommendation information regarding the displayed information data (No in step S331), it returns to step S324. Conversely, if the processing is complete (Yes in step S331), the user gaze calculation unit 107 updates the gaze data 114 according to the corresponding gaze (step S332). The user gaze calculation unit 107 then ends the user gaze calculation process. Figure 4 Step S213).

[0120] Figure 7 It is detailed in Figure 4 The flowchart shows the process performed by the recommendation fitness calculation unit 108 of the user information control server 2 in step S216.

[0121] First, the recommendation fitness calculation unit 108 reads in the interest data 113 (step S341). Next, the recommendation fitness calculation unit 108 reads in the fixation data 114 (step S342).

[0122] The recommendation suitability calculation unit 108 obtains the interest and gaze scores corresponding to each recommendation information in sequence based on the read interest data 113 and gaze scores 114 (step S343).

[0123] The recommendation suitability calculation unit 108 calculates the suitability based on the interest level and gaze level corresponding to each recommendation (step S344). As described above, the suitability is the degree to which the user seems to find each recommendation appropriate when it is displayed. Alternatively, for example, the suitability z_i can be derived using the interest level x_i and gaze level y_i corresponding to a certain recommendation i through the following formula (1).

[0124] z_(i)=ax_(i)+bx_(i)･･･(1)

[0125] Here, coefficients a and b are constants, and the recommendation suitability calculation unit 108 can adjust the coefficients based on the user's response to the recommendation results. Furthermore, interest level x_i and attention level y_i are not limited to scalar values; they can also be set as vector values.

[0126] Then, the recommendation suitability calculation unit 108 determines whether the recommendation suitability calculation process has been completed based on all recommendation information (step S345). As a result, if the processing of all recommendation information has not been completed ("No" in step S345), the recommendation suitability calculation unit 108 returns to step S343.

[0127] On the other hand, if all recommendation information has been processed (Yes in step S345), the recommendation fitness calculation unit 108 updates the fitness data (step S346). The recommendation fitness calculation unit 108 then ends the recommendation fitness calculation process. Figure 4 Step S214).

[0128] Figure 8 It is detailed in Figure 4 The flowchart shows the processing performed by the display information control unit 126 of the recommendation information control server 3 in steps S218 and S220.

[0129] First, the display information control unit 126 reads the information from... Figure 4 The user association data 128 created in steps S212, S213 and S215 (step S361).

[0130] Next, the display information control unit 126 reads the display information data 127 (step S362).

[0131] Next, the display information control unit 126 selects the most relevant recommendation information as suitable information based on the interest level included in the user association data 128 and the recommendation information included in the display information data 127 (step S363). Here, a single recommendation information is selected as suitable information, but suitable information may also consist of multiple recommendations.

[0132] Next, the display information control unit 126 sequentially selects the recommended information contained in the display information data 127 (step S364). That is, the display information control unit 126 repeats the subsequent processing until all the recommended information contained in the display information data 127 has been selected.

[0133] The display information control unit 126 obtains the suitability corresponding to the selected recommendation information and determines whether it is lower than a threshold (step S365). As a result, if the suitability is lower than a certain threshold ("Yes" in step S365), the display information control unit 126 calculates the information similarity between the selected recommendation information and the suitable information (step S366).

[0134] Then, the display information control unit 126 determines whether all the recommended information in the display information data has been selected (step S367).

[0135] In steps S365 and S367, if the suitability is not lower than a certain threshold ("No" in step S365) and if the selection of recommendation information is not completed ("No" in step S367), the display information control unit 126 returns to step S364.

[0136] On the other hand, in step S367, when all the recommended information has been selected ("Yes" in step S367), the display information control unit 126 selects the recommended information with the lowest information similarity to suitable information from the recommended information that has been executed in step S366 so far (step S368), and determines whether the information similarity is below the threshold (step S369).

[0137] As a result, if the information similarity is not below the threshold ("No" in step S369), the display information control unit 126 ends the processing. On the other hand, if the information similarity is below the threshold ("Yes" in step S369), the display information control unit 126 uses the recommended information selected in step S368 as the recommended information for switching sources (step S370). The display information control unit 126 reads the new recommended information data 129 obtained by the recommended information acquisition unit (step S371).

[0138] The display information control unit 126 selects recommended information for switching destinations based on the new recommended information data 129 and the suitability information (step S372). Specifically, for example, it can select recommended information that is highly similar to the suitability information, or it can obtain new recommended information based on the suitability information through the recommendation information acquisition unit 125 in cooperation with the information recommendation system 7.

[0139] Next, the display information control unit 126 updates the display information data 109 by swapping the recommended information for the switching source and the recommended information for the switching destination (step S373). Then, the display information control unit 126 sends the updated display information data 109 to the information display terminal 5 (step S374). Figure 4 (Step S219). The display information control unit 126 obtains the display area of ​​the recommended information for switching sources from the display information data 109 (step S375).

[0140] Next, the display information control unit 126 obtains the gaze intensity contained in the user association data 128 (step S376). Alternatively, it can connect to the user information control server 2 to obtain the latest gaze intensity data 114.

[0141] The display information control unit 126 determines whether the gaze intensity corresponding to the display area of ​​the recommended information is below a threshold (step S377). As a result, if the gaze intensity is not below the threshold ("No" in step S377), the display information control unit 126 returns to step S376. On the other hand, if the gaze intensity is below the threshold, the display information control unit 126 requests a switch in the screen display for the information display terminal 5 (step S378). Figure 4 Step S221), end the process.

[0142] Figure 9 This is a diagram illustrating the internal structure of the display information data 109 / 127. The display information data 109 / 127 shown in the diagram consists of an information display area 401 and recommendation information 402. Furthermore, the information display area 401 includes a display ID 403, a visual recognition rate 404, and display parameters 405.

[0143] Display ID 403 is an ID assigned to each information display area of ​​the information display terminal 5. Here, we show a scenario where the information display terminal 5 has at least four information display areas, and A01, A02, A03, and A04 are respectively assigned as display ID 403. Furthermore, the information display areas can also dynamically change over time.

[0144] Visual recognition rate 404 is the observability of the information display area corresponding to the information display area. For example, it is calculated using user information obtained from the information display terminal 5, based on the ratio of the number of people whose information display area displayed on the information display terminal 5 is contained within the center point of the viewpoint. Furthermore, visual recognition rate 404 can be a constant value or a value that varies in accordance with changes in the information display area.

[0145] Display parameter 405 is a parameter related to the information display area corresponding to display ID 403. Here, assuming the information display area is configured on a two-dimensional plane, it represents the position and size. Furthermore, if the information display area is configured in three-dimensional space, display parameter 405 may also include information related to posture in addition to position and size.

[0146] The recommendation information 402 includes information ID 406, recommendation degree 407, information category 408, corresponding content 409, and relationship tree 410.

[0147] Information ID 406 is an ID assigned to each piece of recommendation information acquired by the recommendation information acquisition unit 125 of the recommendation information control server 3. Here, we show the case where I003, I005, I002, and I007 are assigned as information ID 406 for the information display areas corresponding to A01, A02, A03, and A04, which are display IDs 403, respectively. Furthermore, there can be one or more information IDs 406 assigned to the information display area corresponding to a certain display ID 403.

[0148] The recommendation score of 407 is the degree to which the recommendation information is recommended, obtained from the information recommendation system 7 by the recommendation information control server 3.

[0149] Information category 408 refers to the attributes of recommendation information obtained from the information recommendation system 7 via the recommendation information acquisition department 125. Here, it represents recommendation information for general consumers, such as fashion, beauty appliances, and cosmetics.

[0150] Content 409 corresponds to information about the content associated with the information ID, retrieved from content database 9. Here, the location where the file is stored is shown, but it could also be the address information (URL) used to retrieve the content via the Internet.

[0151] The relationship tree 410 is the association between recommended information obtained from the information recommendation system 7 by the recommendation information acquisition unit 125. For example, it is the information ID of other recommended information that a user who views a certain recommended information frequently views.

[0152] Figure 10A as well as Figure 11A Examples are shown in Figure 4 The diagram shows the situation in step S222 where recommended information is displayed on the fixed information display terminal 5a.

[0153] Here, it is shown that, according to Figure 9 The example shown is a display of recommended information with information IDs A01 to A06 on an information display terminal 5a, which is a fixed signage device. Furthermore, the example shows the display of images as content for recommended information, but the displayed object is not limited to this. That is, the display method of the content corresponding to the recommended information is not limited to image data; it can also be text data, video data, sound data, or a 3D model. There can be more than one piece of content corresponding to each piece of recommended information, and the display methods can be different for each piece.

[0154] Here, Figure 10A This shows a configuration example of recommended information, as well as the activity of the gaze over a certain period of time and the current gaze position. Here, the black circle represents the stationary point of the gaze; the larger the circle, the longer the stationary time. Multiple black circles, or lines entering or exiting a black circle, represent the gaze (in...). Figure 11A (The same applies to China).

[0155] exist Figure 10B In, it is shown in Figure 10A In this case, here is an example of the interest calculated by the user interest calculation unit, the gaze calculated by the user gaze calculation unit, and the suitability calculated by the recommendation suitability calculation unit.

[0156] Next, Figure 11A Shown only after Figure 10A The user's gaze then moves to A04, and the activity and current gaze position are recorded. Figure 11B Shown in Figure 11A Here is an example of the interest calculated by the user interest calculation department, the gaze calculated by the user gaze calculation department, and the suitability calculated by the recommendation suitability calculation department.

[0157] In the first embodiment, the display information control unit 126, for example, uses... Figure 8 The process shown is used to control the displayed information. Figure 10A as well as Figure 10B In this case, the display information data 109 / 127 is set to be used. Figure 9 When using the data illustrated, after selecting I003 corresponding to A01 as the suitable information, the recommended information with a suitability below the threshold becomes the candidate recommended information for the switching source.

[0158] Next, the fitness threshold is set to 0.30. Regarding the recommendation information I007 corresponding to A04, its information similarity to the recommendation information I003 corresponding to A01 is calculated. Here, when referring to the relation tree 410, it is assumed that I003 and I007 have a high information similarity, so I007 corresponding to A04 will not be selected as the recommendation information for switching sources.

[0159] In the change to Figure 11A as well as Figure 11B In the given state, I003, corresponding to A01, is selected as suitable information, and I002, corresponding to A03, is selected as a candidate for recommended information from the switching source. Here, I003 and I002 belong to different information categories 408 and are not in a tree relationship, so there is a possibility of low information similarity. If the information similarity is low, then I002, corresponding to A03, can be selected as the recommended information from the switching source.

[0160] According to the first embodiment, the recommended information displayed in a limited display area can be switched according to the user's gaze and actions, thus improving user convenience and information retrieval efficiency. Furthermore, according to the first embodiment, by switching information at appropriate times based on the user's gaze and actions, information retrieval can be supported without user intervention and without compromising the user experience.

[0161] (2) Second implementation method

[0162] The structure and function of the information display system, which includes the information retrieval support system according to the second embodiment, are substantially the same as those of the information display system S, which includes the information retrieval support system 1 according to the second embodiment. Therefore, the description of the same points is omitted, and the following description focuses on the different points.

[0163] In the second embodiment, the user wears an information display terminal 5b, which is a wearable virtual space display device such as XR goggles, and can access shops and counters in the virtual space via the Internet. Furthermore, when the user accesses the virtual space, the information display terminal 5b configures recommended information within the virtual space based on the user's attributes and activity information, and displays it on the device. At this time, the information retrieval support system 1 and the information display terminal 5 select recommended information of interest to the user based on the user's gaze information, and display new recommended information around the selected recommended information when the user's gaze is directed towards the vicinity of the selected recommended information.

[0164] Even in the second embodiment, the recommended information suitable for the user can be switched at appropriate times using the same method as in the first embodiment. However, unlike the first embodiment, in the second embodiment, the information is switched within the user's line of sight. Therefore, the display control method of the display information control unit 126 of the recommended information control server 3 differs between the first and second embodiments.

[0165] Figure 12 This is detailed in the second embodiment. Figure 4 The flowchart shows the processing performed by the display information control unit 126 of the recommendation information control server 3 in steps S218 and S220. In the second embodiment... Figure 12 The flowchart shown includes steps S384 and S386, which are described later, and this is different from the first embodiment. Figure 8 The flowcharts shown are different.

[0166] First, the display information control unit 126 reads the information from... Figure 4 The user association data 128 created in steps S212, S213, and S215 is then processed (step S381). Next, the display information control unit 126 reads the display information data 127 (step S382).

[0167] Next, the display information control unit 126 selects the recommendation information with the highest suitability as the suitable information based on the suitability included in the user association data 128 and the recommendation information included in the display information data 127 (step S383). Here, the suitable information is selected as a single recommendation information, but the suitable information may also consist of multiple recommendation information.

[0168] Next, the display information control unit 126 determines whether the suitability of the selected suitable information is above a threshold (step S384). As a result, if the suitability of the suitable information is below the threshold ("No" in step S384), the display information control unit 126 ends the process.

[0169] Next, if the suitability of the suitable information is above the threshold (Yes in step S384), the display information control unit 126 sequentially selects each item of the display information data 127 (step S385). That is, the display information control unit 126 repeats the subsequent processing until all the display information data 127 has been selected.

[0170] The display information control unit 126 determines whether the display distance between the selected recommended information and the suitable information is lower than a threshold based on the display area corresponding to the selected recommended information and the display area corresponding to the suitable information (step S386).

[0171] As a result, when the display distance is below a certain threshold ("Yes" in step S386), the display information control unit 126 calculates the information similarity between the selected recommended information and the suitable information (step S387).

[0172] Then, the display information control unit 126 determines whether all the recommended information in the display information data has been selected (step S388).

[0173] In steps S386 and S388, if the display distance is not lower than a certain threshold ("No" in step S386) and if the selection of recommended information is not completed ("No" in step S388), the display information control unit 126 returns to step S385.

[0174] Furthermore, in step S388, if all recommended information selections are completed ("Yes" in step S388), the display information control unit 126 selects the recommended information with the lowest information similarity to suitable information from the recommended information that has been processed in step S387 so far (step S389), and determines whether the information similarity is below a threshold (step S390). As a result, if the information similarity is not below the threshold ("No" in step S390), the display information control unit 126 ends the processing.

[0175] If the information similarity is below the threshold (Yes in step S390), the display information control unit 126 will use the recommended information selected in step S389 as the recommended information for switching sources (step S391).

[0176] The display information control unit 126 reads in the new recommendation information data 129 obtained by the recommendation information acquisition unit (step S392).

[0177] The display information control unit 126 selects recommended information for switching destinations based on the new recommended information data 129 and the suitability information (step S393). Specifically, for example, it can select recommended information that is highly similar to the suitability information, or it can obtain new recommended information based on the suitability information through the recommendation information acquisition unit 125 in cooperation with the information recommendation system 7.

[0178] Next, the display information control unit 126 updates the display information data 109 by swapping the recommended information for the switching source and the recommended information for the switching destination (step S394). Then, the display information control unit 126 sends the updated display information data 109 to the information display terminal 5 (step S395). Figure 4 (Step S219). The display information control unit 126 obtains the display area of ​​the recommended information for switching sources from the display information data 109 (step S396).

[0179] Next, the display information control unit 126 obtains the gaze intensity contained in the user association data 128 (step S397). Alternatively, it can connect to the user information control server 2 to obtain the latest gaze intensity data 114.

[0180] The display information control unit 126 determines whether the gaze intensity corresponding to the display area of ​​the recommended information is above a threshold (step S398). As a result, if the gaze intensity is below the threshold ("No" in step S398), the display information control unit 126 returns to step S397. On the other hand, if the gaze intensity is above the threshold, the display information control unit 126 requests a screen display switch from the information display terminal 5 (step S399). Figure 4 Step S221), end the process.

[0181] In the second embodiment, new recommended information can be displayed around the recommended information that the user cares about, at the moment when the user's gaze is directed, based on the user's gaze information.

[0182] (3. Explanation of information retrieval support methods)

[0183] The information retrieval support system 1 described above performs its processing through the collaboration of software and hardware resources. That is, the processor inside the computer device of the information retrieval support system 1 loads the software that performs the above functions into memory and executes it to perform these functions.

[0184] Therefore, the processing performed by the information retrieval support system 1 can be understood as follows: by executing software recorded in memory through a processor, user information data about the user's attributes, gaze information, and action information is obtained; in cooperation with the information recommendation system 7, user information data about recommendation information and display information data about the information display area is obtained; and based on the user information data and display information data, the user-suitable recommendation information is displayed at appropriate times.

[0185] Furthermore, in this embodiment, from the viewpoint of switching the display of information provided to the user at a time appropriate to the user based on user information data and display information data, it can be understood as a display control method.

[0186] The information display system S according to this embodiment includes at least an information display terminal 5 for displaying recommended information and an information retrieval support system 1 as follows. The information retrieval support system 1 is an information retrieval support system that cooperates with an information recommendation system 7 that provides recommended information to the information display terminal 5 for displaying recommended information to a certain user. It includes: a user information acquisition unit 105 that acquires user information including gaze information indicating the user's gaze towards the information display terminal 5; a user interest calculation unit 106 that, based on the user information acquired over a predetermined period of time, estimates in real time the degree of interest indicating the user's interest in each piece of recommended information; a user gaze calculation unit 107 that, based on the currently acquired user information, estimates in real time the degree of gaze indicating the user's current gaze at each piece of recommended information; a recommendation suitability calculation unit 108 that, based on the interest and gaze, calculates the suitability for each piece of recommended information; a recommended information acquisition unit 125 that, based on the user information and the interest, acquires a new group of recommended information from the information recommendation system 7; and a display information control unit 126 that, based on the suitability, the interest, and the new recommended information, determines the timing for switching to suitable recommended information and requests the information display terminal 5 to switch the displayed recommended information.

[0187] Therefore, since no user operation is required, it is possible to improve the retrieval efficiency of recommended information without compromising user convenience, while switching to and displaying recommended information that is easy to arouse user interest.

[0188] In this embodiment, the information display terminal 5 is at least one of an information display terminal 5a as a fixed signage device and an information display terminal 5b as a user-wearable virtual space display device. The recommended information disposed in the display areas of the information display terminals 5a and 5b is at least one of a two-dimensional surface and a three-dimensional shape. Therefore, the recommended information can be switched to a manner that easily attracts the user's interest without user operation, for example, displayed on a display device of at least one of the information display terminals 5a as a fixed signage device and 5b as a user-wearable virtual space display device.

[0189] In this embodiment, the user information acquisition unit 105 acquires user action information and user attribute information as user information, in addition to gaze information. Therefore, without user intervention, it can switch to and display recommended information that is likely to attract the user's interest based on the user's action information and user attribute information.

[0190] In this embodiment, gaze information is obtained through at least one of a gaze tracking device built into the information display terminal 5 and a device located in the external environment, and is projected as the gaze center point onto a gaze detection surface employing at least one of two-dimensional and three-dimensional shapes. Thus, without user intervention, the gaze center point is projected onto the gaze detection surface employing at least one of two-dimensional and three-dimensional shapes, thereby enabling the display of recommended information that is likely to attract the user's interest.

[0191] In this embodiment, the action information includes at least one of the user's posture information, operation information, heart rate, facial expression, and voice information, which is obtained through at least one device installed in the information display terminal 5 and at least one device worn by the user. Therefore, by obtaining at least one of the user's posture information, operation information, heart rate, facial expression, and voice information through at least one device installed in the information display terminal 5 and at least one device worn by the user, it is possible to switch to and display recommended information that is likely to arouse the user's interest based on action information including at least one of the user's posture information, operation information, heart rate, facial expression, and voice information without user operation.

[0192] In this embodiment, the attribute information includes, for example, at least one of the user's age, gender, hobbies, online purchase history, and browsing history, and is obtained from either the user's device or an external server device (not shown) that has registered the user's attribute information. Therefore, since the information is obtained from either the user's device or an external server device (not shown) that has registered the user's attribute information, it is possible to switch to and display recommended information that is likely to arouse the user's interest based on the attribute information including at least one of the user's age, gender, hobbies, online purchase history, and browsing history without user intervention.

[0193] In this embodiment, the user interest calculation unit 106 uses gaze information. When the center of the gaze is within the display area of ​​the recommendation information, it performs an addition operation on the values ​​for each recommendation information and uses the accumulated value over a certain period of time as the interest level of each recommendation information. Therefore, without user operation, the accumulated value over a certain period of time can be switched to display recommendation information that is likely to arouse the user's interest based on the interest level of each recommendation information.

[0194] In this embodiment, the user gaze degree calculation unit 107 uses the value calculated based on the distance between the current gaze center point obtained using gaze information and the display area of ​​the recommendation information as the gaze degree in each recommendation. Therefore, without user operation, the value calculated based on the distance between the current gaze center point and the display area of ​​the recommendation information can be switched to display recommendation information that is likely to arouse the user's interest, according to the gaze degree in each recommendation.

[0195] In this embodiment, the user interest calculation unit 106 uses gaze information to perform a subtraction operation on the interest level of each recommendation when the gaze center point is not included in the display area of ​​the recommendation information. Therefore, it is possible to switch to and display recommendation information that is more likely to arouse the user's interest without user intervention.

[0196] In this embodiment, the user interest calculation unit 106 uses gaze information, action information, and gaze intensity to adjust the magnitudes of the addition and subtraction operation values. Therefore, it can switch to and display recommendation information that is more likely to arouse the user's interest without user intervention.

[0197] In this embodiment, the display information control unit 126 selects highly interesting recommendation information as suitable information, calculates the information similarity between the suitable information and recommendation information with a suitability score below a threshold, and switches the display of recommendation information with information similarity below the threshold to the display of new recommendation information selected from a new group of recommendation information obtained from the recommendation information acquisition unit 125 based on the suitable information at a time point when the gaze level is below the threshold. Thus, without user operation, it can switch to and display recommendation information that is more likely to arouse the user's interest based on the gaze level.

[0198] In this embodiment, the display information control unit 126 selects recommended information with a suitability threshold or higher as suitable information, calculates the information similarity between the suitable information and new recommended information obtained from the recommended information acquisition unit 125, and switches the display of the suitable information to the display of new recommended information selected from the new recommended information group obtained from the recommended information acquisition unit 125 based on the suitable information at a time point when the gaze level is above the threshold. Thus, without user operation, it is possible to switch to and display recommended information that is more likely to arouse the user's interest based on the gaze level.

[0199] In this embodiment, the display information control unit 126 selects recommended information with a suitability threshold or higher as suitable information. At a time when the gaze level is above the threshold, the display of recommended information that is within a certain distance of the suitable information but has a similarity threshold below the threshold is switched to the display of new recommended information selected from a new group of recommended information obtained from the recommended information acquisition unit 125 based on the suitable information. Thus, without user operation, the display can switch to and show recommended information that is more likely to attract the user's interest based on the gaze level.

[0200] Furthermore, the present invention is not limited to the embodiments described above, but includes various modifications and equivalent structures within the spirit of the appended claims. For example, the embodiments described above are detailed embodiments for ease of understanding of the present invention, and the present invention is not necessarily limited to having all the structures described. In addition, the elements described in parallel in this embodiment may also be connected in series with respect to other elements, with at least one of each element connected in series.

[0201] [Industry Applicability]

[0202] This invention can be applied, for example, to information retrieval support systems, information retrieval support methods, and information display systems related to technologies that use user action information to appropriately retrieve user-suitable recommendation information.

[0203] (Symbol Explanation)

[0204] 1: Information retrieval support system; 2: User information control server; 3: Recommendation information control server; 4: Communication control unit; 5: Information display server; 5a: Signage device; 5b: Virtual space display device; 6: Network; 7: Information recommendation system; 8: User information database; 9: Content database; 10: External input / output device; 101, 121: Communication I / F; 102, 122: CPU; 103, 123: Memory; 104, 124: Storage device; 105: ... 106: User Information Acquisition Department; 107: User Interest Calculation Department; 108: User Gaze Calculation Department; 109, 127: Recommendation Suitability Calculation Department; 110: Display Information Data; 111: Attribute Data; 111: Gaze Data; 112: Action Data; 113: Interest Data; 114: Gaze Data; 115: Suitability Data; 125: Recommendation Information Acquisition Department; 126: Display Information Control Department; 128: User Association Data; 129: New Recommendation Information Data; S: Information Display System.

Claims

1. An information retrieval support system, cooperating with an information recommendation system that provides the recommendation information to an information display terminal for displaying recommendation information to a user, characterized in that, The information retrieval support system has the following features: The user information acquisition unit acquires user information including gaze information representing the user's gaze toward the information display terminal; The user interest calculation unit estimates the degree of interest of the user in each recommended information in real time based on user information obtained over a predetermined period of time. The user gaze calculation unit estimates the gaze degree, representing the degree to which the user is currently gazing at each piece of recommended information, in real time based on the currently obtained user information. The recommendation suitability calculation unit calculates suitability for each recommendation based on the interest level and the gaze level. The recommendation information acquisition unit obtains new recommendation information from the information recommendation system based on the user information and the degree of interest. as well as The display information control unit determines the timing for switching to suitable recommendation information based on the suitability, interest level, and new recommendation information, and requests the information display terminal to switch the displayed recommendation information.

2. The information retrieval support system according to claim 1, characterized in that, The information display terminal is at least one of a fixed display device and a user-wearable display device. The recommended information configured in the display area of ​​the display device is at least one of a two-dimensional surface and a three-dimensional shape.

3. The information retrieval support system according to claim 2, characterized in that, In addition to the gaze information, the user information acquisition unit also acquires the user's action information and the user's attribute information as user information.

4. The information retrieval support system according to claim 3, characterized in that, The gaze information is obtained by at least one of a gaze tracking device built into the information display terminal and a device set in the external environment, and is projected as the gaze center point onto a gaze detection surface employing at least one of two-dimensional and three-dimensional shapes.

5. The information retrieval support system according to claim 4, characterized in that, The action information includes at least one of the user's posture information, operation information, heart rate, facial expression, and voice information. The action information is obtained through at least one of the display device, a device located in the external environment, and a device worn by the user.

6. The information retrieval support system according to claim 5, characterized in that, The attribute information includes at least one of the user's age, gender, hobbies, internet purchase history, and browsing history. The attribute information is obtained from at least one of the user's personal devices or external server devices that have the user's attribute information registered.

7. The information retrieval support system according to claim 6, characterized in that, The user interest calculation unit uses the gaze information to perform an addition operation on the values ​​of each recommendation when the gaze center point is included in the display area of ​​the recommendation information, and uses the cumulative value over a certain period of time as the interest level of each recommendation.

8. The information retrieval support system according to claim 7, characterized in that, The user gaze calculation unit uses the value calculated based on the distance between the current gaze center point obtained using the gaze information and the display area of ​​the recommendation information as the gaze degree in each recommendation information.

9. The information retrieval support system according to claim 7, characterized in that, The user interest calculation unit uses the gaze information to perform a subtraction operation on the interest level for each recommendation when the gaze center point is not included in the display area of ​​the recommendation information.

10. The information retrieval support system according to claim 9, characterized in that, The user interest calculation unit uses the gaze information, the action information, and the gaze level to adjust the magnitude of the addition and subtraction operation values.

11. The information retrieval support system according to claim 2, characterized in that, The display information control unit selects the recommended information with high interest as suitable information, calculates the information similarity between the suitable information and the recommended information with a suitability below a threshold, and at the time point when the gaze level is below the threshold, switches the display of the recommended information with information similarity below the threshold to the display of new recommended information selected from the new recommended information group obtained from the recommended information acquisition unit based on the suitable information.

12. The information retrieval support system according to claim 2, characterized in that, The display information control unit selects the recommended information with a fitness level of 1 or higher as suitable information, calculates the information similarity between the suitable information and the new recommended information obtained from the recommended information acquisition unit, and switches the display of the suitable information to the display of the new recommended information selected from the new recommended information group obtained from the recommended information acquisition unit based on the suitable information at a time point when the gaze level is 1 or higher.

13. The information retrieval support system according to claim 12, characterized in that, The display information control unit selects the recommended information with a suitability of ≥1000 as suitable information. At the time point when the gaze level is ≥10000, the display of recommended information with a similarity of ≥10000 to the suitable information within a certain distance of the suitable information is switched to the display of new recommended information selected from a new group of recommended information obtained by the recommended information acquisition unit based on the suitable information.

14. An information retrieval support method, executed by an information retrieval support system, which collaborates with an information recommendation system that provides the recommendation information to an information display terminal for displaying recommendation information to a user, characterized in that, The information retrieval support method has the following features: The user information acquisition step involves the user information acquisition unit acquiring user information, including gaze information representing the user's gaze toward the information display terminal; The user interest calculation step involves the user interest calculation unit using user information obtained over a predetermined period of time to infer in real time the degree of interest of the user in each recommended information. The user gaze degree calculation step involves the user gaze degree calculation unit using the currently obtained user information to infer in real time the gaze degree representing the degree to which the user is currently gazing at each piece of recommended information. The recommended fit calculation step involves the recommended fit calculation unit calculating the fit for each recommendation based on the interest level and the gaze level. The step of obtaining recommendation information involves the recommendation information obtaining unit obtaining new recommendation information from the information recommendation system based on the user information and the degree of interest. as well as The display information control step involves the display information control unit determining the appropriate timing for switching to suitable recommendation information based on the suitability, interest level, and new recommendation information, and requesting the information display terminal to switch the displayed recommendation information.

15. An information display system, which is an information retrieval support system that collaborates with an information recommendation system that provides recommendation information to an information display terminal for displaying recommendation information to a user, characterized in that, The information display system has the following features: The user information acquisition unit acquires user information including gaze information representing the user's gaze toward the information display terminal; The user interest calculation unit estimates the degree of interest of the user in each recommended information in real time based on user information obtained over a predetermined period of time. The user gaze calculation unit estimates the gaze degree, representing the degree to which the user is currently gazing at each piece of recommended information, in real time based on the currently obtained user information. The recommendation suitability calculation unit calculates suitability for each recommendation based on the interest level and the gaze level. The recommendation information acquisition unit obtains new recommendation information from the information recommendation system based on the user information and the degree of interest. The display information control unit determines the timing for switching to suitable recommendation information based on the suitability, interest level, and new recommendation information, and requests the information display terminal to switch the displayed recommendation information. as well as The information display terminal displays recommended information.

Images