Information retrieval support system, information retrieval support method and information display system

Abstract

To perform display so as to switch to recommendation information that is likely to attract user's interest without damaging user's convenience.SOLUTION: An information retrieval support system includes a user interest degree calculation part for estimating an interest degree representing a degree at which a user is interested in each recommendation information in real time on the basis of user information acquired over a prescribed time, a user gazing degree calculation part for estimating a gazing degree representing a degree at which the user currently gazes each recommendation information in real time on the basis of user information that is currently acquired, a recommendation appropriateness calculation part for calculating appropriateness to each recommendation information on the basis of the interest degree and the gazing degree, a recommendation information acquisition part for acquiring new recommendation information from an information recommendation system on the basis of the user information and the interest degree, and a display information control part for determining a switching time to suitable recommendation information and requesting an information display terminal to switch recommendation information under display on the basis of a suitable degree, the interest degree and the new recommendation information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention is suitable for application to an information retrieval support system, an information retrieval support method, and an information display system relating to a technique for optimally searching for recommended information suitable for a user using user behavior information, for example. [Background technology]

[0002] In recent years, the spread of e-commerce following the COVID-19 pandemic has led to a decline in the number of visitors to commercial facilities with brick-and-mortar stores. Commercial facility operators and tenant operators are promoting the use of digital technologies such as digital signage and metaverse stores in an effort to improve the user experience in brick-and-mortar stores and acquire new revenue sources such as retail media advertising. These digital technologies allow users to gain a deeper understanding of tenants and products. However, for example, in brick-and-mortar stores, the effort required to find the perfect product for a user from the vast selection of products available remains high, and efforts are being made to innovate the user experience by using information recommendation technologies.

[0003] Many e-commerce sites have introduced product recommendation systems that utilize a user's operation history, browsing history, and purchase history to recommend information in response to user operations. Patent Document 1 discloses a product recommendation system that recommends products to a user based on a user's operation history, such as purchasing, browsing, and adding products to an electronic shopping cart. This system displays information tailored to the user's hobbies and preferences in response to user operations. The technology disclosed in this patent document can estimate a user's interests in real time based on the user's operation history, etc., and recommend information suitable for the user. This is expected to reduce the effort required to find recommended product information suitable for the user from the vast number of products available in a store. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 7,113,917 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology described in Patent Document 1 switches information in response to an explicit user operation (touch, scroll, move, etc.), and the recommended information being displayed cannot be switched in real time without a user operation.

[0006] The present invention has been made in consideration of the above points, and aims to propose an information retrieval support system, an information retrieval support method, and an information display system that can switch to and display recommended information that is more likely to attract the user's interest without compromising user convenience. [Means for solving the problem]

[0007] In order to solve this problem, the present invention provides an information search support system that works in conjunction with an information recommendation system that provides recommended information to an information display terminal that displays the recommended information to a user, and that includes a user information acquisition unit that acquires user information including gaze information that indicates the user's gaze toward the information display terminal, a user interest calculation unit that estimates in real time an interest level that indicates the degree to which the user is interested in each piece of recommended information based on user information acquired over a predetermined period of time, a user attention level calculation unit that estimates in real time an attention level that indicates the degree to which the user is currently paying attention to each piece of recommended information based on currently acquired user information, a recommendation suitability calculation unit that calculates a suitability level for each piece of recommended information based on the interest level and the attention level, a recommendation information acquisition unit that acquires new recommended information from the information recommendation system based on the user information and the interest level, and a display information control unit that determines the time to switch to suitable recommended information based on the suitability level, the interest level, and the new recommended information, and requests the information display terminal to switch the recommended information currently being displayed.

[0008] Further, in the present invention, there is provided an information retrieval support method by an information retrieval support system that cooperates with an information recommendation system that provides recommended information to an information display terminal that displays the recommended information to a certain user, the method including: a user information acquisition step in which a user information acquisition unit acquires user information including gaze information that indicates the gaze of the user at the information display terminal; a user interest degree calculation step in which a user interest degree calculation unit estimates in real time an interest degree that indicates the degree of interest the user has in each piece of recommended information based on the user information acquired over a predetermined time period; and a user attention degree calculation step in which a user attention degree calculation unit estimates an interest degree that indicates the degree of interest the user has in each piece of recommended information based on the currently acquired user information. The information recommendation system includes a user attention degree calculation step for estimating in real time an attention degree that indicates the degree to which the user is currently paying attention to the information, a recommendation favorability calculation step for a recommendation favorability calculation unit for calculating a favorability for each piece of recommended information based on the interest degree and the attention degree, a recommendation information acquisition step for a recommendation information acquisition unit for acquiring new recommended information from the information recommendation system based on the user information and the interest degree, and a display information control step for determining the timing to switch to suitable recommended information based on the favorability degree, the interest degree, and the new recommended information, and requesting the information display terminal to switch the recommended information being displayed.

[0009] In addition, the present invention provides an information search support system that works in conjunction with an information recommendation system that provides recommended information to an information display terminal that displays the recommended information to a user, and that includes a user information acquisition unit that acquires user information including gaze information that indicates the user's gaze toward the information display terminal, a user interest calculation unit that estimates in real time an interest level that indicates the degree to which the user is interested in each piece of recommended information based on user information acquired over a predetermined period of time, a user attention level calculation unit that estimates in real time an attention level that indicates the degree to which the user is currently paying attention to each piece of recommended information based on currently acquired user information, a recommendation suitability calculation unit that calculates a suitability level for each piece of recommended information based on the interest level and the attention level, a recommendation information acquisition unit that acquires new recommended information from the information recommendation system based on the user information and the interest level, a display information control unit that determines the time to switch to suitable recommended information based on the suitability level, the interest level, and the new recommended information, and requests the information display terminal to switch the recommended information currently being displayed, and an information display terminal that displays the recommended information. [Effects of the Invention]

[0010] According to the present invention, it is possible to switch to and display recommended information that is likely to attract the user's interest without impairing user convenience. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram showing a schematic configuration of an entire information display system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating a configuration of a user information control server. [Figure 3] FIG. 2 is a diagram illustrating a configuration of a recommendation information control server. [Figure 4] FIG. 10 is a sequence diagram illustrating the operation of the information search and display system. [Figure 5] 5 is a flowchart detailing the processing performed by a user interest degree calculation unit of the user information control server in step S212 of FIG. 4. [Figure 6]5 is a flowchart detailing the process performed by a user gaze degree calculation unit of the user information control server in step S213 of FIG. 4. [Figure 7] 5 is a flowchart detailing the process performed by a recommendation suitability calculation unit of the user information control server in step S216 of FIG. 4. [Figure 8] 5 is a flowchart detailing the processing performed by the display information control unit of the recommendation information control server in steps S218 and S220 of FIG. 4 in the first embodiment. [Figure 9] FIG. 10 is a diagram showing the contents of display information data. [Figure 10A] FIG. 5 is a diagram showing a case where recommended information is displayed on the information display terminal in step S222 of FIG. 4. [Figure 10B] 10B is a diagram showing an example of a preference level calculated based on the interest level and the attention level for each display ID corresponding to the example shown in FIG. 10A. FIG. [Figure 11A] FIG. 5 is a diagram showing a case where recommended information is displayed on the information display terminal in step S222 of FIG. 4. [Figure 11B] 11B is a diagram showing an example of a preference level calculated based on the interest level and the attention level for each display ID corresponding to the example shown in FIG. 11A. FIG. [Figure 12] 5 is a flowchart detailing the processing performed by the display information control unit of the recommendation information control server in step S218 and step S220 of FIG. 4 in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. (1) First embodiment FIG. 1 is a diagram showing an example of a schematic configuration of an entire information display system S including an information retrieval support system 1 according to the first embodiment.

[0013] 1, the information display system S includes at least an information retrieval support system 1 that controls information display based on user information, and information display terminals 5a and 5b that display recommendation results selected by the information retrieval support system 1, and preferably further includes an information recommendation system 7 that provides recommended information to the information retrieval support system 1, a user information database 8 that stores information about users, a content database 9 that stores content to be provided to users, and an external input / output device 10 that acquires information about users. Note that when there is no need to particularly distinguish between the information display terminal 5a, which is a fixed signage device, and the information display terminal 5b, which is a user-wearable virtual space display device, they will be referred to as information display terminal 5.

[0014] An information retrieval support system 1, information display terminals 5a and 5b, an information recommendation system 7, a user information database 8, and an external input / output device 10 are connected via a network 6. Note that the recommended information may be in a form held by the information retrieval support system 1 itself, instead of being provided by the information recommendation system 7.

[0015] The information retrieval support system 1 works in conjunction with an information recommendation system 7 that provides recommended information to an information display terminal 5 that displays recommended information for a user. The information retrieval support system 1 includes a user information control server 2, a recommended information control server 3, and a communication control unit 4. The user information control server 2 acquires information about the user's attributes and behavior, and performs various calculations related to estimating the user's state, such as interest level, attention level, and preference level.

[0016] The recommended information control server 3 acquires recommended information and performs various calculations related to the content and timing of switching the display information based on the user's status. The communication control unit 4 transmits and receives information to and from the information display terminals 5a and 5b, the information recommendation system 7, the user information database 8, the content database 9, and the external input / output device 10 via the network 6.

[0017] The information display terminals 5a and 5b are terminal devices that display a list of recommended information for, for example, a user 15a who is a facility visitor and a user 15b who is a remote visitor, respectively. In the illustrated example, two information display terminals 5a and 5b are shown, but the number of such terminals is not limited to two and may be any number.

[0018] The information display terminal 5 is a device that is set at a predetermined location inside or outside a facility and provides recommended information. Specifically, it is a computer device equipped with a display device such as a general-purpose personal computer (PC), a mobile computer, a smartphone, a tablet, or VR (Virtual Reality) goggles. Here, the fixed signage device is designated as 5a, and the wearable virtual space display device is designated as 5b.

[0019] The information recommendation system 7 selects recommended information to be recommended to the user based on the user information data acquired by the information retrieval support system 1. Here, the recommended information may include the recommendation level, information category, content corresponding to the recommended information, and relationships with other recommended information.

[0020] The user information database 8 stores user information relating to users such as user attributes, preferences, purchase history, browsing history, etc. Here, the user information may be acquired based on a user ID that identifies the user.

[0021] The content database 9 stores content display data, which is data of content to be displayed in recommendation information. Here, the content display data is data including at least one of character data, image data, video data, audio data, and three-dimensional model data.

[0022] The external input / output device 10 is not particularly limited as long as it is a device that acquires information of a user viewing an information display terminal or presents information to a user viewing an information display terminal. For example, in a first embodiment described later, the external input / output device 10 is a device that provides information to a user by voice. For example, in a first embodiment described later, the external input / output device 10 is a device that measures the user's heart rate. The external input / output device 10 can be a computer device such as a PC or a mobile computer.

[0023] The information retrieval support system 1, the information display terminal 5, and the external input / output device 10 each have, for example, a processor such as a CPU (Central Processing Unit) that controls each part through the execution of a program, and a display device such as a display that displays images and other information. The information retrieval support system 1, the information display terminal 5, and the external input / output device 10 also have, for example, an input device such as a touch display that inputs a user's touch operation, flick operation, etc.

[0024] Furthermore, the information search support system 1, the information display terminal 5, and the external input / output device 10 have, for example, a communication interface (hereinafter referred to as "communication I / F") used for communication with external devices, and memory such as RAM (Random Access Memory) or storage in which system data and internal data are stored.

[0025] The network 6 is a communication means used for information communication between the information retrieval support system 1, the information display terminal 5, the information recommendation system 7, the user information database 8, the content database 9, and the external input / output device 10, and is, for example, the Internet, a local area network (LAN), or a wide area network (WAN). The communication lines used for information communication may be wired or wireless, and a combination of these may be used. Furthermore, connections may be made via multiple networks or communication lines using relay devices such as gateway devices or routers.

[0026] First, the information retrieval support system 1 will be described. In this embodiment, a user is located near an information display terminal 5a, which is, for example, a fixed signage device installed in a commercial facility or store. When the user stands in front of the device, the information display terminal 5a displays recommended information so as to be arranged in its display area based on the user's attributes and behavior information. At this time, the information retrieval support system 1 and the information display terminal 5 select recommended information that the user is not interested in based on the user's gaze information, and switch to the selected recommended information when the user's gaze is not directed toward it.

[0027] 2 is a diagram showing the configuration of the user information control server 2. As shown in the figure, the user information control server 2 includes communication I / Fs 101a and 101b, a CPU 102, a memory 103, and a storage device 104. The communication I / Fs 101a and 101b exchange information with the recommendation information control server 3 and the communication control unit 4.

[0028] The CPU 102 performs calculations for executing the processes performed by the user information control server 2. The CPU 102 functions as a display information data acquisition unit that acquires display information data 109 about the recommended information currently being displayed on the information display terminal.

[0029] The memory 103 stores a program for realizing the functions of the user information acquisition unit 105, the user interest calculation unit 106, the user attention calculation unit 107, and the recommendation suitability calculation unit 108, and the CPU 102 executes this program.

[0030] The user information acquiring unit 105 acquires user information including gaze information indicating the user's gaze toward the display area of ​​the information display terminal 5. The user information acquiring unit 105 acquires, as the user information, behavioral information of the user and attribute information of the user in addition to the gaze information.

[0031] The above-mentioned gaze information is acquired by at least one of an gaze tracking device built into the information display terminal 5 and a device installed in the external environment, and is projected as a gaze center point onto a gaze detection surface that takes the form of at least one of a two-dimensional surface and a three-dimensional shape.

[0032] The above-mentioned behavioral information includes at least one of the user's posture information, operation information, heart rate, facial expression, and voice information, and is acquired by at least one of the information display terminals 5a, 5b, a device installed in the external environment, and a device worn by the user.

[0033] The above-mentioned attribute information includes at least one of the user's age, gender, preference information, online purchase history, and browsing history, and is obtained from at least one of a device owned by the user or an external server device on which the user's attribute information is registered.

[0034] The user information acquisition unit 105, which will be described in detail later, acquires user information from the information display terminal 5 and calculates attribute data 110 relating to the user's attributes, gaze data 111 relating to the user's gaze, and behavior data 112 relating to the user's behavior. In addition, part of the user information may be acquired from a user information database 8 connected via the network 6.

[0035] The user interest calculation unit 106, which will be described in detail later, calculates interest level data 113, which is information on the degree of interest the user has in each piece of recommended information, based on user information for a certain period of time. That is, the user interest calculation unit 106 estimates, for example, in real time, the interest level, which indicates the degree of interest the user has in each piece of recommended information, based on user information acquired over a predetermined period of time.

[0036] The user interest calculation unit 106 uses the gaze information to add up values ​​for each piece of recommended information when, for example, the gaze center point is included within the display area of ​​the recommended information, and sets the cumulative value over a certain period of time as the interest level for each piece of recommended information.

[0037] The user interest degree calculation unit 106 uses the gaze information to subtract the degree of interest in each piece of recommended information when, for example, the gaze center point is not included in the display area of ​​the recommended information.

[0038] The user interest degree calculation unit 106 adjusts the magnitude of the added value and the subtracted value using the line of sight information, the behavior information, and the gaze degree, and changes them as necessary.

[0039] The user attention level calculation unit 107, which will be described in detail later, calculates attention level data 114, which is information on the degree to which the user is currently paying attention to each piece of recommended information, based on user information at a certain point in time. That is, the user attention level calculation unit 107 estimates, for example, in real time, the attention level, which indicates the degree to which the user is currently paying attention to each piece of recommended information, based on currently acquired user information.

[0040] The user gaze degree calculation unit 107 calculates a value based on the distance between the current gaze center point obtained using the gaze information and the display area of ​​the recommendation information, and sets the gaze degree for each recommendation information.

[0041] The recommendation suitability calculation unit 108, which will be described in detail later, calculates suitability data 115, which is information on the degree to which the user would find it preferable for certain recommended information to be displayed to the user. That is, the recommendation suitability calculation unit 108 calculates the suitability of each piece of recommended information based on the interest level and the attention level.

[0042] The storage device 104 stores data used by the program executed by the CPU 102. The storage device 104 stores various data, such as display information data 109, attribute data 110, gaze data 111, behavior data 112, interest level data 113, gaze level data 114, and preference level data 115.

[0043] The display information data 109, which will be described in more detail later, is information regarding the recommended information currently being displayed on the information display terminal 5 obtained from the recommended information control server 3, and includes the position and size of the information display area, as well as the content of the recommended information.

[0044] The attribute data 110, which will be described in detail later, includes information about the user's attributes, such as the user's age, gender, preferences, advertising history, purchase history, and browsing history, obtained from the information display terminal 5, the user information database 8, and the external input / output device 10.

[0045] The gaze data 111 includes information relating to the user's gaze, acquired from the information display terminal 5 and the external input / output device 10, as will be described in detail later.

[0046] The behavior data 112 includes information about the user's behavior acquired from the information display terminal 5 and the external input / output device 10, which will be described in detail later.

[0047] As described above, the interest level data 113 is information indicating the degree of interest the user has in each piece of recommended information being displayed, calculated by the user interest level calculation unit 106. The interest level data 113 corresponding to each piece of recommended information may be a single piece of data or may be handled as data having multiple regions.

[0048] As described above, the attention level data 114 is information on the degree to which the user is paying attention to each piece of recommended information being displayed at a certain point in time. The attention level data 114 corresponding to each piece of recommended information may be a single piece of data or may be handled as data having multiple regions.

[0049] As described above, the desirability data 115 is information indicating the degree to which the user feels it is desirable to receive information recommendation for each piece of recommended information being displayed. The desirability data 115 corresponding to each piece of recommended information may be a single piece of data or may be handled as data having multiple regions.

[0050] 3 is a diagram showing the configuration of the recommendation information control server 3. As shown in the figure, the recommendation information control server 3 includes communication I / Fs 121a and 121b, a CPU 122, a memory 123, and a storage device 124. The communication I / Fs 121a and 121b exchange information with the user information control server 2 and the communication control unit 4.

[0051] The CPU 122 performs calculations to execute the processing performed by the recommended information control server 3. The CPU 122 functions as a display information data acquisition unit that acquires display information data 127 about the recommended information currently displayed on the information display terminal from the user information control server 2 as display information data 109. Furthermore, the CPU 122 functions as a content data acquisition unit that acquires content data related to the recommended information acquired by the recommended information acquisition unit 125 from the content database 9 via the network 6.

[0052] The memory 123 stores a program for realizing the functions of the recommended information acquisition unit 125 and the display information control unit 126, and the CPU 122 executes this program.

[0053] The recommended information acquisition unit 125, which will be described in detail later, requests recommended information from the information recommendation system 7 connected via the network 6 based on user-related data 128 including user attribute information and behavioral information acquired from the user information control server 2, acquires the recommended information from the information recommendation system 7, and calculates it as new recommended information data 129. That is, the recommended information acquisition unit 125 acquires new recommended information from the information recommendation system 7 based on the user information and the interest level.

[0054] The display information control unit 126 determines the timing to switch to suitable recommended information based on the suitability level, interest level, and new recommended information, and requests the information display terminal 5 to switch the recommended information being displayed.

[0055] The display information control unit 126 selects recommended information with a high degree of interest as preferred information, calculates the information similarity between the preferred information and recommended information whose preference is below a threshold, and switches the display of recommended information whose information similarity is below the threshold to the display of new recommended information selected based on the preferred information from a new group of recommended information acquired from the recommended information acquisition unit 125 when the degree of attention is below the threshold.

[0056] The display information control unit 126 selects recommended information whose suitability is equal to or greater than a threshold as suitable information, calculates the information similarity between the suitable information and new recommended information acquired from the recommended information acquisition unit 125, and switches the display of the suitable information to the display of new recommended information selected based on the suitable information from the new group of recommended information acquired from the recommended information acquisition unit 125 when the degree of attention is equal to or greater than a threshold.

[0057] The display information control unit 126 selects recommended information whose suitability is equal to or greater than a threshold as suitable information, and switches the display of recommended information that is located within a certain distance from the suitable information and whose information similarity with the suitable information is equal to or less than a threshold to the display of new recommended information selected based on the suitable information from a new group of recommended information acquired from the recommended information acquisition unit 125 when the degree of attention is equal to or greater than a threshold.

[0058] The storage device 124 stores data used by the programs executed by the CPU 122. The storage device 124 stores various data, such as display information data 127, user-related data 128, and new recommended information data 129.

[0059] The display information data 127, which will be described in detail later, is information about the recommended information being displayed on the information display terminal 5, obtained from the recommended information control server 3, and includes the position and size of the information display area and the content of the recommended information. The display information data 127 is data in the same format as the display information data 109.

[0060] The user-related data 128 is information about the user acquired by the user information control server 2, and includes attribute data 110, gaze data 111, behavioral data 112, interest level data 113, attention level data 114, and preference level data 115 stored in the storage device 104 of the user information control server 2.

[0061] The new recommended information data 129 includes recommended information acquired from the recommended information control server 3 that can be displayed next on the information display terminal 5 .

[0062] The information display terminal 5 is, for example, at least one of an information display terminal 5a, which is a fixed signage device, and an information display terminal 5b, which is a user-wearable virtual space display device. The recommendation information arranged in the display area of ​​the information display terminals 5a and 5b is, for example, at least one of a two-dimensional surface and a three-dimensional shape.

[0063] FIG. 4 is a sequence diagram illustrating the operation of the information retrieval support system 1. As shown in FIG. Here, there is shown the operations of the information display terminal 5, the user information control server 2, and the recommended information control server 3. Note that the operations of some of the steps shown in the figure will be described in more detail later.

[0064] First, when a user takes action on the information display terminal 5, for example, when the user stands in front of a fixed information display terminal 5 or when the user puts on a wearable information display terminal 5, the screen display application is launched and information is displayed on the display screen (step S201).

[0065] The information display terminal 5 acquires user information using an attached input device (step S202). At this time, the information display terminal 5 may acquire the user information in cooperation with the external input / output device 10. Next, information relating to the information display area in which the recommended information is displayed is acquired.

[0066] The acquired user information and information display area are sent to the user information control server 2 and received by the user information control server 2 (step S203). Here, the information display area is treated as static data, but it may also be treated as dynamic data and sent to the user information control server 2 periodically.

[0067] The user information control server 2 stores the received user information as attribute data 110, and stores the received information display area as part of the display information data 109. Next, the user information acquisition unit 105 requests attribute information from the user information database 8 based on information included in the user information, for example, the user ID (step S204). The requested attribute information is sent to the user information control server 2, and is received by the user information control server 2. Upon receiving the necessary information, the user information control server 2 updates the attribute data 110. Furthermore, the user information control server 2 transmits the data to the recommendation information control server 3 (step S205).

[0068] Upon receiving the attribute data 110 , the recommendation information control server 3 stores it as part of the user-related data 128 .

[0069] Next, in the recommended information control server 3, the recommended information acquisition unit 125 requests recommended information from the information recommendation system 7 based on the user-related data 128 (step S206). The requested recommended information is sent to the recommended information control server 3, and the recommended information control server 3 receives it as new recommended information data 129.

[0070] Furthermore, in the recommended information control server 3, the display information control unit 126 calculates the display information data 127 based on the new recommended information data 129 (step S207), as will be described in detail later. Next, the display information data 127 is transmitted to the information display terminal 5, and at the same time, a request to switch the display screen is transmitted (step S208).

[0071] The information display terminal 5 receives the display information data 127, and updates the display on the display device upon receiving the request to switch the display screen (step S209).

[0072] The information display terminal 5 acquires user information using an attached input device and the external input / output device 10 (step S210). The user information includes gaze information and behavior information of the user. The acquired user information is sent to the user information control server 2 and is received by the user information control server 2 (step S211).

[0073] The user information control server 2 calculates gaze data 111 and behavior data 112 based on the gaze information and behavior information of the user included in the received user information.

[0074] Next, in the user information control server 2, the user interest calculation unit 106 calculates the user's interest in each piece of recommended information being displayed based on the gaze data 111 and behavior data 112, and stores it as interest data 113 (step S212), as will be described in detail later.

[0075] In the user information control server 2, the user attention degree calculation unit 107 calculates the user's attention degree for each piece of recommended information being displayed based on the gaze data 111 and the behavior data 112, and stores the calculated degree of attention degree data 114 (step S213), as will be described in detail later. In addition, the interest degree data 113 is sent to the recommended information control server 3, and the recommended information control server 3 stores the interest degree data 113 as part of the user-related data 128 (step S214).

[0076] Furthermore, in the recommended information control server 3, the recommended information acquisition unit 125 requests recommended information from the information recommendation system 7 based on the user-related data 128 (step S215). The requested recommended information is sent to the recommended information control server 3, and the recommended information control server 3 receives it as new recommended information data 129.

[0077] Next, in the user information control server 2, the recommendation suitability calculation unit 108 calculates suitability as the degree to which the user would feel that the display of each piece of recommended information currently being displayed is appropriate, based on the interest data 113 and the attention data 114, as will be described in detail later, and stores the calculated suitability as suitability data 115 (step S216).

[0078] The preference data 115 is sent to the recommendation information control server 3, and is stored by the recommendation information control server 3 as part of the user-related data 128 (step S217).

[0079] In the recommended information control server 3, the display information control unit 126 calculates the display information data 127 based on the user-related data 128 and the new recommended information data 129 (step S218), which will be described in detail later. The display information data 127 is transmitted to the information display terminal 5, and the information display terminal 5 receives it (step S219).

[0080] In the recommended information control server 3, the display information control unit 126 determines the timing to switch the recommended information based on the gaze data included in the user-related data 128 (step S220), and sends a request to switch the display screen to the information display terminal 5, which receives it (step S221).

[0081] Upon receiving the request to switch the display screen, the information display terminal 5 updates the display on the display device based on the display information data 127 (step S222). When updating the display, audio information may be presented to the user using an audio output device provided in the information display terminal 5 and the external input / output device 10.

[0082] The subsequent steps S210 to S222 may be repeated.

[0083] FIG. 5 is a flowchart detailing the processing performed by the user interest degree calculation unit 106 of the user information control server 2 in step S212 of FIG.

[0084] First, the user interest degree calculation unit 106 reads the display information data 109 (step S301). As described above, the display information data 109 includes the recommended information acquired from the information recommendation system 7 and an information display area including the display position and size of the recommended information.

[0085] Next, the user interest level calculation unit 106 reads the gaze data 111, the behavior data 112, and the interest level data 113 (step S302). As described above, the gaze data 111 is information about the user's gaze, such as the direction of the user's gaze and the duration of the gaze. As described above, the behavior data 112 is information about the user's behavior, such as the user's body posture information, operation information, heart rate, facial expression, and voice information. As described above, the interest level data 113 is information about the degree to which the user is interested in each piece of recommended information currently being displayed.

[0086] Next, the user interest degree calculation unit 106 sequentially selects recommended information from the display information data 109 (step S303). Next, the user interest degree calculation unit 106 reads the information display area and interest level corresponding to the selected recommended information based on the display information data 109 and the interest level data 113 (step S304). Next, the gaze center point is obtained from the gaze data 111, and it is determined whether or not it is within the read information display area (step S305).

[0087] As a result, if the gaze center point is within the corresponding information display area (Yes in step S305), the user interest degree calculation unit 106 calculates an additional value to be added to the interest degree based on the gaze data 111 and the behavior data 112 (step S306). Specifically, for example, the additional value can be calculated based on the user's heart rate included in the behavior data 112.

[0088] Next, an additional value is added to the corresponding interest level (step S307). If the gaze center point is not within the corresponding information display area (No in step S305), a fixed value is subtracted from the corresponding interest level (step S308).

[0089] After steps S307 and S308, the user interest degree calculation unit 106 determines whether or not the processing has been completed for all the recommended information in the display information data (step S309).

[0090] As a result, if the processing is not completed (No in step S309), the process returns to step S303.

[0091] On the other hand, if the processing is completed (Yes in step S309), the user interest degree calculation unit 106 updates the interest degree data 113 based on the corresponding interest degree (step S310). The user interest degree calculation unit 106 ends the user interest degree calculation process (step S212 in FIG. 4).

[0092] FIG. 6 is a flowchart detailing the process performed by the user gaze degree calculation unit 107 of the user information control server 2 in step S215 of FIG.

[0093] First, the user attention degree calculation unit 107 reads the display information data 109 (step S321). Next, the user attention degree calculation unit 107 reads the gaze data 111, the behavior data 112, and the attention degree data 114 (step S322). As described above, the attention degree data 114 is information on the degree to which the user is paying attention to each piece of recommended information being displayed at a certain point in time.

[0094] Next, the user gaze degree calculation unit 107 reads the gaze center point based on the gaze data 111 (step S323).

[0095] The user attention degree calculation unit 107 sequentially selects recommended information from the display information data 109 (step S324). The user attention degree calculation unit 107 reads an information display area corresponding to the selected recommended information based on the display information data 109 and the interest level data 113 (step S325).

[0096] Next, the user gaze degree calculation unit 107 sequentially selects pixels included in the display area (step S326). The distance between the display position of each pixel and the gaze center point is calculated (step S327). Furthermore, based on the calculated distance and the behavior data 112, an additional value to be added to the gaze degree is calculated (step S328).

[0097] Specifically, for example, the added value can be calculated by calculating a weighting value based on the user's body posture information included in the behavior data 112 using the cosine similarity between a vector representing the user's face direction and a vector representing the gaze data 111. Note that the gaze degree may be calculated using only the distance from the center of the display area.

[0098] Next, the additional value is added to the corresponding gaze degree (step S329).The user gaze degree calculation unit 107 determines whether or not the process has been completed for all pixels in the information display area (step S330).

[0099] As a result, if the calculation process has not been completed for all pixels in the display area (No in step S330), user gaze degree calculation unit 107 returns to step S326.

[0100] When the process is completed (Yes in step S330), the user gaze degree calculation unit 107 determines whether the process is completed for all the recommended information in the display information data (step S331).

[0101] As a result, if the calculation process has not been completed for all the recommendation information in the display information data (No in step S331), the user gaze degree calculation unit 107 returns to step S324. On the other hand, if the process has been completed (Yes in step S331), the user gaze degree calculation unit 107 updates the gaze degree data 114 based on the corresponding gaze degree (step S332). The user gaze degree calculation unit 107 ends the user gaze degree calculation process (step S213 in FIG. 4).

[0102] FIG. 7 is a flowchart detailing the process performed by the recommendation suitability calculation unit 108 of the user information control server 2 in step S216 of FIG.

[0103] First, the recommendation suitability calculation unit 108 reads the interest level data 113 (step S341), and then reads the attention level data 114 (step S342).

[0104] The recommendation suitability calculation unit 108 sequentially acquires the interest level and attention level corresponding to each piece of recommendation information based on the read interest level data 113 and attention level data 114 (step S343).

[0105] The recommendation suitability calculation unit 108 calculates a suitability from the interest level and attention level corresponding to each piece of recommendation information thus obtained (step S344). As described above, the suitability level is the degree to which the user feels that the display of each piece of recommendation information currently being displayed is appropriate. The suitability level z_i can be derived, for example, by the following formula (1) using the interest level x_i and attention level y_i corresponding to a piece of recommendation information i. z_(i)=ax_(i)+bx_(i) (1)

[0106] Here, the coefficients a and b are constants, and the recommendation suitability calculation unit 108 can adjust the coefficients based on the user's reaction to the recommendation result. Note that the interest level x_i and the attention level y_i are not limited to scalar values, and may be vector values.

[0107] Furthermore, the recommendation suitability calculation unit 108 determines whether the recommendation suitability calculation process has been completed for all the recommended information (step S345). As a result, if the process has not been completed for all the recommended information (No in step S345), the recommendation suitability calculation unit 108 returns to step S343.

[0108] On the other hand, if the processing has been completed for all the recommendation information (Yes in step S345), the recommendation suitability calculation unit 108 updates the suitability data (step S346). The recommendation suitability calculation unit 108 ends the recommendation suitability calculation process (step S214 in FIG. 4).

[0109] FIG. 8 is a flowchart detailing the processing performed by the display information control unit 126 of the recommended information control server 3 in steps S218 and S220 of FIG.

[0110] First, the display information control unit 126 reads the user-related data 128 created in steps S212, S213, and S215 in FIG. 4 (step S361). Next, the display information control unit 126 reads the display information data 127 (step S362).

[0111] Next, the display information control unit 126 selects the recommended information with the highest interest level as preferred information based on the interest level included in the user-related data 128 and the recommended information included in the display information data 127 (step S363). Here, the preferred information is selected as a single piece of recommended information, but the preferred information may be composed of multiple pieces of recommended information.

[0112] Next, the display information control unit 126 sequentially selects the recommended information included in the display information data 127 (step S364). That is, the display information control unit 126 repeats the following process until all the recommended information included in the display information data 127 has been selected.

[0113] The display information control unit 126 acquires the preference score corresponding to the selected recommendation information and determines whether it is below a threshold (step S365). If the preference score is below a certain threshold (Yes in step S365), the display information control unit 126 calculates the information similarity between the selected recommendation information and the preference information (step S366).

[0114] Furthermore, the display information control unit 126 determines whether or not the selection of all the recommended information in the display information data has been completed (step S367).

[0115] In steps S365 and S367, if the preference level is not below a certain threshold (No in step S365) and if the selection of all recommended information has not been completed (No in step S367), the display information control unit 126 returns to step S364.

[0116] On the other hand, if the selection of all recommended information has been completed in step S367 (Yes in step S367), the display information control unit 126 selects the recommended information with the lowest information similarity to the preferred information from among the recommended information for which step S366 has been executed so far (step S368), and determines whether the information similarity is below a threshold value (step S369).

[0117] As a result, if the information similarity is not equal to or less than the threshold (No in step S369), the display information control unit 126 ends the process. On the other hand, if the information similarity is equal to or less than the threshold (Yes in step S369), the display information control unit 126 sets the recommended information selected in step S368 as the recommended information to be switched to (step S370). The display information control unit 126 reads new recommended information data 129 acquired by the recommended information acquisition unit (step S371).

[0118] The display information control unit 126 selects the recommended information to switch to based on the new recommended information data 129 and the preferred information (step S372). Specifically, for example, the display information control unit 126 may select recommended information that has a high degree of information similarity to the preferred information, or may acquire new recommended information based on the preferred information in cooperation with the information recommendation system 7 via the recommended information acquisition unit 125.

[0119] Next, the display information control unit 126 updates the display information data 109 by swapping the original recommendation information with the new recommendation information (step S373). Furthermore, the display information control unit 126 transmits the updated display information data 109 to the information display terminal 5 (step S374, step S219 in FIG. 4). The display information control unit 126 acquires the display area of ​​the original recommendation information from the display information data 109 (step S375).

[0120] Next, the display information control unit 126 acquires the gaze degree included in the user-related data 128 (step S376). Note that the display information control unit 126 may connect to the user information control server 2 to acquire the latest gaze degree data 114.

[0121] The display information control unit 126 determines whether the gaze degree corresponding to the display area of ​​the recommendation information is equal to or less than the threshold (step S377). As a result, if the gaze degree is not equal to or less than the threshold (No in step S377), the display information control unit 126 returns to step S376. On the other hand, if the gaze degree is equal to or less than the threshold, the display information control unit 126 requests the information display terminal 5 to switch the screen display (step S378, step S221 in FIG. 4), and ends the process.

[0122] 9 is a diagram showing the contents of the display information data 109 / 127. The display information data 109 / 127 shown in the figure consists of an information display area 401 and recommended information 402. The information display area 401 includes a display ID 403, a visibility rate 404, and display parameters 405.

[0123] Display ID 403 is an ID assigned to each information display area of ​​information display terminal 5. Here, it is shown that information display terminal 5 has at least four information display areas, and A01, A02, A03, and A04 are set for each of them as display ID 403. Note that the information display areas may change dynamically over time.

[0124] The visibility rate 404 is the ease with which the information display area corresponding to each information display area can be seen, and is calculated, for example, using user information acquired by the information display terminal 5, based on the percentage of people whose central point of viewpoint includes the information display area displayed on the information display terminal 5. Note that the visibility rate 404 may use a fixed value, or the value may change in accordance with changes in the information display area.

[0125] Display parameters 405 are parameters related to the information display area corresponding to display ID 403. Here, the position and size are shown on the assumption that the information display area is arranged on a two-dimensional plane. Note that if the information display area is arranged in three-dimensional space, display parameters 405 may include information related to the posture in addition to the position and size.

[0126] The recommendation information 402 includes an information ID 406 , a recommendation level 407 , an information category 408 , corresponding content 409 , and a relationship tree 410 .

[0127] The information ID 406 is an ID assigned to each piece of recommendation information acquired by the recommendation information acquisition unit 125 of the recommendation information control server 3. Here, it is shown that I003, I005, I002, and I007 are set as information IDs 406 for information display areas corresponding to A01, A02, A03, and A04, which are display IDs 403. Note that the number of information IDs 406 set in an information display area corresponding to a certain display ID 403 may be one or more.

[0128] The recommendation level 407 is the degree to which the recommended information is recommended, acquired from the information recommendation system 7 via the recommended information control server 3 .

[0129] The information category 408 is an attribute of the recommended information acquired from the information recommendation system 7 via the recommended information acquisition unit 125. Here, recommended information for general consumers, such as fashion, beauty appliances, and cosmetics, is shown.

[0130] The corresponding content 409 is information on the content corresponding to the information ID, obtained from the content database 9. Here, it indicates the storage location of the file, but it may also be address information (URL) for obtaining the content via the Internet.

[0131] The relationship tree 410 is the relationship between recommended information items acquired from the information recommendation system 7 via the recommended information acquisition unit 125. For example, it is the information ID of another recommended information item that is frequently viewed by a user who has viewed a certain recommended information item.

[0132] 10A and 11A are diagrams each illustrating a case where recommended information is displayed on the fixed information display terminal 5a in step S222 of FIG.

[0133] Here, an example is shown in which recommended information having information IDs A01 to A06 is displayed on an information display terminal 5a, which is a fixed signage device, based on the display information data exemplified in FIG. 9. Note that, although an example is shown in which an image is displayed as the content of the recommended information, the display object does not have to be limited to this. In other words, the display format of the content corresponding to the recommended information is not limited to image data, but may also be text data, video data, audio data, or a three-dimensional model. There may be one or more pieces of content corresponding to each piece of recommended information, and the display format may be different for each piece.

[0134] 10A shows an example of the arrangement of recommendation information, as well as the movement of gaze over a certain period of time and the current gaze position. Here, black circles indicate stationary points of gaze, with larger circles representing longer stationary periods, and lines connecting multiple black circles or entering or exiting black circles represent gaze (similar to FIG. 11A).

[0135] FIG. 10B shows an example of the interest level calculated by the user interest level calculation unit, the attention level calculated by the user attention level calculation unit, and the preference level calculated by the recommendation preference level calculation unit in the case of FIG. 10A.

[0136] Next, Fig. 11A shows the movement of the gaze and the current gaze position when the user's gaze moves to A04 immediately after Fig. 10A. Fig. 11B shows an example of the interest degree calculated by the user interest degree calculation unit, the attention degree calculated by the user attention degree calculation unit, and the preference degree calculated by the recommendation preference degree calculation unit in the case of Fig. 11A.

[0137] In the first embodiment, the display information control unit 126 controls the display information, for example, according to the flow shown in Fig. 8. In the cases of Fig. 10A and Fig. 10B, if the display information data 109 / 127 used is the one exemplified in Fig. 9, I003 corresponding to A01 is selected as the preferred information, and then recommended information whose preferred degree is below the threshold is set as the recommended information candidate to be switched to.

[0138] Next, the threshold value of the suitability is set to 0.30, and the information similarity between the recommended information I007 corresponding to A04 and the recommended information I003 corresponding to A01 is calculated. Here, referring to the relationship tree 410, it is assumed that the information similarity between I003 and I007 is high, so I007 corresponding to A04 is not selected as the recommended information to switch from.

[0139] 11A and 11B, I003 corresponding to A01 will be selected as the preferred information, and I002 corresponding to A03 will be selected as the original recommended information candidate. Here, I003 and I002 belong to different information categories 408 and are not in a tree relationship, so there is a possibility that the information similarity between them is low. If the information similarity is low, I002 corresponding to A03 may be selected as the original recommended information.

[0140] According to the first embodiment, the recommended information displayed in a limited display area can be switched based on the user's line of sight and behavior, thereby improving user convenience and information search efficiency. Furthermore, according to the first embodiment, by switching information at an appropriate time based on the user's line of sight and behavior, information search can be supported without user operation and without impairing the user experience.

[0141] (2) Second embodiment The information display system including the information search support system according to the second embodiment has almost the same configuration and functions as the information display system S including the information search support system 1 according to the second embodiment, so we will omit explanations of the similarities and focus on the differences below.

[0142] In the second embodiment, a user wears an information display terminal 5b, which is a wearable virtual space display device such as XR goggles, and can access stores and sales areas in the virtual space via the Internet. When the user accesses the virtual space, the information display terminal 5 arranges recommended information in the virtual space based on the user's attributes and behavior information and displays it on the device. At this time, the information retrieval support system 1 and the information display terminal 5 select recommended information that the user is interested in based on the user's gaze information, and when the user's gaze is directed around the selected recommended information, display new recommended information around the selected recommended information.

[0143] In the second embodiment, the recommended information suitable for the user can be switched at a suitable time in the same manner as in the first embodiment. However, unlike the first embodiment, the second embodiment switches information within the user's line of sight. Therefore, the display control method of the display information control unit 126 of the recommended information control server 3 differs between the first and second embodiments.

[0144] Fig. 12 is a flowchart detailing the processing performed by the display information control unit 126 of the recommendation information control server 3 in steps S218 and S220 in Fig. 4 in the second embodiment. The flowchart shown in Fig. 12 in the second embodiment differs from the flowchart shown in Fig. 8 in the first embodiment in that it includes steps S384 and S386, which will be described later.

[0145] First, the display information control unit 126 reads the user-related data 128 created in steps S212, S213, and S215 in Fig. 4 (step S381). Next, the display information control unit 126 reads the display information data 127 (step S382).

[0146] Next, the display information control unit 126 selects the recommended information with the highest degree of suitability as the preferred information based on the suitability included in the user-related data 128 and the recommended information included in the display information data 127 (step S383). Here, the preferred information is selected as a single piece of recommended information, but the preferred information may be composed of multiple pieces of recommended information.

[0147] Next, the display information control unit 126 determines whether the suitability of the selected suitable information is equal to or greater than the threshold (step S384). If the suitability of the suitable information is less than the threshold (No in step S384), the display information control unit 126 ends the process.

[0148] Next, if the suitability of the suitable information is equal to or greater than the threshold (Yes in step S384), the display information control unit 126 sequentially selects each item of the display information data 127 (step S385). That is, the display information control unit 126 repeats the subsequent processes until all of the display information data 127 has been selected.

[0149] The display information control unit 126 determines whether the display distance between the selected recommended information and the preferred information is below a threshold based on the display area corresponding to the selected recommended information and the display area corresponding to the preferred information (step S386). As a result, if the display distance is below a certain threshold (Yes in step S386), the display information control unit 126 calculates the information similarity between the selected recommended information and the preferred information (step S387).

[0150] Furthermore, the display information control unit 126 determines whether or not the selection of all the recommended information in the display information data has been completed (step S388). In steps S386 and S388, if the display distance is not below a certain threshold (No in step S386) or if the selection of all recommended information has not been completed (No in step S388), the display information control unit 126 returns to step S385.

[0151] If all the recommended information has been selected in step S388 (Yes in step S388), the display information control unit 126 selects the recommended information having the lowest information similarity with the preferred information from among the recommended information for which step S387 has been executed (step S389), and determines whether the information similarity is equal to or less than a threshold (step S390). If the information similarity is not equal to or less than the threshold (No in step S390), the display information control unit 126 ends the process.

[0152] If the information similarity is equal to or less than the threshold value (Yes in step S390), the display information control unit 126 sets the recommended information selected in step S389 as the recommended information to be switched to (step S391).

[0153] The display information control unit 126 reads the new recommended information data 129 acquired by the recommended information acquisition unit (step S392).

[0154] The display information control unit 126 selects the recommended information to be switched to based on the new recommended information data 129 and the preferred information (step S393). Specifically, for example, the display information control unit 126 may select recommended information that has a high degree of information similarity with the preferred information, or may acquire new recommended information based on the preferred information in cooperation with the information recommendation system 7 via the recommended information acquisition unit 125.

[0155] Next, the display information control unit 126 updates the display information data 109 by swapping the original recommendation information with the new recommendation information (step S394). Furthermore, the display information control unit 126 transmits the updated display information data 109 to the information display terminal 5 (step S395, step S219 in FIG. 4). The display information control unit 126 acquires the display area of ​​the original recommendation information from the display information data 109 (step S396).

[0156] Next, the display information control unit 126 acquires the gaze degree included in the user related data 128 (step S397). Note that the display information control unit 126 may connect to the user information control server 2 and acquire the latest gaze degree data 114.

[0157] The display information control unit 126 determines whether the gaze degree corresponding to the display area of ​​the recommendation information is equal to or greater than a threshold (step S398). As a result, if the gaze degree is below the threshold (No in step S398), the display information control unit 126 returns to step S397. On the other hand, if the gaze degree is equal to or greater than the threshold, the display information control unit 126 requests the information display terminal 5 to switch the screen display (step S399, step S221 in FIG. 4), and ends the process.

[0158] In the second embodiment, based on the user's line of sight information, new recommended information can be displayed at a time when the user's line of sight is directed to the vicinity of recommended information in which the user is interested.

[0159] (3. Explanation of information search support methods) The above-described processes performed by the information retrieval support system 1 are realized by the cooperation of software and hardware resources. That is, a processor within a computer device provided in the information retrieval support system 1 loads into memory and executes software that realizes each of the above-described functions, thereby realizing each of these functions.

[0160] Therefore, the processing performed by the information search support system 1 can be considered as an information search support method in which the processor executes software recorded in memory to obtain user information data regarding the user's attributes, gaze information, and behavioral information, cooperates with the information recommendation system 7 to obtain display information data regarding recommended information and information display areas, and displays recommended information appropriate for the user at an appropriate time based on the user information data and display information data.

[0161] Furthermore, this embodiment can be considered as a display control method from the viewpoint of switching the display of information to be given to a user at a timing suitable for the user based on user information data and display information data.

[0162] The information display system S according to this embodiment includes at least an information display terminal 5 that displays recommended information, and an information search support system 1 as described below. The information search support system 1 is an information search support system that cooperates with an information recommendation system 7 that provides recommended information to an information display terminal 5 that displays recommended information to a user, and includes a user information acquisition unit 105 that acquires user information including gaze information that indicates the user's gaze toward the information display terminal 5, a user interest calculation unit 106 that estimates in real time an interest level that indicates the degree of interest the user has in each piece of recommended information based on user information acquired over a predetermined period of time, a user attention level calculation unit 107 that estimates in real time an attention level that indicates the degree to which the user is currently paying attention to each piece of recommended information based on currently acquired user information, a recommendation suitability calculation unit 108 that calculates a suitability level for each piece of recommended information based on the interest level and attention level, a recommendation information acquisition unit 125 that acquires a new group of recommended information from the information recommendation system 7 based on the user information and the interest level, and a display information control unit 126 that determines the time to switch to suitable recommended information based on the suitability level, interest level, and new recommended information, and requests the information display terminal 5 to switch the recommended information currently being displayed.

[0163] In this way, since no operation by the user is required, the user's convenience is not impaired, and the search efficiency for recommended information is improved, and recommended information that is likely to attract the user's interest can be switched and displayed.

[0164] In this embodiment, the information display terminal 5 is, for example, at least one of the information display terminal 5a, which is a fixed signage device, and the information display terminal 5b, which is a user-wearable virtual space display device, and the recommendation information arranged in the display area of ​​the information display terminals 5a and 5b is at least one of a two-dimensional surface and a three-dimensional shape. In this way, it is possible to switch to recommendation information that is more likely to attract the user's interest without requiring user operation, by displaying it on, for example, at least one display device of the information display terminal 5a, which is a fixed signage device, and the information display terminal 5b, which is a user-wearable virtual space display device.

[0165] In this embodiment, the user information acquiring unit 105 acquires, as user information, not only gaze information but also user behavior information and user attribute information. In this way, it is possible to switch to and display recommended information that is likely to attract the user's interest based on the user behavior information and user attribute information without requiring user operation.

[0166] In this embodiment, the gaze information is acquired by at least one of an eye-tracking device built into the information display terminal 5 and a device installed in the external environment, and is projected as a gaze center point onto a gaze detection surface that takes the form of at least one of a two-dimensional plane and a three-dimensional shape. In this way, by projecting the gaze center point onto the gaze detection surface that takes the form of at least one of a two-dimensional plane and a three-dimensional shape, it is possible to switch to and display recommended information that is more likely to attract the user's interest, without requiring any operation by the user.

[0167] In this embodiment, the behavioral information includes, for example, at least one of the user's posture information, operation information, heart rate, facial expression, and voice information, and is acquired by at least one of a device installed in the information display terminal 5 and a device worn by the user. In this way, at least one of the user's posture information, operation information, heart rate, facial expression, and voice information is acquired by at least one of a device installed in the information display terminal 5 and a device worn by the user, so that recommended information that is likely to attract the user's interest can be switched to and displayed based on the behavioral information including at least one of the user's posture information, operation information, heart rate, facial expression, and voice information, without requiring user operation.

[0168] In this embodiment, the attribute information includes, for example, at least one of the user's age, gender, preference information, online purchase history, and browsing history, and is acquired from a device owned by the user or an external server device (not shown) in which the user's attribute information is registered. In this way, because the attribute information is acquired from a device owned by the user or an external server device (not shown) in which the user's attribute information is registered, it is possible to switch to and display recommended information that is likely to attract the user's interest, without requiring user operation, based on the attribute information including at least one of the user's age, gender, preference information, online purchase history, and browsing history.

[0169] In this embodiment, the user interest degree calculation unit 106 uses the gaze information to add up values ​​for each piece of recommended information when the gaze center point is included in the display area of ​​the recommended information, and sets the cumulative value over a certain period of time as the interest degree for each piece of recommended information. In this way, without requiring user operation, it is possible to switch to and display recommended information that is more likely to attract the user's interest based on the cumulative value over a certain period of time for the interest degree for each piece of recommended information.

[0170] In this embodiment, the user gaze degree calculation unit 107 sets a value calculated based on the distance between the current gaze center point obtained using gaze information and the display area of ​​the recommended information as the gaze degree for each piece of recommended information. In this way, without requiring user operation, it is possible to switch to and display recommended information that is more likely to attract the user's interest based on the gaze degree for each piece of recommended information calculated based on the distance between the current gaze center point and the display area of ​​the recommended information.

[0171] In this embodiment, the user interest degree calculation unit 106 uses the gaze information to subtract the interest degree for each piece of recommended information when the gaze center point is not included in the display area of ​​the recommended information. In this way, it is possible to switch to and display recommended information that is more likely to attract the user's interest without requiring user operation.

[0172] In this embodiment, the user interest degree calculation unit 106 adjusts the magnitude of the added value and the subtracted value using the gaze information, the behavior information, and the gaze degree. In this way, it is possible to switch to and display recommended information that is more likely to attract the user's interest without requiring user operation.

[0173] In this embodiment, the display information control unit 126 selects recommended information with a high degree of interest as preferred information, calculates the information similarity between the preferred information and recommended information with a preference degree equal to or less than a threshold, and switches the display of recommended information with an information similarity equal to or less than the threshold to the display of new recommended information selected based on the preferred information from a new recommended information group acquired from the recommended information acquisition unit 125 when the degree of attention is equal to or less than the threshold. In this way, it is possible to switch to and display recommended information that is more likely to attract the user's interest according to the degree of attention, without requiring user operation.

[0174] In this embodiment, the display information control unit 126 selects recommended information whose suitability is equal to or greater than a threshold as preferred information, calculates the information similarity between the preferred information and new recommended information acquired from the recommended information acquisition unit 125, and switches the display of the preferred information to the display of new recommended information selected based on the preferred information from the new recommended information group acquired from the recommended information acquisition unit 125 when the gaze degree is equal to or greater than the threshold. In this way, it is possible to switch to and display recommended information that is more likely to attract the user's interest according to the gaze degree, without requiring user operation.

[0175] In this embodiment, the display information control unit 126 selects recommended information whose suitability is equal to or greater than a threshold as preferred information, and switches the display of recommended information that is located within a certain distance from the preferred information and whose information similarity with the preferred information is equal to or less than a threshold to the display of new recommended information selected based on the preferred information from a new recommended information group acquired from the recommended information acquisition unit 125, when the gaze degree is equal to or greater than the threshold. In this way, it is possible to switch to and display recommended information that is more likely to attract the user's interest according to the gaze degree, without requiring user operation.

[0176] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, the elements described in parallel in the present embodiment may be configured such that at least one of the elements is connected in series to the other elements. [Industrial Applicability]

[0177] The present invention can be applied to, for example, an information retrieval support system, an information retrieval support method, and an information display system relating to a technique for optimally searching for recommended information suitable for a user using user behavior information. [Explanation of symbols]

[0178] 1... Information search support system, 2... User information control server, 3... Recommendation information control server, 4... Communication control unit, 5... Information display server, 5a... Signage device, 5b... Virtual space display device, 6... Network, 7... Information recommendation system, 8... User information database, 9... Content database, 10... External input / output device, 101, 121... Communication I / F, 102, 122... CPU, 103, 123... Memory, 104, 124... Storage device, 105...User information acquisition unit, 106...User interest level calculation unit, 107...User gaze level estimation unit, 108...Recommendation suitability calculation unit, 109, 127...Display information data, 110...Attribute data, 111...Gaze data, 112...Behavior data, 113...Interest level data, 114...Gaze level data, 115...Preference level data, 125...Recommendation information acquisition unit, 126...Display information control unit, 128...User related data, 129...New recommendation information data, S...Information display system

Claims

1. An information retrieval support system that cooperates with an information recommendation system that provides recommended information to an information display terminal that displays the recommended information to a user, a user information acquisition unit that acquires user information including line-of-sight information indicating a line of sight of the user toward the information display terminal; a user interest degree calculation unit that estimates in real time an interest degree that indicates the degree of interest the user has in each piece of recommended information based on user information acquired over a predetermined period of time; a user attention level calculation unit that estimates in real time an attention level representing the degree to which the user is currently paying attention to each piece of recommended information based on currently acquired user information; a recommendation suitability calculation unit that calculates a suitability for each piece of recommendation information based on the interest level and the attention level; a recommendation information acquisition unit that acquires new recommendation information from the information recommendation system based on the user information and the interest level; a display information control unit that determines a timing for switching to suitable recommended information based on the suitability, the interest, and the new recommended information, and requests the information display terminal to switch the currently displayed recommended information; An information retrieval support system comprising:

2. The information display terminal includes: At least one of a fixed display device and a user-worn display device; The recommendation information arranged in the display area of ​​the display device is at least one of a two-dimensional surface and a three-dimensional shape.

2. The information retrieval support system according to claim 1.

3. The user information acquisition unit As the user information, in addition to the gaze information, behavior information of the user and attribute information of the user are acquired.

3. The information retrieval support system according to claim 2.

4. The line of sight information is The gaze tracking information is acquired by at least one of an eye tracking device built into the information display terminal and a device installed in the external environment, and is projected as a gaze center point onto a gaze detection surface that takes the form of at least one of a two-dimensional surface and a three-dimensional shape.

4. The information retrieval support system according to claim 3.

5. The behavioral information is At least one of posture information, operation information, heart rate, facial expression, and voice information of the user is included, The display device, a device installed in the external environment, and a device worn by the user are acquired by at least one of the display device, the device installed in the external environment, and the device worn by the user.

5. The information retrieval support system according to claim 4.

6. The attribute information is The information includes at least one of the user's age, gender, preference information, online purchase history, and browsing history, The attribute information of the user is acquired from at least one of a device owned by the user or an external server device in which the attribute information of the user is registered.

6. An information retrieval support system according to claim 5.

7. The user interest calculation unit Using the line of sight information, if the line of sight center point is included in the display area of ​​the recommended information, the values ​​for each piece of recommended information are added together, and the cumulative value for a certain period of time is set as the interest level for each piece of recommended information.

7. The information retrieval support system according to claim 6.

8. The user gaze degree calculation unit A value calculated based on the distance between the current gaze center point obtained using the gaze information and the display area of ​​the recommendation information is set as the gaze degree for each recommendation information.

8. The information retrieval support system according to claim 7.

9. The user interest calculation unit Using the line-of-sight information, if the line-of-sight center point is not included in the display area of ​​the recommended information, the interest level for each recommended information is subtracted.

8. The information retrieval support system according to claim 7.

10. The user interest calculation unit The magnitude of the added value and the subtracted value is adjusted using the line-of-sight information, the behavior information, and the gaze level.

10. The information retrieval support system according to claim 9.

11. The display information control unit The recommended information having a high degree of interest is selected as preferred information, an information similarity between the preferred information and the recommended information having a degree of preference equal to or less than a threshold is calculated, and the display of the recommended information having the information similarity equal to or less than the threshold is switched to a display of new recommended information selected based on the preferred information from a new group of recommended information acquired from the recommended information acquisition unit at the time when the degree of attention is equal to or less than the threshold.

3. The information retrieval support system according to claim 2.

12. The display information control unit The recommended information whose suitability is equal to or greater than a threshold is selected as suitable information, an information similarity between the suitable information and new recommended information acquired from the recommended information acquisition unit is calculated, and the display of the suitable information is switched to a display of new recommended information selected based on the suitable information from the new recommended information group acquired from the recommended information acquisition unit at the time when the degree of attention is equal to or greater than a threshold.

3. The information retrieval support system according to claim 2.

13. The display information control unit The recommended information whose suitability is equal to or greater than a threshold is selected as suitable information, and display of recommended information arranged within a certain distance from the suitable information and having information similarity with the suitable information equal to or less than a threshold is switched to display of new recommended information selected based on the suitable information from a new group of recommended information acquired from the recommended information acquisition unit when the degree of attention is equal to or greater than a threshold.

13. The information retrieval support system according to claim 12.

14. 1. An information retrieval support method using an information retrieval support system linked to an information recommendation system that provides recommended information to an information display terminal that displays the recommended information to a user, comprising: a user information acquisition step in which a user information acquisition unit acquires user information including line-of-sight information indicating a line of sight of the user toward the information display terminal; a user interest degree calculation step in which a user interest degree calculation unit estimates in real time an interest degree representing a degree of interest of the user in each piece of recommended information based on user information acquired over a predetermined time period; a user attention degree calculation step in which a user attention degree calculation unit estimates in real time an attention degree representing a degree to which the user is currently paying attention to each piece of recommendation information based on currently acquired user information; a recommendation suitability calculation step in which a recommendation suitability calculation unit calculates a suitability for each piece of recommendation information based on the interest degree and the attention degree; a recommendation information acquisition step in which a recommendation information acquisition unit acquires new recommendation information from the information recommendation system based on the user information and the interest level; a display information control step in which a display information control unit determines the timing for switching to suitable recommended information based on the suitability level, the interest level, and the new recommended information, and requests the information display terminal to switch the recommended information being displayed; 1. An information retrieval support method comprising:

15. An information retrieval support system that cooperates with an information recommendation system that provides recommended information to an information display terminal that displays the recommended information to a user, a user information acquisition unit that acquires user information including line-of-sight information indicating a line of sight of the user toward the information display terminal; a user interest degree calculation unit that estimates in real time an interest degree that indicates the degree of interest the user has in each piece of recommended information based on user information acquired over a predetermined period of time; a user attention level calculation unit that estimates in real time an attention level representing the degree to which the user is currently paying attention to each piece of recommended information based on currently acquired user information; a recommendation suitability calculation unit that calculates a suitability for each piece of recommendation information based on the interest level and the attention level; a recommendation information acquisition unit that acquires new recommendation information from the information recommendation system based on the user information and the interest level; a display information control unit that determines a timing for switching to suitable recommended information based on the suitability, the interest, and the new recommended information, and requests the information display terminal to switch the currently displayed recommended information; an information display terminal that displays recommended information; An information display system comprising:

Images