Display control device, display control method, and program

Abstract

This display control device: acquires a keyword; determines, with respect to a plurality of first virtual objects representing characteristics of a piece of merchandise, a first virtual object to be displayed in a display area and a display mode of the first virtual object, such determination being on the basis of the keyword; and causes the first virtual object to be displayed in the display area in the determined display mode. The display mode of the first virtual object includes any one or more of a display size of the first virtual object, an intensity of display of the first virtual object, and a color of the first virtual object.

Description

Display control device, display control method, and program

[0001] The present disclosure relates to a display control device, a display control method, and a program.

[0002] A device has been developed that provides information about an object to a user by displaying information about the object together with the object in a display area that can be seen by the user. For example, Patent Document 1 discloses a system that provides guidance to a user who is moving in the real world while looking at the display area by placing a guidance message in a virtual space that corresponds to the real world and displaying the message in the virtual space in a display area.

[0003] JP 2019-139424 A

[0004] The information useful to the user is not limited to route guidance. The present disclosure has been made in view of this problem, and one of its objectives is to provide a new technology for providing useful information to the user.

[0005] A display control device according to the present disclosure includes an acquisition unit that acquires a keyword, a first determination unit that determines, for a plurality of first virtual objects representing characteristics of a product, the first virtual objects to be displayed in a display area and a display mode of the first virtual objects based on the keyword, and an output unit that displays the first virtual objects in the display area in the determined display mode, wherein the display mode of the first virtual objects includes one or more of a display size of the first virtual objects, a display density of the first virtual objects, and a color of the first virtual objects.

[0006] A display control method according to the present disclosure is executed by a computer. The method includes an acquisition step of acquiring a keyword, a first determination step of determining, for a plurality of first virtual objects representing characteristics of a product, the first virtual objects to be displayed in a display area and a display mode of the first virtual objects based on the keyword, and an output step of displaying the first virtual objects in the display area in the determined display mode. The display mode of the first virtual objects includes one or more of a display size of the first virtual object, a display density of the first virtual object, and a color of the first virtual object.

[0007] The program of the present disclosure causes a computer to execute the display control method of the present disclosure.

[0008] According to the present disclosure, a new technique for providing useful information to a user is provided.

[0009] FIG. 1 is a diagram illustrating an overview of a display control device. FIG. 2 is a diagram illustrating an example of a display area in which a first type virtual object is displayed. FIG. 3 is a block diagram illustrating an example of the functional configuration of a display control device. FIG. 4 is a block diagram illustrating an example of the hardware configuration of a computer that realizes the display control device. FIG. 5 is a flowchart illustrating an example of a flow of processing executed by the display control device. FIG. 6 is a diagram illustrating an example of first type virtual object information. FIG. 7 is a diagram illustrating an example of rank information. FIG. 8 is a diagram illustrating an example of product information. FIG. 9 is a diagram illustrating an example of the functional configuration of a display control device having a function of guiding a user to a destination. FIG. 10 is a diagram illustrating an example of a display area in which a second type virtual object is displayed. FIG. 11 is a second diagram illustrating a display area in which a second type virtual object is displayed.

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and duplicate explanations will be omitted as necessary for clarity. Furthermore, unless otherwise specified, predetermined values ​​such as predetermined values ​​and threshold values ​​are stored in advance in a storage device accessible from a device that uses the values. Furthermore, unless otherwise specified, the storage unit is composed of one or any number of storage devices.

[0011] <Overview> Fig. 1 is a diagram showing an overview of the display control device 2000. The operation of the display control device 2000 shown in Fig. 1 is an example to facilitate understanding of the display control device 2000. The operations that can be performed by the display control device 2000 are not limited to those shown in Fig. 1.

[0012] The display control device 2000 provides information indicating the features of products to a user of the display control device 2000 in a place where the products are displayed in a store (hereinafter referred to as a display location) or in the vicinity thereof. Here, the store in which the display control device 2000 is used may be a permanent store or a non-permanent store (for example, a product sales outlet set up for a limited time at an event venue).

[0013] The information representing the characteristics of the product is displayed as a virtual object in a display area 60 of the display device. Hereinafter, the virtual object representing the characteristics of the product will be referred to as a first type virtual object 20.

[0014] For example, a user captures an image of a display location using a camera provided on a mobile terminal such as a smartphone. In this case, the display area 60 is a display area of ​​a display device provided on the mobile terminal. The image obtained from the camera is displayed in real time in the display area 60. The display control device 2000, for example, superimposes a first type virtual object 20 on the image of the display location displayed in the display area 60 (each frame constituting the image). In this way, the user can view the first type virtual object 20 representing the characteristics of the product along with the image of the product. Note that, as will be described later, the display area 60 is not limited to the display area of ​​the mobile terminal.

[0015] In order to display information desired by the user in the display area 60, the display control device 2000 uses keywords 10. The keywords 10 are one or more words, phrases, etc. (hereinafter, "words, etc.") that represent the characteristics of a product. For example, the keywords 10 are input by the user under the name of a search word, etc.

[0016] For example, if a user is searching for a rice cooker with a pressure induction heating (IH) function, keyword 10 may be "rice cooker pressure IH."

[0017] The display control device 2000 uses the keywords 10 to determine, from among a plurality of first-type virtual objects 20, the first-type virtual objects 20 to be displayed in the display area 60, and to determine the display mode of the first-type virtual objects 20 to be displayed in the display area 60. The display mode determined here includes at least one or more of the display size, the display darkness, and the display color. Hereinafter, the first-type virtual object 20 determined as the first-type virtual object 20 to be displayed in the display area 60 is also referred to as the "first-type virtual object 20 to be displayed."

[0018] The determination of the first type virtual object 20 to be displayed and the determination of the display mode of the first type virtual object 20 to be displayed are determined based on the degree of association between the first type virtual object 20 and the keyword 10. For example, a first type virtual object 20 that has a relatively high degree of association with the keyword 10 is determined as the first type virtual object 20 to be displayed. Also, for example, the display mode of a first type virtual object 20 that has a higher degree of association with the keyword 10 is determined to be a display mode with a higher degree of emphasis (for example, a larger display size).

[0019] The display control device 2000 outputs each of the first-type virtual objects 20 to be displayed in the display area 60 in the determined display mode. Fig. 2 is a diagram illustrating an example of the display area 60 in which the first-type virtual objects 20 are displayed. The example in Fig. 2 assumes that "rice cooker pressure IH" has been input as the keyword 10.

[0020] The first type virtual objects 20 determined as the first type virtual objects 20 to be displayed based on the keyword 10 are a first type virtual object 20-1 and a first type virtual object 20-2. The first type virtual object 20-1 indicates a characteristic of "pressure induction heating method." The first type virtual object 20-2 indicates a characteristic of "easy to produce burnt rice." Here, the characteristic indicated by the first type virtual object 20-1 has a higher degree of relevance to the keyword 10 than the characteristic indicated by the first type virtual object 20-2. Therefore, the first type virtual object 20-1 is displayed at a larger size than the first type virtual object 20-2.

[0021] <Example of Action and Effect> According to the display control device 2000, a first type virtual object 20 representing the characteristics of a product is displayed in the display area 60. One possible method for providing information about a product to a user is to provide all information related to the product that is within the user's field of view. However, this method is likely to result in too much information being displayed in the user's field of view. The user must then select the information they need from the provided information. Therefore, this method of providing all information to the user places a heavy burden on the user as they must select the information they need from the displayed information.

[0022] In this regard, the display control device 2000 determines the first type virtual objects 20 to be displayed in the display area 60 based on the degree of association between the first type virtual objects 20 and the keyword 10. In this way, by narrowing down the first type virtual objects 20 to be displayed in the display area 60 to a portion based on the keyword 10, only the first type virtual objects 20 that are likely to be useful to the user are displayed in the display area 60. This reduces the burden on the user of selecting and discarding necessary information from the displayed information, thereby increasing convenience. In this way, the display control device 2000 applies a new technology for providing useful information to the user.

[0023] Furthermore, the display control device 2000 determines the display mode of the first type virtual object 20 based on the keyword 10. As a result, the degree of association between the keyword 10 and the first type virtual object 20 is reflected in the display mode of the first type virtual object 20. For example, the first type virtual object 20 having a higher degree of association with the keyword 10 is set to a display mode with a higher degree of emphasis. Therefore, the degree to which the first type virtual object 20 is useful to the user is reflected in the display mode of the first type virtual object 20. This further reduces the burden on the user of selecting and discarding necessary information from displayed information, further improving convenience.

[0024] The display control device 2000 of this embodiment will be described in more detail below.

[0025] 3 is a block diagram illustrating an example of the functional configuration of the display control device 2000. The display control device 2000 has an acquisition unit 2020, a first determination unit 2040, and an output unit 2060. The acquisition unit 2020 acquires a keyword 10. The first determination unit 2040 uses the keyword 10 to determine first-type virtual objects 20 to be displayed and a display mode for each first-type virtual object 20. The output unit 2060 outputs each first-type virtual object 20 to be displayed so that the first-type virtual object 20 is displayed in the display area 60 in the display mode determined for that first-type virtual object 20.

[0026] <Example of Hardware Configuration> Each functional component of the display control device 2000 may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). Below, a case where each functional component of the display control device 2000 is realized by a combination of hardware and software will be further described.

[0027] 4 is a block diagram illustrating an example of the hardware configuration of a computer 1000 that realizes the display control device 2000. The computer 1000 is any computer. For example, the computer 1000 is a stationary computer such as a PC (Personal Computer) or a server machine. Alternatively, the computer 1000 may be a portable computer such as a smartphone or a tablet terminal. The computer 1000 may be a dedicated computer designed to realize the display control device 2000, or may be a general-purpose computer.

[0028] For example, by installing a predetermined application on the computer 1000, the computer 1000 realizes each function of the display control device 2000. The application is configured with a program for realizing each functional component of the display control device 2000. Note that any method for acquiring the program is possible. For example, the program can be acquired from a storage medium on which the program is stored. The storage medium on which the program is stored may be any storage medium such as a DVD (Digital Versatile Disk) or a USB (Universal Serial Bus) memory. Alternatively, the program can be acquired by downloading the program from a server device that manages the storage device on which the program is stored.

[0029] The computer 1000 has a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path for the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 to transmit and receive data to and from each other. However, the method of connecting the processor 1040 and the like to each other is not limited to bus connection.

[0030] The processor 1040 is a processor such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 1060 is a main storage device realized using a random access memory (RAM) or the like. The storage device 1080 is an auxiliary storage device realized using a hard disk, a solid state drive (SSD), a memory card, a read only memory (ROM), or the like.

[0031] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. For example, the input / output interface 1100 is connected to an input device such as a keyboard and an output device such as a display device.

[0032] The network interface 1120 is an interface for connecting the computer 1000 to a network. This network may be a LAN (Local Area Network) or a WAN (Wide Area Network).

[0033] The storage device 1080 stores programs (programs that realize the above-mentioned applications) that realize the functional components of the display control device 2000. The processor 1040 reads these programs into the memory 1060 and executes them to realize the functional components of the display control device 2000.

[0034] The display control device 2000 may be realized by one computer 1000 or by multiple computers 1000. In the latter case, the configurations of the computers 1000 do not need to be the same, and can be different from each other.

[0035] <Regarding the Display Area 60> The display area 60 is a display area capable of displaying the first type virtual object 20 in a manner in which the display location and its surroundings are superimposed on the first type virtual object 20 in the user's field of vision. For example, the display area 60 is a display area in which an image including the display location and its surroundings is displayed.

[0036] The display area 60 in which the image including the display location is displayed is, for example, a display area provided on the display device of the mobile terminal, as described above.

[0037] As another example, the display area 60 in which an image including the display location is displayed is a display area provided in a non-transmissive head-mounted display. The non-transmissive head-mounted display is provided with a camera that captures an image in the direction of the user's field of view. The image generated by the camera is displayed in the display area of ​​the head-mounted display. Therefore, when a user wearing the head-mounted display looks at the display location, the image of the display location is displayed in the display area 60. The display control device 2000 displays the first type virtual object 20 superimposed on the image of the display location displayed in the display area 60.

[0038] An image of the display location does not have to be displayed in the display area 60. In this case, the display area 60 is, for example, a display area provided in a transparent head-mounted display (for example, a glasses-type head-mounted display). In a transparent head-mounted display, the display area 60 is configured to transmit light, like the lenses of glasses. Therefore, the user can see the display location that exists beyond the display area 60 while wearing the head-mounted display. Therefore, the display control device 2000 displays the first type virtual object 20 in the display area 60, so that the first type virtual object 20 and the display location can be superimposed in the user's field of vision.

[0039] The display control device 2000 may be provided integrally with a device having the display area 60 (the mobile terminal or head-mounted display described above), or may be provided separately. In the former case, for example, the display control device 2000 is realized as a mobile terminal or a head-mounted display having the display area 60. In the latter case, for example, the display control device 2000 is realized as a server device that controls the device having the display area 60. The server device is communicably connected to the device having the display area 60 via a network interface 1120 or the like.

[0040] 5 is a flowchart illustrating the flow of processing executed by the display control device 2000. The acquisition unit 2020 acquires a keyword 10 (S102). The first determination unit 2040 uses the keyword 10 to determine first-type virtual objects 20 to be displayed and to determine the display mode of each first-type virtual object 20 (S104). The output unit 2060 outputs each first-type virtual object 20 to be displayed so that it is displayed in the display area 60 in the display mode determined for that first-type virtual object 20 (S106).

[0041] <Acquisition of Keyword 10: S102> The acquisition unit 2020 acquires the keyword 10 (S102). There are various methods for the acquisition unit 2020 to acquire the keyword 10. For example, the acquisition unit 2020 accepts input of the keyword 10 by the user. Here, the method by which the user inputs the keyword 10 may be character input using a keyboard or a touch panel, or voice input using a microphone.

[0042] Alternatively, the display control device 2000 may automatically acquire the keywords 10 from another system. For example, the display control device 2000 may acquire data corresponding to the date and time when the display control device 2000 is used as the keywords 10 from a notepad application, a to-do list application, a scheduler application, or the like.

[0043] <Regarding the First Type Virtual Object 20> The first type virtual object 20 is a virtual object that represents a feature of a product. For example, the first type virtual object 20 is any two-dimensional or three-dimensional virtual object that includes an indication that represents a feature of the product. The two-dimensional virtual object may be a virtual object that represents a two-dimensional shape with a specific name, such as a rectangle or an oval, or may be a virtual object that represents an arbitrary two-dimensional shape without a specific name. The three-dimensional virtual object may be a virtual object that represents a three-dimensional shape with a specific name, such as a rectangular parallelepiped or a cone, or may be a virtual object that represents an arbitrary three-dimensional shape without a specific name.

[0044] There are various methods for representing product features using virtual objects. For example, by displaying text representing the product features on a virtual object, the product features can be represented using the virtual object. Alternatively, by displaying an image such as a pattern that evokes the product features on the virtual object, the product features can be represented using the virtual object. Alternatively, by representing a shape that evokes the product features with the virtual object itself, the product features can be represented using the virtual object.

[0045] Information about each first-type virtual object 20 (hereinafter referred to as first-type virtual object information) is stored in advance in, for example, a storage unit accessible from the display control device 2000. Fig. 6 is a diagram illustrating an example of first-type virtual object information. The first-type virtual object information 200 indicates an identifier 202, a virtual object 204, and a tag 206 for each first-type virtual object 20.

[0046] The identifier 202 indicates the identifier of the first type virtual object 20 .

[0047] The virtual object 204 indicates data representing the first type virtual object 20. The data indicated by the virtual object 204 is, for example, definition information of the first type virtual object 20 that is used to generate the first type virtual object 20. The definition information of the first type virtual object 20 indicates, for example, the type of shape of the virtual object, the type of frame of the virtual object (for example, a solid line or a dotted line), and the color of the virtual object.

[0048] The virtual object 204 may indicate data representing the shape of the virtual object (for example, vector data representing the shape of a two-dimensional object or mesh data representing the shape of a three-dimensional object) instead of the type of shape of the virtual object. Also, the virtual object 204 may indicate image data of a texture to be displayed on the virtual object instead of or in addition to the color of the virtual object.

[0049] Assume that text representing the characteristics of a product is displayed on the first type virtual object 20. In this case, the virtual object 204 further displays the text representing the characteristics of the product. In this case, the virtual object 204 may further include information such as the color and font type of the text.

[0050] It is assumed that image data representing the characteristics of a product is displayed on the first type virtual object 20. In this case, the virtual object 204 further displays image data representing the characteristics of the product.

[0051] The tag 206 indicates a word or the like that has a high degree of association with a product to which the first type virtual object 20 should be attached. As will be described later, the tag 206 is used to calculate the degree of association between the keyword 10 and the first type virtual object 20. The tag 206 may indicate only one word or the like, or may indicate multiple words or the like.

[0052] <Determining the first type virtual object 20 to be displayed and its display mode: S104> The first determination unit 2040 determines the first type virtual object 20 to be displayed and determines the display mode of the first type virtual object 20 (S104). The determination of the first type virtual object 20 to be displayed and the determination of the display mode of each first type virtual object 20 are both performed based on the degree of association between the first type virtual object 20 and the keyword 10. Hereinafter, the index value representing the degree of association between the first type virtual object 20 and the keyword 10 will be referred to as the degree of association.

[0053] The first determination unit 2040 calculates the degree of relevance with the keyword 10 for each of the multiple first-type virtual objects 20. For example, the degree of relevance is calculated for each of all of the first-type virtual objects 20 indicated in the first-type virtual object information 200. Based on the degree of relevance calculated for each first-type virtual object 20, the first determination unit 2040 determines whether or not the first-type virtual object 20 is to be a display target and the display mode of the first-type virtual object 20. A method for calculating the degree of relevance will be described later.

[0054] For example, multiple ranks are determined in advance for the relevance level. In this case, the first determination unit 2040 makes the above-mentioned determinations using the ranks.

[0055] Information representing the correspondence between rank and relevance is called rank information. The rank information indicates a range of relevance in association with each of a plurality of ranks. Furthermore, the rank information indicates information that determines the display mode of the first type virtual object 20 for each rank. A higher rank is associated with a display mode with a higher degree of emphasis (prominence). For example, with regard to the display size, the larger the display size, the higher the degree of emphasis. Furthermore, with regard to the display darkness, the darker the display darkness, the higher the degree of emphasis.

[0056] Fig. 7 is a diagram illustrating an example of rank information. In the example of Fig. 7, three ranks, rank 1 to rank 3, are defined. Also, in the example of Fig. 7, the higher the rank name, the smaller the numerical value assigned to it. Specifically, rank 1 is higher than rank 2, which is higher than rank 3.

[0057] The rank information 300 indicates a rank 302, a relevance range 304, and a display mode 306. The rank 302 indicates the name of the rank. The relevance range 304 indicates the range of relevance. In the example of FIG. 7 , the range of relevance is divided using two thresholds, Th1 and Th2 (Th1>Th2). Specifically, rank 1 corresponds to the range of relevance equal to or greater than Th1. Rank 2 corresponds to the range of relevance equal to or greater than Th2 but less than Th1. Rank 3 corresponds to the range of relevance less than Th2.

[0058] The display mode 306 indicates a display mode set for the first type virtual object 20. For example, the display mode 306 indicates the size, display density, color, etc. of the first type virtual object 20 itself. In addition, for example, the display mode 306 indicates the size, color, font type, etc. of text that is included in the first type virtual object 20 and indicates the characteristics of the product.

[0059] The first determination unit 2040 determines the rank of each first-type virtual object 20 using the rank information 300. Specifically, the first determination unit 2040 determines the rank of the first-type virtual object 20 by determining an association degree range 304 that includes the association degree calculated for the first-type virtual object 20. For example, in the example of FIG. 7 , a first-type virtual object 20 whose association degree with the keyword 10 is greater than Th1 is classified as rank 1.

[0060] Whether or not a first type virtual object 20 is to be a display target is determined based on the rank of the first type virtual object 20. Specifically, a first type virtual object 20 classified into a rank equal to or higher than a threshold value (hereinafter referred to as a rank threshold value) is treated as a display target. The rank threshold value may be determined in advance or may be specified by the user.

[0061] The first determination unit 2040 determines whether the rank of each first-type virtual object 20 is equal to or higher than a rank threshold. If the rank of the first-type virtual object 20 is lower than the rank threshold, the first determination unit 2040 determines that the first-type virtual object 20 is not to be a display target.

[0062] If the rank of the first type virtual object 20 is equal to or higher than the rank threshold, the first determination unit 2040 determines that the first type virtual object 20 is to be a display target. Furthermore, if the rank of the first type virtual object 20 is equal to or higher than the rank threshold, the first determination unit 2040 determines the display mode 306 corresponding to the rank of the first type virtual object 20 as the display mode of the first type virtual object 20.

[0063] 7 , for example, assume that the rank threshold is 2. In this case, the first type virtual objects 20 classified as rank 3 are not targeted for display. On the other hand, the first type virtual objects 20 classified as rank 1 or 2 are targeted for display.

[0064] <<Method of Calculating Degree of Association>> The degree of association between a first-type virtual object 20 and a keyword 10 is represented, for example, by the degree of association between the tag 206 of the first-type virtual object 20 and the keyword 10. The first determination unit 2040 calculates the degree of association between the tag 206 of the first-type virtual object 20 and the keyword 10 for each first-type virtual object 20 indicated in the first-type virtual object information 200.

[0065] There are various methods for calculating the relevance between the tag 206 and the keyword 10. For example, the relevance between the tag 206 and the keyword 10 is calculated using a trained machine learning model. The model used here is called a relevance calculation model. The relevance calculation model is configured to, for example, accept two input texts and output the relevance between the two texts. The relevance calculation model can be configured using any machine learning model such as a neural network.

[0066] The relevance calculation model is trained in advance using multiple training data. The training data includes, for example, two input texts and the relevance between the two input texts (i.e., the ground truth relevance). An apparatus (hereinafter referred to as a training apparatus) that trains the relevance calculation model inputs the two input texts included in the training data to the relevance calculation model to obtain an output value from the relevance calculation model. The training apparatus calculates a loss based on the output value obtained from the relevance calculation model and the ground truth relevance indicated in the training data, and updates the parameters of the relevance calculation model based on the loss, thereby training the relevance calculation model. Note that the above-described training method for the relevance calculation model is an example, and the relevance calculation model may be trained by any other method.

[0067] The first determination unit 2040 obtains the relevance between the tag 206 and the keyword 10 by inputting the tag 206 and the keyword 10 to the relevance calculation model. Here, as described above, the tag 206 may indicate multiple words, etc. In a case where the tag 206 includes multiple words, etc., the first determination unit 2040 calculates the relevance between the tag 206 and the keyword 10 using the multiple words, etc. For example, the first determination unit 2040 calculates the relevance for each of the multiple words, etc. indicated in the tag 206 by inputting the word, etc., and the keyword 10 to the relevance calculation model. Then, the first determination unit 2040 calculates statistical values ​​(such as a simple average, a weighted average, a maximum value, or a minimum value) of the relevance calculated for each of the multiple words, etc. indicated in the tag 206, and uses the statistical values ​​as the relevance between the tag 206 and the keyword 10.

[0068] Here, it is assumed that a weighted average is used as the statistical value. In this case, a weight is predetermined for each of the multiple words, etc. indicated in the tag 206. For example, the tag 206 indicates each word, etc. indicated in the tag 206, together with the weight of the word, etc.

[0069] <Product Determination> In the display area 60, the display control device 2000 displays the first type virtual object 20 to be displayed in the vicinity of the product corresponding to the first type virtual object 20. For example, a first type virtual object 20 including the indication "pressure induction heating" is displayed in the vicinity of a rice cooker or the like that employs a pressure induction heating system.

[0070] The correspondence between the product and the first-type virtual object 20 is, for example, determined in advance. Hereinafter, information indicating the correspondence between the product and the first-type virtual object 20 will be referred to as product information.

[0071] 8 is a diagram illustrating an example of product information. The product information 400 indicates, for each product, a product identifier 402 and a virtual object identifier 404. The product identifier 402 indicates identification information of the product. The virtual object identifier 404 indicates the identifier of the first type virtual object 20.

[0072] Here, one first-type virtual object 20 may be associated with a plurality of products. For example, a first-type virtual object 20 representing information on "pressure induction heating" may be associated with a plurality of types of pressure induction heating rice cookers.

[0073] For each first type virtual object 20 determined as the display target, the first determination unit 2040 identifies one or more products associated with the first type virtual object 20 in the product information 400 as the products that will display the first type virtual object 20.

[0074] The first determination unit 2040 may further narrow down the products for which the first type virtual object 20 is to be displayed, using information other than the product information 400. For example, the display control device 2000 receives input of narrowing-down conditions such as product type, manufacturer, brand, or price range from the user. Of the products identified using the product information 400, the first determination unit 2040 treats only those products that match the narrowing-down conditions specified by the user as products for which the first type virtual object 20 is to be displayed.

[0075] When the above-described narrowing-down conditions are used, various pieces of information (such as product manufacturers) used for narrowing down are stored in advance in, for example, a storage unit accessible from the display control device 2000. For example, various pieces of information about products are included in the product information 400.

[0076] The first determination unit 2040 may use some of the keywords 10 as the narrowing-down conditions. For example, the type of information to be extracted from the keywords 10 as the narrowing-down conditions is determined in advance. In this case, the first determination unit 2040 extracts words or the like representing a predetermined type of information from the keywords 10 and uses them as the narrowing-down conditions.

[0077] For example, the information extracted as a narrowing-down condition from keyword 10 is the type of product. In this case, assume that keyword 10 is "rice cooker pressure induction." In this case, the first determination unit 2040 uses the word "rice cooker" included in keyword 10 as a narrowing-down condition.

[0078] <Displaying First Type Virtual Objects 20: S106> The output unit 2060 displays each first type virtual object 20 determined as a display target in the display area 60 in a display mode determined for the first type virtual object 20. Here, the output unit 2060 displays each first type virtual object 20 in the vicinity of a product corresponding to that first type virtual object 20.

[0079] In order to display the first type virtual objects 20 around the products in the display area 60, the output unit 2060 identifies the position of the products corresponding to the first type virtual objects 20. When a captured image generated by a camera is displayed in the display area 60 (when a mobile terminal, a non-transparent head-mounted display, or the like is used), for example, the output unit 2060 identifies the position of each product included in the image by performing object recognition processing on the image.

[0080] On the other hand, when a captured image is not displayed in the display area 60 (when a see-through head-mounted display or the like is used), for example, the output unit 2060 determines the position of each product on the display area 60 based on the user's current location and the user's line of sight. In this case, the output unit 2060 acquires placement information indicating the placement of each product in the store. Then, for each product that comes into the user's field of view via the display area 60, the output unit 2060 determines the position of that product on the display area 60 based on the user's current location, the user's line of sight, and the position of each product indicated in the placement information. The position of each product on the display area 60 is the position where a straight line connecting the position of that product and the position of the user's eyes intersects with the display area 60.

[0081] The positional relationship between the first type virtual object 20 and the product in the display area 60 can be determined arbitrarily. For example, the relative positional relationship between the first type virtual object 20 and the product is determined in advance. In this case, the first determination unit 2040 displays the first type virtual object 20 in the display area 60 so that the relative positional relationship between the first type virtual object 20 and the product corresponding to that first type virtual object 20 is the predetermined positional relationship. However, if multiple first type virtual objects 20 overlap each other, the first determination unit 2040 may prevent the multiple first type virtual objects 20 from overlapping each other by, for example, shifting the display positions of the first type virtual objects 20.

[0082] The output unit 2060 may determine the arrangement of each first type virtual object 20 two-dimensionally or three-dimensionally in the display area 60. In the former case, the output unit 2060 arranges each first type virtual object 20 to be displayed on a plane corresponding to the display area 60 in the display mode determined for that first type virtual object 20.

[0083] On the other hand, when the arrangement of the first type virtual objects 20 is determined three-dimensionally, for example, the output unit 2060 arranges each of the first type virtual objects 20 to be displayed in a virtual space corresponding to the real world in a display mode determined for that first type virtual object 20. Then, the output unit 2060 displays, in the display area 60, an image of each of the first type virtual objects 20 obtained by projecting each of the first type virtual objects 20 arranged in the virtual space onto a plane corresponding to the display area 60.

[0084] <Guidance to Destination> When a user visits a store to purchase a product, the user often has a destination in mind (i.e., a destination). For example, if a user visits a store to purchase a rice cooker, the rice cooker section is one of the destinations.

[0085] Therefore, the display control device 2000 may have a function of guiding the user to the destination by displaying a virtual object different from the first-type virtual object 20 in the display area 60. Hereinafter, a virtual object used to guide the user to the destination will be referred to as a second-type virtual object. The second-type virtual object is a virtual object that represents a direction (for example, a virtual object that represents an arrow).

[0086] FIG. 9 is a diagram illustrating an example of the functional configuration of a display control device 2000 having a function of guiding a user to a destination. In FIG. 9 , the display control device 2000 further includes a second determination unit 2080 in addition to the functional components of the display control device 2000 shown in FIG. 3 . The second determination unit 2080 determines a second-type virtual object 70 to be displayed in the display area 60 (hereinafter, the second-type virtual object 70 to be displayed) from among a plurality of second-type virtual objects 70 based on the user's travel route to the destination and the user's current location. The second-type virtual object 70 to be displayed in the display area 60 is a virtual object that represents the direction in which the user should currently move. The output unit 2060 displays the second-type virtual object 70 to be displayed in the display area 60.

[0087] <<Determining a Travel Route>> The second determination unit 2080 determines a travel route for the user based on the user's starting point and destination. To do this, the second determination unit 2080 acquires a map of the store (hereinafter referred to as a store map). The second determination unit 2080 then determines a travel route from the current location to the destination based on the starting point, the destination, and the store map. Note that the above-mentioned location information indicating the location of products may be included as part of the store map.

[0088] There are various methods for the second determination unit 2080 to acquire a store map. For example, the second determination unit 2080 acquires a store map by reading out, from a storage unit accessible from the display control device 2000, a store map stored in the storage unit.

[0089] The starting point is, for example, the user's current location at a specific time (for example, the time when keyword 10 is input). There are various methods for the second determination unit 2080 to identify the user's current location. For example, multiple beacons are placed in a store in advance. The second determination unit 2080 identifies the user's current location based on communication between the user's mobile terminal and the beacon. Note that when the user's mobile terminal and the display control device 2000 are implemented as an integrated unit, the display control device 2000 can identify the user's current location based on communication between itself and the beacon.

[0090] Alternatively, for example, the second determination unit 2080 may acquire location information (e.g., GPS (Global Positioning System) coordinates) from the user's mobile terminal to identify the user's current location. When the user's mobile terminal and the display control device 2000 are integrated, the display control device 2000 can acquire location information indicating the user's current location from a GPS sensor provided therein.

[0091] Alternatively, for example, when a captured image generated by a camera is displayed in the display area 60, the second determination unit 2080 may use the captured image to identify the user's current location. For example, the second determination unit 2080 recognizes products included in the captured image by performing object recognition processing on the most recent captured image generated by the camera. The second determination unit 2080 then identifies the location where the recognized products are displayed, thereby identifying the user's current location. The location where each product is displayed can be identified using the above-mentioned placement information.

[0092] There are various methods for the second determination unit 2080 to identify the user's destination point. For example, the second determination unit 2080 identifies the destination point based on a product corresponding to the first-type virtual object 20 to be displayed. As described above, the first determination unit 2040 identifies a product corresponding to each first-type virtual object 20 to be displayed. These identified products are likely to be products displayed at the user's destination point.

[0093] Therefore, the second determination unit 2080 identifies the display location of each product corresponding to each of the first-type virtual objects 20 to be displayed. Furthermore, the second determination unit 2080 identifies the display location identified for each product as the user's destination point. For example, the second determination unit 2080 treats the display location where the largest number of products are displayed among the multiple products corresponding to the first-type virtual objects 20 to be displayed as the user's destination point.

[0094] As a specific example, suppose there are ten first-type virtual objects 20 to be displayed. Suppose that of these ten first-type virtual objects 20, products corresponding to seven of the first-type virtual objects 20 are displayed in the rice cooker section. Suppose also that products corresponding to three of the first-type virtual objects 20 are displayed in the washing machine section. In this case, it is highly likely that the user's destination point is the rice cooker section, which displays the largest number of products (seven) out of the ten products corresponding to the first-type virtual objects 20 to be displayed. Therefore, the second determination unit 2080 specifies the rice cooker section as the destination point.

[0095] The second determination unit 2080 may identify the type of product the user is attempting to purchase and identify a location where that type of product is displayed as the destination point. For example, assume that the keywords 10 include a word or the like that represents a type of product, such as "rice cooker." In this case, the second determination unit 2080 identifies the type of product represented by the word or the like as the type of product the user is attempting to purchase.

[0096] Alternatively, for example, the second determination unit 2080 may accept a user input specifying a destination point, in addition to the keyword 10. In this case, the second determination unit 2080 can identify the destination point based on the user input.

[0097] <<Determining Second-Type Virtual Object 70 to Be Displayed>> The second determination unit 2080 determines, from among multiple second-type virtual objects 70 each representing a different direction, the second-type virtual object 70 that represents the direction in which the user should move, as the second-type virtual object 70 to be displayed. The second determination unit 2080 determines the direction in which the user should move based on the user's current location and the movement route, and determines the second-type virtual object 70 corresponding to that direction as the second-type virtual object 70 to be displayed.

[0098] 10 is a diagram illustrating an example of a display area 60 in which a second type virtual object 70 is displayed. In the example of Fig. 10, the direction in which the user should move is the front direction. Therefore, in the display area 60, a second type virtual object 70 representing the front direction of the user is displayed.

[0099] 10, the user's destination is the rice cooker section, and therefore a virtual object 80 representing the destination is also displayed.

[0100] The output unit 2060 may display, in the display area 60, a second type virtual object 70 representing the direction in which the user should move next, in addition to the second type virtual object 70 representing the direction in which the user should currently move. FIG. 11 is a second diagram illustrating the display area 60 in which the second type virtual object 70 is displayed. In the example of FIG. 11 , the user's movement route is a route in which "after moving forward in the forward direction, the user should move left at the end of the road." Therefore, in addition to the second type virtual object 70-1 representing the forward direction, which is the direction in which the user should currently move, a second type virtual object 70-2 representing the leftward direction, which is the direction in which the user should move next, is displayed in the display area 60.

[0101] <Switching of Virtual Objects Displayed in Display Area 60> The display control device 2000 may simultaneously display both the first type virtual object 20 and the second type virtual object 70 in the display area 60, or may display only one of the first type virtual object 20 and the second type virtual object 70 in the display area 60. In the latter case, for example, the display control device 2000 displays the second type virtual object 70 in the display area 60 while the user is moving. On the other hand, the display control device 2000 displays the first type virtual object 20 in the display area 60 while the user is not moving.

[0102] The display control device 2000 may appropriately switch between a state in which only the first type virtual object 20 is displayed in the display area 60, a state in which only the second type virtual object 70 is displayed in the display area 60, and a state in which both the first type virtual object 20 and the display area 60 are displayed in the display area 60. Specifically, when the user has arrived at the destination, the display control device 2000 displays only the first type virtual object 20 in the display area 60. On the other hand, when the user has not yet arrived at the destination, the display control device 2000 displays only the second type virtual object 70 in the display area 60 if the user is moving, and displays both the first type virtual object 20 and the second type virtual object 70 in the display area 60 if the user is stationary. If the user stops before arriving at the destination, there is a high probability that a product that the user is interested in is displayed at that location. Therefore, in this case, the display control device 2000 uses the display area 60 to provide information about the product while guiding the user along the route to the destination.

[0103] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0104] Each drawing is merely an example for describing one or more embodiments. Each drawing may not relate to only one particular embodiment, but may also relate to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0105] In the present disclosure, a program includes a set of instructions (or software code) that, when loaded into a computer, causes the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0106] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes. (Supplementary Note 1) A display control device comprising: an acquisition means for acquiring a keyword; a first determination means for determining, for a plurality of first virtual objects representing features of a product, the first virtual objects to be displayed in a display area and a display mode of the first virtual objects based on the keyword; and an output means for displaying the first virtual objects in the display area in the determined display mode, wherein the display mode of the first virtual objects includes one or more of a display size of the first virtual objects, a display darkness of the first virtual objects, and a color of the first virtual objects. (Supplementary Note 2) The display control device according to Supplementary Note 1, wherein the first determination means calculates a degree of relevance between each of the first virtual objects and the keyword, and determines, based on the degree of relevance of each of the first virtual objects, the first virtual objects to be displayed in the display area and the display mode of the first virtual objects. (Supplementary Note 3) The display control device according to Supplementary Note 2, wherein the first determination means acquires rank information in which a numerical range of the relevance and a display mode of the first virtual object are associated with each of a plurality of ranks, and when a height of the rank corresponding to the relevance of the first virtual object is equal to or greater than a threshold, determines to display the first virtual object in the display area, and determines the display mode associated with the rank corresponding to the relevance of the first virtual object as the display mode of the first virtual object. (Supplementary Note 4) The display control device according to Supplementary Note 3, wherein the first determination means acquires first virtual object information in which one or more tags representing product features are associated with each of the plurality of first virtual objects, and calculates a relevance between the tag corresponding to the first virtual object and the keyword as a relevance between the first virtual object and the keyword.(Supplementary Note 5) The display control device according to any one of Supplementary Notes 1 to 4, wherein the output means acquires product information in which one or more first virtual objects are associated with each of a plurality of products, and displays the first virtual objects in the display area so as to be positioned around the product corresponding to the first virtual object in the user's field of view. (Supplementary Note 6) The display control device according to any one of Supplementary Notes 1 to 4, further comprising second determination means for identifying a direction in which the user should move based on a movement route to the user's destination and the user's current location, and determining the second-type virtual object representing the direction in which the user should move as the second-type virtual object to be displayed in the display area from among a plurality of second-type virtual objects each representing a different direction, wherein the output means displays the second-type virtual object in the display area. (Supplementary Note 7) The display control device according to Supplementary Note 6, wherein the output means displays the first virtual object in the display area when the user is not moving, and displays the second-type virtual object in the display area when the user is moving. (Supplementary Note 8) A display control method executed by a computer, comprising: an acquisition step of acquiring a keyword; a first determination step of determining, for a plurality of first virtual objects representing features of a product, the first virtual objects to be displayed in a display area and a display mode of the first virtual objects based on the keyword; and an output step of displaying the first virtual objects in the display area in the determined display mode, wherein the display mode of the first virtual objects includes one or more of a display size of the first virtual object, a display darkness of the first virtual object, and a color of the first virtual object. (Supplementary Note 9) The display control method described in Supplementary Note 8, wherein, in the determination step, a degree of relevance between each of the first virtual objects and the keyword is calculated, and the first virtual objects to be displayed in the display area and the display mode of the first virtual objects are determined based on the degree of relevance of each of the first virtual objects.(Supplementary Note 10) A program causing a computer to execute the following steps: an acquisition step of acquiring a keyword; a first determination step of determining, for a plurality of first virtual objects representing features of a product, first virtual objects to be displayed in a display area and a display mode of the first virtual objects based on the keyword; and an output step of displaying the first virtual objects in the display area in the determined display mode, wherein the display mode of the first virtual objects includes one or more of a display size of the first virtual object, a display density of the first virtual object, and a color of the first virtual object.

[0107] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 7 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Notes 8 and 10 in the same dependency relationship as Supplementary Notes 2 to 7. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods.

[0108] This application claims priority based on Japanese Patent Application No. 2024-020976, filed February 15, 2024, the disclosure of which is incorporated herein in its entirety by reference.

[0109] 10 Keyword 20 First type virtual object 60 Display area 70 Second type virtual object 80 Virtual object 200 First type virtual object information 202 Identifier 204 Virtual object 206 Tag 300 Rank information 302 Rank 304 Relevance range 306 Display mode 400 Product information 402 Product identifier 404 Virtual object identifier 1000 Computer 1020 Bus 1040 Processor 1060 Memory 1080 Storage device 1100 Input / output interface 1120 Network interface 2000 Display control device 2020 Acquisition unit 2040 First determination unit 2060 Output unit 2080 Second determination unit

Claims

1. A display control device comprising: an acquisition means for acquiring a keyword; a first determination means for determining, for a plurality of first virtual objects representing characteristics of a product, the first virtual objects to be displayed in a display area and a display mode of the first virtual objects based on the keyword; and an output means for displaying the first virtual objects in the display area in the determined display mode, wherein the display mode of the first virtual objects includes one or more of a display size of the first virtual objects, a display darkness of the first virtual objects, and a color of the first virtual objects.

2. The display control device according to claim 1, wherein the first determination means calculates a degree of relevance between each of the first virtual objects and the keyword, and determines the first virtual object to be displayed in the display area and a display mode of the first virtual object based on the degree of relevance of each of the first virtual objects.

3. The display control device according to claim 2, wherein the first determination means acquires rank information in which a numerical range of the relevance and a display mode of the first virtual object are associated with each of a plurality of ranks, and when the height of the rank corresponding to the relevance of the first virtual object is equal to or greater than a threshold, determines to display the first virtual object in the display area, and determines the display mode associated with the rank corresponding to the relevance of the first virtual object as the display mode of the first virtual object.

4. The display control device according to claim 3, wherein the first determination means acquires first virtual object information in which one or more tags representing product features are associated with each of the plurality of first virtual objects, and calculates the degree of association between the tag corresponding to the first virtual object and the keyword as the degree of association between the first virtual object and the keyword.

5. A display control device according to any one of claims 1 to 4, wherein the output means acquires product information in which one or more first virtual objects are associated with each of a plurality of products, and displays the first virtual objects in the display area so as to be positioned around the product corresponding to the first virtual object in the user's field of vision.

6. A display control device according to any one of claims 1 to 4, further comprising: second determination means for identifying a direction in which the user should move based on a travel route to the user's destination point and the user's current location, and determining, from a plurality of second-type virtual objects each representing a different direction, the second-type virtual object representing the direction in which the user should move as the second-type virtual object to be displayed in the display area; and wherein the output means displays the second-type virtual object in the display area.

7. The display control device according to claim 6, wherein the output means displays the first type of virtual object in the display area when the user is not moving, and displays the second type of virtual object in the display area when the user is moving.

8. A display control method executed by a computer, comprising: an acquisition step of acquiring a keyword; a first determination step of determining, for a plurality of first virtual objects representing characteristics of a product, the first virtual objects to be displayed in a display area and a display mode of the first virtual objects based on the keyword; and an output step of displaying the first virtual objects in the display area in the determined display mode, wherein the display mode of the first virtual objects includes one or more of a display size of the first virtual object, a display density of the first virtual object, and a color of the first virtual object.

9. The display control method according to claim 8, wherein in the determining step, a degree of relevance between each of the first virtual objects and the keyword is calculated, and the first virtual object to be displayed in the display area and a display mode of the first virtual object are determined based on the degree of relevance of each of the first virtual objects.

10. A program that causes a computer to execute the following steps: an acquisition step of acquiring a keyword; a first determination step of determining, for a plurality of first virtual objects representing characteristics of a product, the first virtual objects to be displayed in a display area and a display mode of the first virtual objects based on the keyword; and an output step of displaying the first virtual objects in the display area in the determined display mode, wherein the display mode of the first virtual objects includes one or more of the display size of the first virtual objects, the display darkness of the first virtual objects, and the color of the first virtual objects.

Images