Information processing device and information processing method

Abstract

An information processing device according to one embodiment of the present invention comprises: a line-of-sight direction acquisition unit that acquires a line-of-sight direction of a user; a movement speed acquisition unit that acquires a movement speed of the user; and a control unit that performs, according to the line-of-sight direction and the movement speed, display control processing for displaying an advertisement on an augmented reality space viewed by the user.

Description

Information Processing Apparatus and Information Processing Method

[0001] The present invention relates to an information processing apparatus and an information processing method for displaying advertisements to a user using AR (Augmented Reality) technology.

[0002] There is known a system that causes devices such as smartphones, smart glasses, and head-mounted displays to use AR technology for superimposing and displaying digital information such as CG (Computer Graphics) on real space, and displays advertisements to users. For example, Patent Document 1 discloses a technology for displaying an advertisement image in an AR space. Further, Patent Document 2 discloses a technology for changing the display mode of an object displayed in an AR space.

[0003] Japanese Patent No. 6932224, Japanese Patent No. 5961736 Problem to be Solved

[0004] In conventional AR advertisement systems, the display of advertisements cannot be controlled in accordance with the user's conditions such as the user's line-of-sight direction and movement speed. Therefore, advertisements are displayed at timings and locations inappropriate for the user, resulting in a problem of poor user convenience.

[0005] In contrast, the present invention provides a technology for displaying appropriate advertisements in accordance with the user's state.

[0006] One aspect of the present disclosure provides an information processing apparatus including: a line-of-sight direction acquisition unit that acquires a user's line-of-sight direction; a movement speed acquisition unit that acquires the user's movement speed; and a control unit that performs display control processing for displaying an advertisement in an augmented reality space viewed by the user in accordance with the line-of-sight direction and the movement speed.

[0007] According to the present disclosure, appropriate advertisements can be displayed in accordance with the user's state.

[0008] A diagram illustrating the overall configuration of the information processing system according to the embodiment. A diagram illustrating the functional configuration of the AR glasses according to the embodiment. A diagram illustrating the hardware configuration of the server according to the embodiment. A diagram illustrating the hardware configuration of the AR glasses according to the embodiment. A diagram illustrating the appearance of the AR glasses according to the embodiment. A sequence diagram illustrating the overview of the operation of the information processing system according to the embodiment. An operation flow diagram illustrating the display control process. A diagram illustrating the screen of an AR advertisement. A diagram illustrating the display position of a determined advertisement. A diagram illustrating the display position of an advertisement determined according to surrounding obstacles. A diagram illustrating the display position of an advertisement determined according to surrounding obstacles. A diagram illustrating the size of a determined advertisement. A diagram illustrating the depth perception of a determined advertisement. A diagram illustrating the process of determining the image quality of an advertisement. A diagram illustrating other processes that determine the image quality of an advertisement. A diagram illustrating the process of determining the display of an advertisement according to the user's means of transportation.

[0009] 1. Diagram 1 illustrates the system configuration of the information processing system 1 according to the embodiment. The information processing system 1 is an AR advertising system that displays advertisements on AR glasses 100. The information processing system 1 comprises AR glasses 100 and a server 200. The AR glasses 100 are wearable devices worn by the user that overlay digital information such as CG onto the real world. Existing technologies are used for general processing of the AR glasses 100. The server 200 communicates with the AR glasses 100 via a network 2 such as the Internet and generates digital information to be displayed on the AR glasses 100. Note that the AR glasses 100 are an example of an information processing device.

[0010] Figure 2 is a diagram illustrating the functional configuration of the AR glasses 100 in an embodiment. The AR glasses 100 include a line-of-sight direction acquisition unit 101, a movement speed acquisition unit 102, a display unit 103, a communication unit 105, and a control unit 106.

[0011] The gaze direction acquisition unit 101 acquires the user's gaze direction. "Acquiring the gaze direction" means acquiring information about the gaze direction. The gaze direction is the center (a single point) of the user's field of view on the screen. Information about the gaze direction is, for example, information about the coordinates of the center of the user's field of view on the screen of the AR glasses 100.

[0012] The movement speed acquisition unit 102 acquires the user's movement speed. "Acquiring movement speed" means acquiring information related to movement speed.

[0013] The display unit 103 displays advertisements on the augmented reality space viewed by the user. Augmented reality space is a space in which digital information is superimposed on real space. The storage unit 104 stores various data and programs.

[0014] The communication unit 105 transmits and receives data to and from the server 200 via the network N. The control unit 106 controls the operation of the entire information processing system 1. The control unit 106 also performs display control processing, for example, according to the direction of gaze and movement speed. Display control processing involves generating digital information to be displayed in the augmented reality space.

[0015] Figure 3 illustrates the hardware configuration of the server 200 in an embodiment. Physically, the server 200 is configured as a computer including a processor 251, memory 252, storage 253, communication device 254, input device (optional), display device (optional), and a bus connecting these. Each of these devices operates on power supplied from a battery (not shown). In the following description, the term "device" can be read as a circuit, device, unit, etc. The hardware configuration of the server 200 may include one or more of the devices shown in Figure 3, or it may omit some of the devices. Furthermore, multiple devices with different enclosures may be connected via communication to constitute the server 200.

[0016] Each function in the server 200 is realized by loading predetermined software (programs) onto hardware such as the processor 251 and memory 252, which allows the processor 251 to perform calculations, control communication by the communication device 254, and control at least one of the reading and writing of data in the memory 252 and storage 253.

[0017] The processor 251 controls the entire computer, for example, by running an operating system. The processor 251 may be composed of a central processing unit (CPU) that includes interfaces with peripheral devices, control devices, arithmetic units, registers, etc. Alternatively, a baseband signal processing unit or a call processing unit may be implemented by the processor 251.

[0018] The processor 251 reads programs (program code), software modules, data, etc., from at least one of the storage 253 and the communication device 254 into the memory 252 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described later. Functional blocks of the server 200 may be stored in the memory 252 and implemented by control programs running on the processor 251. Various processes may be executed by one processor 251, or they may be executed simultaneously or sequentially by two or more processors 251. The processor 251 may be implemented by one or more chips. The program may also be transmitted to the server 200 via a telecommunications line.

[0019] The memory 252 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. The memory 252 may also be called a register, cache, main memory, etc. The memory 252 can store executable programs (program code), software modules, etc., for carrying out the method according to this embodiment.

[0020] The storage 253 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 253 may also be called an auxiliary storage device.

[0021] The communication device 254 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc.

[0022] Each device, such as the processor 251 and memory 252, is connected by a bus for communicating information. The bus may be configured using a single bus, or different buses may be used for each device.

[0023] The server 200 may be configured with hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be implemented by such hardware. For example, the processor 251 may be implemented using at least one of these hardware components.

[0024] In this example, the program stored in storage 253 includes a program (hereinafter referred to as the "server program") that causes the computer to function as server 200 in server 200. When the processor 251 is executing the server program, at least one of memory 252 and storage 253 is an example of a storage unit (not shown), the processor 251 is an example of a control unit (not shown), and the communication device 254 is an example of a communication unit (not shown).

[0025] Figure 4 illustrates the hardware configuration of the AR glasses 100 in an embodiment. The AR glasses 100 include a processor 151, memory 152, storage 153, communication device 154, display panel 155, projector 156, camera 157, camera 158, and sensor 159. A detailed explanation of the processor 151, memory 152, storage 153, and communication device 154 is omitted, but the storage 153 stores a program (hereinafter referred to as the "AR program") that causes the computer to function as the AR glasses 100 of the information processing system 1.

[0026] Generally, there are two image display methods for AR glasses: optical see-through and video see-through. The optical see-through method allows the user to directly view the real world through a half-mirror or the like, while the video see-through method displays the real world as a digital image on a display. In this example, AR glasses 100 are optical see-through AR glasses. The display panel 155 overlays digital information onto the real world. The display panel 155 is, for example, a half-mirror that transmits light from the real world. The projector 156 emits light onto the display panel 155 to overlay digital information onto the real world. The camera 157 photographs the area in front of the user wearing the AR glasses 100. The camera 158 photographs the eyes of the user wearing the AR glasses 100. The sensor 159 acquires information such as the user's current location and acceleration information. The sensor 159 is, for example, a GPS (Global Positioning System) and an acceleration sensor.

[0027] In the state where the processor 151 is executing the AR program, the processor 151 is an example of the control unit 106, at least one of the memory 152 and storage 153 is an example of the storage unit 104, and the communication device 154 is an example of the communication unit 105. Also, the display panel 155 and projector 156 are examples of the display unit 103, the camera 158 is an example of the gaze direction acquisition unit 101, and the sensor 159 is an example of the movement speed acquisition unit 102.

[0028] Figure 5 is an example of the appearance of the AR glasses 100. As mentioned above, the AR glasses 100 include a display panel 155, a projector 156, a camera 157, and a sensor 159. In this example, the AR glasses 100 have a spectacle-like shape and include components such as a frame, bridge, and temples.

[0029] 2. Operation diagram 6 is a sequence diagram illustrating the general operation of the information processing system 1 according to the embodiment. The process shown in Figure 6 shows the flow of processing from when the user puts on the AR glasses 100 and the information processing system 1 starts operating until an advertisement is displayed on the AR glasses 100. The process shown in Figure 6 starts, for example, when the information processing system 1 is started up. In the following, hardware such as the AR glasses 100 will be described as the main processing unit, which means that hardware elements such as the processor 151 that executes a program such as an AR program will work in cooperation with other hardware elements to perform the processing.

[0030] When the information processing system 1 is activated, the AR glasses 100 acquire information regarding the user's gaze direction and movement speed (step S1). The AR glasses 100 determine the user's gaze direction based on the image from the camera 158 mounted on the AR glasses 100. Specifically, the AR glasses 100 determine the gaze direction by detecting the user's face orientation and eye movement. The AR glasses 100 also determine the field of view based on the determined gaze direction. Alternatively, the AR glasses 100 may calculate average coordinates from the coordinates on the screen indicating the user's gaze direction at predetermined intervals (e.g., every 3 seconds) and process the calculated average coordinates as the gaze direction.

[0031] Furthermore, the AR glasses 100 calculate the user's movement speed based on location information and acceleration information acquired by sensors 159 (devices such as GPS and acceleration sensors) mounted on the AR glasses 100. Specifically, the AR glasses 100 calculate the movement speed based on the change in the user's location information over a predetermined time interval (for example, 1 second).

[0032] Next, the AR glasses 100 determine the display mode of the advertisement to be displayed on the AR glasses 100 based on the acquired information regarding the direction of gaze and movement speed (step S2). The display mode refers to the configuration of the advertisement to be displayed, and includes elements such as the position, size, depth perception, and temporal changes of the advertisement to be displayed.

[0033] Figure 7 is a flowchart of the display control process. The details of the display control process are described below. The AR glasses 100 determine the display position of the advertisement (step S201). The display position of the advertisement is determined, for example, according to the user's gaze direction and movement speed. The AR glasses 100 selects the display position of the advertisement from among the candidate areas included on the screen, according to the user's gaze direction and movement speed. Selecting "according to the gaze direction" means, for example, selecting a candidate area that is within the field of view identified from the user's gaze direction. Selecting "according to the movement speed" means that the faster the user's movement speed, the more likely it is to select a candidate area that is closer to the center of the screen.

[0034] Figure 8 illustrates a candidate area on the screen of the AR glasses 100. In the example in Figure 8, the candidate area is indicated by a shaded trapezoid. The candidate area is an area where an advertisement may be displayed. The conditions for an area to be a candidate area are predetermined. The candidate area is, for example, a surface of an object such as a building, road, sign, utility pole, or streetlamp, and has an area greater than or equal to a threshold. The AR glasses 100 identifies the candidate area by recognizing an object from an image captured by the camera 157, for example. For the sake of simplicity in this diagram, the advertisement content (i.e., the content of the product or service) is omitted, and the same applies to other figures. Furthermore, the method for determining the candidate area for the advertisement is not limited to this. Existing technologies such as machine learning may also be used for object recognition.

[0035] The AR glasses 100 assign a unique location ID to each candidate area and a unique advertisement ID to each advertisement to be displayed, and manage the candidate areas and the advertisements to be displayed in those candidate areas (i.e., those assigned to those candidate areas) using a database (not shown). As a result, once an advertisement is assigned to a candidate area, it will move in accordance with the movement of the object related to that candidate area on the screen, as long as the object related to that candidate area is present on the screen of the AR glasses 100.

[0036] Figure 9 illustrates a candidate area. Figure 9 shows an example of a screen displayed on the AR glasses 100. In the example in Figure 9, the peripheral field of view is represented by a gray circle, and the candidate areas are represented by a diagonal trapezoid. The peripheral field of view refers to a predetermined range (extent) of area based on the direction of gaze. The center of the gray circle represents the user's direction of gaze. In this example, the candidate areas where advertisements are displayed are limited to those within the peripheral field of view. Figure 9 does not show candidate areas outside the peripheral field of view. If there are multiple candidate areas within the peripheral field of view, the candidate areas may be further narrowed down according to a predetermined priority order. For example, a higher priority is given to candidate areas that are surfaces of objects closer to the user.

[0037] In this example, the display position of the advertisement is determined based on the user's movement speed, selecting one candidate area from a narrowed-down list of candidates according to the user's gaze direction. The AR glasses 100, for example, select a candidate area that is further away from the user as the user's movement speed increases. The display position of the advertisement is determined, for example, according to a movement speed coefficient. For example, the movement speed coefficient is set to a range of "0 ≤ x ≤ 1" for the user's movement speed, with a predetermined movement speed (e.g., 20 km / h) being "1", and the value approaches "0" as the user's movement speed decreases, with "0" representing the state where the user is stationary. The higher the value of the movement speed coefficient (i.e., the closer the value is to "1"), the further away the candidate area from the user is selected.

[0038] The AR glasses 100 acquire information about the user's movement speed and current location information (e.g., latitude and longitude coordinates) based on information acquired by the sensor 159, for example. Generally, it is considered preferable to display an advertisement for about 5 seconds. For example, if the user is moving at 20 km / h (approximately 5.6 m / s), the AR glasses 100 will display an advertisement in one candidate area selected from candidate areas located at least 28 m (5 (seconds) x 5.6 (m / s)) away from the user's current location. Also, for example, if the user is moving at 7 km / h (approximately 1.4 m / s), the advertisement will be displayed in one candidate area selected from candidate areas located at least 7 m (5 (seconds) x 1.4 (m / s)) away from the user's current location. Note that the length of the advertisement to be displayed and the candidate area for the advertisement to be displayed are not limited to these examples.

[0039] If the user's gaze direction changes, the display position of the advertisement may be changed to follow the user's gaze direction. For example, the AR glasses 100 determine the display position of the advertisement based on candidate areas identified in the peripheral field of view after the movement.

[0040] Furthermore, the AR glasses 100 may, for example, re-determine the display position of the advertisement after selecting a candidate area, depending on surrounding obstacles. Obstacles include, for example, people, bicycles, cars, utility poles, street trees, signs, and other objects. The display position of the advertisement may be determined, for example, depending on the presence or absence of obstacles. For example, if there are obstacles around the user, the AR glasses 100 may exclude the area where the advertisement and the obstacle overlap from the candidate area when determining the display position of the advertisement.

[0041] FIG. 10 is a diagram illustrating an example of advertisement display positions determined according to surrounding obstacles. In the example of FIG. 10, a utility pole exists on the roadside as an obstacle. In the example of FIG. 10A, the space shown in gray overlaps the advertisement and the utility pole. Therefore, as shown in the example of FIG. 10B, the AR glasses 100 exclude candidate areas that overlap the utility pole. Note that the advertisement display position may be determined according to the type of obstacle.

[0042] Returning to FIG. 7, the description is resumed. Next, the AR glasses 100 determine the size of the advertisement (step S202). The size of the advertisement is determined, for example, according to the user's line-of-sight direction and moving speed. For example, the AR glasses 100 display the advertisement smaller than the reference size (100%) as the distance from the user to an object having a surface serving as a candidate area (hereinafter simply referred to as "distance to the candidate area") is farther. The reference size is the size when the advertisement is displayed as it is (without display control processing). The reference size is, for example, the area size of the identified candidate area.

[0043] The size of the advertisement is determined, for example, according to a distance coefficient. For example, as the distance coefficient, with the distance to the candidate area falling within the range of 0≤y≤1, a predetermined first distance (for example, 50 m or more) is set to "1"; the closer the distance to the candidate area is, the closer the coefficient is to "0", and a predetermined second distance (for example, 1 m or less) is set to "0". The higher the value of the distance coefficient (that is, the closer the value is to "1"), the smaller the advertisement is displayed compared to the reference size.

[0044] The AR glass 100 calculates the distance to the candidate area based on, for example, information about the user's line-of-sight direction. A well-known calculation method such as a method based on SLAM (Simultaneous Localization and Mapping) is used to calculate the distance to the candidate area, for example. For example, when reducing the size of an advertisement by up to 50% from the reference size, if the distance to the candidate area is 50 m or more (that is, when the distance coefficient is "1"), the AR glass 100 displays the advertisement 50% smaller than the reference size (100(%) - 50(constant) × 1(coefficient)). Further, for example, when the distance to the candidate area is approximately 25 m, the AR glass 100 displays the advertisement 25% smaller than the reference size (100(%) - 50(constant) × 0.5(coefficient)). Furthermore, for example, when the distance to the candidate area is 1 m or less, the AR glass 100 displays the advertisement at the reference size (100(%) - 50(constant) × 0(coefficient)). Note that the size of the displayed advertisement is not limited thereto.

[0045] FIG. 11 is a diagram illustrating the determined size of an advertisement. FIG. 11 shows an example of a screen displayed on the AR glass 100. In the example of FIG. 11, since the distance to the candidate area is, for example, far (the distance to the candidate area is 50 m or more), the AR glass 100 displays the advertisement smaller than the reference size.

[0046] The size of the advertisement is determined based on the moving speed of the user. For example, the AR glass 100 displays the advertisement in a smaller size as the moving speed of the user is faster. The size of the advertisement is determined according to, for example, a moving speed coefficient. For the moving speed coefficient, the higher the value thereof (that is, the closer the value is to "1"), the smaller the advertisement is displayed compared to the reference size.

[0047] The AR glasses 100 acquire information about the user's movement speed, for example. When the AR glasses 100 reduces the size of an advertisement by up to 50% from the standard size, for example, if the user is moving at 20 km / h (i.e., the speed coefficient is "1"), the advertisement is displayed 50% smaller than the standard size (100 (%) - 50 (constant) × 1 (coefficient)). Also, for example, if the user is moving at 10 km / h, the advertisement is displayed 75% smaller than the standard size (100 (%) - 50 (constant) × 0.5 (coefficient)). Also, for example, if the user is stationary (i.e., the speed coefficient is "0"), the advertisement is displayed at the standard size (100% - 50 (constant) × 0 (coefficient)). The size of the advertisement to be displayed is not limited to these examples.

[0048] Furthermore, if the user's gaze direction changes, the size of the advertisement may change to follow the user's gaze direction. For example, if the user's gaze direction moves to a predetermined advertisement, the AR glasses 100 will display that advertisement at its standard size.

[0049] The size of the advertisement may be determined according to the distance coefficient and the speed coefficient. The AR glasses 100 will, for example, display the advertisement smaller than the standard size as the sum of the distance coefficient and the speed coefficient, "0 ≤ x + y ≤ 2", increases (i.e., as the sum approaches "2").

[0050] Let's return to Figure 7 and resume the explanation. Next, the AR glasses 100 determine the sense of depth of the advertisement (step S203). Sense of depth (i.e., three-dimensionality) is a visual effect that creates a sense of distance in flat displays such as screens. Sense of depth is determined, for example, based on the user's line of sight. For example, the AR glasses 100 displays the advertisement smaller than the standard sense of depth the further away the user's line of sight is. The standard sense of depth is the sense of depth when the advertisement is displayed as is (without display control processing). The standard sense of depth is, for example, a sense of depth predetermined by the advertiser (advertiser) for each advertisement.

[0051] The sense of depth in an advertisement is determined by a distance coefficient. The higher the value of the distance coefficient (i.e., the closer the value is to "1"), the smaller the advertisement will appear compared to the standard sense of depth.

[0052] The AR glasses 100 calculate the distance to the candidate area based on information regarding the user's gaze direction. For example, when an advertisement is to be up to 50% smaller than the standard depth perception, if the distance to the candidate area is 50m or more (i.e., the distance coefficient is "1"), the AR glasses 100 will display the advertisement 50% smaller than the standard depth perception (100 (%) - 50 (constant) × 1 (coefficient)). Also, for example, if the distance to the candidate area is approximately 25m, the advertisement will be displayed 75% smaller than the standard depth perception (100 (%) - 50 (constant) × 0.5 (coefficient)). Also, for example, if the distance to the candidate area is 1m or less, the advertisement will be displayed at the standard depth perception (100 (%) - 50 (constant) × 0 (coefficient)). Note that the depth perception of the displayed advertisement is not limited to these examples.

[0053] Figure 12 illustrates the determined depth perception of an advertisement. In the example in Figure 12, the user's line of sight is represented by a gray circle, and the advertisement is represented by a diagonal trapezoid. In this example, because the user's line of sight is, for example, far away (distance to the candidate area is 50m or more), the AR glasses 100 display the advertisement smaller than the standard depth perception.

[0054] Furthermore, the sense of depth of the advertisement is determined according to the user's movement speed. For example, the AR glasses 100 will display the advertisement smaller than the standard sense of depth the faster the user moves. The sense of depth of the advertisement is determined according to, for example, the movement speed coefficient. The higher the value of the movement speed coefficient (i.e., the closer the value is to "1"), the smaller the advertisement will be displayed than the standard sense of depth.

[0055] The AR glasses 100 acquire information about the user's movement speed. For example, when the depth perception of an advertisement is reduced by up to 50% from the standard depth perception, the AR glasses 100 will display the advertisement 50% smaller (100 (%) - 50 (constant) × 1 (coefficient)) than the standard depth perception when the user is moving at 20 km / h (i.e., the speed coefficient is "1"). Also, for example, when the user is moving at 10 km / h, the advertisement will be displayed 75% smaller (100 (%) - 50 (constant) × 0.5 (coefficient)) than the standard depth perception. Also, for example, when the user is stationary (i.e., the speed coefficient is "0"), the advertisement will be displayed with the standard depth perception (100 (%) - 50 (constant) × 0 (coefficient)). Note that the depth perception of the displayed advertisement is not limited to these examples.

[0056] Furthermore, if the user's gaze direction changes, the sense of depth of the advertisement may be changed to follow the user's gaze direction. For example, if the user's gaze direction moves to a specific advertisement, the AR glasses 100 will display that advertisement with a standard sense of depth.

[0057] The sense of depth of an advertisement may be determined according to a distance coefficient and a movement speed coefficient. For example, the AR glasses 100 will display the advertisement smaller than the standard sense of depth as the sum of the distance coefficient and movement speed coefficient, "0 ≤ x + y ≤ 2", increases (i.e., as the sum approaches "2").

[0058] Let's return to Figure 7 and resume the explanation. Next, the AR glasses 100 determine the time change of the advertisement (step S204). Time change refers to the change in the state of the advertisement over time. Specifically, the composition of the advertisement, such as text, shapes, colors, and size, changes over time. The degree of time change is determined, for example, based on the direction of gaze. For example, the AR glasses 100 displays the advertisement later than the reference time change the further away the user's gaze is directed. The reference time change is the time change when the advertisement is displayed as is (without display control processing). The reference time change is a time change predetermined by the advertiser (advertiser) for each advertisement.

[0059] The AR glasses 100 display advertisements more slowly than the standard time change the further away the user's gaze is directed. The time change is determined, for example, according to a distance coefficient. The higher the value of the distance coefficient (i.e., the closer the value is to "1"), the slower the advertisement is displayed compared to the standard time change. Note that "displaying advertisements slower than the standard time change" means, for example, that the frequency or speed of advertisement switching, advertisement playback speed, and advertisement animation speed are reduced.

[0060] The AR glasses 100 calculate the distance to the candidate area based on information about the user's gaze direction. For example, when the time change of an advertisement is set to be up to 50% slower than the standard time change, if the distance to the candidate area is 50m or more (i.e., the distance coefficient is "1"), the AR glasses 100 will display the advertisement 50% (100 (%) - 50 (constant) × 1 (coefficient)) slower than the standard time change. Also, for example, if the distance to the candidate area is approximately 25m, the advertisement will be displayed 75% (100 (%) - 50 (constant) × 0.5 (coefficient)) slower than the standard time change. Also, for example, if the distance to the candidate area is 1m or less, the advertisement will be displayed at the standard time change (100 (%) - 50 (constant) × 0 (coefficient)). Note that the time change of the displayed advertisement is not limited to these examples.

[0061] Furthermore, the time progression of the advertisement is determined according to the user's movement speed. For example, the AR glasses 100 will display the advertisement slower than the standard time progression the faster the user is moving. Also, the sense of depth of the advertisement is determined according to, for example, the movement speed coefficient. The higher the value of the movement speed coefficient (i.e., the closer the value is to "1"), the slower the advertisement will be displayed compared to the standard time progression.

[0062] The AR glasses 100 acquire information about the user's movement speed, for example. For example, when the time change of an advertisement is reduced by up to 50% compared to the standard time change, the AR glasses 100 displays the advertisement 50% smaller (100 (%) - 50 (constant) × 1 (coefficient)) than the standard time change when the user is moving at 20 km / h (i.e., the movement speed coefficient is "1"). Also, for example, when the user is moving at 10 km / h, the advertisement is displayed 75% smaller (100 (%) - 50 (constant) × 0.5 (coefficient)) than the standard time change. Also, for example, when the user is stationary (i.e., the speed coefficient is "0"), the advertisement is displayed at the standard time change (100% - 50 (constant) × 0 (coefficient)). Note that the time change of the displayed advertisement is not limited to these examples.

[0063] Furthermore, if the user's gaze direction changes, the time-dependent changes in the advertisement may be modified to follow the user's gaze direction. For example, if the user's gaze direction moves to a predetermined advertisement, the AR glasses 100 will display that advertisement with a standard time-dependent change.

[0064] The time change of the advertisement may be determined according to the distance coefficient and the movement speed coefficient. For example, the AR glasses 100 will display the advertisement smaller than the reference time change as the sum of the distance coefficient and the movement speed coefficient "0 ≤ x + y ≤ 2" increases (i.e., as the sum approaches "2").

[0065] Returning to Figure 6, we resume the explanation. Next, the AR glasses 100 receive information about the advertisement to be displayed from the server 200 (step S3). Next, the AR glasses 100 overlay the received advertisement onto the real space and display it on the display unit 103 according to the determined display mode (step S4).

[0066] According to the information processing system 1 described above, advertisements can be displayed effectively in a manner that does not interfere with the user's ability to view the surrounding scenery. As a result, even when the user is on the move, both the safety of the user's movement and the persuasive effect of the advertisement can be achieved.

[0067] 3. Modifications The present invention is not limited to the embodiments described above, and various modifications are possible. Several modifications are described below. Two or more of the matters described below may be combined and applied.

[0068] (1) Determination of Advertisement Image Quality Figures 13 and 14 illustrate the process of determining the image quality of an advertisement. The AR glasses 100 may determine the image quality of the advertisement. The image quality of the advertisement may be determined, for example, according to the user's surrounding environment. The surrounding environment refers to, for example, the environment such as brightness and weather. Image quality refers to elements such as luminance, contrast, saturation, and resolution. For example, the AR glasses 100 will display the luminance of the advertisement higher the brighter the user's surroundings are. Also, for example, the AR glasses 100 will display the luminance of the advertisement lower the darker the user's surroundings are.

[0069] The brightness of an advertisement is determined, for example, according to a brightness coefficient. For example, the brightness coefficient is the ambient light around the user (candela: cd / m²). 2 ) is set to the range "0 ≤ z ≤ 1", and a predetermined brightness (for example, 1,000 cd / m²) is defined. 2 A value of "1" represents a certain level of brightness around the user, with the value approaching "0" as the brightness around the user decreases, and "0" representing a state where there is no light around the user. The higher the value of the brightness coefficient (i.e., the closer the value is to "1"), the brighter the advertisement will appear.

[0070] The AR glasses 100 acquire information about the brightness around the user, for example. For example, when determining the brightness of an advertisement, the AR glasses 100 will acquire information about the brightness around the user, such as 1,000 cd / m². 2 In the above case (i.e., when the luminance coefficient is "1"), the brightness of the advertisement is determined and displayed as 100% (100 (%) × 1 (coefficient)). Also, for example, if the ambient light around the user is 1 cd / m² 2 In the following cases, the brightness of the advertisement will be set to 0% (100 (%) × 0 (coefficient)) and displayed. However, the brightness of the displayed advertisement is not limited to this.

[0071] As shown in Figure 13, for example, if it is around 8 p.m. and the user's surroundings are dark, the AR glasses 100 will display the advertisement with reduced brightness.

[0072] In the example in Figure 13, the entire area around the user was dark, but in the example in Figure 14, there is a streetlamp along the roadside, one advertisement is located in a bright area illuminated by the streetlamp, and the other advertisements are located in a dark area not illuminated by the streetlamp. The AR glasses 100 may, for example, acquire information about the brightness of each candidate area and determine the brightness for each advertisement. In this example, the AR glasses 100 determines a high brightness for the advertisement illuminated by the streetlamp, and a low brightness for the other advertisements not illuminated by the streetlamp, and displays the advertisements.

[0073] (2) Determination of display mode according to means of transportation The AR glasses 100 may further have a means of transportation identification unit (not shown). The means of transportation identification unit identifies the user's means of transportation based on the user's gaze direction and speed of movement. Means of transportation include, for example, walking, vehicles, ships, etc. Vehicles include, for example, bicycles, electric scooters, motorcycles, automobiles, buses, trucks, railway vehicles, ships, aircraft, etc. In addition, if the means of transportation identification unit identifies the user's means of transportation as a vehicle based on the user's gaze direction and speed of movement, it may also identify whether the user is driving the vehicle themselves.

[0074] The AR glasses 100, for example, identify that the user's means of transportation is a vehicle if the speed of movement exceeds a predetermined speed. The predetermined speed is, for example, 10 km / h or more. However, the predetermined speed is not limited to this. Furthermore, the AR glasses 100 also identify that the user is driving the vehicle if, for example, the image from the camera 157 shows components installed in the driver's seat of the vehicle (e.g., steering wheel, instrument panel, or front pillar).

[0075] The manner in which advertisements are displayed may be determined based on the mode of transportation of the identified user. For example, if the user's mode of transportation is a vehicle and the user is driving the vehicle, the AR glasses 100 may display the advertisement above the user's eye level, reduce elements such as the size, depth, and temporal changes of the advertisement, and reduce the number of advertisements. Conversely, if the user is not driving (i.e., the mode of transportation is walking, or the user is in a vehicle but is not driving), the AR glasses 100 may display the advertisement near the user's eye level, increase elements such as the size, depth, and temporal changes of the advertisement, and increase the number of advertisements.

[0076] Figure 15 illustrates the process of determining how advertisements are displayed according to the user's mode of transportation. Figure 15 shows the image from camera 157, which displays components of the driver's seat of a vehicle (automobile), specifically the steering wheel, instrument panel, and front pillar. Therefore, the AR glasses 100 identify that the user's mode of transportation is a vehicle (automobile) and that the user is driving. The AR glasses 100 also reduce the size of the advertisement elements and the number of advertisements displayed.

[0077] Furthermore, the manner in which advertisements are displayed may be determined based on the distance between vehicles driven by the user. For example, when the distance between vehicles is short, the AR glasses 100 may reduce the size, depth perception, and temporal changes of the advertisements, as well as the number of advertisements. Conversely, when the distance between vehicles is far, the AR glasses 100 may increase the size, depth perception, and temporal changes of the advertisements, as well as the number of advertisements. In this way, by displaying advertisements in a manner that minimizes obstruction of the driver's view, reduction of concentration, and other factors that hinder driving operations, it is possible to achieve both user (and passenger) safety and the appealing effect of the advertisements.

[0078] In the above embodiment, we described a case where elements such as position, size, depth perception, and temporal changes are determined as the display manner of the advertisement, but the invention is not limited to this. As long as the display manner of the advertisement can be determined, other elements that constitute the advertisement may also be determined.

[0079] (3) Candidate Region In the above embodiment, the candidate region was the surface of an object, but is not limited thereto. The candidate region may be limited to an area without an object, such as the sky (air).

[0080] (4) Advertisement display position In the above embodiment, the advertisement display position moved in accordance with the movement of objects on the screen (following the direction of the user's gaze), but is not limited to this. The advertisement display position may, for example, remain displayed at a specific position on the screen (even if the direction of the user's gaze changes).

[0081] (5) Identification of Candidate Regions In the above embodiment, when there are multiple candidate regions in the peripheral field of view, a higher priority was given to the candidate region which is the surface of an object that is closer to the user, but this is not limited to this. When there are multiple candidate regions in the peripheral field of view, for example, a higher priority may be given to the candidate region which is closest to the user's line of sight.

[0082] (6) Types of AR glasses 100 In the above embodiment, the AR glasses 100 were optical see-through type smart glasses, but are not limited to this. The AR glasses 100 may be, for example, video see-through type.

[0083] (7) Shape of AR glasses The shape of the AR glasses 100 is not limited to the glasses-type shape shown in Figure 5. The shape of the AR glasses 100 may be any shape, such as goggles, helmets, or headsets.

[0084] (8) Other programs executed by the processor 151 may be provided by download via a network such as the Internet, or they may be provided recorded on a computer-readable non-temporary recording medium such as a DVD-ROM. Each processor may be, for example, a CPU, an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit).

[0085] The block diagrams used in the description of the above embodiments show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining software with the one or more of the above devices.

[0086] Functions include, but are not limited to, judgment, decision, determination, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmission unit or transmitter. In all cases, as mentioned above, the method of implementation is not particularly limited.

[0087] For example, the information processing system 1 in one embodiment of the present disclosure may function as a computer that performs the processing described in the present disclosure.

[0088] Each aspect or embodiment described in this disclosure may be applied to at least one of the following: LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (new Radio), W-CDMA®, GSM®, CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi®), IEEE 802.16 (WiMAX®), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth®, and other appropriate systems, as well as next-generation systems extended based thereon. Furthermore, multiple systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A with 5G).

[0089] The processing procedures, sequences, flowcharts, etc., of each aspect or embodiment described in this disclosure may be reordered, provided they do not contradict each other. For example, the methods described in this disclosure present various step elements in an exemplary order and are not limited to the specific order presented.

[0090] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be sent to other devices.

[0091] The determination may be made by a value represented by one bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, by comparing with a predetermined value).

[0092] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0093] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name. Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technologies (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technologies (such as infrared or microwave), at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0094] The information, signals, etc., described herein may be represented using any of the following different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc., which may be referred to throughout the above description, may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof. Terms used herein and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meaning.

[0095] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0096] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0097] Any reference to elements using the designations “First,” “Second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to the First and Second elements do not imply that only two elements may be employed, or that the First element must precede the Second element in any way.

[0098] In the above-described configuration of each device, the term "part" may be replaced with "means," "circuit," "device," etc.

[0099] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.

[0100] In this disclosure, if articles are added by translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0101] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different."

[0102] 100...AR glasses, 101...Eyew direction acquisition unit, 102...Movement speed acquisition unit, 103...Display unit, 104...Storage unit, 105...Communication unit, 106...Control unit, 151...Processor, 152...Memory, 153...Storage, 154...Communication device, 155...Display panel, 156...Projector, 157...Camera, 158...Camera, 159...Sensor, 200...Server

Claims

1. An information processing device comprising: a gaze direction acquisition unit that acquires the direction of the user's gaze; a movement speed acquisition unit that acquires the speed of the user's movement; and a control unit that performs display control processing to display an advertisement on the augmented reality space viewed by the user according to the gaze direction and the speed of movement.

2. The information processing apparatus according to claim 1, wherein the control unit determines the display position of the advertisement in the augmented reality space.

3. The information processing apparatus according to claim 1, wherein the control unit determines the display size of the advertisement in the augmented reality space.

4. The information processing apparatus according to claim 1, wherein the control unit determines the sense of depth of the advertisement in the augmented reality space.

5. The information processing apparatus according to claim 1, wherein the control unit determines the time change of the advertisement in the augmented reality space.

6. The information processing apparatus according to claim 1, wherein the control unit displays the advertisement in accordance with the obstacles surrounding the user.

7. The information processing apparatus according to claim 1, wherein the display control process displays the advertisement according to the user's surrounding environment.

8. The information processing apparatus according to claim 1, wherein the control unit displays the advertisement according to the user's means of transportation.

9. An information processing method comprising the steps of: obtaining the direction of the user's gaze; obtaining the speed of the user's movement; and performing display control processing to display an advertisement in the augmented reality space viewed by the user according to the direction of the gaze and the speed of movement.

Images