Information processing device and information processing method

Abstract

To appropriately determine the display conditions for virtual representation information in real space. [Solution] The information processing device 100 acquires first information relating to the real space, second information relating to the real display information displayed in the real space, and third information relating to the request for displaying virtual display information on the real space. Based on the first information, second information, and third information, the display conditions for the virtual display information are determined.

Description

Technical Field

[0001] The present invention relates to a technique for displaying virtual information in the real space.

Background Art

[0002] A mixed reality (MR) technique for superimposing a video of the real space and virtual contents (such as CG and characters) on an information terminal such as an HMD (Head Mounted Display) or a smartphone is known. The virtual contents include various advertisements.

[0003] Patent Document 1 discloses a method of associating the real space with the virtual space and displaying a virtual advertisement as virtual display information at a location corresponding to the real space in the virtual space.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when displaying a virtual advertisement in the real space, it is necessary to consider the relationship between the real advertisement already existing in the real space and the virtual advertisement.

[0006] The present invention provides an information processing apparatus and an information processing method capable of appropriately determining display conditions for a virtual advertisement (virtual display information) with respect to the real space.

Means for Solving the Problems

[0007] An information processing device as one aspect of the present invention is characterized by comprising: a first acquisition means for acquiring first information relating to real space; a second acquisition means for acquiring second information relating to real display information displayed in real space; a third acquisition means for acquiring third information relating to a request for displaying virtual display information on real space; and a determination means for determining display conditions for virtual display information based on the first information, the second information, and the third information.

[0008] Furthermore, another aspect of the present invention is an information processing method characterized by comprising the steps of: acquiring first information relating to real space; acquiring second information relating to real display information displayed in real space; acquiring third information relating to a request for displaying virtual display information on real space; and determining display conditions for virtual display information based on the first information, the second information, and the third information. [Effects of the Invention]

[0009] According to the present invention, the display conditions for virtual display information in real space can be appropriately determined. [Brief explanation of the drawing]

[0010] [Figure 1] A diagram showing an example of an application scenario in Example 1. [Figure 2] A diagram showing the hardware configuration of the information processing device in Example 1. [Figure 3] A diagram showing the logical configuration of the information processing device in Example 1. [Figure 4] A flowchart showing the process performed by the information processing device in Example 1. [Figure 5] A diagram showing an example of real-world advertising information in Example 1. [Figure 6] A diagram showing an example of real space in Example 1. [Figure 7] A diagram showing an example of real-world spatial information in Example 1. [Figure 8] A diagram showing an example of a request for displaying an MR advertisement in Example 1. [Figure 9]A flowchart showing the determination process in Example 1. [Figure 10] A flowchart showing the display condition determination process in Example 1. [Figure 11] A diagram showing the logical configuration of the information processing device in Example 2. [Figure 12] A flowchart showing the process performed by the information processing device in Example 2. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. [Examples]

[0012] The information processing device of Example 1 is used by a client (hereinafter referred to as the MR advertisement display client) who wants viewers wearing a display device to see advertisements as virtual content displayed in the real world (virtual display information: hereinafter referred to as MR advertisements). The MR advertisement display client uses the information processing device to obtain the display conditions for MR advertisements. The display conditions for MR advertisements are various conditions for displaying MR advertisements in the real world, and are determined considering the impact that the display of MR advertisements has on advertisements existing in the real world (real display information: hereinafter referred to as real advertisements). Having obtained the display conditions for MR advertisements, the MR advertisement display client can solicit advertisers and other providers who wish to display MR advertisements under those display conditions, or propose the display of M5R advertisements under those display conditions to specific providers.

[0013] In this embodiment, the real-world space where the MR advertisement is displayed is represented by an image displayed on a display device such as a video see-through HMD. However, it may also be the actual real-world space that is not represented on an optical see-through display device such as AR (Augmented Reality) glasses.

[0014] FIG. 1 shows an example of a usage scene of the information processing apparatus according to Embodiment 1. The information processing apparatus is constituted by a personal computer, a server computer, or the like, and is used for an MR advertisement requester to acquire display conditions of an MR advertisement.

[0015] In FIG. 1, a viewer 101 wearing an HMD is standing beside an intersection in an actual real space and viewing virtual space content including an image of the real space displayed on an HMD screen 102 and an MR advertisement 103 superimposed thereon. In this example, the MR advertisement 103 is displayed on the real space above a building 106 that actually exists within the HMD screen 102.

[0016] Also, in the real space, a real advertisement 104 is displayed (posted) on the side surface of a building 107. Also, a real advertisement 105 is displayed on the rooftop of a building 108. If the MR advertisement 103 is directly displayed when the MR advertisement requester 110 requests the MR advertisement 103 to be displayed on the real space above the building 106, the MR advertisement 103 will be displayed overlapping the real advertisement 104 within the HMD screen 102. That is, the visibility of the real advertisement 104 by the viewer 101 is affected by the display of the MR advertisement 103. In particular, if the visibility of the real advertisement is affected when the MR advertisement is an advertisement with different content (such as different products) from the real advertisement or is an advertisement displayed by a different business operator than the real advertisement, it is inconvenient for the advertiser of the real advertisement, etc.

[0017] FIG. 2 shows the hardware configuration of the information processing apparatus according to Embodiment 1. The information processing apparatus has a CPU 10. The CPU 10 is connected to an input I / F 40, an output I / F 50, a ROM 20, and a RAM 30 via a bus 60. The input I / F 40 acquires an input signal in a form that can be processed by the CPU 10 from an external device (such as an imaging device, a display device, and an input device) not shown. The output I / F 50 outputs an output signal in a form that can be processed by an external device (such as a display device) not shown to the external device.

[0018] ROM20 stores the operating system (OS) for the CPU10 to run, device drivers for the input I / F40 and output I / F50 to run, and processing programs for the CPU10 to execute the processes described later. RAM30 temporarily stores the processing programs read from ROM20, and the CPU10 executes the processes described later according to the processing programs stored in RAM30. Alternatively, processing may be performed by hardware, including an arithmetic unit and various circuits, instead of software processing using the CPU10.

[0019] Figure 3 shows the configuration of the information processing device 100 (CPU 10) in Embodiment 1. The information processing device 100 includes a real-world advertising information acquisition unit 301, a real-world space information acquisition unit 302, an MR advertising request input unit 303, an MR advertising determination unit 304, and an MR advertising decision unit 305. The MR advertising decision unit 305 is connected to the display unit 33. The display unit 33 is an HMD, and displays real-world images and virtual space content on the HMD screen 102 as described above.

[0020] The real-space information acquisition unit 302, as the first acquisition means, creates information relating to the real space (first information: hereinafter referred to as real-space information). The real-space information includes information about the owner of the real space, such as the owner's name and ownership period (unlimited or finite dates or periods). The owner of the real space is the owner of the real space itself, regardless of whether there are objects in the real space, and also includes the owner of air rights, which are the right to use a portion of the airspace above the land.

[0021] Furthermore, real-world spatial information includes attribute information of the real-world space, such as information indicating where that real-world space belongs, such as being inside a building or on a road. In addition, real-world spatial information also includes viewing feasibility information indicating whether or not the viewer can view the HMD in that real-world space.

[0022] The second acquisition method, the Real Advertisement Information Acquisition Unit 301, is created by acquiring information related to real advertisements (second information: hereinafter referred to as "real advertisement information") from the requester of the real advertisement. The real advertisement information includes information such as the location, orientation and size of the real advertisement, the requester or rights holder of the real advertisement, the display period of the real advertisement, and the conditions under which the display of MR advertisements superimposed on the real advertisement is permitted. The real advertisement information may also include information indicating the content of the real advertisement (such as what kind of product is being advertised).

[0023] The MR advertisement request input unit 303, which serves as a third acquisition method, acquires MR advertisement requests (third-party information regarding requests for MR advertisements in real space) entered by the MR advertisement display requester. The MR advertisement display request includes the location and size of the space where the MR advertisement will be displayed, the method of displaying the MR advertisement, and the display period, etc., as requested by the MR advertisement display requester.

[0024] The MR ad determination unit 304 determines whether the display of an MR ad in response to an MR ad display request affects the viewer's 101 visibility of the real ad (i.e., whether the display of the MR ad affects the visibility of the real ad). The effect on the visibility of the real ad means that the display of the MR ad changes the viewer's visibility of the real ad, making the real ad invisible or difficult to see.

[0025] The MR advertisement determination unit 305 determines the display conditions for the MR advertisement according to the results determined by the MR advertisement judgment unit 304. The display conditions for the MR advertisement include the display range as the range of the real space in which the MR advertisement is displayed, the display period for displaying the MR advertisement (such as always or a limited period or time), the range of positions in which viewers can view the MR advertisement (hereinafter referred to as the viewing range), and the payment (contract) conditions for the fee to display the MR advertisement. The determination means is composed of the MR advertisement judgment unit 304 and the MR advertisement determination unit 305.

[0026] The flowchart in Figure 4 shows the process (information processing method) that the information processing device 100, used by the MR advertisement display requester, executes according to the processing program. In step (process) S400, the information processing device 100 performs initialization processing.

[0027] Next, in step S401, the real-world advertisement information acquisition unit 301 acquires real-world advertisement information. Specifically, for each real-world advertisement, it creates real-world advertisement information by collecting information such as the display location, orientation, size, display requester, display rights holder, display period, and permission conditions for MR advertisements.

[0028] Next, in step S402, the real-space information acquisition unit 302 acquires real-space information. Specifically, it acquires owner information, ownership period, attribute information, and viewing availability information for each of the arbitrarily divided real-space areas.

[0029] Next, in step S403, the MR advertisement request input unit 303 obtains the MR advertisement request entered by the MR advertisement display requester. The MR advertisement request must include the location and size of the space where the MR advertisement will be displayed and the display period, as mentioned above. In addition, it may include display methods such as whether or not the MR advertisement will be displayed in the same space at all times and whether or not the space where the MR advertisement will be displayed will be moved. It may also include requests regarding contract terms such as the amount that can be paid for the display of the MR advertisement.

[0030] Next, in step S404, the MR ad determination unit 304 determines whether the display of the MR ad will affect the visibility of the real ad, based on the real ad information, real space information, and MR ad request obtained in the previous step. If it determines that there will be an impact on the visibility of the real ad, it generates and outputs an impact determination result for that real ad, which is associated with the permission conditions for the MR ad and the MR ad request from the real ad information. Details of the impact determination process in this step will be described later.

[0031] Next, in step S405, the MR advertisement determination unit 305 uses the impact determination result output by the MR advertisement judgment unit 304 to determine and output the display conditions for the MR advertisement. Details of the display condition determination process in this step will be described later.

[0032] Next, in step S406, the information processing device 100 determines whether or not it has been instructed to terminate this process, and repeats the processes from step S403 to step S405 until it is instructed to terminate. If it is instructed to terminate, it terminates this process.

[0033] Through the above process, the MR ad display requester can obtain display conditions for the MR ad that take into account the impact on the visibility of the real-world ad.

[0034] Although the explanation described the use of the information processing device 100 by an MR advertisement display requester who wants to obtain the display conditions for MR advertisements, it can also be used by advertising agencies, etc., that contract with MR advertisement display requesters using the obtained display conditions for MR advertisements and receive payment.

[0035] Note that the processes of multiple steps included in the flowchart in Figure 4 and the flowcharts in other figures described later may be performed as a single step, or the processes of each step may be further subdivided into multiple steps.

[0036] Figure 5 shows an example of real-world ad information 500. Real-world ad information 500 includes ad ID 501, display position and orientation 502, display size 503, ad requester 504, ad rights holder 505, display period 506, and MR ad permission conditions 507 for each real-world ad.

[0037] The display position is expressed in terms of latitude, longitude, and altitude, indicating where the actual advertisement will be displayed. The display orientation is expressed in terms of direction. The display size 503 represents the size (shape) of the displayed actual advertisement, expressed as a rectangular prism with dimensions of width a × height b × depth c. If it cannot be represented as a rectangular prism, a rectangular prism enclosing the actual advertisement is defined, and the size of that rectangular prism is calculated.

[0038] Display rights holder 505 is a person who owns the right to display an actual advertisement at the above-mentioned display location, for example, a business that owns an advertising sign installed on the roof of a building and solicits advertisements to be displayed on said sign. Display requester 504 is a person (for example, an advertiser) who requests (demands) the display of an actual advertisement from display rights holder 505.

[0039] Display period 506 refers to the period or date (start date, etc.) during which the actual advertisement is displayed.

[0040] MR advertising permission condition 507 is a condition for allowing the display requester 504 to display an MR advertisement in the same position as the actual advertisement being displayed, superimposed on the actual advertisement. Specifically, it includes conditions regarding fees, such as permitting the superimposition of the MR advertisement on the actual advertisement if a usage fee is paid to both or one of the display requester 504 and the display rights holder 505. In this case, the amount of the fee and the percentage of payment to the display requester 504 and the display rights holder 505 may also be included in the conditions. If the display requester 504 and the display rights holder 505 are the same person and do not permit the superimposition of the MR advertisement on the actual advertisement regardless of the conditions, MR advertising permission condition 507 may be denied. Furthermore, MR advertising permission condition 507 may also include conditions regarding display time, such as permitting superimposition within a certain period of time each day, and conditions regarding visibility range, such as permitting the display of the MR advertisement only in a position and size that does not significantly affect the viewer's perception of the actual advertisement.

[0041] Figure 6 shows various examples of real spaces in this embodiment. Real space 601 is the space above building 106, real space 602 is the space above building 107, real space 603 is the space above building 108, and real space 604 is the space above building 109. Real space 605 is the space in front of building 106, and real space 606 is the space in front of building 108. Real spaces 607 and 608 are the sidewalk spaces facing the road.

[0042] Figure 7 shows an example of real-space information 700 acquired by the real-space information acquisition unit 302 in this embodiment. The real-space information 700 includes spatial identification information (hereinafter referred to as spatial ID) 701, spatial name 702, owner information 703, attribute information 704, and viewability information 705. The real-space information 700 is specified using spatial ID 701, which is identification information that divides the real space and does not overlap in all divided spaces. A predefined and database-stored spatial ID 701 is used.

[0043] Owner information 703 indicates the owner of the land or building and the owner of the physical space if the ownership of the land or building is clear. If the land and the owner of the physical space are different, the information indicates the owner of the physical space, not the owner of the land. If the physical space is not owned by an individual, such as a public road, but is owned by a local government that has jurisdiction over the physical space, owner information 703 indicates the local government.

[0044] For example, the owner of the real-world space 601 shown in Figure 6 is the owner of building 106. Real-world space 603 is the space above building 108, but its owner is not the owner of building 108, but the person who requested the display of the real-world advertisement 105. Real-world space 607, which is the space on the sidewalk, is a space managed by municipality A. Therefore, the owner of real-world space 607 is municipality A.

[0045] Attribute information 704 for real space indicates "air" for real space above a building, and indicates "rooftop" or "sidewalk" for areas accessible to people, such as rooftops or sidewalks. Additionally, if there is a real advertisement in that real space, attribute information 704 indicates "real advertisement".

[0046] Viewability information 705 indicates whether the viewer can view the video through the HMD from that real space (i.e., whether the viewer can enter that real space).

[0047] Figure 8 shows an example of an MR advertisement request 800 entered into the MR advertisement request input unit 303 by the MR advertisement display requester in this embodiment. The MR advertisement request 800 includes the desired space 801 from which the display conditions for the MR advertisement should be obtained, the desired period 802 for which the MR advertisement should be displayed, the desired display method 803 for the MR advertisement, the amount of money that can be paid 804, and the priority order 805 for these. In the example in Figure 8, the highest priority is to display the MR advertisement using the desired space with space IDs 0010 to 0100, and the second highest priority is a display method that keeps the MR advertisement constantly displayed without moving it from the desired space within the desired display period. The amount of money that can be paid is entered as an even lower priority.

[0048] The flowchart in Figure 9 shows the details of the impact determination process in step S404 of Figure 4. In step S900, the MR advertisement determination unit 304 determines the position and size of the real space 601 in Figure 6, which is the desired space where the MR advertisement is to be displayed. Specifically, the MR advertisement determination unit 304 calculates the position and size of the real space 601 from the position information of the spatial ID 701 of the real space 601 and the number of said spatial IDs included in the real space information 700 acquired by the real space information acquisition unit 302. In this case, for example, if the shape of the real space 601 is represented as a rectangular prism, its size is calculated by calculating the length, width, and height of the rectangular prism. If the shape of the real space 601 is represented as a sphere or ellipsoid, the radius of the sphere and the lengths of the major and minor axes of the ellipsoid are calculated. Furthermore, if the shape of the real space 601 is a triangular pyramid or other special shape, its size may be calculated by approximating it as an ellipsoid.

[0049] Next, in step S901, the MR advertisement determination unit 304 determines the range in which a viewer can see the MR advertisement when it is displayed in the desired space (hereinafter referred to as the MR advertisement viewing range). The MR advertisement viewing range can be determined by calculating the distance at which a viewer can see the advertisement on the HMD screen relative to the position and size of the real space 601 determined in step S900 (hereinafter referred to as the viewing distance). The viewing distance for a person as a viewer is the distance at which an object can be drawn on the HMD screen at a size that a person can see. Specifically, first, the size of an object that a viewer can see on the HMD screen is defined. Next, the distance at which an object is drawn at that size is calculated as the viewing distance.

[0050] In other words, the viewing distance D can be calculated using the formula D = S × f / s, given the focal length f of the HMD, the size s of the object on the HMD screen, and the actual size S of the object. The MR advertisement viewing range is defined as the area inside a cylinder with a radius equal to the viewing distance D, calculated from the center point of the desired space. Alternatively, the MR advertisement viewing range may be defined as the area inside a sphere with a radius equal to the viewing distance D.

[0051] Next, in step S902, the MR ad determination unit 304 identifies one or more real ads (hereinafter referred to as "viewable real ads") that are located within the MR ad viewing range identified in step S901, based on the real ad information acquired by the real ad information acquisition unit 301. The viewable real ads are identified as follows.

[0052] First, the MR ad determination unit 304 identifies one or more spatial IDs that are included within the MR ad viewing range from the real-world spatial information. Next, it extracts the spatial IDs from the identified spatial IDs whose attribute information is "real-world ad". For example, if the MR ad viewing range is determined to include the area on the opposite side of the intersection in the real-world space 601 where viewer 101 is standing, as shown in Figure 1, then real-world ads 104 and 105 are identified as viewable real-world ads from the real-world ad information.

[0053] Next, in step S903, the MR ad determination unit 304 determines whether displaying the MR ad 103 in the real space 601 shown in Figure 6 affects (changes in) the visibility of the viewable real ads 104 and 105 by viewers who view it within the MR ad viewing range. Whether or not there is an effect on the visibility of the viewable real ads can be determined by determining whether or not the MR ad is superimposed in front of the viewable real ads. In other words, it can be determined by whether or not the viewer's line of sight when viewing the viewable real ads passes through the real space 601 where the MR ad is displayed. The viewer's line of sight information and the determination result of whether or not the line of sight passes through the real space 601 where the MR ad is displayed (whether or not the display of the MR ad affects the visibility of the real ads) correspond to information about the effect.

[0054] Specifically, the viewer's position within the MR advertisement viewing range (hereinafter referred to as the viewing position) is first identified by extracting a spatial ID for which the viewing availability information included in the real-world spatial information is "available". The viewing position is, for example, the center of the bottom surface of the real-world space corresponding to the extracted spatial ID. The viewer's line of sight when viewing the real-world advertisement 104 from the viewing position can be calculated from the coordinates indicating the viewing position and the position of the real-world advertisement 104.

[0055] The coordinates of the real advertisement 104 are calculated from its position, orientation, and size. Specifically, the center position of the real advertisement 104 is determined from the coordinates indicating its position, and the display range in which the real advertisement 104 is displayed is identified from its orientation and size relative to that center position. Then, the spatial ID of one or more spatial regions within this display range is determined, and the center position of each spatial region (hereinafter referred to as the real advertisement region) is taken as the coordinates of the real advertisement 104.

[0056] The MR ad determination unit 304 determines the gaze of the viewer looking at the real-world ad 104 for all combinations of viewing positions and the coordinates of the real-world ad (one or more real-world ad areas) 104. It then determines whether the distance between the coordinates of the center position of each real-world ad area and the gaze is less than or equal to half the size of that real-world ad area, and if so, it determines that the gaze has passed through that real-world ad area. By determining that the gaze has passed through at least one real-world ad area in this way, it determines that the gaze has passed through the real-world space 601. The MR ad determination unit 304 then determines that there is an impact on the visibility of the real-world ad 104 for viewing positions where the gaze has passed through the real-world space 601. The MR ad determination unit 304 performs the above determination for all viewing positions.

[0057] If the MR ad determination unit 304 determines in step S903 that there is an impact on the visibility of the real ad, it performs the process in step S904. If it determines that there is no impact on visibility, it performs the process in step S903 for other visible real ads.

[0058] In step S904, the MR advertisement determination unit 304 creates an impact determination result that associates the determination result from step S903 with the MR advertisement permission conditions and the MR advertisement request.

[0059] Next, in step S905, the MR ad determination unit 304 determines whether or not the processes in steps S903 and S904 have been performed on all of the visible real ads identified in step S902 (for example, real ads 104 and 105). If it has been performed, the process ends.

[0060] The flowchart in Figure 10 shows the details of the display condition determination process in step S405 of Figure 4. In step S1000, the MR advertisement determination unit 305 creates candidate display conditions for the MR advertisement. The display conditions for the MR advertisement are determined based on the display range of the MR advertisement, the viewing range where the viewer is located, and the contractual conditions between the MR advertisement display requester, the real advertisement display requester, and the owner of the real space. Specifically, eight candidate conditions are created, consisting of combinations of whether or not there are restrictions on the display range, whether or not there are restrictions on the viewing range, and whether or not there are contractual conditions.

[0061] Option 1 has no contractual conditions and fulfills all MR advertising requirements. Option 2 has contractual conditions and fulfills all MR advertising requirements. Option 3 has no contractual conditions and fulfills the requirements to display MR advertisements in the desired space (display range) included in the MR advertising requirements, limited to a viewing range that does not affect the visibility of real advertisements. Option 4 has contractual conditions and fulfills the requirements to display MR advertisements in the desired space included in the MR advertising requirements, limited to a viewing range that does not affect the visibility of real advertisements. Option 5 has no contractual conditions and fulfills the requirements to display MR advertisements without restricting the viewing range, limited to a display range that does not affect the visibility of real advertisements. Option 6 has contractual conditions and fulfills the requirements to display MR advertisements without restricting the viewing range, limited to a display range that does not affect the visibility of real advertisements. Option 7 has no contractual conditions and fulfills the requirements to display MR advertisements limited to a display range and viewing range that do not affect the visibility of real advertisements. Option 8 is a display condition that, after setting contractual conditions, displays MR advertisements with restrictions on the display and viewing ranges so as not to affect the visibility of real advertisements.

[0062] In step S1001, the MR advertisement decision unit 305 calculates a score for each of the candidate display conditions created in step 1000. The score is calculated according to a calculation method based on the MR advertisement permission conditions and MR advertisement requests associated with the impact assessment results created in step S904.

[0063] This section explains the calculation method using the deduction method. First, standard scores are set for the display range, viewing range, and contract conditions of the MR advertisement. Then, points are deducted from the standard scores if the MR advertisement violates the conditions for permitting the MR advertisement compared to the actual advertisement. In addition, the deduction amount increases in order of priority for items that violate the MR advertisement request.

[0064] For example, the standard score for display range, viewing range, and contract terms is set at 10 points, with a penalty of -6 points for violating the MR advertising permission conditions, and the penalty for items violating the MR advertising request is -5 points, -3 points, -2 points, and -1 point in order of priority. For example, candidate 1 might score 13 points, candidate 2 18 points, candidate 3 16 points, candidate 4 21 points, candidate 5 14 points, candidate 6 19 points, candidate 7 17 points, and candidate 8 22 points. Note that the method of calculating the score for each candidate is not limited to this example.

[0065] Next, in step S1002, the MR advertisement determination unit 305 determines the display conditions for the MR advertisement to be adopted using the score calculated in step S1001. Specifically, the candidate with the highest score is determined as the display conditions for the MR advertisement. In the example score above, candidate 8 is determined as the display conditions for the MR advertisement. That is, the MR advertisement can be superimposed on the real advertisement 104 by paying a usage fee to both the requester and the rights holder of the real advertisement 104. The amount of the usage fee to be paid to both the requester and the rights holder of the real advertisement 104 is the same for both, in accordance with the MR advertisement permission conditions. Furthermore, the MR advertisement can be displayed in the real space 602 when viewed from a viewing range that does not affect the viewer's perception of the real advertisement 105. In addition, if there is a possibility that the viewer's perception of the real advertisement 105 may be affected, the MR advertisement can be displayed only in a viewing range within the real space 602 that does not affect the viewer's perception of the real advertisement 105.

[0066] According to the embodiment described above, the client requesting the display of the MR advertisement can obtain display conditions for the MR advertisement that take into account the impact of the MR advertisement's display on the visibility of the real advertisement.

[0067] [Variation 1-1] In step 1000 of Figure 10, the MR advertisement decision unit 305 may create fewer or more candidates for the display conditions of the MR advertisement, rather than creating eight candidates. For example, a candidate display condition may be one that aims for a synergistic effect of the advertisement by allowing viewers to see both the real advertisement and the MR advertisement. Alternatively, a candidate display condition may be one in which the MR advertisement moving within the desired space is superimposed on the real advertisement intermittently, not continuously, but at regular intervals. In this case, the score of the candidate may be determined from the proportion of time the superimposed display takes place. Furthermore, a generative AI may be used to create the candidates. The generative AI may be used to create the display conditions for the MR advertisement, and the client requesting the display of the MR advertisement may be allowed to choose whether or not to adopt them.

[0068] Alternatively, instead of selecting the candidate with the highest score from multiple candidates as the display condition for the MR advertisement, only candidates that match the MR advertisement permission conditions may be selected. In this case, if it is not possible to narrow down the candidates to one, the person requesting the display of the MR advertisement may be allowed to make the selection.

[0069] The method for assigning points to each candidate may also be to calculate points using an additive or subtractive method, rather than deducting points for violations of the MR advertising request and approval conditions. In an additive method, points may be added for matches with the MR advertising request and approval conditions, or the points added may vary according to the priority of the MR advertising request.

[0070] Alternatively, scores may be calculated by assigning weights to each item of the MR advertising request according to its priority. Alternatively, scores may be determined based on the proportion of viewing areas that influence the visibility of the real-world advertisement. Since the impact on the effectiveness of the real-world advertisement is greater when viewing locations where viewers tend to gather are significant, the degree of influence may be calculated by assigning weights to viewing locations for each type of real-world advertisement effectiveness and reflecting this in the score.

[0071] In Example 1, the display conditions for the MR ad were determined without considering the impact that the MR ad would have on the actual ad's content or its advertiser or other providers. In contrast, the display conditions for the MR ad may be determined based on the results of a determination of the impact that the MR ad would have on the actual ad's content or advertiser. For example, if it is determined that the display of the MR ad would have an adverse effect on the content or advertiser of the actual ad, the display conditions for the MR ad may be restricted to eliminate or minimize such adverse effects. Conversely, if it is determined that the display of the MR ad would have a positive effect on the content or advertiser of the actual ad due to synergistic effects between the two ads, the display conditions for the MR ad may be determined to further enhance such positive effects. In these cases, it is preferable to include information on the content and advertiser of the MR ad in the MR ad request. Furthermore, when considering the impact on the content and advertiser of the actual ad, the display conditions for the MR ad include not only the display conditions when the MR ad is superimposed on the actual ad, but also the display conditions when the MR ad is displayed in a real space close to the actual ad.

[0072] Furthermore, without performing any processing to determine the impact on the visibility of real-world advertisements (and the impact on the content and providers mentioned above) as in Example 1, the display conditions for MR advertisements may be determined based on real-world spatial information, real-world advertisement information, and MR advertisement requests. In this case, information regarding the impact may be obtained from the relationship between real-world spatial information, real-world advertisement information, and MR advertisement requests, and the display conditions for MR advertisements may be determined based on this information. For example, the attribute information of the real-world space where the display of MR advertisements is desired and the real-world advertisement information displayed there may be used as information regarding the impact, and the display conditions for MR advertisements may be determined based on this.

[0073] Furthermore, the information processing device 100 of Example 1 can also be used by MR advertisement display requesters who only want to obtain the display conditions for MR advertisements, without actually displaying the MR advertisements. In this case, the determined MR advertisement display conditions may include information that limits who can utilize those display conditions.

[0074] Furthermore, while Example 1 described a case where a requester of an MR advertisement prioritizes items such as the desired space, desired period, desired display method, and payable amount for displaying the MR advertisement, items that are not of high priority do not need to be prioritized. For items that do not need to be prioritized, the MR advertisement decision unit 305 may determine the priority.

[0075] [Variation 1-2] The real-world advertisement information acquisition unit 301 may acquire real-world advertisement information that has been previously registered in a database by the person requesting the display of the real-world advertisement, or it may acquire real-world advertisement information by image recognition processing from an image obtained by capturing the currently displayed real-world advertisement with a camera mounted on the HMD. Alternatively, real-world advertisement information may be acquired by image recognition processing from an image acquired by another imaging device such as a surveillance camera. In this case, the position, orientation, size, and information of the person requesting the display of the real-world advertisement can be acquired.

[0076] The real-world advertisement information acquisition unit 301 may also acquire real-world advertisement information after restricting the location of real-world advertisements for which real-world advertisement information can be acquired.

[0077] In Example 1, latitude, longitude, and altitude were used as the location of the real-world advertisement, but spatial IDs or addresses may also be used as the location of the real-world advertisement, similar to real-world spatial information. Orientation may also be added to the location of the real-world advertisement. The orientation of the real-world advertisement is a factor that determines whether or not a viewer can see the real-world advertisement from their viewing position. Since viewing positions include not only the ground but also various locations such as building rooftops, the orientation of the real-world advertisement can be used to determine whether or not the real-world advertisement can be seen.

[0078] [Modification 1-3] The MR advertisement determination unit 304 may define the viewing range in which the MR advertisement 103 can be viewed as a range other than the range inside the cylinder with the viewing distance as described above. Information on viewing positions within the viewing range where people are likely to look around, such as rooftops of buildings or windows of buildings, may be collected in advance, and the unit may determine whether the display of the MR advertisement affects the visibility of the real advertisement for each viewing position within the viewing range. Alternatively, if even one viewing position within the viewing range has an impact on the visibility of the real advertisement, the unit may determine that the display of the MR advertisement affects the visibility of the real advertisement for that viewing range.

[0079] Furthermore, the MR advertisement determination unit 304 does not have to determine whether the display of the MR advertisement affects the visibility of the real advertisement simply by whether or not the viewer's gaze passes through the space where the MR advertisement is displayed. For example, if the angle of the gaze passing through the space where the MR advertisement is displayed is very large, it may be difficult to see the real advertisement as well. In such a case, a field of view angle in which the real advertisement can be seen can be defined relative to the orientation of the real advertisement, and the determination may be made based on whether or not the gaze within that field of view passes through the space where the MR advertisement is displayed.

[0080] Furthermore, the MR advertisement determination unit 304 may also exclude viewing positions that are too close to the real advertisement when determining the viewing range for the real advertisement. For example, as shown in Figure 6, if a viewer tries to view a real advertisement installed on the roof of a building, such as the real advertisement 105, from a position directly in front of the building 108, the building 108 will obstruct their line of sight, and they will not be able to see the real advertisement 105. In this way, viewing positions where the angle of the line of sight becomes very large in the height direction may be excluded from the viewing range. In this case, the radius of the range of viewing positions to be excluded may be determined, and a viewing range excluding the central portion having that radius may be obtained.

[0081] Furthermore, the MR ad determination unit 304 may determine the line of sight in a manner other than determining the coordinates of the center of the real-world ad area identified by the spatial ID. For example, if the real-world ad has a polygon, the line of sight may be determined to the coordinates of each vertex of the polygon, or a bounding box large enough to cover the real-world ad may be defined, and the line of sight may be determined to the coordinates of the vertices of the bounding box.

[0082] [Modifications 1-4] The MR advertisement determination unit 304 may acquire real-world spatial information by means other than using spatial IDs. For example, it may define real-world spaces as spaces with specific sizes, such as the real-world spaces 601 to 607 shown in Figure 6. In this case, the vertex coordinates, reference coordinates, and dimensions (length, width, and height) of each space may be acquired as real-world spatial information.

[0083] [Variations 1-5] The MR advertisement decision unit 305 may determine contractual conditions other than paying the same usage fee to the requester and the rights holder of the real advertisement 104 as one of the display conditions for the MR advertisement. For example, it may decide to pay a usage fee not only to the requester and the rights holder of the real advertisement, but also to the owner of the real space where the real advertisement will be displayed. In this case, the usage fee may be determined according to the wishes of the requester and the rights holder of the real advertisement, the owner of the real space, and the requester of the MR advertisement. [Examples]

[0084] Next, we will describe Example 2. Example 1 described the case where display conditions for MR advertisements are determined considering the impact on the visibility of real-world advertisements. In contrast, Example 2 describes the case where the MR advertisement has been created in advance as virtual space content (hereinafter referred to as MR content), and the MR advertisement display requester, who is specifically aware of the content of the MR advertisement, uses an information processing device.

[0085] Figure 11 shows the system configuration including the information processing device 100' of Example 2. Components of this example that are the same as those of Example 1 are denoted by the same reference numerals as in Example 1. The information processing device 100' is the information processing device 100 of Example 1 with the addition of an MR content acquisition unit 1101 and an MR content generation unit 1102.

[0086] The MR content acquisition unit 1101 is connected to the MR content management unit 31, which is provided separately from the information processing device 100' and manages MR content, and acquires MR advertisements as MR content from the MR content management unit 31. The MR content generation unit 1102, acting as a generation means, generates MR content to be displayed on the HMD by modifying the MR advertisement acquired by the MR content acquisition unit 1101 according to the display conditions of the MR advertisement determined by the MR advertisement determination unit 305.

[0087] The flowchart in Figure 12 shows the processing (information processing method) that the information processing device 100' executes according to the processing program. In this flowchart, the same reference numerals are used for the same steps as in the flowchart of Example 1 (Figure 4).

[0088] In step S1201, after the processing from step S400 to step S402 has been completed, the MR content acquisition unit 1101 acquires an MR advertisement from the MR content management unit 31 that determines the display conditions from among a plurality of pre-created MR contents (MR advertisements). At this time, the MR content acquisition unit 1101 also acquires the size (display range), display method, and information of the display requester (third information) of the MR advertisement from the MR content management unit 31. The display range is acquired by the number of spatial IDs (number in the vertical, horizontal, and height directions). The display method is the same as in Embodiment 1, such as always displaying or moving the space on which the MR advertisement is displayed.

[0089] After steps S1201 and S403 through S405 are processed, step S1202 is performed. In step S1202, the MR content generation unit 1102 changes the display range, display period, and display method of the MR advertisement acquired by the MR content acquisition unit 1101 according to the display conditions of the MR advertisement determined by the MR advertisement determination unit 305. This generates the MR content (MR advertisement) to be presented to the MR advertisement display requester. Then this process ends.

[0090] According to this embodiment, when an MR advertisement has been created in advance, it is possible to obtain display conditions for the MR advertisement that take into account the impact on the visibility of the real advertisement, and furthermore, to obtain the MR advertisement in accordance with said display conditions.

[0091] [Modification 2-1] The desired space included in the MR advertisement request may not be limited to one real space, but may be multiple real spaces, and the display conditions for the MR advertisement may be determined for each real space. In this case, the impact assessment in step S404 and the determination of the MR advertisement display conditions in step S405 are performed for multiple real spaces. This allows the MR advertisement requester to select a more appropriate real space and display conditions for displaying the MR advertisement.

[0092] [Modification 2-2] The MR content generation unit 1102 may generate an MR advertisement to be presented to an MR advertisement requester by changing the display range, display period, and display method of the MR advertisement acquired by the MR content acquisition unit 1101 using a generation AI.

[0093] Furthermore, the MR content generation unit 1102 may generate new MR advertisements to be presented to MR advertisement display requesters without using the MR advertisements acquired by the MR content acquisition unit 1101.

[0094] Furthermore, depending on the MR advertisement display conditions determined by the MR advertisement determination unit 305, the MR content generation unit 1102 may present the MR advertisement acquired by the MR content acquisition unit 1101 to the MR advertisement display requester without making any changes.

[0095] In the above embodiments, we have described cases where the actual display information and virtual display information are advertisements. However, the actual display information and virtual display information may also be information other than advertisements, such as guides, news, and video works.

[0096] The above embodiments include the following configurations.

[0097] (Composition 1) A first acquisition means for acquiring first information about the real space, A second acquisition means for acquiring second information relating to the reality display information displayed in the aforementioned real space, A third acquisition means for acquiring third information relating to the request for displaying virtual display information in the aforementioned real space, An information processing apparatus characterized by having determination means for determining the display conditions of the virtual display information based on the first information, the second information, and the third information. (Configuration 2) The information processing device according to configuration 1, characterized in that the virtual display information is different from the actual display information. (Composition 3) The information processing apparatus according to configuration 1 or 2, characterized in that the determination means acquires information regarding the impact of the display of the virtual display information on the actual display information based on the first information, the second information, and the third information, and determines the display conditions based on the information regarding the impact. (Composition 4) The information processing device according to configuration 3, characterized in that the aforementioned effect is an effect on the viewer's perception of the reality display information. (Composition 5) The information processing apparatus according to configuration 3, characterized in that the aforementioned effect is an effect on the content of the actual display information. (Composition 6) The information processing apparatus according to configuration 3, characterized in that the aforementioned effect is an effect on the provider of the reality display information. (Composition 7) An information processing device according to any one of configurations 1 to 6, characterized in that the actual display information and the virtual display information include advertisements. (Composition 8) The information processing device according to any one of configurations 1 to 7, characterized in that the first information includes at least one of the following: identification information of the real space, a space name, an owner, and attributes, and information indicating whether or not viewing by a viewer is possible in the real space. (Composition 9) The information processing device according to any one of configurations 1 to 8, characterized in that the second information includes at least one of the following: the position, orientation, and size in which the actual display information is displayed; the person who requested the display of the actual display information; the owner of the right to display the actual display information; the display period of the actual display information; and the permission conditions for displaying the virtual display information in the same position as the actual display information. (Composition 10) The information processing device according to any one of configurations 1 to 9, characterized in that the third information includes at least one piece of information from either the real space and the display period for which the virtual display information is requested to be displayed, the display method for which the virtual display information is requested, and the amount payable to request the display of the virtual display information. (Composition 11) The information processing device according to any one of configurations 1 to 10, characterized in that the display conditions include at least one of the range of the real space in which the virtual display information is displayed and the display period, the position of the viewer who views the virtual display information, and the payment conditions for the display of the virtual display information. (Composition 12) An information processing apparatus according to any one of configurations 1 to 11, further comprising a generation means for generating the virtual display information according to the aforementioned display conditions.

[0098] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0099] The embodiments described above are merely representative examples, and various modifications and changes can be made to each embodiment when implementing the present invention. [Explanation of Symbols]

[0100] 100,100′ Information Processing Device 101 viewers 102 HMD screens 103 MR Advertisement 104,105 Reality Ads 301 Real-world advertising information acquisition department 302 Real-world spatial information acquisition unit 303 MR Advertisement Request Input Section 304 MR Advertising Judgment Department 305 MR Advertising Decision-Making Department

Claims

1. A first acquisition means for acquiring first information about the real space, A second acquisition means for acquiring second information relating to the reality display information displayed in the aforementioned real space, A third acquisition means for acquiring third information relating to the request for displaying virtual display information in the aforementioned real space, An information processing apparatus characterized by having determination means for determining the display conditions of the virtual display information based on the first information, the second information, and the third information.

2. The information processing apparatus according to claim 1, characterized in that the virtual display information is different from the actual display information.

3. The information processing apparatus according to claim 1, characterized in that the determination means acquires information regarding the influence of the display of the virtual display information on the actual display information based on the first information, the second information, and the third information, and determines the display conditions based on the information regarding the influence.

4. The information processing apparatus according to claim 3, characterized in that the aforementioned effect is an effect on the viewer's perception of the reality display information.

5. The information processing apparatus according to claim 3, characterized in that the aforementioned effect is an effect on the content of the actual display information.

6. The information processing apparatus according to claim 3, characterized in that the aforementioned effect is an effect on the provider of the reality display information.

7. The information processing apparatus according to claim 1, characterized in that the actual display information and the virtual display information include advertisements.

8. The information processing device according to claim 1, characterized in that the first information includes at least one of the following: identification information of the real space, a space name, an owner, and attributes, and information indicating whether or not viewing by a viewer is possible in the real space.

9. The information processing device according to claim 1, wherein the second information includes at least one of the following: the position, orientation, and size in which the actual display information is displayed; the person who requested the display of the actual display information; the holder of the right to display the actual display information; the display period of the actual display information; and the permission conditions for displaying the virtual display information in the same position as the actual display information.

10. The information processing apparatus according to claim 1, wherein the third information includes at least one of the following: a real space and a display period for which the display of the virtual display information is requested; a display method for which the display of the virtual display information is requested; and an amount of money payable to request the display of the virtual display information.

11. The information processing apparatus according to claim 1, characterized in that the display conditions include at least one of the range of the real space in which the virtual display information is displayed and the display period, the position of the viewer who views the virtual display information, and the payment conditions for the display of the virtual display information.

12. The information processing apparatus according to claim 1, further comprising a generation means for generating the virtual display information in accordance with the aforementioned display conditions.

13. The process of acquiring first information about the real space, A step of acquiring second information relating to the real-world display information shown in the aforementioned real-world space, A step of acquiring third information relating to the request for displaying virtual display information in the aforementioned real space, An information processing method characterized by comprising the step of determining the display conditions of the virtual display information based on the first information, the second information, and the third information.

14. A program characterized by causing a computer to execute processing according to the information processing device described in claim 13.

Images