Information processing device, program, and information processing method

Abstract

To provide an information processing device, a program, and an information processing method that send an AR (Augmented Reality) content to a user terminal according to the situation of the user terminal and make the user terminal display it.SOLUTION: In a system 10 including a management device and a user terminal, the management device 100 comprises a user situation acquisition unit that acquires the situation of the user terminal that includes location information indicating the location of the user terminal 200 that is a spectacle type device, an application start unit that starts up multiple applications corresponding to the situation of the user terminal acquired by the user situation acquisition unit, an application selection unit that selects at least one application from the multiple applications based on information from the user terminal, and an information transmission unit that transmits an AR content for AR streaming generated by the application selected by the application selection unit to the user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a program, and an information processing method. [Background technology]

[0002] Patent Document 1 describes AR (Augmented Reality), a technology that reflects (extends) virtual reality in the real world, MR (Mixed Reality), a technology that allows you to experience a virtual world by overlaying it on the real world, and XR (X Reality (Cross Reality)), a general term for these technologies. [Prior art document] [Patent documents] [Patent Document 1] JP 2022-105942 A Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment of the present invention, there is provided an information processing device. The information processing device may include a content acquisition unit that acquires content corresponding to a status of a user terminal. The information processing device may include a display control unit that displays the content on a display unit of the user terminal. The information processing device may include a reaction acquisition unit that acquires reaction information indicating a reaction of a user of the user terminal to the content displayed on the display unit. The information processing device may include a selection unit that selects content to be provided to a user from a plurality of contents based on a learning result using a plurality of learning data including content information of the content, attribute information and environmental information of the user, and the reaction information.

[0004] The information processing device may further include a model generation unit that performs learning using the plurality of learning data to generate a learning model that receives the content information, the attribute information, and the environmental information as input and outputs the reaction information, and the selection unit may input content information of the plurality of contents, the user's attribute information, and environmental information into the learning model and select content to be provided to the user from the plurality of contents based on the reaction information output from the learning model, and the display control unit may display the content selected by the selection unit on the display unit of the user terminal.

[0005] In any of the information processing devices, the reaction acquisition unit may acquire the reaction information generated by at least one of head tracking, eye tracking, and hand tracking of the user by the user terminal. The reaction acquisition unit may acquire the reaction information indicating whether the direction of the user's face is facing the content. The reaction acquisition unit may acquire the reaction information indicating whether the user's line of sight is directed toward the content. The reaction acquisition unit may acquire the reaction information indicating whether the user's hand has moved toward the content.

[0006] In any of the information processing devices, the content information may include the type of the content, the attribute information may include the address, occupation, and age of the user, and the environmental information may include situation information when the user viewed the content and spatial information of the space where the user viewed the content. The situation information may include time information, information about the user's surroundings, and information about the user's behavior, and the spatial information may include the size of the space, the brightness of the space, and information about the location where the user viewed the content.

[0007] Any of the information processing devices may further include an application launch unit that launches multiple applications corresponding to the status of the user terminal, and an application selection unit that selects at least one application from the multiple applications based on information from the user terminal, and the content acquisition unit may acquire the content generated by the application selected by the application selection unit.

[0008] According to one embodiment of the present invention, there is provided a program for causing a computer to function as the information processing device.

[0009] According to one embodiment of the present invention, there is provided an information processing device. The information processing device may include an application launch unit that launches a plurality of applications corresponding to a status of a user terminal. The information processing device may include an application selection unit that selects at least one application from the plurality of applications based on information from the user terminal. The information processing device may include an information transmission unit that transmits information generated by the application selected by the application selection unit to the user terminal.

[0010] In the information processing device, the application selection unit may select at least one application from the plurality of applications in accordance with selection information for selecting one of the plurality of applications received from the user terminal.

[0011] In any of the information processing devices, the application selection unit may select at least one application from the plurality of applications based on user-related information related to the user received from the user terminal.

[0012] In any of the information processing devices, the application launching unit may launch a plurality of applications selected from a plurality of applications corresponding to a status of the user terminal. The application launching unit may launch the plurality of applications selected from the plurality of applications corresponding to the status of the user terminal in descending order of the probability of use by the user of the user terminal. The application launching unit may launch the plurality of applications selected from the plurality of applications corresponding to the status of the user terminal in descending order of the probability of use within a range of an allocated resource amount allocated to the user terminal. The application launching unit may launch the plurality of applications selected from the plurality of applications corresponding to the status of the user terminal in a manner that launches as many applications as possible within a range of an allocated resource amount allocated to the user terminal.

[0013] In any of the information processing devices, the application launch unit may launch a corresponding VM for an application among the plurality of applications corresponding to the status of the user terminal, the application having a usage probability by the user of the user terminal higher than a first threshold, and launch the application on the VM, and if the usage probability is lower than the first threshold and higher than a second threshold lower than the first threshold, launch the VM corresponding to the application.

[0014] According to one embodiment of the present invention, there is provided a program for causing a computer to function as the information processing device.

[0015] According to one embodiment of the present invention, there is provided an information processing method executed by a computer. The information processing method may include a content acquisition step of acquiring content corresponding to a status of a user terminal. The information processing method may include a display control step of displaying the content on a display unit of the user terminal. The information processing method may include a reaction acquisition step of acquiring reaction information indicating a reaction of a user of the user terminal to the content displayed on the display unit. The information processing method may include a selection step of selecting content to be provided to a user from a plurality of contents based on a learning result using a plurality of learning data including content information of the content, attribute information and environmental information of the user, and the reaction information.

[0016] According to one embodiment of the present invention, there is provided an information processing method executed by a computer. The information processing method may include an application launch step of launching a plurality of applications corresponding to a status of a user terminal. The information processing method may include an application selection step of selecting at least one application from the plurality of applications based on information from the user terminal. The information processing method may include an information transmission step of transmitting information generated by the application selected in the application selection step to the user terminal.

[0017] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0018] [Figure 1] An example of a system 10 is shown schematically. [Figure 2] FIG. 2 is an explanatory diagram for explaining an example of processing content in the system 10. [Figure 3] FIG. 2 is an explanatory diagram for explaining an example of processing content in the system 10. [Figure 4] 2 illustrates an example of a functional configuration of a management device 100. [Figure 5]FIG. 2 is an explanatory diagram for explaining an example of processing content in the system 10. [Figure 6] FIG. 2 is an explanatory diagram for explaining an example of processing content in the system 10. [Figure 7] 2 illustrates an example of a functional configuration of a management device 100. [Figure 8] FIG. 2 is an explanatory diagram for explaining standby control by the management device 100. [Figure 9] 1 shows an example of a hardware configuration of a computer 1200 that functions as the management device 100. DETAILED DESCRIPTION OF THE INVENTION

[0019] Compared to the general web, current AR lacks access to practical information sources, and most content is limited to closed entertainment or experiences within handmade spaces. A system that easily incorporates real-time, practical information into AR and VR spaces is needed. Furthermore, if AR and VR devices become as commonplace as smartphones and are worn at all times, information overload is likely. In particular, XR experiences, unlike smartphones, passively receive information from the entire field of view, so information selection methods must be considered. Active selection via a user interface is undesirable for XR experiences; instead, it is preferable for information to be automatically selected based on user behavior and meta information. In response to this situation, for example, AI can select and discard information collected from the web using user information, behavior, and context. Since this information is typically numerical or textual, it can be presented in the XR space using a mechanism that generates three-dimensional objects from that information. The user's reactions to the information are collected from the XR device's sensors, and the AI ​​optimizes the selection process for each individual user.

[0020] Traditional AR technology runs applications on smartphones or wearable devices, enriching our lives by adding information to the real world. However, as AR becomes more widespread, it can become overwhelmed with information and cause stress. Furthermore, wearable devices like AR glasses have low processing power, making it difficult to launch multiple apps simultaneously. While AR streaming is a technology that launches AR apps on a server, such as the cloud, and sends only the video to the device, even with AR streaming, server-side resources are depleted. To address this issue, a switcher could be provided from the cloud that can simultaneously run multiple apps and seamlessly switch between streaming videos according to user requests and circumstances. To prevent server-side resource depletion, a switcher could be provided that analyzes trends in information needs based on user circumstances using user switching logs and pre-launches only highly usable apps.

[0021] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0022] 1 is a schematic diagram of an example of a system 10. The system 10 includes a management device 100. The management device 100 may be an example of an information processing device. The system 10 includes a user terminal 200.

[0023] The management device 100 manages information to be provided to the user 20. For example, the management device 100 transmits AR content according to the state of the user terminal 200 to the user terminal 200, and causes the AR content to be displayed.

[0024] The user terminal 200 may be any terminal as long as it has a communication function and a function to display information to the user 20. For example, the user terminal 200 is an eyeglass-type device such as AR glasses. For example, the user terminal 200 is a smartphone, a tablet terminal, or the like.

[0025] The user terminal 200 displays the AR content received from the management device 100. The content managed by the management device 100 may be in any data format. For example, the content is 3D CG (Computer Graphics) data. For example, the content is a still image. For example, the content is a video. For example, the content is text. The content may include a combination of these.

[0026] If the user terminal 200 is an eyeglass-type device, it may be a device that can give the user 20 the sensation that the content is located in real space by displaying the content on transparent or semi-transparent glasses. In other words, the user terminal 200 may be a so-called transparent (see-through) device.

[0027] When the user terminal 200 is a smartphone or a tablet terminal, the user terminal 200 may be a video pass-through type device. In this case, the user terminal 200 may display a captured image on a non-transparent display and superimpose the content. When the user terminal 200 is an eyeglass-type device, the user terminal 200 may be a video pass-through type device.

[0028] 2 and 3 are explanatory diagrams for explaining an example of the processing content in the system 10. Here, the flow of processing between the user terminal 200 and the curation AI 300 included in the management device 100 will be explained.

[0029] The user terminal 200 includes a display unit 202 and a detection unit 204. The detection unit 204 includes multiple types of sensors. Examples of sensors included in the detection unit 204 include, but are not limited to, a position sensor, an image sensor, a gyro sensor, an acceleration sensor, a gaze sensor, and an audio sensor. The position sensor may include at least one of a GNSS (Global Navigation Satellite System) positioning function such as a GPS (Global Positioning System) positioning function, a network positioning function, a VPS (Visual Positioning System) positioning function, and a UWB (Ultra Wide Band) positioning function. Note that the positioning functions of the position sensor are not limited to these, and the positioning function may include only a positioning function other than these, or may include at least one of these positioning functions and another positioning function. The detection unit 204 may realize head tracking of the user 20, eye tracking of the user 20, hand tracking of the user 20, etc., using multiple types of sensors.

[0030] The user terminal 200 first transmits attribute information 316 of the user 20 and environmental information 310 including the position and orientation of the user terminal 200 detected by the detection unit 204 to the curation AI 300. The information selection function 302 of the curation AI 300 selects information corresponding to the environmental information 310 and the attribute information 316 from the information source 40. The curation AI 300 may manage the information source 40 by organizing external information using an API (Application Programming Interface) and a DB (Database).

[0031] 3D generation 324 is performed on the information 322 selected by the information selection function 302 and displayed on the display unit 202 of the user terminal 200. The 3D generation 324 may be a function that converts the selected information 322 to match the position and posture of the user 20 so that it does not feel strange to the user 20. The 3D generation 324 may be performed by the curation AI 300. The 3D generation 324 may also be performed by the user terminal 200.

[0032] The detection unit 204 detects user behavior 332 with respect to the information displayed on the display unit 202 and transmits the detected user behavior 332 to the curation AI 300. The user behavior 332 may be information indicating the reaction of the user 20 to the information displayed on the display unit 202. The detection unit 204 transmits to the curation AI 300, for example, user behavior 332 such as whether the user 20 turned toward the information, followed the information with their eyes, reacted to the information by touching it with their hand, ignored it, or manually switched to other information.

[0033] The learning and optimization function 304 of the curation AI 300 uses the information displayed on the display unit 202, the environmental information 310, the attribute information 316, and the user behavior 332 to perform learning and optimization to select information suitable for the user 20 from multiple pieces of information corresponding to the situation of the user terminal 200.

[0034] 3, the environment information 310 may include situation information 312 and space information 314. The situation information 312 includes behavioral information indicating what behavior the user 20 is performing, time information, and ambient information related to the user 20's surroundings. The space information 314 includes information on the location where the user 20 viewed the information, information on the size of the space, information on the brightness of the space, and the like. The attribute information 316 includes information such as the address, occupation, and age of the user 20. The content information 42 related to the information displayed on the display unit 202 includes the type of content.

[0035] The learning and optimization function 304 performs learning based on this information. For example, for each attribute, situation, and space condition of the user 20, if the user 20 has a positive reaction to displayed information, the learning and optimization function 304 makes it more likely that the information or information of the same type as the information is selected under those conditions, and if the user 20 has a negative reaction, makes it less likely that the information or information of the same type as the information is selected under those conditions.

[0036] The information selection function 302 selects information to be provided to the user 20 based on the learning results of the learning and optimization function 304. This makes it possible to automatically and appropriately select information suitable for the user 20.

[0037] 4 illustrates an example of the functional configuration of the management device 100. The management device 100 includes an environment information acquisition unit 102, an attribute information acquisition unit 104, a content acquisition unit 106, a display control unit 108, a reaction acquisition unit 110, a model generation unit 112, and a selection unit 114.

[0038] The environment information acquisition unit 102 acquires environment information from the user terminal 200. The environment information includes position information indicating the position of the user terminal 200. The environment information includes posture information indicating the posture of the user terminal 200. The environment information may include situation information. The environment information may include spatial information.

[0039] The situation information may include time information indicating the time period, etc. The situation information may include behavior information indicating what behavior the user 20 is engaged in. The situation information may include surrounding information regarding the surroundings of the user 20. The surrounding information may include information regarding people, objects, etc. present around the user 20.

[0040] The spatial information may include location information indicating the type of location, the size of the space, the brightness of the space, or the weather information of the location where the user 20 is located.

[0041] The attribute information acquisition unit 104 acquires attribute information of the user 20. The attribute information may include the name of the user 20. The attribute information may include the username of the user 20. The attribute information may include the address of the user 20. The attribute information may include the gender of the user 20. The attribute information may include the age of the user 20. The attribute information may include the occupation of the user 20. The attribute information may include the blood type of the user 20. The attribute information may include the nationality of the user 20.

[0042] The content acquisition unit 106 acquires content corresponding to the situation of the user terminal 200. The content acquisition unit 106 may acquire multiple pieces of content that match the situation of the user terminal 200 from the information source 40. The content acquisition unit 106 acquires content registered for the range included in the field of view of the user 20, for example, from the position and posture of the user terminal 200.

[0043] The display control unit 108 causes the content acquired by the content acquisition unit 106 to be displayed on the display unit 202 of the user terminal 200. The display control unit 108 may transmit the content acquired by the content acquisition unit 106 and a display instruction to the user terminal 200.

[0044] The reaction acquisition unit 110 acquires reaction information indicating the reaction of the user 20 to the content displayed on the display unit 202. The reaction acquisition unit 110 may receive, from the user terminal 200, reaction information generated by the detection unit 204 of the user terminal 200.

[0045] The reaction acquisition section 110 may acquire reaction information generated by at least one of head tracking, eye tracking, and hand tracking of the user 20 by the user terminal 200.

[0046] The reaction acquisition unit 110 acquires reaction information indicating, for example, whether the user 20's face is facing directly toward the content. Facing the user 20's face directly toward the content may be an example of a positive reaction. The reaction acquisition unit 110 acquires reaction information indicating, for example, whether the user 20's face is turned away from the content. Facing the user 20's face away from the content may be an example of a negative reaction.

[0047] The reaction acquisition unit 110 acquires, for example, reaction information indicating whether or not the gaze of the user 20 is directed toward the content. Directing the gaze of the user 20 toward the content may be an example of a positive reaction. The reaction acquisition unit 110 may acquire reaction information indicating whether or not the gaze of the user 20 has followed the content. Following the gaze of the user 20 toward the content may be an example of a positive reaction. The reaction acquisition unit 110 acquires, for example, reaction information indicating whether or not the gaze of the user 20 has been turned away from the content. Turning the gaze of the user 20 away from the content may be an example of a negative reaction.

[0048] The reaction acquisition unit 110 acquires reaction information indicating, for example, whether or not the user 20's hands have moved in response to content. The reaction acquisition unit 110 acquires reaction information indicating, for example, whether or not the user 20's hands have operated the content. The user 20's hand moving in response to content may be an example of a positive reaction. The user 20's not moving in response to content may be considered neither a positive reaction nor a negative reaction, or may be treated as a negative reaction.

[0049] The model generation unit 112 generates a learning model that receives the content information, attribute information, and environmental information as input and outputs the reaction information by performing learning using a plurality of pieces of learning data including content information of the content displayed on the display unit 202, attribute information and environmental information of the user 20, and reaction information. By using this learning model, it is possible to estimate how a user 20 with attributes indicated by the attribute information will react when viewing the content indicated by the content information in the environment indicated by the environmental information.

[0050] The selection unit 114 selects content to be provided to the user 20 from the plurality of contents acquired by the content acquisition unit 106 based on the learning result by the model generation unit 112. For example, the selection unit 114 inputs content information of the plurality of contents, attribute information and environmental information of the user 20 into a learning model generated by the model generation unit 112, and selects content to be provided to the user 20 from the plurality of contents based on reaction information output from the learning model. The selection unit 114 may select content for which the output reaction information indicates a more positive reaction as content to be provided to the user 20. The display control unit 108 may display the content selected by the selection unit 114 on the display unit 202 of the user terminal 200.

[0051] This allows, when there is a large amount of content that corresponds to the situation of the user terminal 200, to provide only the content to which the user 20 is likely to respond more positively, thereby improving convenience for the user 20.

[0052] 5 and 6 are explanatory diagrams for explaining an example of processing content in the system 10. Here, an example of processing content will be explained, focusing on the switcher function 420 provided in the management device 100.

[0053] In this example, the user terminal 200 transmits positioning data 402 to the management device 100. The positioning data 402 includes, for example, a GPS positioning result and an image of the surroundings of the user terminal 200 captured by the user terminal 200. The positioning function 404 and VPS / UWB 406 of the management device 100 use the positioning data 402 to identify the position and attitude of the user terminal 200.

[0054] The management device 100 manages a plurality of applications 412. Examples of the applications 412 include, but are not limited to, social networking service (SNS) applications, business applications, entertainment applications, map applications, navigation applications, and game applications.

[0055] The management device 100 starts up, of the multiple applications 412, multiple applications 412 that correspond to the location of the user terminal 200. Here, an example will be mainly described in which the management device 100 starts up a VM (Virtual Machine) 410 for each application 412 and causes the VM 410 to execute the application 412, but this is not limiting. The management device 100 may cause one VM 410 to execute multiple applications 412. Furthermore, the management device 100 may cause multiple containers to execute multiple applications 412.

[0056] The switcher function 420 switches between outputs from a plurality of applications 412 according to information received from the user terminal 200 and delivers the output to the user terminal 200 .

[0057] The switcher function 420 switches between outputs from a plurality of apps 412 and delivers the output to the user terminal 200, for example, in accordance with a switching instruction from the user terminal 200. The switcher function 420 may deliver the output of one app 412 from among the plurality of apps 412 to the user terminal 200, or may deliver the output of a plurality of apps 412 to the user terminal 200. The switcher function 420 may select an app 412 from the plurality of apps 412 that corresponds to the status of the user terminal 200, and deliver the output from the selected app 412 to the user terminal 200.

[0058] In this way, the management device 100 simultaneously launches multiple apps 412 and seamlessly switches the information to be AR streamed depending on the request or situation of the user 20. Since wearable devices such as AR glasses have low processing power, it is difficult to simultaneously launch multiple apps 412. However, according to the system 10 according to the present embodiment, the apps 412 are launched on the management device 100 side, making simultaneous launch possible. Furthermore, by adopting a configuration in which multiple apps 412 are launched simultaneously and switched depending on information from the user terminal 200, it is possible to prevent the user 20 from experiencing stress due to the waiting time for app launch, compared to when an app 412 is launched every time an instruction is received from the user terminal 200.

[0059] In the system 10, the location information of the user terminal 200 may be registered in the location data DB 430. The system 10 may further include an emergency control function 442. In response to acquiring external disaster information 440 such as an earthquake early warning, the emergency control function 442 requests the switcher function 420 to deliver the external disaster information 440 to the user terminal 200. The switcher function 420 stops delivering the output of the app 412 to the user terminal 200 and delivers the external disaster information 440 to the user terminal 200. This allows only the external disaster information 440 to be displayed on the user terminal 200, reducing the possibility that the user 20 will not be able to grasp the disaster because the external disaster information 440 is obscured or difficult to see due to other content. After switching to delivering only the external disaster information 440, the switcher function 420 may resume delivering the output of the app 412 in response to a request from the user terminal 200.

[0060] In response to acquiring the external disaster information 440, the emergency control function 442 may request the PCF 52 to change the priority of the user terminal 200. In response to the request from the emergency control function 442, the PCF 52 may set a policy corresponding to the user terminal 200.

[0061] The emergency control function 442 may be provided in the management device 100. The emergency control function 442 may also be provided in a device other than the management device 100.

[0062] The management device 100 starts, for example, all of the applications 412 corresponding to the status of the user terminal 200. This eliminates the waiting time for application startup.

[0063] The management device 100 may activate only some of the apps 412 among all the apps 412 corresponding to the status of the user terminal 200. This reduces the amount of resources of the management device 100 consumed by constantly activating the apps 412. The management device 100 may execute dynamic standby control of the apps 412 based on a user log.

[0064] 6 is an explanatory diagram for explaining standby control by the management device 100. The management device 100 may include a switching log 452, a position log 454, an AI 456, and a standby control function 460.

[0065] The switching log 452 stores a switching log by the switcher function 420. The position log 454 stores a position log of the user terminal 200. The AI ​​456 uses the switching log 452 and the position log 454 to analyze the tendency of the user 20 to switch apps 412. The AI ​​456 performs learning that takes, for example, the location and time period of the user terminal 200 as input and outputs the probability of use by the user 20 for each of the multiple apps 412. As a specific example, the AI ​​456 performs learning to output the probability of use of each app 412 within a predetermined period, such as within one hour, based on, for example, the location of the user terminal 200, the time period in which the app 412 was used, the position in space of the output information of the app 412, the most recently viewed content, and switching details such as which app 412 was switched from to which other app 412, etc.

[0066] The standby control function 460 uses the learning results of the AI ​​456 to acquire the usage probability of each of the multiple apps 412 corresponding to the status of the user terminal 200, and determines whether to classify each of the multiple apps 412 as Hot or Cold based on the usage probability. For example, the standby control function 460 classifies apps 412 with a usage probability higher than a predetermined threshold as Hot, and apps 412 with a lower usage probability as Cold. The standby control function 460 starts up the apps 412 that have been classified as Hot. This makes it possible to start up only the apps 412 with a high usage probability among the multiple apps 412 corresponding to the status of the user terminal 200, thereby appropriately reducing the resources consumed by the management device 100.

[0067] When one application 412 is executed per VM, the standby control function 460 may determine whether to set each of the multiple applications 412 to Hot, Warm, or Cold based on the usage probability. For example, the standby control function 460 sets an application 412 whose usage probability is higher than a predetermined first threshold as Hot, sets an application 412 whose usage probability is lower than the first threshold and higher than a second threshold that is lower than the first threshold as Warm, and sets an application 412 whose usage probability is lower than the second threshold as Cold. For an application 412 set to Hot, the standby control function 460 starts a corresponding VM to start the application 412, and for an application 412 set to Warm, the standby control function 460 starts a corresponding VM but does not start the application 412. This enables smooth switching of the application 412 set to Hot, and reduces the startup wait time for the application 412 set to Warm by omitting the time required to start a VM.

[0068] The standby control function 460 may predetermine the amount of allocated resources for each user terminal 200, and may activate the application 412 within the range of the allocated resource amount.

[0069] 7 is an explanatory diagram for explaining an example of standby control by the management device 100. Here, an example will be explained in which the apps 412 corresponding to the status of the user terminal 200 are app A, app B, app C, and app D, and the usage probabilities of the apps, respectively, are 97%, 96%, 23%, and 10%.

[0070] The management device 100 starts applications in descending order of usage probability within the allocated resource amount. In the example shown in Fig. 7, the management device 100 starts applications A, B, and C, but does not start application D, which would exceed the allocated resource amount.

[0071] 7, when the management device 100 receives an instruction to start application D from the user terminal 200, the management device 100 terminates applications 412 in ascending order of usage probability until it has secured the amount of resources necessary to start application D. In the example shown in FIG. 7, the management device 100 terminates application C and starts application D.

[0072] 8 shows an example of the functional configuration of the management device 100. The management device 100 includes a user status acquisition unit 122, an application launch unit 124, an application selection unit 126, and an information transmission unit 128.

[0073] The user status acquisition unit 122 acquires the status of the user terminal 200. The user status acquisition unit 122 may acquire location information of the user terminal 200. For example, the user status acquisition unit 122 receives the location information of the user terminal 200 measured by the user terminal 200 from the user terminal 200. For example, the user status acquisition unit 122 receives positioning data 402 from the user terminal 200 and acquires the location information of the user terminal 200 using the positioning data 402.

[0074] The user state acquiring unit 122 may acquire posture information of the user terminal 200. The user state acquiring unit 122 may receive, from the user terminal 200, posture information indicating the posture of the user terminal 200 identified by the detection unit 204 included in the user terminal 200.

[0075] The application launching unit 124 launches a plurality of applications 412 corresponding to the status of the user terminal 200 acquired by the user status acquiring unit 122 .

[0076] For example, the application launch unit 124 launches a plurality of VMs and launches the application 412 in each of the plurality of VMs. The application launch unit 124 may launch one application 412 for each VM. The application launch unit 124 may launch multiple applications 412 for each VM.

[0077] For example, the application launch unit 124 launches multiple containers and launches the application 412 in each of the multiple containers. The application launch unit 124 may launch one application 412 for each container. The application launch unit 124 may launch multiple applications 412 for each container.

[0078] The application selection unit 126 selects at least one application 412 from the plurality of applications 412 launched by the application launch unit 124 based on information from the user terminal 200 .

[0079] For example, the application selecting unit 126 receives selection information for selecting one of the plurality of applications 412 from the user terminal 200. The application selecting unit 126 selects at least one application 412 from the plurality of applications 412 in accordance with the selection information.

[0080] The application selection unit 126 may receive user-related information related to the user 20 from the user terminal 200, and may select at least one application 412 from the plurality of applications 412 based on the user-related information. For example, the user-related information indicates the state of the user 20, and the application selection unit 126 selects at least one application 412 that is more suitable for the state of the user 20 from the plurality of applications 412.

[0081] The information transmitting unit 128 transmits information generated by the application 412 selected by the application selecting unit 126 to the user terminal 200. When multiple applications 412 are selected by the application selecting unit 126, the information transmitting unit 128 transmits information generated by each of the multiple applications 412 to the user terminal 200.

[0082] The application launch unit 124 may launch a plurality of applications 412 selected from a plurality of applications 412 corresponding to the status of the user terminal 200. This reduces the amount of resources of the management device 100 required to keep a plurality of applications 412 running.

[0083] For example, the application launching unit 124 launches a plurality of applications 412 selected from a plurality of applications 412 corresponding to the status of the user terminal 200 in descending order of the probability of use by the user 20 of the user terminal 200. The application launching unit 124 calculates the use probability of each of the plurality of applications 412, for example, based on the switching log 452 and the position log 454. The application launching unit 124 calculates the use probability of the plurality of applications 412 in a geographical area by, for example, using a usage history or switching history of the applications 412 by the user 20 in the geographical area, by increasing the use probability of the application 412 that the user 20 has used more frequently in the past in the geographical area. Furthermore, for example, when the user 20 is using a first application 412 and has a history of using a second application 412 next in the past, the application launching unit 124 calculates the use probability so that the use probability of the second application 412 in the situation where the user 20 is using the first application 412 is higher. Any other method may be used to calculate the usage probability of the application 412 using the usage history or switching history of the application 412.

[0084] The application launch unit 124 may select a plurality of applications 412 from a plurality of applications 412 corresponding to the status of the user terminal 200 within the range of the amount of allocated resources allocated to the user terminal 200 .

[0085] For example, the application launching unit 124 launches a plurality of applications 412 selected in descending order of usage probability within the allocated resource amount from a plurality of applications 412 corresponding to the status of the user terminal 200. The application launching unit 124 may launch the applications 412 in descending order of usage probability within the allocated resource amount.

[0086] For example, the application launching unit 124 launches a plurality of applications 412 selected from the plurality of applications 412 corresponding to the status of the user terminal 200 so as to launch as many applications 412 as possible within the allocated resource amount. For example, when there are five applications 412, namely, applications A, B, C, D, and E, corresponding to the status of the user terminal 200, and the combination of applications A, B, and C and the combination of applications A, B, D, and E can be launched within the allocated resource amount, the application launching unit 124 launches applications A, B, D, and E.

[0087] For example, the application launch unit 124 launches applications 412 one by one from among a plurality of applications 412 corresponding to the status of the user terminal 200 in order of the highest probability of use, and after launching applications A and B, if it is possible to launch one application C or two applications D and E within the remaining allocated resource amount, it launches applications D and E.

[0088] The application launch unit 124 may launch the applications 412 one by one in descending order of usage probability within the allocated resource amount from among the multiple applications 412 corresponding to the status of the user terminal 200, and when selecting the final one, if the use probabilities of the candidate applications 412 are the same or the difference is smaller than a predetermined threshold, select the application 412 that requires the least amount of resources to launch the application 412. This can reduce the amount of resources consumed by the management device 100.

[0089] When the application launching unit 124 launches multiple VMs and launches an application 412 in each of the multiple VMs, with one application 412 launched per VM, the application launching unit 124 may launch a VM corresponding to an application 412 corresponding to the status of the user terminal 200, for which a usage probability by the user 20 of the user terminal 200 is higher than a first threshold, and launch the application on the VM. If the usage probability is lower than the first threshold and higher than a second threshold lower than the first threshold, the application launching unit 124 may launch the VM corresponding to the application. That is, if the usage probability is higher than the first threshold, the application launching unit 124 launches the VM and the application, and if the usage probability is lower than the first threshold but higher than the second threshold, the application launching unit 124 launches only the VM corresponding to the application. This allows applications with a high usage probability to be quickly switched to, and reduces the time required to launch a VM for applications with a relatively high usage probability.

[0090] The management device 100 may include an environmental information acquisition unit 102, an attribute information acquisition unit 104, a content acquisition unit 106, a display control unit 108, a reaction acquisition unit 110, a model generation unit 112, and a selection unit 114, in addition to the user situation acquisition unit 122, the application launch unit 124, the application selection unit 126, and the information transmission unit 128. In this case, for example, the application launch unit 124 launches a plurality of applications 412 according to the situation of the user terminal 200, the application selection unit 126 selects at least one application 412 from the plurality of applications 412, the content acquisition unit 106 acquires a plurality of pieces of content generated by the application 412 selected by the application selection unit 126, the selection unit 114 selects content to be provided to the user 20 from the plurality of pieces of content acquired by the content acquisition unit 106, and the display control unit 108 displays the selected content on the display unit 202 of the user terminal 200.

[0091] 9 schematically illustrates an example of the hardware configuration of a computer 1200 that functions as the management device 100. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "units" of the device according to the present embodiment, or can cause the computer 1200 to perform operations associated with the device according to the present embodiment or one or more "units," and / or can cause the computer 1200 to perform a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0092] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communications interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes a ROM 1230 and legacy input / output units such as a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0093] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into the graphics controller itself, and causes the image data to be displayed on the display device 1218.

[0094] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0095] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0096] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information in accordance with the use of the computer 1200.

[0097] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0098] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and may perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0099] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0100] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0101] The blocks in the flowcharts and block diagrams in the present embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, including integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0102] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture, including instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory stick, integrated circuit card, etc.

[0103] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0104] Computer-readable instructions may be provided locally or over a wide area network (WAN) such as a local area network (LAN), the Internet, etc. to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or programmable circuitry, such that the processor or programmable circuitry executes the computer-readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0105] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0106] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0107] 10 System, 20 User, 40 Information Source, 52 PCF, 100 Management Device, 102 Environmental Information Acquisition Unit, 104 Attribute Information Acquisition Unit, 106 Content Acquisition Unit, 108 Display Control Unit, 110 Response Acquisition Unit, 112 Model Generation Unit, 114 Selection Unit, 122 User Status Acquisition Unit, 124 Application Launch Unit, 126 Application Selection Unit, 128 Information Transmission Unit, 200 User Terminal, 202 Display Unit, 204 Detection Unit, 300 Curation AI, 302 Information Selection Function, 304 Learning and Optimization Function, 310 Environmental Information, 312 Situation Information, 314 Spatial Information, 316 Attribute Information, 322 Selected Information, 324 3D Generation, 332 User Behavior, 402 Positioning Data, 404 Positioning Function, 406 VPS / UWB, 410 VM, 412 application, 420 switcher function, 430 position data DB, 440 external disaster information, 442 emergency control function, 452 switching log, 454 position log, 456 AI, 460 standby control function, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphics controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims

1. a user status acquisition unit that acquires a status of a user terminal, the user terminal being an eyeglass-type device, including location information indicating the location of the user terminal; an application launch unit that launches a plurality of applications corresponding to the status of the user terminal acquired by the user status acquisition unit; an application selection unit that selects at least one application from the plurality of applications based on information from the user terminal; an information transmission unit that transmits AR content for AR streaming generated by the application selected by the application selection unit to the user terminal; An information processing device comprising:

2. The information processing device according to claim 1 , wherein the user status acquisition unit acquires the status of the user terminal including position information of the user terminal and attitude information of the user terminal.

3. The information processing device according to claim 1 , wherein the application selection unit selects at least one application from the plurality of applications in accordance with selection information for selecting one of the plurality of applications received from the user terminal.

4. The information processing apparatus according to claim 1 , wherein the application selection unit selects at least one application from the plurality of applications based on user-related information related to a user of the user terminal received from the user terminal.

5. The information processing device according to claim 1 , wherein the application launching unit launches a plurality of applications selected from a plurality of applications corresponding to a status of the user terminal.

6. The information processing device according to claim 5 , wherein the application launch unit launches the plurality of applications selected from the plurality of applications corresponding to the status of the user terminal in descending order of probability of use by the user of the user terminal.

7. 7. The information processing device according to claim 6, wherein the application launch unit launches the plurality of applications selected from the plurality of applications corresponding to the status of the user terminal in descending order of the usage probability within a range of an allocated resource amount allocated to the user terminal.

8. 8. The information processing device according to claim 7, wherein the application launch unit launches a plurality of selected applications from the plurality of applications corresponding to a status of the user terminal so as to launch as many applications as possible within a range of an allocated resource amount allocated to the user terminal.

9. 6. The information processing device according to claim 5, wherein the application launch unit calculates a usage probability of each of the plurality of applications corresponding to a status of the user terminal based on a location log of the user terminal and a log of application switching by the user of the user terminal, and launches the plurality of applications selected in descending order of the calculated usage probability.

10. 10. The information processing device according to claim 9, wherein the application launch unit calculates the usage probability of the plurality of applications in a specific geographical area by using a usage history and a switching history of applications by the user in the specific geographical area and increasing the usage probability of an application that the user has used more frequently in the past in the specific geographical area.

11. 10. The information processing device according to claim 9, wherein the application launch unit calculates the usage probability such that the probability of the second application being used in a situation in which the user is using the first application is higher when the user has a history of using a second application in the past while using the first application.

12. 2. The information processing device according to claim 1, wherein the application launch unit launches a corresponding VM and launches the application on the VM for an application among the plurality of applications corresponding to the status of the user terminal, the application having a usage probability by the user of the user terminal higher than a first threshold, and when the usage probability is lower than the first threshold and higher than a second threshold lower than the first threshold, the application launch unit launches the VM corresponding to the application but does not launch the application.

13. A program for causing a computer to function as the information processing device according to any one of claims 1 to 12.

14. 1. A computer-implemented information processing method, comprising: a user status acquisition step of acquiring a status of the user terminal including location information indicating a location of the user terminal, which is a glasses-type device; an application launching step of launching a plurality of applications corresponding to the status of the user terminal acquired in the user status acquiring step; an application selection step of selecting at least one application from the plurality of applications based on information from the user terminal; an information transmitting step of transmitting AR content for AR streaming, generated by the application selected in the application selecting step, to the user terminal; An information processing method comprising:

Images