Information processing device and program and information processing method
The system addresses information overload and resource management issues in AR and VR by using AI to select and display relevant information based on user reactions, and manages applications on a cloud server to optimize resource usage and reduce startup times.
Patent Information
- Application Number
- JP2024143387
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-08-24
AI Technical Summary
Current AR and VR technologies lack effective mechanisms to incorporate real-time practical information seamlessly and efficiently, leading to information overload and resource exhaustion, especially in wearable devices with limited processing power.
An information processing system that utilizes AI to analyze user reactions and environmental data to selectively display relevant information in XR environments, and manages multiple applications on a cloud-based server to optimize resource usage and reduce startup times.
Enhances user convenience by automatically selecting and displaying relevant information based on user interactions, while efficiently managing multiple applications to prevent resource exhaustion and reduce startup times.
Smart Images

Figure 2024164179000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, a program, and an information processing method. [Background technology]
[0002] Patent Document 1 describes AR (Augmented Reality), a technology that reflects (extends) virtual reality in the real world, MR (Mixed Reality), a technology that allows you to experience a virtual world by overlaying it on the real world, and XR (X Reality (cross reality)), a general term for these technologies. [Prior art document] [Patent documents] [Patent Document 1] JP 2022-105942 A Summary of the Invention [Means for solving the problem]
[0003] According to one embodiment of the present invention, an information processing device is provided. The information processing device may include a content acquisition unit that acquires content corresponding to a status of a user terminal. The information processing device may include a display control unit that displays the content on a display unit of the user terminal. The information processing device may include a reaction acquisition unit that acquires reaction information indicating a reaction of a user of the user terminal to the content displayed on the display unit. The information processing device may include a selection unit that selects content to be provided to a user from a plurality of contents based on a learning result using a plurality of learning data including content information of the content, attribute information and environmental information of the user, and the reaction information.
[0004] The information processing device may further include a model generation unit that generates a learning model that receives the content information, the attribute information, and the environmental information and outputs the reaction information by performing learning using the multiple learning data, and the selection unit may input content information of the multiple contents, the user's attribute information and environmental information to the learning model and select content to be provided to the user from the multiple contents based on the reaction information output from the learning model, and the display control unit may display the content selected by the selection unit on the display unit of the user terminal.
[0005] In any of the information processing devices, the reaction acquisition unit may acquire the reaction information generated by at least one of head tracking, eye tracking, and hand tracking of the user by the user terminal. The reaction acquisition unit may acquire the reaction information indicating whether the direction of the user's face is facing the content. The reaction acquisition unit may acquire the reaction information indicating whether the user's line of sight is directed toward the content. The reaction acquisition unit may acquire the reaction information indicating whether the user's hand has moved toward the content.
[0006] In any of the information processing devices, the content information may include the type of the content, the attribute information may include the address, occupation, and age of the user, and the environmental information may include situation information when the user viewed the content and spatial information of the space where the user viewed the content. The situation information may include time information, information about the user's surroundings, and behavior information of the user, and the spatial information may include the size of the space, the brightness of the space, and information about the location where the user viewed the content.
[0007] Any of the information processing devices may further include an app launch unit that launches multiple apps corresponding to a status of the user terminal, and an app selection unit that selects at least one app from the multiple apps based on information from the user terminal, and the content acquisition unit may acquire the content generated by the app selected by the app selection unit.
[0008] According to one embodiment of the present invention, there is provided a program for causing a computer to function as the information processing device.
[0009] According to an embodiment of the present invention, there is provided an information processing device. The information processing device may include an application launching unit that launches a plurality of applications corresponding to a status of a user terminal. The information processing device may include an application selecting unit that selects at least one application from the plurality of applications based on information from the user terminal. The information processing device may include an information transmitting unit that transmits information generated by the application selected by the application selecting unit to the user terminal.
[0010] In the information processing device, the application selection unit may select at least one application from the plurality of applications in accordance with selection information for selecting one of the plurality of applications received from the user terminal.
[0011] In any of the information processing devices, the application selection unit may select at least one of the plurality of applications based on user-related information related to the user received from the user terminal.
[0012] In any of the information processing devices, the application launching unit may launch the multiple applications selected from the multiple applications corresponding to a status of the user terminal. The application launching unit may launch the multiple applications selected from the multiple applications corresponding to the status of the user terminal in descending order of the probability of use by a user of the user terminal. The application launching unit may launch the multiple applications selected from the multiple applications corresponding to the status of the user terminal in descending order of the probability of use within a range of an allocated resource amount allocated to the user terminal. The application launching unit may launch the multiple applications selected from the multiple applications corresponding to the status of the user terminal in a manner that launches as many applications as possible within a range of an allocated resource amount allocated to the user terminal.
[0013] In any of the information processing devices, the application launch unit may launch a corresponding VM for an application among the plurality of applications corresponding to a status of the user terminal, the application having a usage probability by a user of the user terminal higher than a first threshold, and launch the application on the VM, and if the usage probability is lower than the first threshold and higher than a second threshold lower than the first threshold, launch the VM corresponding to the application.
[0014] According to one embodiment of the present invention, there is provided a program for causing a computer to function as the information processing device.
[0015] According to one embodiment of the present invention, there is provided an information processing method executed by a computer. The information processing method may include a content acquisition step of acquiring content corresponding to a status of a user terminal. The information processing method may include a display control step of displaying the content on a display unit of the user terminal. The information processing method may include a reaction acquisition step of acquiring reaction information indicating a reaction of a user of the user terminal to the content displayed on the display unit. The information processing method may include a selection step of selecting content to be provided to a user from a plurality of contents based on a learning result using a plurality of learning data including content information of the content, attribute information and environmental information of the user, and the reaction information.
[0016] According to one embodiment of the present invention, there is provided an information processing method executed by a computer. The information processing method may include an application launching step of launching a plurality of applications corresponding to a status of a user terminal. The information processing method may include an application selection step of selecting at least one application from the plurality of applications based on information from the user terminal. The information processing method may include an information transmission step of transmitting information generated by the application selected in the application selection step to the user terminal.
[0017] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. [Brief description of the drawings]
[0018] [Figure 1] An example of a system 10 is shown diagrammatically. [Diagram 2] FIG. 2 is an explanatory diagram for explaining an example of processing contents in the system 10. [Diagram 3] FIG. 2 is an explanatory diagram for explaining an example of processing contents in the system 10. [Figure 4] 2 illustrates an example of a functional configuration of a management device 100. [Diagram 5]FIG. 2 is an explanatory diagram for explaining an example of processing contents in the system 10. [Figure 6] FIG. 2 is an explanatory diagram for explaining an example of processing contents in the system 10. [Figure 7] 2 illustrates an example of a functional configuration of a management device 100. [Figure 8] FIG. 2 is an explanatory diagram for explaining standby control by the management device 100. [Figure 9] 1 illustrates an example of a hardware configuration of a computer 1200 that functions as the management device 100. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0019] Compared to the general web, current AR lacks access to practical information sources, and most content is limited to closed entertainment or experiences in handmade spaces. A mechanism is needed to easily incorporate real-time practical information into AR and VR spaces. In addition, if AR and VR devices become as common as smartphones and it becomes commonplace to wear them all the time, information overload is likely. In particular, unlike smartphones, XR experiences passively receive information from the entire field of view, so it is necessary to consider how to select information. As an XR experience, it is not desirable to have users actively make selections using a user interface, and it is preferable for the information displayed to be automatically selected based on the user's behavior and meta information. In such a situation, for example, AI can select information collected from the web using user information, behavior, and situation. Since the information is numerical or textual information, the information is presented in the XR space using a mechanism that generates three-dimensional objects from that information. The reaction of the user who viewed the information is obtained from the sensors of the XR device, and the AI optimizes the content of the selection for each user.
[0020] Conventional AR displays information by running applications on smartphones or wearable devices, enriching our lives by adding information to the real world, but as it becomes more widespread, it becomes overflowing with information, which causes stress. In addition, wearable devices such as AR glasses have low processing power, making it difficult to launch multiple apps at the same time. There is a technology called AR streaming, which launches an AR app on the server side, such as the cloud, and sends only the video to the device, but even if AR streaming is used to improve efficiency, server-side resources are exhausted. In response to this situation, for example, a switcher that can run multiple apps simultaneously and seamlessly switch the streaming video according to the user's request or situation can be provided from the cloud. In addition, in order to prevent resource exhaustion on the server side, it is possible to analyze the trends in information needs for each user's situation from the user's switching log and control it so that only apps with high usability are launched in advance.
[0021] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.
[0022] 1 illustrates an example of a system 10. The system 10 includes a management device 100. The management device 100 may be an example of an information processing device. The system 10 includes a user terminal 200.
[0023] The management device 100 manages information to be provided to the user 20. For example, the management device 100 transmits AR content according to the state of the user terminal 200 to the user terminal 200 to display it.
[0024] The user terminal 200 may be any terminal as long as it has a communication function and a function of displaying information to the user 20. For example, the user terminal 200 is a glasses-type device such as AR glasses. For example, the user terminal 200 is a smartphone, a tablet terminal, or the like.
[0025] The user terminal 200 displays the AR content received from the management device 100. The content managed by the management device 100 may be in any data format. For example, the content is 3D CG (Computer Graphics) data. For example, the content is a still image. Also, for example, the content is a video. For example, the content is text. The content may include a combination of these.
[0026] When the user terminal 200 is a glasses-type device, it may be a device that can give the user 20 the sensation that the content is placed in real space by displaying the content on transparent or semi-transparent glasses. In other words, the user terminal 200 may be a so-called transparent (see-through) device.
[0027] When the user terminal 200 is a smartphone or a tablet terminal, the user terminal 200 may be a video pass-through type device. In this case, the user terminal 200 may display a captured image on a non-transparent display and superimpose the content. When the user terminal 200 is a glasses-type device, the user terminal 200 may be a video pass-through type device.
[0028] 2 and 3 are explanatory diagrams for explaining an example of the processing contents in the system 10. Here, the flow of processing between the user terminal 200 and the curation AI 300 included in the management device 100 will be explained.
[0029] The user terminal 200 includes a display unit 202 and a detection unit 204. The detection unit 204 includes multiple types of sensors. Examples of sensors included in the detection unit 204 include, but are not limited to, a position sensor, an image sensor, a gyro sensor, an acceleration sensor, a gaze sensor, and a sound sensor. The position sensor may include at least one of a GNSS (Global Navigation Satellite System) positioning function such as a GPS (Global Positioning System) positioning function, a network positioning function, a VPS (Visual Positioning System) positioning function, and a UWB (Ultra Wide Band) positioning function. The positioning functions included in the position sensor are not limited to these, and the positioning function may include only a positioning function other than these, or may include at least one of these positioning functions and a positioning function other than these. The detection unit 204 may realize head tracking of the user 20, eye tracking of the user 20, hand tracking of the user 20, and the like, by using multiple types of sensors.
[0030] The user terminal 200 first transmits attribute information 316 of the user 20 and environmental information 310 including the position and orientation of the user terminal 200 detected by the detection unit 204 to the curation AI 300. An information selection function 302 of the curation AI 300 selects information corresponding to the environmental information 310 and the attribute information 316 from the information source 40. The curation AI 300 may manage the information source 40 by organizing information from outside using an API (Application Programming Interface) and a DB (DataBase).
[0031] 3D generation 324 is performed on the information 322 selected by the information selection function 302, and is displayed on the display unit 202 of the user terminal 200. The 3D generation 324 may be a function that converts the selected information 322 to match the position and posture of the user 20 so that it does not feel strange to the user 20. The 3D generation 324 may be performed by the curation AI 300. The 3D generation 324 may also be performed by the user terminal 200.
[0032] The detection unit 204 detects user behavior 332 with respect to the information displayed on the display unit 202 and transmits it to the curation AI 300. The user behavior 332 may be information indicating the reaction of the user 20 to the information displayed on the display unit 202. The detection unit 204 transmits to the curation AI 300 the user behavior 332, such as whether the user 20 turned toward the information, followed the information with their eyes, reacted to the information by touching it with their hands, ignored it, or manually switched to other information.
[0033] The learning and optimization function 304 of the curation AI 300 uses the information displayed on the display unit 202, the environmental information 310, the attribute information 316, and the user behavior 332 to perform learning and optimization to select information suitable for the user 20 from multiple pieces of information corresponding to the situation of the user terminal 200.
[0034] 3, the environment information 310 may include situation information 312 and space information 314. The situation information 312 includes behavioral information indicating what behavior the user 20 is performing, time information, and surrounding information related to the surroundings of the user 20. The space information 314 includes information on the location where the user 20 viewed the information, information on the size of the space, information on the brightness of the space, and the like. The attribute information 316 includes information such as the address, occupation, and age of the user 20. The content information 42 related to the information displayed on the display unit 202 includes the type of content.
[0035] The learning and optimization function 304 executes learning based on this information. For example, for each attribute, situation, and space condition of the user 20, when the user 20 shows a positive reaction to the displayed information, the learning and optimization function 304 makes the information or the same type of information more likely to be selected under the condition, and when the user 20 shows a negative reaction, makes the information or the same type of information less likely to be selected under the condition.
[0036] The information selection function 302 selects information to be provided to the user 20 based on the learning results of the learning and optimization function 304. This makes it possible to automatically and appropriately select information suitable for the user 20.
[0037] 4 illustrates an example of a functional configuration of the management device 100. The management device 100 includes an environment information acquisition unit 102, an attribute information acquisition unit 104, a content acquisition unit 106, a display control unit 108, a reaction acquisition unit 110, a model generation unit 112, and a selection unit 114.
[0038] The environmental information acquisition unit 102 acquires environmental information from the user terminal 200. The environmental information includes position information indicating the position of the user terminal 200. The environmental information includes posture information indicating the posture of the user terminal 200. The environmental information may include situation information. The environmental information may include spatial information.
[0039] The situation information may include time information indicating a time period, etc. The situation information may include behavioral information indicating how the user 20 is behaving. The situation information may include surrounding information regarding the surroundings of the user 20. The surrounding information may include information regarding people, objects, etc. present around the user 20.
[0040] The spatial information may include location information indicating a type of location. The spatial information may include information on the size of the space. The spatial information may include information on the brightness of the space. The spatial information may include information on the weather of the location where the user 20 is located.
[0041] The attribute information acquisition unit 104 acquires attribute information of the user 20. The attribute information may include the name of the user 20. The attribute information may include the username of the user 20. The attribute information may include the address of the user 20. The attribute information may include the gender of the user 20. The attribute information may include the age of the user 20. The attribute information may include the occupation of the user 20. The attribute information may include the blood type of the user 20. The attribute information may include the nationality of the user 20.
[0042] The content acquisition unit 106 acquires content corresponding to the status of the user terminal 200. The content acquisition unit 106 may acquire a plurality of pieces of content suitable for the status of the user terminal 200 from the information source 40. The content acquisition unit 106 acquires content registered for a range included in the field of view of the user 20, for example, from the position and posture of the user terminal 200.
[0043] The display control unit 108 causes the display unit 202 of the user terminal 200 to display the content acquired by the content acquisition unit 106. The display control unit 108 may transmit to the user terminal 200 the content acquired by the content acquisition unit 106 and a display instruction.
[0044] The reaction acquisition unit 110 acquires reaction information indicating a reaction of the user 20 to the content displayed on the display unit 202. The reaction acquisition unit 110 may receive, from the user terminal 200, reaction information generated by the detection unit 204 of the user terminal 200.
[0045] The reaction acquisition section 110 may acquire reaction information generated by at least one of head tracking, eye tracking, and hand tracking of the user 20 by the user terminal 200.
[0046] The reaction acquisition unit 110 acquires reaction information indicating, for example, whether the direction of the face of the user 20 is facing directly toward the content. The direction of the face of the user 20 facing directly toward the content may be an example of a positive reaction. The reaction acquisition unit 110 acquires reaction information indicating, for example, whether the direction of the face of the user 20 is turned away from the content. The direction of the face of the user 20 is turned away from the content may be an example of a negative reaction.
[0047] The reaction acquisition unit 110 acquires, for example, reaction information indicating whether or not the gaze of the user 20 is directed toward the content. The fact that the gaze of the user 20 is directed toward the content may be an example of a positive reaction. The reaction acquisition unit 110 may acquire reaction information indicating whether or not the gaze of the user 20 has followed the content. The fact that the gaze of the user 20 has followed the content may be an example of a positive reaction. The reaction acquisition unit 110 acquires, for example, reaction information indicating whether or not the gaze of the user 20 has been averted from the content. The fact that the gaze of the user 20 has been averted from the content may be an example of a negative reaction.
[0048] The reaction acquisition unit 110 acquires reaction information indicating, for example, whether or not the hand of the user 20 has moved on the content. The reaction acquisition unit 110 acquires reaction information indicating, for example, whether or not the hand of the user 20 has operated the content. The hand of the user 20 moving on the content may be an example of a positive reaction. The hand of the user 20 not moving on the content may be neither a positive reaction nor a negative reaction, or may be treated as a negative reaction.
[0049] The model generation unit 112 executes learning using a plurality of learning data including content information of the content displayed on the display unit 202, attribute information and environmental information of the user 20, and reaction information, to generate a learning model in which the content information, attribute information, and environmental information are input and the reaction information is output. By using the learning model, it is possible to estimate how a user 20 with attributes indicated by the attribute information will react when viewing the content indicated by the content information in the environment indicated by the environmental information.
[0050] The selection unit 114 selects content to be provided to the user 20 from the plurality of contents acquired by the content acquisition unit 106 based on the learning result by the model generation unit 112. For example, the selection unit 114 inputs content information of the plurality of contents, attribute information and environmental information of the user 20 into a learning model generated by the model generation unit 112, and selects content to be provided to the user 20 from the plurality of contents based on reaction information output from the learning model. The selection unit 114 may select content for which the output reaction information indicates a more positive reaction as content to be provided to the user 20. The display control unit 108 may cause the display unit 202 of the user terminal 200 to display the content selected by the selection unit 114.
[0051] As a result, when there is a large amount of content that corresponds to the situation of the user terminal 200, it is possible to provide the user 20 with only the content to which the user 20 is likely to respond more positively, thereby improving convenience for the user 20.
[0052] 5 and 6 are explanatory diagrams for explaining an example of the processing contents in the system 10. Here, an example of the processing contents will be explained focusing on the switcher function 420 provided in the management device 100.
[0053] In this example, the user terminal 200 transmits positioning data 402 to the management device 100. The positioning data 402 includes, for example, a GPS positioning result and an image of the surroundings of the user terminal 200 captured by the user terminal 200. The positioning function 404 and VPS / UWB 406 of the management device 100 use the positioning data 402 to identify the position and attitude of the user terminal 200.
[0054] The management device 100 manages a plurality of applications 412. Examples of the applications 412 include, but are not limited to, social networking service (SNS) applications, business applications, entertainment applications, map applications, navigation applications, and game applications.
[0055] The management device 100 starts up a plurality of applications 412 corresponding to the location of the user terminal 200, out of the plurality of applications 412. Here, an example in which the management device 100 starts up a VM (Virtual Machine) 410 for each application 412 and causes the VM 410 to execute the application 412 will be mainly described, but is not limited to this. The management device 100 may cause one VM 410 to execute a plurality of applications 412. Furthermore, the management device 100 may cause a plurality of containers to execute a plurality of applications 412.
[0056] The switcher function 420 switches between outputs from a plurality of applications 412 according to information received from the user terminal 200 and delivers the outputs to the user terminal 200 .
[0057] The switcher function 420 switches between outputs from a plurality of apps 412 and distributes the outputs to the user terminal 200 in accordance with, for example, a switching instruction from the user terminal 200. The switcher function 420 may distribute the output of one app 412 among the plurality of apps 412 to the user terminal 200, or may distribute the outputs of a plurality of apps 412 to the user terminal 200. The switcher function 420 may select an app 412 corresponding to the status of the user terminal 200 from the plurality of apps 412, and distribute the output of the selected app 412 to the user terminal 200.
[0058] In this way, the management device 100 starts multiple apps 412 simultaneously and seamlessly switches the information to be AR streamed according to the request or situation of the user 20. Since a wearable device such as AR glasses has low processing power, it is difficult to start the apps 412 simultaneously. However, according to the system 10 according to the present embodiment, the apps 412 are started on the management device 100 side, so that the apps can be started simultaneously. Furthermore, by adopting a configuration in which multiple apps 412 are started simultaneously and switched according to information from the user terminal 200, it is possible to prevent the user 20 from feeling stressed by the waiting time for app start, compared to the case in which the app 412 is started every time an instruction is received from the user terminal 200.
[0059] In the system 10, the location information of the user terminal 200 may be registered in the location data DB 430. The system 10 may further include an emergency control function 442. In response to acquiring external disaster information 440 such as an emergency earthquake alert, the emergency control function 442 requests the switcher function 420 to distribute the external disaster information 440 to the user terminal 200. The switcher function 420 stops the distribution of the output of the application 412 to the user terminal 200 and distributes the external disaster information 440 to the user terminal 200. This allows the user terminal 200 to display only the external disaster information 440, and reduces the possibility that the user 20 will not be able to grasp the disaster because the external disaster information 440 is not visible or is difficult to see due to other content. After switching to the distribution of only the external disaster information 440, the switcher function 420 may resume the distribution of the output of the application 412 in response to a request from the user terminal 200.
[0060] In response to acquiring the external disaster information 440, the emergency control function 442 may request the PCF 52 to change the priority of the user terminal 200. In response to the request from the emergency control function 442, the PCF 52 may set a policy corresponding to the user terminal 200.
[0061] The emergency control function 442 may be provided in the management device 100. The emergency control function 442 may be provided in a device other than the management device 100.
[0062] The management device 100, for example, activates all of the applications 412 corresponding to the status of the user terminal 200. This can eliminate waiting time for application activation.
[0063] The management device 100 may activate only some of the apps 412 among all the apps 412 corresponding to the status of the user terminal 200. This can reduce the amount of resources of the management device 100 consumed by constantly activating the apps 412. The management device 100 may execute dynamic standby control of the apps 412 based on a user log.
[0064] 6 is an explanatory diagram for explaining standby control by the management device 100. The management device 100 may include a switching log 452, a position log 454, an AI 456, and a standby control function 460.
[0065] The switching log 452 stores a switching log by the switcher function 420. The position log 454 stores a position log of the user terminal 200. The AI 456 uses the switching log 452 and the position log 454 to analyze the switching tendency of the user 20 to the app 412. The AI 456 executes learning to input, for example, the location and time period of the user terminal 200 and output the probability of use by the user 20 for each of the multiple apps 412. As a specific example, the AI 456 executes learning to output the probability of use of each app 412 within a predetermined period, such as within one hour, based on, for example, the location of the user terminal 200, the time period in which the app 412 was used, the position in space of the output information by the app 412, the immediately preceding viewing content, and switching details such as which app 412 was switched to which other app 412, etc.
[0066] The standby control function 460 acquires the usage probability of each of the multiple applications 412 corresponding to the status of the user terminal 200 using the learning result by the AI 456, and determines whether to classify each of the multiple applications 412 as Hot or Cold based on the usage probability. For example, the standby control function 460 classifies the application 412 having a usage probability higher than a predetermined threshold as Hot, and classifies the application 412 having a low usage probability as Cold. The standby control function 460 starts the application 412 that has been set as Hot. As a result, it is possible to start only the application 412 having a high usage probability among the multiple applications 412 corresponding to the status of the user terminal 200, and it is possible to appropriately reduce the resource consumption of the management device 100.
[0067] In a case where one application 412 is executed for one VM, the standby control function 460 may determine whether each of the multiple applications 412 is to be set to Hot, Warm, or Cold based on the usage probability. For example, the standby control function 460 sets the application 412 whose usage probability is higher than a predetermined first threshold as Hot, sets the application 412 whose usage probability is lower than the first threshold and higher than a second threshold lower than the first threshold as Warm, and sets the application 412 whose usage probability is lower than the second threshold as Cold. For the application 412 set to Hot, the standby control function 460 starts the corresponding VM to start the application 412, and for the application 412 set to Warm, the standby control function 460 starts the corresponding VM and does not start the application 412. As a result, a smooth switching is realized for the application 412 set to Hot, and the startup waiting time can be reduced for the application 412 set to Warm by omitting the time to start the VM.
[0068] The standby control function 460 may predetermine an amount of allocated resources for each user terminal 200, and may activate the application 412 within the range of the allocated resource amount.
[0069] 7 is an explanatory diagram for describing an example of standby control by the management device 100. Here, an example will be described in which the apps 412 corresponding to the status of the user terminal 200 are app A, app B, app C, and app D, and the usage probabilities of the apps are 97%, 96%, 23%, and 10%, respectively.
[0070] The management device 100 starts applications in descending order of usage probability within the allocated resource amount. In the example shown in Fig. 7, the management device 100 starts applications A, B, and C, but does not start application D, which would exceed the allocated resource amount.
[0071] 7, when the management device 100 receives an instruction to start application D from the user terminal 200, the management device 100 terminates applications 412 in ascending order of usage probability until a resource amount sufficient to start application D is secured. In the example shown in FIG. 7, the management device 100 terminates application C and starts application D.
[0072] 8 illustrates an example of a functional configuration of the management device 100. The management device 100 includes a user status acquisition unit 122, an application launch unit 124, an application selection unit 126, and an information transmission unit 128.
[0073] The user status acquiring unit 122 acquires the status of the user terminal 200. The user status acquiring unit 122 may acquire position information of the user terminal 200. For example, the user status acquiring unit 122 receives the position information of the user terminal 200 measured by the user terminal 200 from the user terminal 200. For example, the user status acquiring unit 122 receives positioning data 402 from the user terminal 200 and acquires the position information of the user terminal 200 using the positioning data 402.
[0074] The user status acquiring unit 122 may acquire posture information of the user terminal 200. The user status acquiring unit 122 may receive, from the user terminal 200, posture information indicating the posture of the user terminal 200 identified by the detection unit 204 included in the user terminal 200.
[0075] The application launching unit 124 launches a plurality of applications 412 corresponding to the status of the user terminal 200 acquired by the user status acquiring unit 122 .
[0076] For example, the application launch unit 124 launches a plurality of VMs and launches the application 412 in each of the plurality of VMs. The application launch unit 124 may launch one application 412 for each VM. The application launch unit 124 may launch multiple applications 412 for each VM.
[0077] For example, the application launch unit 124 launches a plurality of containers and launches the application 412 in each of the plurality of containers. The application launch unit 124 may launch one application 412 for each container. The application launch unit 124 may launch multiple applications 412 for each container.
[0078] The application selection unit 126 selects at least one application 412 from the multiple applications 412 launched by the application launch unit 124 based on information from the user terminal 200 .
[0079] For example, the application selecting unit 126 receives selection information for selecting one of the multiple applications 412 from the user terminal 200. The application selecting unit 126 selects at least one application 412 from the multiple applications 412 in accordance with the selection information.
[0080] The application selection unit 126 may receive user-related information related to the user 20 from the user terminal 200, and select at least one application 412 from the multiple applications 412 based on the user-related information. For example, the user-related information indicates a state of the user 20, and the application selection unit 126 selects at least one application 412 that is more suitable for the state of the user 20 from the multiple applications 412.
[0081] The information transmitting unit 128 transmits information generated by the application 412 selected by the application selecting unit 126 to the user terminal 200. When a plurality of applications 412 are selected by the application selecting unit 126, the information transmitting unit 128 transmits information generated by each of the plurality of applications 412 to the user terminal 200.
[0082] The application launch unit 124 may launch a plurality of applications 412 selected from the plurality of applications 412 corresponding to the status of the user terminal 200. This can reduce the amount of resources of the management device 100 required to keep the plurality of applications 412 running.
[0083] For example, the application launch unit 124 launches a plurality of applications 412 selected from a plurality of applications 412 corresponding to the situation of the user terminal 200 in descending order of the probability of use by the user 20 of the user terminal 200. The application launch unit 124 calculates the probability of use of each of the plurality of applications 412 based on, for example, the switching log 452 and the position log 454. The application launch unit 124 calculates the probability of use of the plurality of applications 412 in a geographical area by, for example, increasing the probability of use of the application 412 that the user 20 has used more frequently in the past in the geographical area, using the usage history and switching history of the applications 412 by the user 20 in the geographical area. In addition, for example, when the user 20 uses a first application 412 and there is a history of using a second application 412 next in the past, the application launch unit 124 calculates the probability of use so that the probability of use of the second application 412 in the situation where the user 20 uses the first application 412 is increased. As a method for calculating the usage probability of the application 412 using the usage history or switching history of the application 412, any other method may be adopted.
[0084] The application launch unit 124 may select a plurality of applications 412 from the plurality of applications 412 corresponding to the status of the user terminal 200 within the range of the amount of allocated resources allocated to the user terminal 200 .
[0085] For example, the application launching unit 124 launches a plurality of applications 412 selected in descending order of usage probability within the range of the allocated resource amount from a plurality of applications 412 corresponding to the status of the user terminal 200. The application launching unit 124 may launch the applications 412 in descending order of usage probability within the range not exceeding the allocated resource amount.
[0086] For example, the application launching unit 124 launches a plurality of applications 412 selected from the plurality of applications 412 corresponding to the status of the user terminal 200 so as to launch as many applications 412 as possible within the range of the allocated resource amount. For example, when there are five applications 412, namely, applications A, B, C, D, and E, corresponding to the status of the user terminal 200, and a combination of applications A, B, and C and a combination of applications A, B, D, and E can be launched within the range of the allocated resource amount, the application launching unit 124 launches applications A, B, D, and E.
[0087] For example, the application launch unit 124 launches applications 412 one by one from among a plurality of applications 412 corresponding to the status of the user terminal 200 in order of the highest probability of use, and after launching applications A and B, if it is possible to launch one application C or two applications D and E within the remaining allocated resource amount, then it launches applications D and E.
[0088] The application launch unit 124 launches the applications 412 one by one in descending order of usage probability within the range of the allocated resource amount from among the multiple applications 412 corresponding to the status of the user terminal 200, and when selecting the last one, if the application 412 candidates have the same usage probability or the difference between the probabilities is smaller than a predetermined threshold, the application launch unit 124 may select the application 412 that requires a smaller amount of resources to launch the application 412. This can reduce the amount of resources consumed by the management device 100.
[0089] In the case where the application launch unit 124 launches a plurality of VMs and launches an application 412 in each of the plurality of VMs, and launches one application 412 for each VM, the application launch unit 124 may launch a corresponding VM for an application whose usage probability by the user 20 of the user terminal 200 is higher than a first threshold among the plurality of applications 412 corresponding to the status of the user terminal 200, launch the application on the VM, and launch the VM corresponding to the application when the usage probability is lower than the first threshold and higher than a second threshold lower than the first threshold. That is, the application launch unit 124 launches the VM and the application when the usage probability is higher than the first threshold, and launches only the VM corresponding to the application when the usage probability is lower than the first threshold but higher than the second threshold. This allows an application with a high usage probability to be immediately switched over, and the time to launch the VM can be omitted for an application with a relatively high usage probability.
[0090] The management device 100 may include an environmental information acquisition unit 102, an attribute information acquisition unit 104, a content acquisition unit 106, a display control unit 108, a reaction acquisition unit 110, a model generation unit 112, and a selection unit 114 in addition to the user status acquisition unit 122, the application launch unit 124, the application selection unit 126, and the information transmission unit 128. In this case, for example, the application launch unit 124 launches a plurality of applications 412 according to the status of the user terminal 200, the application selection unit 126 selects at least one application 412 from the plurality of applications 412, the content acquisition unit 106 acquires a plurality of contents generated by the application 412 selected by the application selection unit 126, the selection unit 114 selects a content to be provided to the user 20 from the plurality of contents acquired by the content acquisition unit 106, and the display control unit 108 causes the display unit 202 of the user terminal 200 to display the selected content.
[0091] 9 is a schematic diagram showing an example of a hardware configuration of a computer 1200 functioning as the management device 100. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to the present embodiment, or cause the computer 1200 to execute operations associated with the device according to the present embodiment or one or more "parts", and / or cause the computer 1200 to execute a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.
[0092] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive, a solid state drive, etc. The computer 1200 also includes a legacy input / output unit such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.
[0093] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.
[0094] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.
[0095] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or a program that depends on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.
[0096] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing operations or processing of information according to the use of the computer 1200.
[0097] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.
[0098] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.
[0099] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.
[0100] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.
[0101] The blocks in the flowcharts and block diagrams in the present embodiment may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.
[0102] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.
[0103] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages such as the “C” programming language or similar programming languages.
[0104] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.
[0105] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.
[0106] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, it does not mean that the process must be performed in this order. [Explanation of symbols]
[0107] 10 system, 20 user, 40 information source, 52 PCF, 100 management device, 102 environmental information acquisition unit, 104 attribute information acquisition unit, 106 content acquisition unit, 108 display control unit, 110 reaction acquisition unit, 112 model generation unit, 114 selection unit, 122 user status acquisition unit, 124 application launch unit, 126 application selection unit, 128 information transmission unit, 200 user terminal, 202 display unit, 204 detection unit, 300 curation AI, 302 information selection function, 304 learning and optimization function, 310 environmental information, 312 situation information, 314 spatial information, 316 attribute information, 322 selected information, 324 3D generation, 332 user behavior, 402 positioning data, 404 positioning function, 406 VPS / UWB, 410 VM, 412, application, 420, switcher function, 430, position data DB, 440, external disaster information, 442, emergency control function, 452, switching log, 454, position log, 456, AI, 460, standby control function, 1200, computer, 1210, host controller, 1212, CPU, 1214, RAM, 1216, graphic controller, 1218, display device, 1220, input / output controller, 1222, communication interface, 1224, storage device, 1230, ROM, 1240, input / output chip
Claims
1. a user status acquisition unit that acquires a status of a user terminal including location information indicating a location of the user terminal, the user terminal being a glasses-type device; an application launch unit that launches a plurality of applications corresponding to the status of the user terminal acquired by the user status acquisition unit; an application selection unit that selects at least one application from the plurality of applications based on information from the user terminal; an information transmission unit that transmits, to the user terminal, AR content for AR streaming that is generated by the application selected by the application selection unit; An information processing device comprising:
2. The information processing device according to claim 1 , wherein the user status acquisition unit acquires a status of the user terminal including position information of the user terminal and attitude information of the user terminal.
3. The information processing apparatus according to claim 1 , wherein the application selection unit selects at least one application from the plurality of applications in accordance with selection information for selecting one of the plurality of applications received from the user terminal.
4. The information processing apparatus according to claim 1 , wherein the application selection unit selects at least one of the applications from the plurality of applications on the basis of user-related information related to a user of the user terminal received from the user terminal.
5. The information processing apparatus according to claim 1 , wherein the application launch unit launches a plurality of applications selected from a plurality of applications corresponding to a status of the user terminal.
6. The information processing device according to claim 5 , wherein the application launch unit launches the plurality of applications selected from the plurality of applications corresponding to a status of the user terminal in descending order of a probability of use by a user of the user terminal.
7. The information processing device according to claim 6, wherein the application launch unit launches the plurality of applications selected from the plurality of applications corresponding to a status of the user terminal in order of the usage probability, within a range of an allocated resource amount allocated to the user terminal.
8. The information processing device according to claim 7 , wherein the application launch unit launches a plurality of selected applications from the plurality of applications corresponding to a status of the user terminal so as to launch as many applications as possible within a range of an allocated resource amount allocated to the user terminal.
9. 6. The information processing device according to claim 5, wherein the application launch unit calculates a usage probability of each of the plurality of applications corresponding to a status of the user terminal based on a location log of the user terminal and a log of application switching by a user of the user terminal, and launches the plurality of applications selected in order of the calculated usage probability.
10. 10. The information processing device according to claim 9, wherein the application launch unit calculates a usage probability of the plurality of applications in a specific geographical area by using a usage history and switching history of applications by the user in the geographical area to increase a usage probability of an application that the user has used more frequently in the past in the geographical area.
11. 10. The information processing device according to claim 9, wherein the application launch unit calculates the usage probability such that a probability of the second app being used in a situation in which the user is using the first app is higher when the user has a history of using a second app in the past while using the first app.
12. 2. The information processing device according to claim 1, wherein the application launch unit, for an application among the plurality of applications corresponding to the status of the user terminal that has a usage probability by the user of the user terminal higher than a first threshold, launches a corresponding VM and launches the application on the VM, and if the usage probability is lower than the first threshold and higher than a second threshold that is lower than the first threshold, launches the VM corresponding to the application and does not launch the application.
13. A program for causing a computer to function as the information processing device according to any one of claims 1 to 12.
14. 1. A computer-implemented information processing method, comprising: A user status acquisition stage for acquiring a status of a user terminal including location information indicating a location of the user terminal, the user terminal being a glasses-type device; an application launch step of launching a plurality of applications corresponding to the status of the user terminal acquired in the user status acquisition step; an application selection step of selecting at least one application from the plurality of applications based on information from the user terminal; an information transmitting step of transmitting AR content for AR streaming, generated by the application selected in the application selecting step, to the user terminal; An information processing method comprising:
Citation Information
Patent Citations
Location based policy system and method for changing computing environment
JP2009181568A
Information processing device, information processing method, and information processing program
JP2019105931A
Content providing system and content providing method
JP2022020039A
Information processor and method for processing information
JP2022071801A
Augmented reality in-application advertisements
US20200219319A1
