Method, apparatus, and program product for deploying visual resources

Abstract

The present disclosure relates to methods, devices and program products for deploying visual resources. In one method, a resource requirement of a visual application for a visual resource in a network system is obtained. Based on the resource requirement, the visual resource to be invoked by the visual application is predicted. Based on processing capabilities of respective edge devices in the network system and the visual resource, an edge device in the network system located in proximity to a terminal device for running the visual application is identified. Based on a time requirement in the resource requirement, the visual resource is deployed to the edge device. Further, corresponding devices and program products are provided. With exemplary implementations of the present disclosure, a visual resource to be used by a visual application can be predicted and deployed to an edge device in proximity to a terminal device in advance. In this way, latency of the terminal device can be reduced and user experience can be improved.

Description

Technical Field

[0001] Implementations of the present disclosure relate to resource management, and more particularly, to methods, devices, and computer program products for deploying visual resources in devices in a network system. Background Art

[0002] With the development of computer technology and network technology, various types of visual services can be provided to terminal devices such as portable computing devices. For example, a large number of virtual reality (VR) applications, augmented reality (AR) applications, and mixed reality applications have been developed. Users can install these visual applications on terminal devices and access a variety of application services provided by the network system. Applications such as VR / AR usually involve a large amount of data, which means that the creation, rendering, and transmission of visual content require a lot of time, especially for applications involving large three-dimensional models. This places higher demands on the computing power, storage capacity, and data transmission capacity of the network system. Summary of the Invention

[0003] Therefore, it is desirable to develop and implement a technical solution for deploying visual resources in a network system in a more efficient manner. It is desirable that the technical solution can deploy visual resources in a more convenient and efficient manner, increase the speed at which visual applications access visual resources, and thus reduce user waiting delays.

[0004] According to a first aspect of the present disclosure, a method for deploying visual resources is provided. In this method, resource requirements of a visual application for the visual resources in a network system are obtained. Based on the resource requirements, the visual resources to be called by the visual application are predicted. Based on the processing capabilities of each edge device in the network system and the visual resources, an edge device located near a terminal device in the network system is identified, and the terminal device is used to run the visual application. Based on the time requirements in the resource requirements, the visual resources are deployed to the edge device.

[0005] According to a second aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; a volatile memory; and a memory coupled to the at least one processor, the memory having instructions stored therein, which, when executed by the at least one processor, cause the device to perform a method according to the first aspect of the present disclosure.

[0006] According to a third aspect of the present disclosure, there is provided a computer program product tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions for executing the method according to the first aspect of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The features, advantages and other aspects of the various implementations of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings, which illustrate several implementations of the present disclosure in an illustrative and non-limiting manner. In the accompanying drawings:

[0008] Figure 1 A block diagram schematically illustrates an application environment in which exemplary implementations of the present disclosure may be implemented;

[0009] Figure 2 A block diagram schematically illustrates a process for deploying visual resources according to an exemplary implementation of the present disclosure;

[0010] Figure 3 Schematically shows a flow chart of a method for deploying visual resources according to an exemplary implementation of the present disclosure;

[0011] Figure 4 A block diagram schematically illustrates different types of deployment of visual resources according to an exemplary implementation of the present disclosure;

[0012] Figure 5 A block diagram schematically illustrates a process for deploying visual resources to various devices in a network system according to an exemplary implementation of the present disclosure;

[0013] Figure 6 A block diagram schematically illustrates a process for migrating visual resources between various devices in a network system according to an exemplary implementation of the present disclosure; and

[0014] Figure 7 A block diagram schematically shows a device for deploying visual resources according to an exemplary implementation of the present disclosure. DETAILED DESCRIPTION

[0015] The following describes preferred implementations of the present disclosure in more detail with reference to the accompanying drawings. Although preferred implementations of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the implementations described herein. Rather, these implementations are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0016] As used herein, the term "including" and its variations represent open inclusion, i.e., "including but not limited to." Unless otherwise stated, the term "or" means "and / or." The term "based on" means "based at least in part on." The terms "an example implementation" and "an implementation" mean "at least one example implementation." The term "another implementation" means "at least one additional implementation." The terms "first," "second," etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0017] For ease of description, first see Figure 1 An application environment according to an exemplary implementation of the present disclosure is described. Figure 1 Schematically illustrates a block diagram of an application environment 100 in which exemplary implementations of the present disclosure may be implemented. Figure 1 As shown, network system 110 may include a core network 130 and an edge network 120. Here, core network 130 may include computing devices 132, ..., and storage devices 134 with higher hardware configurations, and edge network 120 may include edge devices 122, ..., 124 with lower hardware configurations. Terminal device 140 may run various types of applications, and edge devices may process some or all requests from various applications. If the performance of an edge device does not meet the application's requirements, the request may be processed by a device in core network 130.

[0018] The terminal device 140 can run a visual application 150 (e.g., a VR application, an AR application, and a mixed reality application). Compared to ordinary applications, visual applications 150 involve a larger amount of data transmission and have higher requirements for real-time transmission. Therefore, how to deploy the various visual resources involved in visual applications 150 in the network system 110 has become a research hotspot.

[0019] Edge computing-based technical solutions have been proposed. For example, required visual resources can be pre-loaded onto one or more edge devices based on the current terminal device's call, providing the required visual resources to the terminal device. However, because visual applications involve large amounts of data and require a long time to transmit, existing technical solutions cannot guarantee that visual applications will receive visual resources in a timely manner, resulting in delays in visual applications.

[0020] It will be understood that early prediction is crucial for visual applications that require low latency. In order to address the above-mentioned shortcomings, an exemplary implementation of the present disclosure proposes a method for deploying visual resources. Specifically, the concept of a prediction engine is proposed, which can predict events that will occur in the future and require the deployment, update and / or redeployment of visual resources. Utilizing the exemplary implementation of the present disclosure, the required visual resources can be predicted in advance, and the visual resources can be transmitted in a timely and accurate manner, thereby providing a smooth user experience. Here, the prediction engine can work continuously to monitor changes in visual resource demand and network status, thereby further optimizing the overall efficiency of the network system.

[0021] See first Figure 2 An exemplary implementation according to the present disclosure is described. Figure 2 Schematically shows a block diagram of a process 200 for deploying visual resources according to an exemplary implementation of the present disclosure. Figure 2 As shown, resource requirements 210 of the vision application 150 for vision resources in the network system 110 can be obtained (e.g., based on a reservation status 212 and a history status 214 of the vision application 150). Furthermore, a prediction engine 220 can be used to predict vision resources 230 to be called by the vision application 150. Here, the vision resources 230 can include both application programs 232 and scene data 234.

[0022] Furthermore, based on the processing capabilities of each edge device in the network system 110 and the required visual resources 230, a suitable edge device located near the terminal device 140 can be found so as to deploy the visual resources 230 to the edge device based on the time requirement in the resource requirement 210. Figure 2 As shown by arrow 240 in FIG, application 232 can be deployed to edge device 122; and as shown by arrow 242, scenario data 234 can be deployed to edge device 124.

[0023] With the exemplary implementation of the present disclosure, the edge device can have sufficient time to process the transmission of the visual resource 230 and the subsequent processing (e.g., decompression, decoding, instantiation, etc.) of the visual resource 230. In this way, pauses and interruptions of the visual application 150 can be avoided.

[0024] In addition, the prediction engine 220 can provide sufficient time for the capability evaluation and selection of edge devices. The capabilities of various devices in the network system 110 (e.g., bandwidth capability, computing capability, storage capability, etc.) can be comprehensively considered, and all resources of the network system 110 can be aggregated and shared. In this way, the latency of the vision application 150 can be reduced by pre-deploying the vision resources 230 without increasing the hardware facilities of the network system. Figure 3 More details of an exemplary implementation according to the present disclosure are described.

[0025] Figure 3 A flowchart of a method 300 for deploying visual resources according to an exemplary implementation of the present disclosure is schematically shown. At block 310, resource requirements 210 of a visual application 150 for visual resources in a network system 110 are obtained. It will be appreciated that the operation of a visual application 150 may occur in two situations: the visual application 150 is already running on a terminal device 140, and the visual application 150 is not yet running on the terminal device 140. The two situations will be described below.

[0026] According to an exemplary implementation of the present disclosure, the visual application 150 has not yet been started, and it can be determined that the visual application 150 has not yet been run by the terminal device 140. Reservations for visual resources in the network system 110 can be obtained from the network event calendar of the network system 110. It will be understood that the network event calendar can describe various events that will occur in the network system 110. Individual users and / or group users can add appointment requests to the network event calendar to request that a certain visual application be run at a specified time. Assuming that a community is going to hold a VR game competition at 10 am on Saturday, the organizer can add an appointment to the network event calendar. Using the exemplary implementation of the present disclosure, various game-related visual resources can be loaded to the edge device near the access point of the community before 10 o'clock (for example, one day or several hours in advance).

[0027] According to an exemplary implementation of the present disclosure, the time requirement can be determined based on the appointment time (e.g., 10:00 a.m. on Saturday), and the resource requirement 210 of the visual resource can be determined based on the appointment. More details of the event can be further determined based on the appointment, such as the time, season, location, venue size, etc. Linear regression can be used to predict the scale of the event, or a distance function measurement algorithm can be used to find the recent historical records. In this way, basic information about the event can be obtained to determine the name of the visual application, the location and number of access points of a large number of terminal devices that will run the visual application, the required uplink and downlink bandwidth, etc.

[0028] According to an exemplary implementation of the present disclosure, if the visual application 150 has been started, it can be determined that the visual application 150 is already running on the terminal device 140. The relevant time requirements can be determined based on the current time. Further, the historical status of the resources occupied by the visual application 150 during the historical time period can be obtained, and the resource requirements 210 related to this call can be determined based on the historical status. It will be understood that resources can include any one of computing resources, storage resources, and bandwidth resources. For example, the requirements of this call for application and scene data can be determined.

[0029] At block 320, based on the resource requirements 210, the visual resources 230 to be called by the visual application 150 are predicted. According to an exemplary implementation of the present disclosure, the prediction can be performed based on a statistical analysis / machine learning method. The visual resources 230 to be called by the terminal device 140 can be determined based on the prediction model and the historical status. A variety of technologies that have been developed in the past and / or will be developed in the future can be used to train the prediction model so that the prediction model can include the association between the status of the resources occupied by the terminal device and the visual resources to be called by the terminal device. Then, the prediction engine 220 can determine the visual resources 230 required in the future based on the current resource requirements 210 and the prediction model.

[0030] In the example of the VR game competition above, historical data related to the VR game can be obtained to determine the relevant location and user behavior preferences, etc. Further, the next peak data period can be predicted, and ARIMA or long short-term memory networks can be used to predict the upcoming visual resource calls of specific edge devices. For example, the data traffic related to VR games every day in the past 10 days can be used as input to predict the data traffic during the VR game competition. According to an exemplary implementation of the present disclosure, it is even possible to accurately predict the peak data traffic in a specific time period (for example, 10:30 AM-11:00 AM). In the example where the VR game has been started, the changes in resource occupancy can be monitored in real time, and the visual resources 230 that the terminal device 140 needs to call in the present and future can be predicted. It can be determined which visual resources the network system 110 will provide through the prediction engine and based on the resource requirements collected by the perception module.

[0031] According to an exemplary implementation of the present disclosure, reservations and real-time monitoring can be combined. For example, calendar events for upcoming visual applications, status statistics for already launched visual applications, and other detectable data inputs can be monitored for prediction. This detected data can be fused to form a feature vector representing resource demand 210. This feature vector can then be input into a pre-acquired prediction model to determine the visual resources 230 to be called upon.

[0032] Utilizing the exemplary implementations of the present disclosure, the visual resources 230 required by the visual application 150 can be continuously predicted based on the needs of the visual application 150 (e.g., long-term, short-term, and current needs). Subsequently, the visual resources 230 to be called upon in the future can be deployed, updated, or redeployed at various devices in the network system 110. In this way, the terminal device 140 can more intelligently and effectively collaborate with various devices in the network system 110, fully utilizing the computing power, storage capacity, and bandwidth capabilities of each device, thereby providing proactive optimization services and reducing user waiting time.

[0033] In the following, we will refer to Figure 4 More details about the distribution of visual resources 230 are described. Figure 4 A block diagram schematically illustrates different types of deployment 400 of visual resources according to an exemplary implementation of the present disclosure. According to an exemplary implementation of the present disclosure, a visual resource library may be pre-established. Figure 4 As shown, the visual resource library may include applications 232 and scene data 234 that will be called by various visual applications. For example, the visual resources of a VR game may include applications related to computing and rendering called by the VR game, as well as 3D models related to various game scenes loaded by the VR game, as well as related texture, material, and lighting data.

[0034] According to an exemplary implementation of the present disclosure, a deployment engine 410 may be used to deploy various visual resources in the network system 110. For example, the types of deployment may include: new content 412, updated content 414, and load adjustment 416. The deployment engine 410 may first determine to which destination the visual resource 230 needs to be deployed, and then deploy the visual resource 230 to the destination. Figure 3 See blocks 330 and 340 for more details regarding deployment engine 410 .

[0035] exist Figure 3 At box 330, based on the processing capabilities of each edge device and the visual resources 230 in the network system 110, the edge devices in the network system 110 that are located near the terminal device 140 are identified. The terminal device 140 here can run the visual application 150, and the visual application 150 may or may not have been run yet. The visual resources 230 to be called by the visual application 150 can have a variety of virtual network functions. For example, the visual application 150 may include an application for performing the following functions: perception, drawing, and encoding / decoding. At the same time, the various devices in the network system 110 may have different processing capabilities. At this time, a suitable device can be selected as the deployment destination based on both the functions of the visual resources 230 and the processing capabilities of the devices.

[0036] Taking an AR tour guide application for historical sites as an example, the application can sense the geographic location of the terminal device 140 and present the three-dimensional original appearance of the historical site in real time based on changes in the geographic location. For another example, the visual application 150 can include artificial intelligence (AI) and machine learning (ML) libraries to respond to user input and / or location changes based on edge computing technology and provide related services in the virtual world. For another example, for AR applications, the mapping relationship between the real world and the virtual world can be determined based on the AI / ML library, and so on.

[0037] Figure 5 Schematically shows a block diagram of a process 500 for deploying visual resources 230 to various devices in the network system 110 according to an exemplary implementation of the present disclosure. Figure 5 As shown, terminal device 140 can be connected (e.g., via an access point) to edge network 120, and further connected to core network 130. Each edge device 122, ..., 124 can have different processing capabilities. In this case, a suitable deployment destination can be selected from network system 110 based on the type of visual resources and the processing capabilities of each device.

[0038] According to an exemplary implementation of the present disclosure, different types of visual applications have different requirements for functions and performance. For example, strong interactive VR applications require low-latency voice and video encoding services; although weak interactive VR applications do not require extremely low latency, they still have special requirements for transmission rate and packet loss rate. At this time, the resource requirements can be mapped to a resource template corresponding to the visual application, so as to identify the edge device based on the comparison of the processing capabilities of the various edge devices with the resource template. For example, a template can be defined for each visual application, and the actual requirements can be decomposed into aspects such as computing resources, storage resources, and bandwidth resources. For example, a template can be defined in the manner of Table 1 below.

[0039] Table 1 Template examples for visual applications

[0040]

[0041]

[0042] It will be understood that Table 1 above only schematically illustrates the demand templates for basic three-dimensional services. According to an exemplary implementation of the present disclosure, intermediate demand templates, advanced demand templates, and so on can also be provided for visual applications with other requirements. According to an exemplary implementation of the present disclosure, the functions provided by each device in the network system can be matched with the demand templates to determine the most suitable destination for visual resource deployment. According to an exemplary implementation of the present disclosure, edge devices can be identified based on a comparison of the processing capabilities of each edge device with the resource template. Using the exemplary implementation of the present disclosure, the strategy for visual resource deployment can be determined according to the requirements of different types of visual applications.

[0043] According to an exemplary implementation of the present disclosure, at least any one of the priority and the frequency of change associated with the resource demand can be further obtained. Subsequently, the edge device can be identified based on at least any one of the priority and the frequency of change, and the location of the access point of the terminal device to the network system. For example, a higher priority can be set for resource demands with higher real-time requirements so as to provide services for these resource demands in priority. According to an exemplary implementation of the present disclosure, VR / AR resources can have different access frequencies, some content is relatively fixed, and some content is constantly updated with changes in geographical location and time. For frequently accessed VR / AR resources, they can be placed near the terminal device (for example, an edge device close to the terminal device in the edge network) to ensure that the terminal device can quickly obtain these resources. Since the content of these resources is relatively fixed, the terminal device does not need to access these resources frequently, thereby reducing the bandwidth occupied by resource transmission.

[0044] According to an exemplary implementation of the present disclosure, the visual resources 230 can be encapsulated into different data packets. The requirements of each data packet in terms of computing resources, storage resources, and bandwidth resources can be determined. Further, the priority, update frequency, or frequency of interaction with the user related to the content in such data packets can be determined, and then a suitable destination can be selected for each data packet. Generally speaking, the closer the network distance between the visual resource 230 and the terminal device 140, the shorter the time the terminal device 140 takes to access the visual resource 230, and thus the visual resource 230 is preferentially sent to the edge device closest to the terminal device 140. Priority can be set for each data packet separately to make full use of the various storage devices in the network system 110 to minimize latency, reduce costs, and improve end-user satisfaction with the virtual experience.

[0045] According to an exemplary implementation of the present disclosure, the bandwidth requirements of the visual application 150 can be obtained in real time. If it is determined that a large-scale event will occur, it can be checked whether the current edge device can meet the bandwidth requirements. The visual application 150 can request bandwidth allocation from the current edge device through the bandwidth management (BWM) API. Specifically, the visual application 150 can use the BWM API to register the required bandwidth allocation in a static and / or dynamic manner. If the edge device does not have sufficient available bandwidth resources, the bandwidth management service can inform the visual application 150 that the edge device cannot support the required bandwidth request. Subsequently, the visual application 150 can request bandwidth resources from other edge devices in the surrounding area.

[0046] The visual application 150 can select which edge device to perform mobile edge computing through the Edge Network Information Service (ENIS). The ENIS service can provide a list of the computing capabilities of each edge device, so that the visual application 150 can obtain the configuration and status of each edge device. Specifically, a suitable edge device can be selected based on a combination of the following conditions: the distance between the edge device and the adjacent edge device, the available bandwidth resources, the hardware configuration (including CPU, memory resources, etc.), and whether it has a special basic computing accelerator (such as GPU, TPU or codec accelerator, etc.).

[0047] exist Figure 3 At block 340, based on the time requirement in the resource requirement, the visual resources 230 are deployed to the edge devices identified at block 330. Generally speaking, the visual resources placed on the closest edge devices require low latency, and different visual applications may have different priorities. For visual applications with relatively low latency requirements, the visual resources involved may be moved or copied to a terminal device that is slightly farther away from the terminal device 140 (e.g., Figure 5 Edge device 124 is shown. In this case, the edge device closest to terminal device 140 (e.g., edge device 122) can be assigned to vision applications that require extremely low latency, thereby ensuring low latency and high throughput for the entire network system. For example, AI / ML applications can be deployed on devices with higher computing power. For another example, vision applications and high-performance rendering applications with very strict time requirements can be deployed on edge device 122 close to terminal device 140, thereby reducing network latency.

[0048] like Figure 5 As shown, visual resources 230 can be deployed at edge devices in edge network 120 and various devices in core network 130. Figure 5As shown, edge device 122 is the device closest to terminal device 140 and may include VR applications 510, AR applications 512, AI / ML applications 514, frequently changing data 516, and rendering applications 518. Edge device 124 is a neighboring device of edge device 122 and may include AI / ML applications 520 that are less sensitive to latency, applications related to codecs 522, and the like. Furthermore, computing device 132 in core network 130 may include an AI / ML engine 530 with high computing performance requirements, and storage device 134 may include less frequently changing data 540, and so on.

[0049] According to an exemplary implementation of the present disclosure, a list of visual resources can be created to manage all visual resources associated with the visual application 150. When the visual application 150 needs to call a certain visual resource, it can search the list to find the required visual resource from the corresponding device. It will be understood that the contents of the list will change over time. Therefore, the list can be updated when the deployment of visual resources changes.

[0050] The above only describes the process of deploying visual resources 230 using the newly created content 412 type as an example. According to an exemplary implementation of the present disclosure, deployment may also involve updating content 414. Here, updating content 414 refers to the need to redeploy visual resources in order to present changed content to the user during the operation of the visual application 150. For example, when a VR game user moves to a new geographical location in the virtual world, the surrounding scene data needs to be supplemented or updated. The edge device associated with the visual resource to be updated can be determined in the manner described above, and then the updated visual resource is deployed to the determined edge device. For another example, a user of an AR guide application can move forward in the real world, which will cause the user's position in the virtual world to change. Therefore, it is necessary to provide new visual resources to the visual application based on the changed position. For example, there are ruins of a building in front of the user. When the user moves towards the ruins, a three-dimensional restoration model of the building can be displayed at a position corresponding to the ruins in the virtual world.

[0051] According to an exemplary implementation of the present disclosure, the developer of the visual application 150 may continuously update the visual resources involved in the visual application 150. For example, as a VR game version is updated, the game may include new game scenes. The updated visual resources can be redeployed to selected edge devices. Alternatively and / or additionally, an appropriate edge device can be selected for the new game scene.

[0052] It will be appreciated that edge devices can have limitations in terms of computational resources, storage resources, and bandwidth, and thus the deployment of those visual resources to edge devices can be determined based on the capability detection of the edge devices. The destination that is most suitable for deployment of visual resources can be determined based on the capabilities of the individual edge devices. In particular, during the execution of visual applications 150, the workloads of the individual devices in the network system 110 can change constantly, resulting in a load adjustment 416 type of deployment. Load adjustment 416 refers to the need for redeployment of visual resources when the workload (e.g., including any of computational resources, storage resources, bandwidth resources) of a certain device is about to exceed a safety threshold. For example, when the number of game players is increasing and causing the workload of a certain device to exceed a safety threshold, the service provided to a portion of the players can be "offloaded" to another device in the network system. With the example implementations of the present disclosure, the deployment of visual resources can be adjusted constantly through real-time monitoring and prediction.

[0053] Figure 6 A block diagram of a process 600 for migrating visual resources between individual devices in a network system according to an example implementation of the present disclosure is schematically shown. As shown, when the workload of edge device 122 is found to be too high and not suitable for rendering 518 applications, the rendering applications 516 can be migrated from edge device 122 to edge device 124 with lower workload, as shown by arrow 630. In this way, edge device 124 can provide more abundant computational resources to support real-time rendering of rendering applications 518. Figure 6

[0054] According to one example implementation of the present disclosure, the deployment of visual resources can be retained or terminated according to predefined rules. For example, the visual resources to be invoked can be deployed 230 to the corresponding devices before the scheduled visual applications 150 are started. When the visual applications 150 have been started, the status of the visual applications 150 can be continuously monitored to predict the visual resources to be invoked by the visual applications based on the historical status that has occurred, and the loading process can be performed accordingly. According to one example implementation of the present disclosure, the monitoring of the status of the visual applications can be stopped.

[0055] The above has been described with reference to Figures 2 to 6 ​A process of deploying visual resources 230 in the network system 110 is described. In the following, the process of deploying visual resources 230 is described in general with examples of a VR game tournament and an AR tour application, respectively. In one example, a VR game tournament is expected to be held at 10:00 AM on Saturday, an organizer can first make a reservation in a calendar. For example, the name of the specific VR game involved in the game tournament, the venue, the number of participants to be scheduled, the bandwidth, and the like can be specified. The prediction engine 220 can receive the subscription calendar and prepare for the game tournament on Saturday based on the reservation event in the calendar. For example, the prediction engine 220 can count the bandwidth, computing resource, and storage resource that each user can occupy based on the historical data related to the VR game, and in turn pre-deploy the corresponding visual resources for the game tournament. Further, one or more edge devices near the venue can be determined in advance (e.g., 1 day in advance) according to various resource requirements, and the corresponding visual resources are deployed to these edge devices.

[0056] According to one example implementation of the present disclosure, various edge devices can be ranked according to the amount of resources that each edge device can provide. A suitable edge device can be selected according to both the distance between each edge device and the access point of the venue and the amount of resources. At this time, the visual resources required for the game tournament will be pre-stored in these edge devices, and the requirements of the corresponding bandwidth, computing resource, and storage resource are met. Further, since there can be a large number of concurrent data accesses during the game tournament, the visual resources can be cached at multiple edge devices in the edge network. The data packets of the visual resources can be divided and deployed according to the access frequency of the visual resources, and the data packets that are frequently accessed can be deployed at the edge device closest to the access point.

[0057] The rendering application can be deployed at the edge device closest to the access point as much as possible in order to maximize the advantage of low latency and high throughput. The visual resources involving high computing power requirements can be deployed at devices with better hardware configurations (e.g., edge devices with higher configurations in the edge network or computing devices in the core network) as much as possible. The visual resources involving data uploading and sharing can be deployed at locations close to the access point to avoid network congestion and bandwidth bottlenecks.

[0058] In another example, a user can use an AR tour application to tour a historical monument. The user can use the terminal device 140 to take a picture of the scene in front (e.g., historical relics) in real time, at which time the AR application will superimpose a three-dimensional restoration model of the historical relics corresponding to the current location in the display of the terminal device. The tour route can be pre-set from the AR application, and the visual resources are pre-deployed before the user actually enters each area.

[0059] Using the exemplary implementation of the present disclosure, the long-term, short-term, and current needs of the vision application 150 can be continuously monitored to continuously predict the vision resources 230 required by the vision application 150. In this way, the required resources can be pre-loaded to a storage location that is easily accessible to the terminal device 140, thereby reducing the delay caused by data transmission.

[0060] See above for Figures 2 to 6 An example of the method according to the present disclosure is described in detail, and the implementation of the corresponding device will be described below. According to the exemplary implementation of the present disclosure, a device for deploying visual resources is provided. The device includes: obtaining the resource requirements of the visual application for the visual resources in the network system; based on the resource requirements, predicting the visual resources to be called by the visual application; based on the processing capabilities of each edge device in the network system and the visual resources, identifying the edge device in the network system that is located near the terminal device, the terminal device is used to run the visual application; and based on the time requirements in the resource requirements, deploying the visual resources to the edge device. According to the exemplary implementation of the present disclosure, the device further includes a module for executing the other steps in the method 300 described above.

[0061] Figure 7 Schematically illustrates a block diagram of a device 700 for managing data patterns according to an exemplary implementation of the present disclosure. As shown, the device 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 702 or loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0062] Various components in device 700 are connected to I / O interface 705, including an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, optical disk, etc.; and a communication unit 709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0063] The various processes and procedures described above, such as method 300, may be performed by processing unit 701. For example, in some implementations, method 300 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 708. In some implementations, part or all of the computer program may be loaded and / or installed onto device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by CPU 701, one or more steps of method 300 described above may be performed. Alternatively, in other implementations, CPU 701 may be configured in any other suitable manner to implement the aforementioned processes / methods.

[0064] According to an exemplary implementation of the present disclosure, an electronic device is provided, comprising: at least one processor; a volatile memory; and a memory coupled to the at least one processor, the memory having instructions stored therein, which, when executed by the at least one processor, cause the device to perform a method for deploying visual resources. The method comprises: obtaining resource requirements of a visual application for visual resources in a network system; predicting visual resources to be called by the visual application based on the resource requirements; identifying edge devices in the network system that are located near a terminal device, based on the processing capabilities and visual resources of each edge device in the network system, the terminal device being used to run the visual application; and deploying the visual resources to the edge devices based on the time requirements in the resource requirements.

[0065] According to an exemplary implementation of the present disclosure, obtaining resource requirements includes: in response to determining that the visual application has not yet been run by the terminal device, obtaining a reservation for visual resources in the network system from a network event calendar of the network system; determining a time requirement for the resource requirements based on the time of the reservation; and determining the resource requirement for the visual resources based on the reservation.

[0066] According to an exemplary implementation of the present disclosure, obtaining resource requirements includes: in response to determining that the visual application has been run by the terminal device, determining the time requirement of the resource requirement based on the current time; obtaining the historical status of resources occupied by the visual application during the historical time period, the resources including any one of computing resources, storage resources and bandwidth resources; and determining the resource requirement based on the historical status.

[0067] According to an exemplary implementation of the present disclosure, predicting visual resources to be called by a visual application includes: determining the visual resources to be called by a terminal device based on a prediction model and a historical status, the prediction model including an association relationship between the status of resources occupied by the terminal device and the visual resources to be called by the terminal device.

[0068] According to one example implementation of the disclosure, identifying the edge device further includes obtaining at least one of a priority and a frequency of change associated with the resource requirement, and identifying the edge device based on at least one of the priority and the frequency of change and a location of an access point of the network system to which the terminal device accesses.

[0069] According to one example implementation of the disclosure, the method further includes updating the edge device based on a change in the location of the access point.

[0070] According to one example implementation of the disclosure, identifying the edge device includes mapping the resource requirement to a resource template corresponding to the visual application, and identifying the edge device based on a comparison of a processing capability of each edge device with the resource template.

[0071] According to one example implementation of the disclosure, the method further includes redeploying the updated visual resource to the edge device in response to determining that the visual resource is updated.

[0072] According to one example implementation of the disclosure, the visual resource includes at least one of an application to be invoked by the visual application and scene data to be loaded by the visual application.

[0073] According to one example implementation of the disclosure, the visual application includes at least one of a virtual reality application, an augmented reality application, and a mixed reality application.

[0074] According to an example implementation of the disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-transitory computer readable medium and includes machine executable instructions for performing a method according to the disclosure.

[0075] According to an example implementation of the disclosure, a computer readable medium is provided. The computer readable medium has stored thereon machine executable instructions, which when executed by at least one processor, cause the at least one processor to implement a method according to the disclosure.

[0076] The disclosure can be a method, an apparatus, a system, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for

[0077] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0078] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0079] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some implementations, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, which may be executed by the computer-readable program instructions to implement various aspects of the present disclosure.

[0080] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0081] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0082] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0083] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0084] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the implementations disclosed herein.

Claims

1. A method for deploying visual resources, comprising: Obtaining resource requirements of a visual application for the visual resources in a network system; Based on the resource requirements, predicting the vision resources to be used by the vision application; identifying an edge device in the network system that is located near a terminal device based on processing capabilities of each edge device in the network system and the visual resources, the terminal device being used to run the visual application; as well as deploying the visual resource to the edge device based on the time requirement in the resource requirement, Wherein identifying the edge device further comprises: obtaining at least any one of a priority and a change frequency associated with the resource demand; as well as The edge device is identified based on at least any one of the priority and the change frequency, and a location of an access point through which the terminal device accesses the network system.

2. The method according to claim 1, wherein obtaining the resource requirement comprises: In response to determining that the visual application has not been executed by the terminal device, Obtaining, from a network event calendar of the network system, a reservation for the visual resource in the network system; determining the time requirement of the resource requirement based on the scheduled time; as well as The resource requirement of the visual resource is determined based on the reservation.

3. The method according to claim 1, wherein obtaining the resource requirement comprises: In response to determining that the visual application is already being executed by the terminal device, determining the time requirement of the resource requirement based on a current time; Obtaining a historical status of resources occupied by the visual application during a historical time period, wherein the resources include any one of computing resources, storage resources, and bandwidth resources; and The resource requirement is determined based on the historical status.

4. The method of claim 3, wherein predicting the vision resource to be called by the vision application comprises: The visual resource to be called by the terminal device is determined based on a prediction model and the historical status, wherein the prediction model includes an association relationship between the status of resources occupied by the terminal device and the visual resource to be called by the terminal device.

5. The method according to claim 1, further comprising: The edge device is updated based on the change in the location of the access point.

6. The method of claim 1 , wherein identifying the edge device further comprises: mapping the resource requirements to a resource template corresponding to the vision application; as well as The edge devices are identified based on a comparison of the processing capabilities of the respective edge devices with the resource template.

7. The method according to claim 1, further comprising: In response to determining that the visual resource is updated, the updated visual resource is redeployed to the edge device. 8 . The method according to claim 1 , wherein the visual resource comprises at least any one of the following: an application program to be called by the visual application and scene data to be loaded by the visual application.

9. The method according to claim 8, wherein the visual application comprises at least any one of the following: a virtual reality application, an augmented reality application, and a mixed reality application.

10. An electronic device comprising: at least one processor; Volatile memory; as well as a memory coupled to the at least one processor, the memory having instructions stored therein, the instructions, when executed by the at least one processor, causing the apparatus to perform a method for deploying visual resources, the method comprising: Obtaining resource requirements of a visual application for the visual resources in a network system; Based on the resource requirements, predicting the vision resources to be used by the vision application; Identifying edge devices in the network system that are located near a terminal device based on processing capabilities of each edge device in the network system and the vision resource, the terminal device being used to run the vision application; and deploying the visual resource to the edge device based on the time requirement in the resource requirement, Wherein identifying the edge device further comprises: obtaining at least any one of a priority and a change frequency associated with the resource demand; and The edge device is identified based on at least any one of the priority and the change frequency, and a location of an access point through which the terminal device accesses the network system.

11. The apparatus according to claim 10, wherein obtaining the resource requirement comprises: In response to determining that the visual application has not been executed by the terminal device, Obtaining, from a network event calendar of the network system, a reservation for the visual resource in the network system; determining the time requirement of the resource requirement based on the scheduled time; as well as The resource requirement of the visual resource is determined based on the reservation.

12. The apparatus of claim 10, wherein obtaining the resource requirement comprises: In response to determining that the visual application is already being executed by the terminal device, determining the time requirement of the resource requirement based on a current time; Obtaining a historical status of resources occupied by the visual application during a historical time period, wherein the resources include any one of computing resources, storage resources, and bandwidth resources; and The resource requirement is determined based on the historical status.

13. The apparatus of claim 12, wherein predicting the visual resource to be called by the visual application comprises: The visual resource to be called by the terminal device is determined based on a prediction model and the historical status, wherein the prediction model includes an association relationship between the status of resources occupied by the terminal device and the visual resource to be called by the terminal device.

14. The apparatus of claim 10, wherein the method further comprises: The edge device is updated based on the change in the location of the access point.

15. The device of claim 10, wherein identifying the edge device further comprises: mapping the resource requirements to a resource template corresponding to the vision application; as well as The edge devices are identified based on a comparison of the processing capabilities of the respective edge devices with the resource template.

16. The apparatus of claim 10, further comprising: In response to determining that the visual resource is updated, the updated visual resource is redeployed to the edge device.

17. The device according to claim 10, wherein the visual resources include at least any one of the following: an application to be called by the visual application and scene data to be loaded by the visual application, and wherein the visual application includes at least any one of the following: a virtual reality application, an augmented reality application, and a mixed reality application.

18. A computer program product tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions for performing the method according to any one of claims 1 to 9.

Images