End-cloud cooperative work method, related device and communication system

Abstract

The invention discloses an end-cloud cooperative work method, a related device and a communication system. And the end-side equipment and the cloud-side equipment cooperatively process the service so as to cooperate to meet the service requirements of the user. According to the scheme, partial or all services are moved to the cloud, so that not only can stronger and higher-quality services be provided, but also the computing power and energy consumption of the end-side equipment can be saved, the endurance time of the end-side equipment can be prolonged, and better terminal use experience can be provided for users.

Description

Technical Field

[0001] The present application relates to the field of terminals and cloud computing, and in particular to methods, related devices and communication systems for end-cloud collaboration. Background Art

[0002] With the continuous improvement of chip computing power, mobile phones and other electronic devices can provide users with a better user experience, such as smoother, higher quality, more types of dynamic effects, richer colors, etc. However, the improvement of computing power mainly depends on the iterative upgrade of the process. Moore's Law has slowed down, and the speed of edge computing power improvement has also slowed down. In addition, there is still competition between backward process N and advanced process N+2 in the chip field. Users' pursuit of user experience is constantly improving. How to meet user needs under limited or certain edge computing power is a direction worth studying. Summary of the invention

[0003] The present application provides a method, related devices and communication system for end-cloud collaborative work, which can provide users with high-quality services, save the computing power and energy consumption of end-side devices, and extend the battery life of end-side devices.

[0004] In a first aspect, a communication system is provided, which includes a first device and a first node, wherein the first device is used to start a first application and send first data to the first node, the first data including control data and / or first business data of the first application; the first node is used to allocate a first runtime to the first device, start the first application through the first runtime, receive the first data through the first application, generate second data based on the first data through the first application, and return the second data to the first device through the first application, the second data including second business data of the first application; the first device is also used to provide services based on the second data.

[0005] By implementing the communication system provided in the first aspect, the terminal-side device moves part or all of the services to the cloud-side node, and the cloud-side node allocates a dedicated runtime to the terminal-side device to execute the part or all of the services. This can not only provide users with stronger and higher-quality services, but also save the computing power and energy consumption of the terminal-side device, extend the battery life of the terminal-side device, and provide users with a better terminal usage experience.

[0006] In conjunction with the first aspect, in some implementations, the first device is further used to, after starting the first application, send a first notification message to the first node, the first notification message including an identifier of the first application; the first node is specifically used to, after receiving the first notification message, start the first application through the first runtime. In this way, the cloud-side node starts the first application in advance, and when the end-side device has a need for collaboration during the running of the first application, the cloud-side node can quickly respond to its needs by starting the first application.

[0007] In combination with the first aspect, in some implementations, the control data includes a search instruction, and the second service data includes a result searched by the first node according to the search instruction;

[0008] Alternatively, the first service data includes an original image captured by the first device, and the second service data includes an image obtained after the first node processes the original image;

[0009] Alternatively, the control data includes the game operation instructions of the user, and the second business data includes the game screen rendered by the first node according to the game operation instructions.

[0010] In combination with the first aspect, in some embodiments, the communication system further includes a second device, the second device is used to start the second application, and send third data to the first node, the third data includes control data and / or third business data of the second application; the first node is also used to allocate a second runtime to the second device, start the second application through the second runtime, receive the third data through the second application, generate fourth data based on the third data, and return the fourth data to the second device, the fourth data includes fourth business data of the second application, the second runtime is different from the first runtime; the second device is also used to provide services based on the fourth data. That is, the same cloud-side node can provide collaborative services for multiple end-side devices, improving the terminal usage experience of multiple users.

[0011] In combination with the first aspect, in some embodiments, the communication system also includes a manufacturer cloud server, and the first device is further used to log in to the manufacturer cloud server before starting the first application; the manufacturer cloud server is used to assign a first node among multiple nodes to the first device.

[0012] In some embodiments, the first node meets any of the following conditions:

[0013] Among the multiple nodes, the first node is closest to the first device;

[0014] The multiple nodes include nodes deployed in the access network, and among the nodes deployed in the access network, the first node is closest to the first device;

[0015] The multiple nodes do not include nodes deployed in the access network, and include nodes deployed in the bearer network, and among the nodes deployed in the bearer network, the first node is closest to the first device;

[0016] The multiple nodes do not include nodes deployed on the access network or the bearer network, and include centrally deployed nodes, and among the centrally deployed nodes, the first node is closest to the first device;

[0017] The distance between the first node and the first device is within a first value.

[0018] In some embodiments, the communication system further includes a second node, and the manufacturer's cloud server is further used to allocate a second node among the multiple nodes to the first device after the first device moves; the second node is used to allocate a third runtime to the first device; and the first node is further used to release the first runtime allocated to the first device. After the end-side device moves, by reallocating the cloud computing nodes, it is possible to better provide collaborative services for the end-side device, and the initial cloud computing node releases the resources allocated to the end-side device, which can save resources in the initial cloud computing node and avoid resource waste.

[0019] In combination with the first aspect, in some implementations, after the first device moves, the distance between the first device and the second node is smaller than the distance between the first device and the first node, which is equivalent to allocating a cloud computing node closer to the end-side device to the end-side device, so as to better provide collaborative services for the end-side device.

[0020] In combination with the first aspect, in some embodiments, the first node is further used to, after allocating the first runtime to the first device, install some or all of the applications installed on the first device on the first runtime, and some or all of the applications include the first application.

[0021] In conjunction with the first aspect, in some implementations, the applications installed in the first device include applications installed in the first runtime of the first node. In some implementations, the applications installed in the first device may be more than the applications installed in the first runtime of the first node.

[0022] In conjunction with the first aspect, in some implementations, the communication system further includes: a cloud server providing services for the first application; the first node is specifically configured to, after receiving the first data through the first application and before returning the second data to the first device, communicate with the cloud server through the first application running in the first runtime to generate the second data based on the first data. That is, the service provided by the terminal side device also requires support from the cloud server on the network side.

[0023] In combination with the previous implementation, in some implementations, the cloud server is further used to periodically send a heartbeat message to the first node, and the heartbeat message is used to maintain the communication connection between the cloud server and the first node. In this way, the cloud computing node is responsible for the heartbeat connection with the cloud server, while the heartbeat connection with the terminal side is reduced, which does not affect the service experience, but also saves the power of the terminal side and saves the air interface resource occupation.

[0024] In combination with the first aspect, in some implementations, the first runtime includes any one or more of the following: a basic environment for running application code, and a runtime library.

[0025] In combination with the first aspect, in some implementations, the first application in the first device and the first application in the first node include different processing logics.

[0026] In combination with the first aspect, in some implementations, when the first device includes a fourth runtime, the first device is specifically configured to launch the first application through the fourth runtime.

[0027] In the second aspect, a method for end-cloud collaboration is provided, which is applied to a first device. The method may include: sending the identifiers of some or all of the installed applications to the first node, where some or all of the applications include the first application, and the identifiers of some or all of the applications are used by the first node to install some or all of the applications on the first runtime allocated to the first device; starting the first application; sending first data to the first node, where the first data includes control data and / or first business data of the first application; receiving second data returned by the first node, where the second data is generated by the first node based on the first data through the first application running on the first node, and the second data includes the second business data of the first application; and providing services based on the second data.

[0028] In combination with the second aspect, in some embodiments, after starting the first application, the method further includes: sending a first notification message to the first node, the first notification message including an identifier of the first application, and the identifier of the first application is used by the first node to start the first application through the first runtime.

[0029] In conjunction with the second aspect, in some implementations, the control data includes a search instruction, and the second service data includes a result searched by the first node according to the search instruction;

[0030] Alternatively, the first service data includes an original image captured by the first device, and the second service data includes an image obtained after the first node processes the original image;

[0031] Alternatively, the control data includes the game operation instructions of the user, and the second business data includes the game screen rendered by the first node according to the game operation instructions.

[0032] In combination with the second aspect, in some embodiments, before starting the first application, the method also includes: logging in to the manufacturer's cloud server; receiving allocation information sent by the manufacturer's cloud server, where the allocation information is used to indicate that the first node is allocated to the first device.

[0033] In some embodiments, the first node meets any of the following conditions:

[0034] Among the multiple nodes, the first node is closest to the first device;

[0035] The multiple nodes include nodes deployed in the access network, and among the nodes deployed in the access network, the first node is closest to the first device;

[0036] The multiple nodes do not include nodes deployed in the access network, and include nodes deployed in the bearer network, and among the nodes deployed in the bearer network, the first node is closest to the first device;

[0037] The multiple nodes do not include nodes deployed on the access network or the bearer network, and include centrally deployed nodes, and among the centrally deployed nodes, the first node is closest to the first device;

[0038] The distance between the first node and the first device is within a first value.

[0039] In some implementations, the method further includes: after the position is moved, receiving new allocation information sent by the manufacturer's cloud server, where the new allocation information is used to indicate that the second node is allocated to the first device.

[0040] In some implementations, after the first device moves, the distance between the first device and the second node is smaller than the distance between the first device and the first node.

[0041] In combination with the second aspect, in some implementations, the second data is specifically generated by the first node after the first application running on the first runtime communicates with a cloud server, and the cloud server is used to provide services for the first application.

[0042] In conjunction with the second aspect, in some implementations, the application installed in the first device includes an application installed in the first runtime of the first node.

[0043] In combination with the second aspect, in some implementations, the first runtime includes any one or more of the following: a basic environment for running application code, and a runtime library.

[0044] In conjunction with the second aspect, in some implementations, the first application in the first device and the first application in the first node include different processing logics.

[0045] In combination with the second aspect, in some implementations, when the first device includes a fourth runtime, the first device is specifically configured to launch the first application through the fourth runtime.

[0046] In a third aspect, a method for end-cloud collaboration is provided, which is applied to a first node. The method may include: allocating a first runtime to a first device; starting a first application through the first runtime; receiving first data sent by the first device through the first application, the first data including control data and / or first business data of the first application in the first device; generating second data based on the first data through the first application, the second data including second business data of the first application; and returning the second data to the first device through the first application.

[0047] In combination with the third aspect, in some implementations, before starting the first application through the first runtime, it also includes: receiving a first notification message sent by the first node after starting the first application, the first notification message including an identifier of the first application.

[0048] In combination with the third aspect, in some embodiments, the method also includes: allocating a second runtime to the second device, the second runtime being different from the first runtime; starting a second application through the second runtime; receiving third data sent by the second device through the second application, the third data including control data and / or third business data of the second application in the second device; generating fourth data based on the third data, the fourth data including fourth business data of the second application; and returning the fourth data to the second device.

[0049] In combination with the third aspect, in some embodiments, before allocating the first runtime to the first device, the method further includes: receiving allocation information sent by the manufacturer's cloud server, where the allocation information is used to indicate that the first node is allocated to the first device.

[0050] In some embodiments, the first node meets any of the following conditions:

[0051] Among the multiple nodes, the first node is closest to the first device;

[0052] The multiple nodes include nodes deployed in the access network, and among the nodes deployed in the access network, the first node is closest to the first device;

[0053] The multiple nodes do not include nodes deployed in the access network, and include nodes deployed in the bearer network, and among the nodes deployed in the bearer network, the first node is closest to the first device;

[0054] The multiple nodes do not include nodes deployed on the access network or the bearer network, and include centrally deployed nodes, and among the centrally deployed nodes, the first node is closest to the first device;

[0055] The distance between the first node and the first device is within a first value.

[0056] In combination with the third aspect, in some embodiments, the method also includes: when receiving a second notification message sent by the manufacturer's cloud server, the second notification message is used to instruct the first device to release the first runtime allocated to the first device, and the second notification message is sent by the manufacturer's cloud server after knowing the mobile location of the first device; releasing the first runtime allocated to the first device.

[0057] In combination with the third aspect, in some embodiments, after allocating the first runtime to the first device, the method further includes: installing some or all of the applications installed on the first device on the first runtime, some or all of the applications including the first application.

[0058] In combination with the third aspect, in some implementations, a first application running in a first runtime communicates with a cloud server to generate second data based on first data, and the cloud server is used to provide services for the first application.

[0059] In some implementations, a heartbeat message sent by the cloud server is periodically received, and the heartbeat message is used to maintain a communication connection between the cloud server and the first node.

[0060] In combination with the third aspect, in some implementations, the first runtime includes any one or more of the following: a basic environment for running application code, and a runtime library.

[0061] In conjunction with the third aspect, in some implementations, the first application in the first node and the first application in the first device include different processing logics.

[0062] In a fourth aspect, a method for end-cloud collaboration is provided, which is applied to a communication system including a first device and a first node. The method may include: the first device starts a first application and sends first data to the first node, the first data including control data and / or first business data of the first application; the first node allocates a first runtime to the first device, starts the first application through the first runtime, receives the first data through the first application, generates second data based on the first data through the first application, and returns the second data to the first device through the first application, the second data including the second business data of the first application; the first device provides services based on the second data.

[0063] The optional implementation manner of the fourth aspect may refer to the above-mentioned second aspect and the optional implementation manner of the second aspect, which will not be repeated here.

[0064] In a fifth aspect, an electronic device is provided, comprising: a memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute a method executed by the electronic device in the second aspect or any one of the embodiments of the second aspect.

[0065] In a sixth aspect, an electronic device is provided, comprising: a memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute a method executed by the electronic device in the third aspect or any one of the embodiments of the third aspect.

[0066] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute a method executed by the electronic device in the second aspect or any one of the embodiments of the second aspect.

[0067] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute a method executed by the electronic device in the third aspect or any one of the embodiments of the third aspect.

[0068] In a ninth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method executed by the electronic device in the second aspect or any one of the embodiments of the second aspect.

[0069] In the tenth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method executed by the electronic device in the third aspect or any one of the embodiments of the third aspect.

[0070] In the eleventh aspect, an embodiment of the present application provides a chip system, which includes at least one processor for implementing the method executed by the electronic device in the second aspect or any one of the embodiments of the second aspect.

[0071] In the twelfth aspect, an embodiment of the present application provides a chip system, which includes at least one processor for implementing the method executed by the electronic device in the third aspect or any one of the embodiments of the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A schematic diagram of the structure of a communication system 10 provided in an embodiment of the present application;

[0073] Figure 2 A schematic diagram of the process flow of the end-cloud collaborative working method provided in the embodiment of the present application;

[0074] Figure 3 A schematic diagram of the installation status when running on multiple devices provided by an embodiment of the present application;

[0075] Figure 4 An image comparison diagram provided for the embodiments of the present application;

[0076] Figure 5 A hardware structure block diagram of the terminal device 100 provided in an embodiment of the present application;

[0077] Figure 6 The software architecture of the terminal device 100 provided in the embodiment of the present application;

[0078] Figure 7 A hardware structure block diagram of a cloud computing node 200 provided in an embodiment of the present application;

[0079] Figure 8 This is a hardware structure block diagram of the vendor cloud server 400 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0080] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0081] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0082] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java, extensible markup language (XML), JS (JavaScript), TS / eTS (Type Script / enhanceType Script), C / C++, etc. The interface source code is parsed and rendered on the electronic device, and finally presented as content that the user can recognize. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icon, button, menu, tab, text box, dialog box, status bar, navigation bar, widget, etc. displayed on the display screen of an electronic device.

[0083] The combination of end and cloud for business processing is a solution that can well meet the business needs of users. For example, a cloud-end collaborative service platform 2 connected to the cloud server 1 and the user terminal 3 can be set up, and the cloud-end collaborative service platform 2 has strong computing, analysis and processing capabilities. The cloud-end collaborative service platform 2 can monitor the user terminal 3 in real time. If the user operation exceeds the processing capacity of the user terminal 3, such as large-scale computing, graphics rendering, etc., the cloud-end collaborative service platform 2 can migrate the computing task to the cloud server 1, use the powerful computing power of the cloud server 1 to perform the calculation, and then return the calculation result to the user terminal 3.

[0084] The above-mentioned end-cloud combined business processing solution needs to rely on the cloud-end collaborative service platform 2 to monitor the business processing status in the user terminal 3, and is responsible for task migration and result return between the end-cloud. This solution requires coordination among multiple parties and the execution process is relatively complicated.

[0085] The present application provides a method, related apparatus and communication system for end-cloud collaboration. In this method, the end-side device and the cloud-side device work together to jointly process services and collaborate to meet the business needs of users. The cloud-side device can be regarded as a shadow node or external expansion device of the end-side device, which is used to undertake part or all of the services originally performed on the end-side device. It is equivalent to that the end-side device and the cloud-side device form a "new terminal" for end-cloud collaborative processing of services, and jointly provide services to users. The services provided to users depend on user needs, for example, may include but are not limited to games, image processing, video processing, search services, application startup animations, transition animations, etc.

[0086] For example, after the camera application of the end-side device captures the original image through the camera, it processes the photo based on its own computing power (which may include simple processing such as noise reduction, cropping, and sharpening) to generate a photo; then, the end-side device uploads the photo / photo original data to the cloud-side device, and the cloud-side device uses the strong computing power of the cloud-side device to further refine the photo / original data (which may include adjusting the facial features of the person in the photo, changing the background light and shadow, adding background, and other complex processing), and then returns the refined photo to the end-side device. In this way, the end-side device can use the computing power of the cloud-side device to refine the photo and provide users with better quality photos, or even high-quality photos that the end-side cannot provide due to insufficient computing resources.

[0087] Before introducing in detail the end-cloud collaborative working method provided by the present application, the communication system provided by an embodiment of the present application is first introduced.

[0088] Figure 1 The communication system 10 provided in the present application is provided as an example.

[0089] like Figure 1As shown, the communication system 10 may include: one or more end-side devices 100, one or more cloud computing nodes 200, one or more cloud servers (service providers, SP) 300 for providing content, and one or more vendor cloud servers 400.

[0090] The end-side device 100 is a smart terminal device, which can be of various types. The embodiments of the present application do not limit its specific type. For example, the end-side device 100 can be a mobile phone, and can also include a tablet computer, a desktop computer, a desktop computer with a touch-sensitive surface or a touch panel, a laptop computer (laptop), a handheld computer, a notebook computer, a smart screen, a wearable device (such as a smart watch, a smart bracelet, etc.), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a car machine, a smart headset, a game console, and can also be an Internet of Things (IOT) device or a smart home device such as a smart water heater, a smart lamp, a smart air conditioner, etc.

[0091] The terminal device 100 is used to directly interact with the user, thereby providing the user with the services he needs. For example, the terminal can display game screens, processed pictures, videos, search results, application startup effects, transition effects, etc.

[0092] The terminal device 100 has one or more APPs installed, or can access a web page or a mini-program, and can use an account to log in to the SP 300 corresponding to the APP, web page, or mini-program to use various services provided by the SP 300. For example, the APPs installed in the terminal device 100 may include browsers, game applications, image processing applications, and the like.

[0093] The APP installed in the terminal device 100, the web pages visited, the mini-programs, etc., can be provided or operated by the network content provider, or provided by the device manufacturer of the terminal device 100. That is, the APP installed in the terminal device 100 can be a third-party application or a system application. The APP installed, the web pages visited, the mini-programs, etc. in different terminal devices 100 can be different, depending on user needs. The device manufacturers mentioned later all refer to the device manufacturers of the terminal device 100.

[0094] There may be multiple terminal devices 100. Multiple terminal devices 100 may be provided by the same device manufacturer or may belong to different device manufacturers, which is not limited here. For example, multiple terminal devices 100 may specifically include terminal device 101 and terminal device 102, and terminal device 101 and terminal device 102 are provided by the same device manufacturer.

[0095] Cloud devices such as cloud nodes and cloud servers in the communication system 10 can provide corresponding services in an on-demand and easily scalable manner through the network, which may include virtualized resources jointly constructed by multiple physical servers, that is, cloud devices can be distributed on multiple different physical devices. Of course, in some embodiments, cloud devices can also be implemented as physical servers or physical server systems.

[0096] The cloud computing node 200 is a device used to form a "new terminal" with the terminal device 100 in the embodiment of the present application.

[0097] The cloud computing node 200 has relatively abundant processing resources, such as powerful computing power (including the computing power of various processors such as CPU, GPU, NPU, etc.), more storage space, etc.

[0098] The cloud computing node 200 can be implemented as an edge computing node, that is, the cloud computing node is deployed to a location closer to the party in need (such as a location close to the data source or user of the application), thereby shortening the response time and reducing the delay in service delivery. In the embodiment of the present application, the cloud computing node 200 may include at least the following deployment methods:

[0099] Deployment method 1. The cloud computing node 200 is deployed at the edge of the network (such as the access network) in the manner of mobile edge computing (MEC), for example, at the base station provided by the operator. The cloud computing node 200 deployed in the MEC manner can use the real-time information on the mobile network side at the edge of the network to provide collaborative services for the terminal device 100. In one possible implementation method, since there may be multiple base stations provided by different operators, and the operator to which the terminal device 100 belongs is determined by the user, the cloud computing node 200 can be deployed at the base stations provided by multiple different operators. When the cloud computing node 200 is subsequently allocated to the terminal device 100, only the cloud computing node 200 at the base station provided by the operator to which the terminal device 100 belongs will be allocated, and the cloud computing node 200 at other operators will not be allocated to it, which is enough to ensure fast communication between the terminal device 100 and the cloud computing node 200.

[0100] Deployment method 2. The cloud computing node 200 is deployed in a bearer network, for example, it can be deployed in a packet transport network (PTN) or an optical transport network (OTN).

[0101] Deployment mode 3. Centralized deployment, multiple cloud computing nodes 200 are deployed uniformly and are not dependent on other network devices.

[0102] The cloud computing nodes 200 may be deployed in other ways, such as micro cloud nodes, fog computing nodes, etc., which are not limited to the above deployment methods. In a communication system, multiple deployment methods of cloud computing nodes 200 may exist at the same time.

[0103] The number of cloud computing nodes 200 in the communication system 10 can be multiple, for example, cloud computing nodes can be deployed in different regions, so as to provide collaborative services for the terminal devices 100 in different regions. The regions here can be divided into countries, cities, districts under the jurisdiction of cities, etc., and the embodiment of the present application does not limit the size of a region. For example, the first region can be deployed with cloud computing nodes 201, and the second region can be deployed with cloud computing nodes 202, and the first region is different from the second region.

[0104] One cloud computing node 200 can be used to connect to multiple terminal devices 100 and provide collaborative services for the multiple terminal devices 100.

[0105] The cloud computing node 200 may be provided and managed by a device manufacturer, or may be provided and managed by a middleman authorized by the device manufacturer.

[0106] SP 300 is used to provide content services, and specifically can be used to receive requests sent by the end-side device 100 or the cloud computing node 200, process the request and return the processing result to the end-side device 100 or the cloud computing node 200. SP 300 can be regarded as a server, and the end-side device 100 and the corresponding cloud computing node 200 can be regarded as clients.

[0107] The number of SP 300 in the communication system 10 may be one or more. One SP 300 may be used to provide services for an APP, a webpage, or a small program. Different SP 300s may be used to provide services for different APPs, different webpages, or different small programs. Different SP 300s may be provided or operated by different network content providers. For example, the multiple SPs 300 in the communication system specifically include SP 301, SP 302, and SP 303, wherein SP 301 may provide services for a browser APP, SP 302 may provide services for a game APP, and SP 303 may provide services for an image processing APP.

[0108] One SP 300 can connect to multiple end-side devices 100 and cloud computing nodes 200 and provide services for the multiple end-side devices 100 and cloud computing nodes 200. The multiple end-side devices 100 and cloud computing nodes 200 can be devices using the APP, webpage or applet corresponding to the one SP 300.

[0109] The vendor cloud server 400 is used to manage multiple end-side devices 100 of the corresponding vendor, and is also used to manage the cloud computing nodes 200 of the corresponding vendor. The vendor cloud server 400 is provided by the device manufacturer. If the communication system 10 includes end-side devices of different vendors, the communication system 10 may also include cloud servers of different vendors. Figure 1 A vendor cloud server 400 is exemplarily shown in the figure, and the vendor cloud server 400 can be used to manage the terminal device 101 and the terminal device 102.

[0110] In an embodiment of the present application, after the end-side device 100 is turned on, a system account (such as an account assigned by the manufacturer to which the end-side device belongs) can be registered, and the corresponding manufacturer cloud server 400 can be logged in through the system account, or the manufacturer cloud server 400 can be directly logged in through an existing system account. Afterwards, the manufacturer cloud server 400 can allocate a cloud computing node 200 to the end-side device 100. The strategy for allocating cloud computing nodes 200 can refer to the detailed introduction of the subsequent method embodiments. For example, if the end-side device 101 and the end-side device 102 are both located in the first area, the manufacturer cloud server 400 can allocate the cloud computing node 201 located in the first area to the end-side device 101 and the end-side device 102.

[0111] In some implementations, if the area where the end-side device 100 is located changes, the vendor cloud server 400 may allocate a new cloud computing node 200 to the end-side device 100 , thereby providing better collaborative services for the end-side device 100 .

[0112] After the manufacturer's cloud server 400 allocates a cloud computing node 200 to the terminal device 100, the cloud computing node 200 will also synchronously install some or all of the applications installed on the terminal device 100. In addition, as the user installs new applications on the terminal device 100, the cloud computing node 200 will also synchronously install new applications. Although the terminal device 100 and the cloud computing node 200 install applications synchronously, the implementation and functions of the same application on different devices may be different. For a detailed description, please refer to the subsequent method embodiments. An application that can be installed in both the terminal device 100 and the cloud computing node 200 can provide processing logic for terminal-cloud collaboration, and the application can also be referred to as a first application.

[0113] After the client device 100 is assigned the corresponding cloud computing node 200, the two can collaboratively process business, thereby providing services to users.

[0114] One situation is that the business collaboratively processed by the end-side device 100 and the cloud computing node 200 requires the support of SP 300. Any one or more of them can initiate a service request to SP 300. After SP 300 responds to the request and returns the result, the end-side device 100 and the cloud computing node 200 can present the content the user wants based on the result.

[0115] In another case, the services collaboratively processed by the end-side device 100 and the cloud computing node 200 do not require support from the SP 300, and the two can jointly process user needs and present desired content to the user.

[0116] For the specific coordination manner between the terminal device 100 and the cloud computing node 200, please refer to the detailed description of the subsequent method embodiments.

[0117] In the embodiment of the present application, the data structure and content between the "new terminals" are defined, that is, the data structure and content of the interaction between the terminal device 100 and the corresponding cloud computing node 200 are defined. At the same time, the application program interface (API) and the corresponding programming paradigm between the "new terminals" (that is, between the terminal device 100 and the cloud computing node 200) are defined. The API is used for communication between the terminal device 100 and the cloud computing node 200, and the programming paradigm is used to indicate the code writing method for the application on the terminal device and the application on the cloud computing node. The above-mentioned definition operation can be performed by the device manufacturer, and the application developer uses the definition to write the program code of the application. The embodiment of the present application does not limit the specific implementation of the API and the programming paradigm. Based on this, the terminal device 100 and the corresponding cloud computing node 200 in the present application can communicate based on the defined API to complete collaborative work.

[0118] The data structure and content between the terminal device 100 and the SP 300, or the data structure and content between the cloud computing node 200 and the SP 300, remain consistent with the data structure and content between the terminal device 100 and the SP 300 in the solution of the prior art in which there is only the terminal device 100 but no cloud computing node 200. In this way, the SP 300 can provide the required services for the "new terminal" without being aware of the "new terminal".

[0119] The above-mentioned devices in the communication system 10 can communicate through a variety of technologies, and different technologies can be used to communicate between different devices, which is not limited in the embodiments of the present application. For example, the terminal device 100 and the cloud computing node 200 can communicate through wireless local area networks (WLAN), cellular networks, etc. The terminal device 100 and the cloud computing node 200 can each communicate with the SP 300 through a limited wide area network, etc. The terminal device 100 and the manufacturer's cloud server 400 can also communicate through a limited wide area network, etc. The SP 300 and the manufacturer's cloud service 400 can communicate through wired connections, etc.

[0120] Figure 1 The protocols used for communication between the various devices in the communication system 10 shown and the formats of the communication messages may be pre-defined or negotiated by the providers of the various devices.

[0121] Figure 1 The communication system 10 shown is only an example. In a specific implementation, the communication system 10 may include devices with more or fewer functions, or may combine or split some devices. For example, the number of the terminal device 100 may be one, and the number of the cloud computing node 200 may also be one.

[0122] Regarding the role of each device in the communication system 10 and the specific implementation of the operations performed by each device, please refer to the relevant description of the subsequent method embodiments, which will not be repeated here.

[0123] based on Figure 1 The communication system 10 of the present application will be described in detail below. The method for end-cloud collaborative work provided by the embodiment of the present application is described in detail below.

[0124] Figure 2 An exemplary flow chart of the method for end-cloud collaboration is provided. Figure 2 As shown, the method may include the following steps:

[0125] S101, the terminal device 100 logs in to the manufacturer's cloud server 400.

[0126] The end-side device 100 can first register a system account, and then use the registered system account to log in to the manufacturer cloud server 400. Alternatively, the end-side device 100 can also log in to the manufacturer cloud server 400 based on the user's registered system account. The manufacturer cloud server 400 here is a cloud server provided by the manufacturer to which the end-side device 100 belongs.

[0127] The embodiment of the present application does not specifically limit the way in which the terminal device 100 registers a system account and uses the system account to log in to the manufacturer cloud server 400. For example, the terminal device 100 can display an account login interface provided by the setting application, and after the user enters a Huawei account in the interface, the terminal device 100 can be triggered to use the Huawei account to log in to the manufacturer cloud server 400 provided by the device manufacturer.

[0128] When the end-side device 100 logs in to the manufacturer's cloud server 400 , it can send its own device identification and location information to the manufacturer's cloud server 400 .

[0129] Alternatively, the terminal device 100 may also send its own device identification to the manufacturer's cloud server 400 during the process of logging into the manufacturer's cloud server 400, and then send the location information to the manufacturer's cloud server 400 after logging in.

[0130] Alternatively, the terminal device 100 may also send its own device identification and location information to the manufacturer's cloud server 400 after logging in.

[0131] The device identifier of the terminal device 100 can be an internet protocol (IP) address, a media access control (MAC) address, a universally unique identifier (UUID), an international mobile subscriber identity (IMSI), a mobile equipment identifier (MEID), an international mobile equipment identity (IMEI), or a system account for logging in to the manufacturer's cloud server 400, etc., which is not limited in the embodiments of the present application.

[0132] In some implementations, after the end-side device 100 logs in to the vendor cloud server 400 , it can also send information about each application installed by itself (such as application identification) to the vendor cloud server 400 so that the vendor cloud server 400 can be informed of the application installation status of the end-side device 100 .

[0133] S102 , the manufacturer's cloud server 400 allocates a cloud computing node 200 to the terminal device 100 .

[0134] The vendor cloud server 400 may select a cloud computing node 200 from the multiple cloud computing nodes 200 managed by it and allocate it to the terminal device 100. In some other implementations, the vendor cloud server 400 may also allocate multiple cloud computing nodes 200 to the terminal device 100 according to actual needs.

[0135] Specifically, the vendor cloud server 400 is used to manage multiple cloud computing nodes 200 of the corresponding vendor, and thus can know the location and deployment mode of each of the multiple cloud computing nodes 200. Based on this, the vendor cloud server 400 can select a cloud computing node 200 from the location information of the terminal device 100 and any one or more of the information of the multiple cloud computing nodes 200 and allocate it to the terminal device 100.

[0136] The strategy for allocating the cloud computing node 200 to the terminal device 100 may include any of the following:

[0137] Strategy 1.

[0138] From among the multiple cloud computing nodes 200 , a cloud computing node 200 that is closest to the end-side device 100 is selected and allocated to the end-side device 100 .

[0139] In the process of end-cloud collaborative processing of services, the communication distance between the end-side device and the cloud computing node is a key factor affecting the quality of communication service (QoS), that is, a key factor affecting user experience. Based on strategy 1, the cloud computing node 200 closest to the end-side device 100 is assigned to the end-side device 100, and the distance between the two is shortened as much as possible, which can improve the QoS of the end-cloud collaborative processing of services, make the end-cloud collaboration experience close to or consistent with the experience on a pure end-side device, and allow users to enjoy high-quality services without noticing. In particular, after accessing the high-bandwidth and low-latency 5G cellular network and subsequent better networks, the end-cloud collaboration experience is almost the same as the experience on a pure end-side device.

[0140] Strategy 2.

[0141] The cloud computing nodes deployed in the access network (such as cloud computing nodes deployed in the base station in MEC mode) are preferably used. From the cloud computing nodes in the access network, the cloud computing node 200 closest to the terminal device 100 is selected and allocated to the terminal device 100.

[0142] If the multiple cloud computing nodes 200 do not include the cloud computing nodes deployed in the access network, then secondly, among the cloud computing nodes deployed in the bearer network, the cloud computing node 200 closest to the terminal device 100 is selected and allocated to the terminal device 100;

[0143] If the multiple cloud computing nodes 200 do not include cloud computing nodes deployed on the bearer network, then the cloud computing node 200 closest to the terminal device 100 is selected from the centrally deployed cloud computing nodes and allocated to the terminal device 100.

[0144] Through strategy 2, the cloud computing node in the access network closest to the terminal device 100 is preferred, and the cloud computing node in the bearer network closest to the terminal device 100 is selected secondly. Since the terminal device 100 is closest to the access network and then the bearer network during the network access process, the distance between the terminal device 100 and the cloud computing node 200 assigned to it can be shorter and the number of transit devices can be less. Therefore, strategy 2 can also improve the QoS of the end-cloud collaborative processing business, making the end-cloud collaborative experience close to or consistent with the experience on pure terminal devices, allowing users to enjoy high-quality services without noticing.

[0145] Strategy 3.

[0146] The cloud computing nodes 200 whose distances to the end-side device 100 are within a certain value (such as a first value) are allocated to the end-side device 100 .

[0147] Alternatively, on the basis of Strategy 1 and Strategy 2, a distance condition is added, and the cloud computing node 200 is allocated to the end-side device 100 only when the distance between the cloud computing node 200 and the end-side device 100 determined in Strategy 1 or Strategy 2 is within a certain value (such as the first value).

[0148] This certain value can be set by the manufacturer cloud server 400 as needed and is not limited here.

[0149] Strategy 3 can ensure that the distance between the cloud computing node 200 allocated to the end-side device 100 and the end-side device 100 is less than or equal to a certain value, thereby ensuring the QoS of the end-cloud collaborative processing service.

[0150] Not limited to the three strategies listed above, the vendor cloud server 400 can also allocate cloud computing nodes 200 to the end-side device 100 according to other strategies. For example, the available storage space and computing power of each cloud computing node 200 under the vendor cloud server 400 can also be considered, and cloud computing nodes 200 are allocated based on this, so that it can be ensured that the cloud computing nodes 200 allocated to the end-side device 100 can provide collaborative services for the end-side device 100 well. For another example, the region where the end-side device 100 is located can also be considered. The vendor cloud server 400 can only select one or more cloud computing nodes 200 deployed in the region and allocate them to the end-side device 100. The embodiment of the present application does not specifically limit the strategy for allocating cloud computing nodes.

[0151] After determining the cloud computing node 200 to be allocated to the end-side device 100, the vendor cloud server 400 can complete the allocation work by the following operations:

[0152] The vendor cloud server 400 records the allocation relationship. For example, the vendor cloud server 400 can associate and record the device information of the end-side device 100 and the device information of the cloud computing node 200 allocated to the end-side device 100. The device information of both of the above may include any one or more of the device identification, location information, communication address (such as IP address, MAC address), etc. Recording the allocation relationship facilitates the vendor cloud server 400 to manage multiple end-side devices and multiple cloud computing nodes 200 under it.

[0153] The vendor cloud server 400 sends the device information of the terminal device 100 to the cloud computing node 200 assigned thereto, so that the cloud computing node 200 can know for which terminal devices 100 it will provide collaborative services.

[0154] The vendor cloud server 400 sends the device information of the determined cloud computing node 200 to the terminal device 100. This allows the terminal device 100 to obtain the information of the cloud computing node 200 allocated to it.

[0155] It can be seen that the process of allocating a cloud computing node to an end-side device can also be considered as a binding process, that is, binding the end-side device 100 and the cloud computing node 200 allocated thereto.

[0156] After the end-side device 100 and the cloud computing node 200 obtain each other's information, the two parties can establish a communication connection to support subsequent end-cloud collaborative work.

[0157] S103 , the cloud computing node 200 allocates resources to the terminal device 100 .

[0158] The bound cloud computing node 200 allocates resources to the terminal device 100 for providing collaborative services to the terminal device 100 in the future. The allocated resources may include but are not limited to: storage resources, computing resources, etc. In some embodiments, the cloud computing node 200 may also allocate resources on demand, and coordinate resources to provide services to the terminal device 100 when the terminal device 100 needs it.

[0159] In some embodiments, the cloud computing node 200 can provide a runtime for the terminal device 100. The definition of the runtime may include: 1. The runtime system, which is the basic environment or platform for the application code to run, which is built on the operating system (OS) of the cloud computing node 200 and is used to support the running of the application code and the processing of application data. 2. The runtime library that can be used on demand, which includes function libraries, object libraries, class libraries, etc., to support input and output, memory management, etc. when the application code runs on it.

[0160] If a cloud computing node 200 is allocated to a plurality of different end-side devices 100 , the cloud computing node 200 can allocate resources to different end-side devices 100 respectively, that is, allocate different runtimes to different end-side devices 100 .

[0161] The cloud computing node 200 can complete the operation of allocating resources to the terminal device 100 by the following operations: associate and record the device information of the terminal device 100 and the runtime information provided for the terminal device 100. The runtime information may include, for example, the identifier of the runtime, the storage location of the runtime in the cloud computing node 200, the size of the storage space, etc.

[0162] In addition to the cloud computing node 200 providing a runtime for the terminal device 100 , the terminal device 100 itself may also include a runtime for supporting the running of the application code and the processing of application data of the terminal device 100 itself.

[0163] Figure 3 The example shows the runtime installation on multiple devices.

[0164] like Figure 3 As shown, if cloud computing node 201 is allocated to terminal device 101 and terminal device 102, cloud computing node 201 will provide a runtime 1 for terminal device 101 and another runtime 2 for terminal device 102. In addition, terminal device 101 and terminal device 102 each have a corresponding runtime.

[0165] In the embodiment of the present application, the end-side device 100 can be referred to as the first device, and the cloud computing node 200 initially allocated to the end-side device 100 by the vendor cloud server 400 can be referred to as the first node. The runtime allocated by the first node to the first device can be referred to as the first runtime.

[0166] S104 , the cloud computing node 200 installs some or all of the existing applications on the terminal device 100 .

[0167] After the end-side device 100 logs in to the manufacturer's cloud server 400, it can maintain communication with the manufacturer's cloud server 400 and send the status of its installed applications to the manufacturer's cloud server 400.

[0168] The cloud computing node 200 may install part or all of the installed applications on the runtime provided for the terminal device 100 according to the applications installed on the terminal device 100 .

[0169] In some implementations, after allocating the cloud computing node 200 to the terminal device 100, the vendor cloud server 400 may notify the application store (such as the application store server) provided by the device vendor to send the installation packages of some or all applications on the terminal device 100 to the cloud computing node 200 for installation by the cloud computing node 200. The vendor cloud server 400 may also send the device identification of the terminal device 100 in association, so that the cloud computing node 200 can find the corresponding runtime according to the device identification and then install the application on the runtime.

[0170] In some other embodiments, after allocating a cloud computing node 200 to the terminal device 100, the manufacturer's cloud server 400 can send the application installation status in the terminal device 100 to the cloud computing node 200, and then the cloud computing node 200 will download the installation package from the application store provided by the device manufacturer (such as an application store server) and install the application on the runtime provided for the terminal device 100.

[0171] In some other embodiments, after the terminal device 100 obtains the information of the cloud computing node 200 assigned to it, it can also directly send its own application installation status to the cloud computing node 200, and then the cloud computing node 200 will download the installation package from the application store provided by the device manufacturer (such as an application store server) and install the application on the runtime provided for the terminal device 100.

[0172] In S104, some or all of the applications installed in the cloud computing node 200 may include applications that have been adaptively improved for the end-cloud collaborative working method provided in this application. For applications in the end-side device 100 that have not been adaptively improved for this application, the cloud computing node 200 does not need to be installed. The manner in which the application is adaptively improved for the end-cloud collaborative working method provided in this application, the improved application form and function, etc., can be referred to the relevant description of the subsequent steps, which will not be repeated here.

[0173] In some other implementations, some or all of the applications installed in the cloud computing node 200 may include, among the applications that have been adaptively improved for the present application, a preset number of applications that are used most frequently on the terminal device 100, or a preset number of applications that have been recently used by the terminal device 100. In this way, applications can be installed on the cloud computing node 200 according to the user's usage frequency or usage habits, which can better meet user needs and save space on the cloud computing node.

[0174] S105: The terminal device 100 installs a new application.

[0175] In a possible implementation, after the cloud computing node 200 is allocated to the terminal device 100, the terminal device 100 can install new applications according to user needs as the user uses the terminal device 100. The terminal device 100 can download new applications from the application store provided by the device manufacturer.

[0176] S106, the cloud computing node 200 synchronously installs a new application.

[0177] If the new application installed by the terminal device 100 is an application that has been adapted for this application, the cloud computing node 200 will install the application synchronously. If the new application installed by the terminal device 100 is not an application that has been adapted, but a regular application, the cloud computing node 200 does not need to install the application synchronously.

[0178] In some implementations, after receiving the application download request from the terminal device 100, the application store provided by the device manufacturer can send the installation package to the terminal device 100 and the cloud computing node 200 at the same time, so that the cloud computing node 200 can also install the new application synchronously.

[0179] In other implementations, after the manufacturer's cloud server 400 learns that a new application has been installed on the terminal device 100, it may notify the application store to simultaneously send the application installation package to the cloud computing node 200, or notify the cloud computing node 200 of the new application and allow the cloud computing node 200 to download and install the application on its own.

[0180] Through S105-S106, as new applications are installed on the end-side device 100, the corresponding cloud computing node 200 also installs new applications synchronously, so that the applications at runtime in the cloud computing node 200 and the applications at runtime in the end-side device 100 can be kept matched, preparing for subsequent end-cloud collaboration.

[0181] The above steps S105-S106 are optional steps. If the user has not installed a new application on the terminal device 100, there is no need to execute S105-S106.

[0182] Similar to S105-S106, if the terminal device 100 deletes part of the application under the user's instruction, then the cloud computing node 200 will also delete the part of the application during the corresponding runtime.

[0183] It can be seen from S104-S106 that as the user uses the terminal device 100, various applications that meet the user's needs can be installed on the terminal device 100 and the corresponding cloud computing node 200, and ultimately a personalized runtime matching the user will be formed on the terminal device 100 and the cloud computing node 200.

[0184] From the above S104-S106, it can be seen that various applications (including applications that provide end-cloud collaborative services with adaptive collaboration for this application and ordinary applications that have not been improved) will be installed on the end-side device 100 according to the actual needs of the user, while the corresponding cloud computing node 200 only installs applications that provide end-cloud collaborative services with adaptive collaboration for this application, so the applications installed on the cloud computing node 200 are included in the applications installed on the end-side device 100. In other words, it can be considered that the applications installed on the end-side device 100 are the full set, and the applications installed on the cloud computing node 200 corresponding to the end-side device 100 are a subset of the full set on the end side.

[0185] S107, the terminal device 100 and the cloud computing node 200 both run the first application and work together in the process.

[0186] In some implementations, if the cloud computing nodes 200 allocated by the vendor cloud server 400 to the terminal device 100 in S102 include multiple ones, then in S107, the terminal device 100 can select one cloud computing node 200 from them to work together according to the following strategy: select cloud computing nodes according to the real-time requirements of the service, specifically, select cloud computing nodes deployed in the access network (such as cloud computing nodes deployed in the base station in MEC mode) for strong real-time services, select cloud computing nodes deployed in the bearer network for services with less real-time requirements, and select centrally deployed cloud computing nodes for services without real-time requirements. In this way, the user's real-time requirements for services can be met while taking into account user experience and cost.

[0187] The terminal device 100 and the cloud computing node 200 both run the first application at runtime, work together during the running of the first application, and jointly provide the user with the services required by the user.

[0188] The first application refers to the above-mentioned application that has been adaptively improved for the end-cloud collaborative working method provided by this application. The first application can be an application pre-loaded in the end-side device 100, or it can be an application that the user subsequently downloads during the use of the end-side device 100. The type of the first application can include multiple types, such as game applications, browsers, image processing applications, video applications, etc., which are not limited in this embodiment of the present application.

[0189] Although both are called first applications, the first application in the terminal device 100 and the first application in the cloud computing node 200 are not exactly the same. For example, the program code of the first application in the terminal device 100 and the program code of the first application in the cloud computing node 200 may be different.

[0190] The first application in the terminal device 100 provides processing logic on the terminal device, and the first application in the cloud computing node 200 provides processing logic on the cloud computing node. In addition, the first application in the terminal device 100 and the first application in the cloud computing node 200 can cooperate and collaborate with each other to jointly provide services for users.

[0191] The function and implementation of the first application in the terminal device 100 and the function and implementation of the first application in the cloud computing node 200 can be defined independently by the application developer, or can also be defined by negotiation between the application developer and the device manufacturer. The function and implementation of the first application on the terminal device and the cloud computing node can refer to the respective operations of the two in the collaborative work described later.

[0192] The difference between the first application in the terminal device 100 and the first application in the cloud computing node 200 determines how the two work together, that is, determines which data is processed on the terminal side and which data is processed on the cloud side.

[0193] The following describes the collaborative working process of the terminal device 100 and the cloud computing node 200.

[0194] In some implementations, after the end-side device 100 starts the first application, it can send a first notification message carrying the identifier of the started first application to the cloud computing node 200 to notify the cloud computing node 200 to start the first application, and then the cloud computing node 200 also runs the first application accordingly through the runtime to support the end-cloud collaborative work. In this way, the cloud computing node 200 starts the first application in advance, and when the end-side device 100 has a need for collaboration, it can quickly respond to its needs through the started first application.

[0195] In other embodiments, after the terminal device 100 starts the first application, it communicates with the cloud computing node 200 when there is a need for collaboration (for example, sending relevant control data or business data to the cloud computing node 200). After the cloud computing node 200 learns that the terminal device 100 has a need for collaboration, it runs the corresponding runtime and runs the first application, thereby responding to the collaboration needs of the terminal device 100.

[0196] In an embodiment of the present application, the end-side device 100 and the cloud computing node 200 can decide how to work together according to the scheduling strategy, that is, decide which operations each of them performs during the business processing. In other embodiments, the end-side device 100 and the cloud computing node 200 can negotiate to decide whether the two need to work together and how the two work together, that is, jointly decide which operations each of them performs during the business processing. Alternatively, the end-side device 100 can also decide whether the two need to work together, and the end-side device 100 can instruct the cloud computing node 200 how to work.

[0197] Factors that affect the collaborative working mode of the end-side device 100 and the cloud computing node 200 may include any one or more of the following: application load, computing power requirements, communication QoS between the end and the cloud, service processing delay, service real-time requirements, etc.

[0198] in:

[0199] Application Load

[0200] Application load refers to the amount of data that needs to be processed in the current business logic stage. The load of different business logic stages of the same application can be different, and the load of different applications can also be different. Optionally, when the application load is larger, more operations can be moved to the cloud computing node for execution.

[0201] In some embodiments, when the application load is heavy and there is no need to respond to user operations in real time, the business can be processed first in the cloud computing node 200, and then the terminal device 100 directly obtains the results from the cloud computing node 200 and presents the results to the user.

[0202] For example, for the search service of the browser application, after receiving the user operation (such as the search operation) of the user triggering the service, the terminal device 100 can send control data (such as the search instruction) to the cloud computing node 200 to instruct the cloud computing node 200 to complete the search work, and the cloud computing node 200 will then return the search results to the terminal device 100. Alternatively, if the cloud computing node 200 has performed the same search operation in the past, it can directly return the search results to the terminal device 100 after receiving the control data, without the need for repeated search, which can save search time.

[0203] Since the cloud computing node 200 has powerful computing power and fast processing speed, most operations are moved to the cloud side for execution, so the latency is low and can almost be considered to be zero.

[0204] Computing power requirements

[0205] The computing power requirement refers to the amount of computing resources required to support the current application load. The computing power requirements of different business logic stages of the same application can be different, and the computing power requirements of different applications can also be different. Optionally, the greater the computing power requirement, the more operations can be moved to the cloud computing node for execution. This can save the computing power of the end-side device 100, or reduce the requirements for the end-side device 100, while also providing good services to users.

[0206] QoS for communication between end and cloud

[0207] The communication QoS depends on the current network conditions, and the communication QoS at different times may be different. Optionally, the better the communication QoS, the more operations can be moved to the cloud computing node for execution.

[0208] Business processing delay

[0209] The processing delay of a service depends on the service itself and can be regarded as the time required to process the service, such as the time required to process an image or a video. Optionally, services with longer processing delays can be moved to cloud computing nodes for execution, thereby saving computing power on the end-side device.

[0210] Real-time business requirements

[0211] The real-time requirements of a service depend on the user's experience requirements for the service. For example, for browser search services or image processing services, real-time response is not required; for gaming services, real-time response is required on the end cloud. Optionally, for services with strong real-time requirements, cloud computing nodes can complete most of the business logic and collaborate with end-side devices to provide users with the required services; for services with low real-time requirements but high computing power requirements, cloud computing nodes can complete most of the workload.

[0212] In some implementations, services that are not very real-time can be processed serially by the end-cloud, which is equivalent to the end-side device 100 completing the basic operation, and the cloud computing node 100 performs further enhanced operations on this basis, providing differentiated results different from the end-side processing solution alone.

[0213] For example, for the end-cloud collaborative photography service, after receiving the user's photography operation, the end-side device 100 can generate a user preview photo and original photo data (rawdata) (also called original image), and send the original photo data to the cloud computing node 200, which automatically performs further enhancement processing on the photo (such as automatic beauty, enhanced clarity, etc.). Alternatively, the user can also input some processing operations based on the original photo (such as personalized editing requirements, adding filters, defogging, separation of group photos, etc.), and the end-side device 100 sends the user's operation instructions to the cloud computing node 200, and then the cloud computing node 200 performs targeted processing on the original photo data according to the user operation. Afterwards, the cloud computing node 200 returns the processed photo (also called image) to the end-side device 100, and the end-side device 100 obtains a high-quality photo processed by the cloud side. Usually, there is a certain time between the user clicking to take a photo and viewing the photo. This time is enough for the cloud computing node 200 to return the processed photo and hand it over to the end-side device 100 for display, so that the user can get a better photo experience.

[0214] refer to Figure 4 , Figure 4 An example is shown of the comparison of images processed by the end and the cloud. Figure 4 The photo on the left may be an original photo generated by the terminal device 100. Figure 4 The photo on the right side may be a refined photo after further processing by the cloud computing node 200. Comparing the two photos, it can be seen that the photo processed by the cloud computing node 200 has been finely processed such as increasing brightness, smoothing skin, removing facial blemishes, and slimming the body.

[0215] Since the cloud computing node 200 has strong computing power and fast processing speed, the above-mentioned end-cloud collaborative photo processing solution can reduce the overall latency of the photo-taking service.

[0216] In some implementations, for services with strong real-time requirements, most of the processing operations can also be performed by the cloud side. For example, the terminal device 100 can continuously receive operation instructions input by the user, and the cloud computing node 200 can also continuously process services according to these operation instructions to achieve real-time collaboration effects.

[0217] For example, for the end-cloud collaborative online game service, the end-side device 100 is used to interact with the user, receive the operation instructions input by the user (such as switching the perspective of the game map, attack operation, etc.), and then send the indication information of these operation instructions to the cloud computing node 200, and then the cloud computing node 200 renders a new game screen according to the operation instruction, and then returns the rendering result to the end-side device 100 for presentation. As long as the time for the cloud computing node 200 to return the game screen can meet the game refresh rate requirements of the end-side device 100, the real-time requirements of the online game service can be met, and a smooth game experience can be provided to the user. Here, the computing power of the cloud computing node 200 is powerful, which can improve the quality of the rendering results and provide users with a high-quality game experience. For example, the cloud computing node 200 can render effective light and shadow effects based on global illumination, which can reflect the lighting changes of the game screen. In this way, the end-side device 100 with low hardware configuration and unable to present high-quality game effects alone can also use the end-cloud collaborative solution of this application to provide users with a higher quality game experience.

[0218] In online gaming, the smoothness and stability of the images are important factors that affect the user experience. Since the online gaming in this application can be processed by the end-cloud collaboration, the game image frame returned by the cloud computing node 200 can contain a timestamp, so that the end-side device 100 can arrange the returned multiple frames of images in order without confusion, and the end-side device 100 can also use the vertical synchronization signal (V-sync) to keep the display refresh rate consistent with the frame rate processed by the end-cloud to maintain the stability of the output image quality.

[0219] Since the cloud computing node 200 has strong computing power and fast processing speed, the above-mentioned end-cloud collaborative processing of online game services can reduce the overall service latency.

[0220] The above examples are only examples, and other scheduling strategies can be constructed according to actual business needs, which depends on the definition of the application developer. The embodiments of the present application do not specifically limit the scheduling strategy.

[0221] The same application can have different scheduling strategies at different business logic stages. This means that the same application can have multiple different scheduling strategies. Of course, different applications can also have different scheduling strategies.

[0222] Summarizing the above-mentioned scheduling strategies, it can be seen that the data sent by the terminal device 100 to the cloud computing node 200 may include two categories: 1. Control data, which is used to instruct the cloud computing node 200 to perform certain operations. For example, the control data indicating the search sent by the terminal device 100 to the cloud computing node 200 in the browser search service, and the control data containing user operation instructions sent by the terminal device 100 to the cloud computing node 200 in the game service. 2. Business data, which is used to be handed over to the cloud computing node 200 for further processing. For example, the original photo data (rawdata) sent by the terminal device 100 to the cloud computing node 200 in the photo service. The business data sent by the terminal device 100 to the cloud computing node 200 can be called the first business data.

[0223] The data returned by the cloud computing node 200 to the terminal device 100 is generally the processed business data. The business data returned by the cloud computing node 200 to the terminal device 100 may be referred to as second business data. For example, the second business data may include search results, images processed from original images, rendered game screens, and the like.

[0224] Based on the above introduction to the end-cloud collaborative work process, it can be seen that in the collaborative work, the end-side device 100 and the cloud computing node 200 have their own division of labor. Both run the first application, but perform different operations and can process different data and data volumes. This can make up for the lack of computing power on the end side and provide users with an enhanced experience. For example, when taking photos, you can use the cloud-side computing power to do advanced retouching for portraits, and you can choose to enlarge a single person or object for group photos for further processing.

[0225] In the process of end-cloud collaboration, if the service provided by the end-side device 100 and the cloud computing node 200 to the user requires network support, the cloud computing node 200 may also interact with the SP 300 corresponding to the first application during the collaboration. For example, the cloud computing node 200 may initiate a service request to the SP 300, and after the SP 300 returns the data, the end-side device and the cloud computing node 200 may collaboratively process the data. Alternatively, the end-side device 100 may initiate a service request to the SP 300, and after the SP 300 returns the data, the end-side device and the cloud computing node 200 may collaboratively process the data.

[0226] In other embodiments, during the end-cloud collaborative work process, the services provided to users by the end-side device 100 and the cloud computing node 200 do not require network-side support. The two can work together to provide users with the services they need, so there is no need to interact with SP300.

[0227] A cloud computing node 200 can provide services for multiple end-side devices 100, one of which can be referred to as a first device, and another end-side device 100 can be referred to as a second device. Similar to the process in which the cloud computing node 200 provides collaborative services for one of the end-side devices 100 (i.e., the first device), in the process in which the cloud computing node 200 provides collaborative services for another end-side device (i.e., the second device), the second device is used to start the second application and send third data to the first node, the third data including the control data and / or third business data of the second application; the first node is also used to allocate a second runtime to the second device, start the second application through the second runtime, receive the third data through the second application, generate fourth data based on the third data, and return the fourth data to the second device, the fourth data including the fourth business data of the second application, the second runtime is different from the first runtime; the second device is also used to provide services based on the fourth data. The second application here can be the same as or different from the first application mentioned above.

[0228] In optional step S108, the terminal device 100 crosses cloud computing nodes.

[0229] The situations where the terminal device 100 crosses cloud computing nodes may include the following:

[0230] Case 1. The cloud computing node 200 allocated to the end-side device 100 is deployed at the base station. If the end-side device 100 roams across base stations, a new cloud computing node 200 needs to be allocated. Roaming means that the end-side device 100 leaves the registration area of ​​the cellular mobile service and enters another area. This means that the location of the end-side device 100 has changed significantly.

[0231] Case 2. The cloud computing node 200 allocated to the terminal device 100 is deployed in the bearer network and is the cloud computing node 200 closest to the access network (such as a base station). If the terminal device 100 roams across the bearer network, a new cloud computing node 200 needs to be allocated.

[0232] Case 3. After the end-side device 100 moves, a cross-cloud computing node situation occurs. Specifically, after the end-side device 100 moves, if the cloud computing node is reallocated for the end-side device 100 at the new location according to the allocation strategy in S102 (such as strategy 1 with the shortest distance), and the cloud computing node 200 corresponding to the current end-side device 100 is different, it is considered that a cross-cloud computing node situation occurs. In this case, the subsequent step S109 can be executed.

[0233] In optional step S109 , the manufacturer cloud server 400 allocates a new cloud computing node 200 to the terminal device 100 .

[0234] Since the vendor cloud server 400 and the end-side device 100 maintain communication, the cross-cloud computing node status of the end-side device 100 can be known, and whether to execute S109 can be decided based on the cross-cloud computing node status.

[0235] The vendor cloud server 400 can re-allocate a new cloud computing node 200 to the terminal device 100 according to the new position of the terminal device 100 after crossing the cloud computing node. The strategy and specific operations of allocating cloud computing nodes can refer to the description in S102. In some embodiments, the distance between the terminal device 100 and the new cloud computing node 200 is less than the distance between the terminal device 100 and the original cloud computing node 200.

[0236] In order to distinguish the original cloud computing node from the new cloud computing node, the original cloud computing node is marked as cloud computing node 201 and the new cloud computing node is marked as cloud computing node 202 .

[0237] The new cloud computing node 200 may be referred to as a second node. The runtime allocated by the second node to the end-side device 100 may be referred to as a third runtime.

[0238] Optional step S110, during runtime migration between the new and old cloud computing nodes.

[0239] Runtime migration can include the following operations:

[0240] The new cloud computing node 202 allocates resources to the terminal device 100 and deploys a runtime for the terminal device 100. The cloud computing node 202 can learn about the application installation status in the terminal device 100 from the manufacturer's cloud server 400, download the corresponding application from the application store, and install it on the runtime deployed for the terminal device 100. Alternatively, the cloud computing node 202 can also directly obtain the application code that is personalized and installed for the terminal device 100 from the original cloud computing node 201, and install the corresponding application on its own runtime accordingly. In this way, the new cloud computing node 202 also has the conditions to provide collaborative services for the terminal device 100, and the new cloud computing node 202 can provide collaborative services for the terminal device 100 in the future.

[0241] Optionally, the original cloud computing node 201 releases the runtime deployed for the terminal device 100. The original cloud computing node 201 can release the runtime resources originally deployed for the terminal device 100, clear the various application codes and related business data originally installed for it, and release the storage resources and computing resources originally allocated to the terminal device 100. By releasing resources, resources in each cloud computing node can be saved and resource waste can be avoided.

[0242] For example, if a user uses a mobile phone in District B of City A, the user has a "cloud runtime" on the cloud computing node in District B. When the user uses the mobile phone in District M of City K, a new "cloud runtime" is deployed on the cloud computing node in District M, and the "cloud runtime" resources of the cloud computing node in District B are released. When the user returns to City A or arrives in City X, the "cloud runtime" of the cloud computing node in District M is released, and a new "cloud runtime" is allocated to collaborate with the terminal to provide services for the user.

[0243] S108-S110 may occur at any time point, depending on the movement of the user when carrying the terminal device 100. Therefore, S108-S110 may be executed not only after S107, but also at any time point before S107. After executing S108-S110, the new cloud computing node will work with the terminal device 100 to provide the user with the services he needs.

[0244] Figure 2 The end-cloud collaborative working method shown has at least the following technical effects:

[0245] 1. Use the capabilities of cloud computing nodes to process business, with low requirements for the capabilities of the end-side devices. Regardless of the capabilities of the end-side devices, high-quality user experience can be guaranteed with the support of cloud computing nodes.

[0246] If the terminal device has low capabilities, the cloud computing node can make up for the shortcomings of the terminal device. Compared with relying solely on the terminal device to provide services to users, this application can provide stronger and higher-quality services. In this way, even terminal devices with lower configurations can provide users with high-quality services, thereby ensuring the user experience.

[0247] 2. Using the capabilities of cloud computing nodes to process business is equivalent to moving part or all of the business to the cloud, thereby saving the computing power and energy consumption of the terminal equipment, extending the battery life of the terminal equipment, and providing users with a better terminal usage experience.

[0248] 3. If the end-cloud collaborative processing business requires support from other service providers, then since the end-side device and cloud computing nodes form a "new terminal" for end-cloud collaborative processing business, the data format and content of the "new terminal" communicating with other service providers can keep the original end-side device itself consistent with the other service providers. Therefore, the implementation of this application solution does not require other service providers to be aware of it, nor does it require other service providers to make adaptive improvements. It can be easily and low-costly connected to a large number of applications, ensuring that multiple applications can provide users with high-quality and excellent user experience based on this application.

[0249] 4. A cloud computing node can connect to multiple end-side devices and coordinate multiple end-side devices to process services. Compared with end-side devices providing services by improving configuration, in this application solution, a cloud computing node can assist multiple end-side devices to provide high-quality services, which can reduce the overall solution cost while ensuring user experience.

[0250] 5. Currently, users obtain and process information through terminals to improve the efficiency and quality of their work and life. Terminals are personal information assistants, and users need to understand, screen, judge and process a lot of information. In this application, a large model is deployed during cloud operation, and the general model provided by each service is coordinated to allow the new terminal of end-cloud collaboration to understand the different intentions of different users in different scenarios, and provide personalized services and applications for them through multi-modal interaction (voice, touch, keyboard and mouse, gesture, gaze, ...), becoming the user's intelligent assistant, and providing users with a one-step direct solution based on the large model, so that they can obtain a unique user experience. For example, a conference secretary needs to set an alarm, start recording, transcribe text, and take minutes in multiple steps. Based on the new terminal, it can be completed in one step through voice interaction: Please help me start the meeting mode at 9:00 and provide meeting minutes for review at 11 o'clock. It is equivalent to, with the strong computing power of cloud computing nodes, the "new terminal" can install more powerful applications, thereby providing users with higher quality services than before, and can also provide personalized services for different users.

[0251] 6. In addition, each application of the terminal now maintains a heartbeat message with the SP cloud service of the application, so that the cloud service can be delivered in time when it needs to notify the user, but for the terminal, it greatly wastes power to passively receive messages, and also increases the use of precious air interface resources between the mobile network and the terminal. Each application can be deployed on the cloud operation in this application to be responsible for the heartbeat connection with the SP, while reducing the heartbeat connection with the terminal side, which does not affect the service experience, but also saves the power of the terminal and saves air interface resource occupancy.

[0252] The following describes the hardware and software architecture of each device provided in the embodiments of the present application.

[0253] refer to Figure 5 , Figure 5 The hardware structure diagram of the terminal device 100 provided in the embodiment of the present application. The terminal device 100 may be Figure 5 The terminal side device 100 in the communication system is shown.

[0254] like Figure 5As shown, the terminal device 100 may include: the terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0255] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0256] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0257] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0258] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0259] The charging management module 140 is used to receive charging input from a charger.

[0260] The power management module 141 is used to connect the battery 142 , the charging management module 140 and the processor 110 .

[0261] The wireless communication function of the terminal device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0262] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0263] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be set in the same device as at least some of the modules of the processor 110.

[0264] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0265] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0266] In some embodiments, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0267] The end device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0268] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel.

[0269] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0270] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0271] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; non-volatile memory may include disk storage devices and flash memory.

[0272] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; can be divided into single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; can be divided into universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

[0273] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0274] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0275] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the terminal device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0276] The terminal device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0277] In the embodiments of the present application:

[0278] The processor 110 is used to install one or more applications, including the first application mentioned in the above embodiment.

[0279] The mobile communication module 150, the wireless communication module 160, the antenna, etc. are used to support the communication between the terminal device 100 and other devices (such as the manufacturer's cloud server 400, the cloud computing node 200, the application store server, etc.). The communication process between the terminal device 100 and other devices can refer to the detailed description of the method embodiment above.

[0280] The internal memory 121 is used to store the implementation code of the end-cloud collaboration provided by the present application on the end-side device 100, and the processor 110 is used to read the code to complete the implementation of the end-cloud collaboration method on the end-side device 100. The internal memory 121 can also be used to store the implementation code of each application (including the first application) loaded in the end-side device 100.

[0281] Display screen 194 can be used to interact with users, receive various operations input by users, and can also be used to display the results of end-cloud collaboration, such as displaying game screens, processed pictures and videos, search results, application startup animations, transition animations, etc.

[0282] Information input devices such as microphone 170C and camera 193 can also be used to receive various interactive operations input by the user.

[0283] The terminal device 100 may be configured with a software operating system (OS), which may include, but is not limited to, etc.

[0284] The software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes a mobile operating system with a layered architecture as an example to exemplify the software structure of the terminal device 100.

[0285] Figure 6 The software architecture of the terminal device 100 according to an embodiment of the present application is shown.

[0286] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the mobile operating system is divided into four layers, from top to bottom: application layer, program framework layer / core service layer, underlying library and runtime, and kernel layer.

[0287] The application layer can include a series of application packages.

[0288] like Figure 6 As shown, the application package may include a first application, a camera, a gallery, a calendar, a call, a map, a navigation, a WLAN, a Bluetooth, music, a video, a short message, and other applications. For the implementation and function of the first application in the terminal device 100, reference may be made to the relevant description in the above method embodiment.

[0289] The program framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The program framework layer includes some predefined functions.

[0290] like Figure 6 As shown, the program framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0291] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0292] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0293] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0294] The phone manager is used to provide communication functions for the terminal device, such as the management of call status (including answering, hanging up, etc.).

[0295] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0296] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, the terminal device vibrates, and the indicator light flashes.

[0297] The runtime can refer to all code libraries, frameworks, etc. required for the program to run. For example, for the C language, the runtime includes a series of function libraries required for the C program to run. For the Java language, in addition to the core library, the runtime also includes the virtual machine required for the Java program to run. The above core library may include the function functions that the Java language needs to call.

[0298] The underlying library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0299] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0300] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0301] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0302] A 2D graphics engine is a drawing engine for 2D drawings.

[0303] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0304] Figure 7 It is a hardware structure diagram of the cloud computing node 200 provided in an embodiment of the present application. Figure 7 The cloud computing node shown may be the cloud computing node 200 in the communication system 10 described above.

[0305] like Figure 7As shown, the cloud computing node may include: one or more processors 201, memory 202, communication interface 203, transmitter 205, receiver 206, coupler 207 and antenna 208. These components may be connected via bus 204 or other means. Figure 7 Take bus connection as an example.

[0306] The communication interface 203 can be used for the cloud computing node to communicate with other communication devices. Specifically, the communication interface 203 can be a 3G communication interface, a long-term evolution (LTE) (4G) communication interface, a 5G communication interface, a WLAN communication interface, a WAN communication interface, etc. Not limited to a wireless communication interface, the network device 300 can also be configured with a wired communication interface 203 to support wired communication, for example, the backhaul link between the cloud computing node and other cloud computing nodes can be a wired communication connection.

[0307] In some embodiments of the present application, the transmitter 205 and the receiver 206 can be regarded as a wireless modem. The transmitter 205 can be used to transmit and process the signal output by the processor 201. The receiver 206 can be used to receive the signal. In the cloud computing node, the number of the transmitter 205 and the receiver 206 can be one or more. The antenna 208 can be used to convert the electromagnetic energy in the transmission line into electromagnetic waves in the free space, or to convert the electromagnetic waves in the free space into electromagnetic energy in the transmission line. The coupler 207 can be used to divide the mobile communication signal into multiple paths and distribute it to multiple receivers 206. It can be understood that the antenna 208 of the network device can be implemented as a large-scale antenna array.

[0308] The memory 202 is coupled to the processor 201 and is used to store various software programs and / or multiple sets of instructions. Specifically, the memory 202 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices.

[0309] The memory 202 may store an operating system (hereinafter referred to as system), such as an embedded operating system such as uCOS, VxWorks, RTLinux, etc. The memory 202 may also store a network communication program, which may be used to communicate with one or more other devices.

[0310] In the embodiment of the present application, the memory 202 may store in association the implementation code of the end-cloud collaborative working method provided in the embodiment of the present application on the cloud computing node side.

[0311] In the embodiment of the present application, the processor 201 can be used to read and execute computer-readable instructions. Specifically, the processor 201 can be used to call a program stored in the memory 202, such as an implementation program of the end-cloud collaborative working method provided by the present application on the cloud computing node side, and execute the instructions contained in the program.

[0312] Need to explain, Figure 7 The cloud computing node shown is only one implementation of the embodiment of the present application. In actual applications, the cloud computing node may also include more or fewer components, which is not limited here.

[0313] In the embodiments of the present application:

[0314] The memory 202 may be used to store the correspondence between the cloud computing node and the terminal device, that is, to store the device information of the terminal device 100 assigned to the cloud computing node.

[0315] The processor 201 is used to allocate storage resources and computing resources to the end-side device 100 and deploy a runtime for it. The runtime is used to run one or more applications (such as the first application) that are the same as those in the end-side device 100.

[0316] The communication interface 203, the transmitter 205, the receiver 206, the antenna 208, etc. are used to support the communication between the cloud computing node and other devices (such as the end-side device 100, the vendor cloud server 400, the SP 300, etc.), especially the communication in the collaborative work with the end-side device 100. The communication process between the cloud computing node and other devices can refer to the detailed description of the method embodiment above.

[0317] Figure 8 It is a hardware structure diagram of the vendor cloud server 400 provided in an embodiment of the present application. Figure 8 The vendor cloud server shown may be the vendor cloud server 400 in the aforementioned communication system 10 .

[0318] like Figure 8 As shown, the vendor cloud server may include: one or more processors 301, memory 302, communication interface 303, transmitter 305, receiver 306, coupler 307 and antenna 308. These components may be connected via bus 304 or other means. Figure 8 Take bus connection as an example.

[0319] The communication interface 303 can be used for the vendor cloud server to communicate with other communication devices. Specifically, the communication interface 303 can be a 3G communication interface, a long-term evolution (LTE) (4G) communication interface, a 5G communication interface, a WLAN communication interface, a WAN communication interface, etc. Not limited to a wireless communication interface, the network device 300 can also be configured with a wired communication interface 303 to support wired communication, for example, the backhaul link between the vendor cloud server and other vendor cloud servers can be a wired communication connection.

[0320] In some embodiments of the present application, the transmitter 305 and the receiver 306 can be regarded as a wireless modem. The transmitter 305 can be used to transmit and process the signal output by the processor 301. The receiver 306 can be used to receive signals. In the manufacturer's cloud server, the number of transmitters 305 and receivers 306 can be one or more. The antenna 308 can be used to convert electromagnetic energy in the transmission line into electromagnetic waves in free space, or to convert electromagnetic waves in free space into electromagnetic energy in the transmission line. The coupler 307 can be used to divide the mobile communication signal into multiple paths and distribute it to multiple receivers 306. It can be understood that the antenna 308 of the network device can be implemented as a large-scale antenna array.

[0321] The memory 302 is coupled to the processor 301 and is used to store various software programs and / or multiple sets of instructions. Specifically, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices.

[0322] The memory 302 may store an operating system (hereinafter referred to as system), such as embedded operating systems such as uCOS, VxWorks, RTLinux, etc. The memory 302 may also store a network communication program, which may be used to communicate with one or more other devices.

[0323] In an embodiment of the present application, the memory 302 may be associated with the storage device 302 to store the implementation code of the end-cloud collaborative working method provided in the embodiment of the present application on the manufacturer's cloud server side.

[0324] In the embodiment of the present application, the processor 301 can be used to read and execute computer-readable instructions. Specifically, the processor 301 can be used to call a program stored in the memory 302, such as the implementation program of the end-cloud collaborative working method provided by the present application on the manufacturer's cloud server side, and execute the instructions contained in the program.

[0325] Need to explain, Figure 8 The manufacturer cloud server shown is only one implementation method of the embodiment of the present application. In actual applications, the manufacturer cloud server may also include more or fewer components, which is not limited here.

[0326] In the embodiments of the present application:

[0327] The memory 302 can be used to store information about each end-side device 100 managed by the vendor's cloud server (such as location, information about installed applications, etc.), and information about each cloud computing node managed by it (such as device information, deployment location, etc.), and can also be used to store the corresponding allocation relationship between cloud computing nodes and end-side devices. The processor 301 is used to allocate cloud computing nodes to each logged-in end-side device 100. The allocation strategy and allocation method can refer to the detailed description of the method embodiment above.

[0328] The communication interface 303, the transmitter 305, the receiver 306, the antenna 308, etc. are used to support the communication between the vendor cloud server and other devices (such as the end-side device 100, the cloud computing node 200, etc.). The communication process between the vendor cloud server and other devices can refer to the detailed description of the method embodiment above.

[0329] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0330] The present application also provides an electronic device, which may include: a memory and a processor. The memory may be used to store a computer program; the processor may be used to call the computer program in the memory, so that the electronic device executes the method executed by the terminal device 100 or the cloud computing node 200 or the vendor cloud server 400 in any of the above embodiments.

[0331] The present application also provides an electronic device, which may include: a memory and a processor. The memory may be used to store a computer program; the processor may be used to call the computer program in the memory, so that the electronic device executes the method executed by the terminal device 100 or the cloud computing node 200 or the vendor cloud server 400 in any of the above embodiments.

[0332] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the terminal device 100 or the cloud computing node 200 or the manufacturer's cloud server 400 in any of the above embodiments.

[0333] In one possible design, the chip system also includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0334] The chip system may be composed of the chip, or may include the chip and other discrete devices.

[0335] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0336] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0337] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0338] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which, when executed, enables the computer to execute the method executed by the terminal device 100 or the cloud computing node 200 or the vendor cloud server 400 in any of the above-mentioned embodiments.

[0339] The present application also provides a computer-readable storage medium, which stores a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the terminal device 100 or the cloud computing node 200 or the vendor cloud server 400 in any of the above embodiments.

[0340] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

[0341] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

[0342] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0343] In short, the above is only an embodiment of the technical solution of this application, and is not intended to limit the protection scope of this application. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of this application shall be included in the protection scope of this application.

Claims

1. A communication system, characterized in that: The communication system comprises a first device and a first node, wherein: The first device is used to start a first application and send first data to the first node, where the first data includes control data and / or first service data of the first application; The first node is used to allocate a first runtime to the first device, start the first application through the first runtime, receive the first data through the first application, generate second data according to the first data through the first application, and return the second data to the first device through the first application, wherein the second data includes second service data of the first application; The first device is further configured to provide a service according to the second data.

2. The communication system according to claim 1, characterized in that The first device is further configured to, after starting the first application, send a first notification message to the first node, where the first notification message includes an identifier of the first application; The first node is specifically configured to start the first application through the first runtime after receiving the first notification message.

3. The communication system according to claim 1 or 2, characterized in that: The control data includes a search instruction, and the second service data includes a result searched by the first node according to the search instruction; Alternatively, the first service data includes an original image captured by the first device, and the second service data includes an image obtained after the first node processes the original image; Alternatively, the control data includes a game operation instruction of a user, and the second business data includes a game screen rendered by the first node according to the game operation instruction.

4. The communication system according to any one of claims 1 to 3, characterized in that: The communication system further includes a second device, The second device is used to start a second application and send third data to the first node, where the third data includes control data and / or third service data of the second application; The first node is further configured to allocate a second runtime to the second device, start the second application through the second runtime, receive the third data through the second application, generate fourth data according to the third data, and return the fourth data to the second device, the fourth data including fourth service data of the second application, and the second runtime is different from the first runtime; The second device is further configured to provide a service according to the fourth data.

5. The communication system according to any one of claims 1 to 4, characterized in that: The communication system also includes a vendor cloud server, The first device is further used to log in to the vendor cloud server before starting the first application; The vendor cloud server is used to allocate the first node among multiple nodes to the first device.

6. The communication system according to claim 5, characterized in that: The first node meets any of the following conditions: Among the multiple nodes, the first node is closest to the first device; The multiple nodes include nodes deployed in an access network, and among the nodes deployed in the access network, the first node is closest to the first device; The multiple nodes do not include nodes deployed in the access network, but include nodes deployed in the bearer network, and among the nodes deployed in the bearer network, the first node is closest to the first device; The multiple nodes do not include nodes deployed in an access network or a bearer network, but include centrally deployed nodes, and among the centrally deployed nodes, the first node is closest to the first device; The distance between the first node and the first device is within a first value.

7. The communication system according to claim 5 or 6, characterized in that: The communication system further comprises a second node, The vendor cloud server is further configured to allocate the second node among the plurality of nodes to the first device after the first device moves; The second node is used to allocate a third runtime to the first device; The first node is further used to release the first runtime allocated to the first device.

8. The communication system according to claim 7, characterized in that: After the first device moves, the distance between the first device and the second node is smaller than the distance between the first device and the first node.

9. The communication system according to any one of claims 1 to 8, characterized in that: The first node is further configured to, after allocating the first runtime to the first device, install part or all of the applications installed on the first device on the first runtime, wherein the part or all of the applications include the first application.

10. A method for end-cloud collaboration, characterized in that: The method is applied to a first device, and the method includes: Sending identifiers of some or all of the installed applications to a first node, wherein the some or all of the applications include a first application, and the identifiers of the some or all of the applications are used by the first node to install the some or all of the applications on a first runtime allocated to the first device; Starting the first application; Sending first data to the first node, where the first data includes control data and / or first service data of the first application; receiving second data returned by the first node, where the second data is generated by the first node according to the first data through the first application running on the first node, and the second data includes second service data of the first application; Provide a service based on the second data.

11. The method according to claim 10, characterized in that After starting the first application, the method further includes: A first notification message is sent to the first node, where the first notification message includes an identifier of the first application, and the identifier of the first application is used by the first node to start the first application through the first runtime.

12. The method according to claim 10 or 11, characterized in that: The control data includes a search instruction, and the second service data includes a result searched by the first node according to the search instruction; Alternatively, the first service data includes an original image captured by the first device, and the second service data includes an image obtained after the first node processes the original image; Alternatively, the control data includes a game operation instruction of a user, and the second business data includes a game screen rendered by the first node according to the game operation instruction.

13. The method according to any one of claims 10 to 12, characterized in that: Before starting the first application, the method further includes: Log in to the vendor’s cloud server; Allocation information sent by the vendor cloud server is received, where the allocation information is used to indicate that the first node is allocated to the first device.

14. The method according to claim 13, characterized in that The first node meets any of the following conditions: Among the multiple nodes, the first node is closest to the first device; The multiple nodes include nodes deployed in an access network, and among the nodes deployed in the access network, the first node is closest to the first device; The multiple nodes do not include nodes deployed in the access network, but include nodes deployed in the bearer network, and among the nodes deployed in the bearer network, the first node is closest to the first device; The multiple nodes do not include nodes deployed in an access network or a bearer network, but include centrally deployed nodes, and among the centrally deployed nodes, the first node is closest to the first device; The distance between the first node and the first device is within a first value.

15. The method according to claim 13 or 14, characterized in that The method further comprises: After the position is moved, new allocation information sent by the manufacturer's cloud server is received, and the new allocation information is used to indicate that the second node is allocated to the first device.

16. The method according to claim 15, characterized in that After the first device moves, the distance between the first device and the second node is smaller than the distance between the first device and the first node.

17. The method according to any one of claims 10 to 16, characterized in that: The second data is specifically generated by the first node after the first application running on the first runtime communicates with a cloud server, and the cloud server is used to provide services for the first application.

18. The method according to claims 10-17, characterized in that The first runtime includes any one or more of the following: a basic environment for running application code, and a runtime library.

19. The method according to any one of claims 10 to 17, characterized in that: The first application in the first device and the first application in the first node include different processing logics.

20. The method according to any one of claims 10 to 19, characterized in that: The first device includes a fourth runtime, and the first device is specifically configured to start the first application through the fourth runtime.

21. A method for end-cloud collaboration, characterized in that: The method is applied to a first node, and the method comprises: assigning a first runtime to a first device; Launching a first application through the first runtime; receiving, through the first application, first data sent by the first device, where the first data includes control data and / or first service data of the first application in the first device; generating second data according to the first data by the first application, where the second data includes second business data of the first application; The second data is returned to the first device through the first application.

22. The method according to claim 21, characterized in that Before starting the first application through the first runtime, the method further includes: A first notification message sent by the first node after starting the first application is received, where the first notification message includes an identifier of the first application.

23. The method according to claim 21 or 22, characterized in that The control data includes a search instruction, and the second service data includes a result searched by the first node according to the search instruction; Alternatively, the first service data includes an original image captured by the first device, and the second service data includes an image obtained after the first node processes the original image; Alternatively, the control data includes a game operation instruction of a user, and the second business data includes a game screen rendered by the first node according to the game operation instruction.

24. The method according to any one of claims 21 to 23, characterized in that: The method further comprises: assigning a second runtime to a second device, the second runtime being different from the first runtime; launching a second application through the second runtime; receiving, through the second application, third data sent by the second device, the third data including control data and / or third service data of the second application in the second device; generating fourth data according to the third data, wherein the fourth data includes fourth service data of the second application; The fourth data is returned to the second device.

25. The method according to any one of claims 21 to 24, characterized in that: Before allocating the first runtime to the first device, the method further includes: Allocation information sent by the manufacturer's cloud server is received, where the allocation information is used to indicate that the first node is allocated to the first device.

26. The method according to claim 25, characterized in that The first node meets any of the following conditions: Among the multiple nodes, the first node is closest to the first device; The multiple nodes include nodes deployed in an access network, and among the nodes deployed in the access network, the first node is closest to the first device; The multiple nodes do not include nodes deployed in the access network, but include nodes deployed in the bearer network, and among the nodes deployed in the bearer network, the first node is closest to the first device; The multiple nodes do not include nodes deployed in an access network or a bearer network, but include centrally deployed nodes, and among the centrally deployed nodes, the first node is closest to the first device; The distance between the first node and the first device is within a first value.

27. The method according to any one of claims 21 to 26, characterized in that: The method further comprises: receiving a second notification message sent by the vendor cloud server, where the second notification message is used to instruct the first device to release the first runtime allocated to the first device, and the second notification message is sent by the vendor cloud server after learning the mobile location of the first device; The first runtime allocated for the first device is released.

28. The method according to any one of claims 21 to 27, characterized in that: After allocating the first runtime to the first device, the method further includes: Some or all of the applications installed on the first device are installed on the first runtime, wherein the some or all of the applications include the first application.

29. The method according to any one of claims 21 to 28, characterized in that: Generating second data according to the first data specifically includes: The first application running through the first runtime communicates with the cloud server to generate second data according to the first data, and the cloud server is used to provide services for the first application.

30. An electronic device, characterized in that: include: A memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method as described in any one of claims 10-20.

31. An electronic device, characterized in that: include: A memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method as described in any one of claims 21-29.

32. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device executes the method as described in any one of claims 10-20 and 21-29.

33. A chip system, the chip system is applied to electronic equipment, the chip system comprises one or more processors, characterized in that: The processor is configured to call computer instructions so as to execute the method according to any one of claims 10-20, 21-29.

Images