Transferring heterogeneous state using generative models

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for transferring heterogeneous state using generative models. One of the methods includes receiving a natural language description of parameters of a state of a first platform to be used to configure a state of a second platform; determining, using a receiver generative model, relevant parameters of the state of the second platform based on the natural language description of the parameters of the state of the first platform; and configuring the relevant parameters of the state of the second platform using the parameters of the state of the first platform.

Description

[0001] Attorney Docket No. 56113-0795WO1

[0002] TRANSFERRING HETEROGENEOUS STATE USING GENERATIVE MODELS

[0003] BACKGROUND

[0004] [1] This specification relates to transferring state information between different platforms.

[0005] [2] People sometimes need to change their platforms, e.g., devices, applications, or services, because of technological advancements, software updates, performance issues, or aesthetic and design preferences. For example, users may update their phones, purchase additional devices, try new mobile applications, or subscribe to new services. Whenever this change happens, there is a need to transfer a state, e.g., status, settings, or configurations, of the old platform to the new platform.

[0006] [3] Generative models have demonstrated state of the art performance across a wide range of tasks, such as text generation (e.g., writing, summarization, translation, coding), image generation, and audio generation. Some generative models use very large neural network models that are trained on vast amounts of data. For example, a large language model (LLM) can include a transformer-based neural network model with self-attention capabilities and can achieve general-purpose language understanding and generation in response to a query. Thus, generative models are being deployed in various systems, platforms, or applications, e.g., as a coding assistant, as an email writing assistant, and for generating images in a presentation.

[0007] SUMMARY

[0008] [4] Transferring state information between different platforms is relatively straightforward when there is a standard format between the state of the old platform and the state of the new platform. For example, an old mobile device and a new mobile device made by the same company can have a homogeneous state format that is uniform and composed of similar elements. When transferring settings from the old mobile device to the new mobile device made by the same company, it is possible to export the settings of the old mobile device in a well-defined format and then use the exported settings to configure the settings on the new mobile device.

[0009] [5] However, some platforms can have heterogeneous state format, and thus the state information being transferred may not be directly mapped between different platforms. For example, devices having different operating systems or made by different companies can have heterogeneous state format. Some settings on the second platform can have a different label from the label of the corresponding setting on the first platform, can be in a different Attorney Docket No. 56113-0795WO1 category from the category of the corresponding setting on the first platform, or can require configuring a combination of settings on the second platform instead of a single setting on the first platform. Rule-based approaches cannot transfer such divergent settings without enumerating all possibilities (e.g., specifying corresponding labels or categories of settings between platforms), and enumerating all possibilities may not be feasible in practice. Rulebased approaches can get outdated over time and may not generalize well to future changes. For example, mappings between parameters of the states of the two platforms can change if the operating system of one of the platforms is updated.

[0010] [6] This specification describes systems and techniques for transferring heterogeneous state between platforms using generative models that are associated with the platforms. In particular, the generative models can be configured to communicate with each other in natural language and understand the heterogeneous state in natural language. In some cases, the generative models can be configured to generate natural language descriptions of tests and perform the tests to ensure that the heterogeneous state is correctly transferred.

[0011] [7] In general, one innovative aspect of the subject matter described in this specification can be embodied in methods that include the actions of receiving a natural language description of parameters of a state of a first platform to be used to configure a state of a second platform; determining, using a receiver generative model, relevant parameters of the state of the second platform based on the natural language description of the parameters of the state of the first platform; and configuring the relevant parameters of the state of the second platform using the parameters of the state of the first platform. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardw are, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0012] [8] The foregoing and other embodiments can each optionally include one or more of the following features, alone or in combination. In particular, one embodiment includes all the following features in combination. The actions can include receiving text data describing an integration test for verify ing whether the second platform is correctly configured; and after Attorney Docket No. 56113-0795WO1 configuring the relevant parameters of the state of the second platform, performing, using the receiver generative model, the integration test based on the text data describing the integration test to verify whether the second platform has been correctly configured. The text data describing the integration test describes a test environment of the second platform, an action to be performed, and an expected result in response to the action if the second platform is correctly configured, and performing, using the receiver generative model, and the integration test includes: setting the test environment of the second platform using the text data describing the test environment: performing the action described in the text data: obtaining a result generated in response to the action; and determining an integration test result by comparing the result with the expected result described in the text data. The actions include determining that the second platform has not been correctly configured; in response to determining that the second platform has not been correctly configured, generating, using the receiver generative model, text data describing a failed integration test and a clarification request; and sending the text data describing the failed integration test and the clarification request to the first platform. The natural language description of the parameters of the state of the first platform is generated by a sender generative model of the first platform in response to receiving a request for transferring the parameters of the state of the first platform to the second platform. Determining, using the receiver generative model, the relevant parameters of the state of the second platform includes: generating a prompt input including the natural language description of the parameters of the state of the first platform; processing the prompt input using the receiver generative model to obtain data describing candidate parameters of the state of the second platform; and determining the relevant parameters of the state of the second platform based on the data describing the candidate parameters of the state of the second platform. The first platform is a first device, and the second platform is a second device. The first platform is a first software application, and the second platform is a second software application. The first platform is a first service, and the second platform is a second service.

[0013] [9] In general, another innovative aspect of the subject matter described in this specification can be embodied in methods that include the actions of receiving a request for transferring a state of a first platform to a second platform, wherein the state of the first platform is to be used to configure a state of the second platform; generating, using a sender generative model, a natural language description of parameters of the state of the first platform; and sending the natural language description of the parameters of the state of the first platform to the second platform, wherein a receiver generative model of the second Attorney Docket No. 56113-0795WO1 platform is used to determine relevant parameters of the state of the second platform based on the natural language description of the parameters of the state of the first platform, and the relevant parameters of the state of the second platform is configured using the parameters of the state of the first platform. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0014]

[0010] The foregoing and other embodiments can each optionally include one or more of the following features, alone or in combination. In particular, one embodiment includes all the following features in combination. The action can include generating, using the sender generative model, text data describing an integration test for verifying whether the second platform is correctly configured; and sending the text data describing the integration test to the second platform, wherein after the relevant parameters of the state of the second platform is configured, the receiver generative model performs the integration test based on the text data describing the integration test to verify whether the second platform has been correctly configured. The text data describing the integration test describes a test environment of the second platform, an action to be performed, and an expected result in response to the action if the second platform is correctly configured. The actions can include receiving text data describing a failed integration test and a request for clarification; generating, using the sender generative model and based on the text data describing the failed integration test, an updated natural language description of the parameters of the state of the first platform; and sending the updated natural language description of the parameters of the state of the first platform to the second platform. The actions can include receiving text data describing a failed integration test and a request for clarification; generating, using the sender generative model and based on the text data describing the failed integration test, an updated natural language description of the integration test; and sending the updated natural language description of the integration test to the second platform.

[0015]

[0011] Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages. Attorney Docket No. 56113-0795WO1

[0016]

[0012] Rather than manually transferring heterogeneous state information (which might be more error prone) or using rule-based approaches enumerating complex mapping between the parameters of the states of the different platforms, the systems and methods described in this specification can robustly and automatically transfer heterogeneous state between platforms using one or more generative models associated with the platforms. A sender generative model associated with the originating platform and a receiver generative model associated with the receiving platform can map the state of the originating platform to the state of the receiving platform through one or multiple rounds of exchange of text data in natural language. For example, the systems and techniques can efficiently and accurately perform transferring of settings from one device to another device. In some implementations, the sender generative model can generate one or more integration tests for the heterogeneous state transfer, and the receiver generative model can perform the one or more integration tests to validate whether the state information is correctly transferred. In some implementations, if the integration test does not pass, the receiver generative model can generate and send a failure description to the sender generative model, and can ask the sender generative model to generate a different descnption of the heterogeneous state, a different description of the integration test, a description of a different integration test, or a combination of these.

[0017]

[0013] Compared with the rule-based approaches which may not generalize well for future updates of the platforms, the systems and techniques described herein using generative models can adapt well to future updates of the platforms and can also work in previously unseen environments. In some implementations, the systems and techniques can improve the efficiency of the communication between the platforms for transferring the heterogeneous parameters, thus saving the network and computation resources of the systems. For example, if the goal is to mute all notifications on the second device, the rule-based approaches need to enumerate notification settings for all the apps in a first mobile device and map them to the notification settings for all the apps in the second mobile device. Using the generative model based approach described herein, the systems only need to communicate a short natural language description of the request of “muting all notifications at the second mobile device,” thus saving the network communication resources and the computation resources at both the first and the second platforms.

[0018]

[0014] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims. Attorney Docket No. 56113-0795WO1

[0019] BRIEF DESCRIPTION OF THE DRAWINGS

[0020]

[0015] FIG. 1 A is a diagram of an example environment for transferring heterogeneous states between platforms.

[0021]

[0016] FIG. IB is a diagram of an example environment for transferring settings from one mobile device to another mobile device.

[0022]

[0017] FIG. 2 is a flow chart of an example process for sending heterogeneous states from an originating platform.

[0023]

[0018] FIG. 3 is a flow chart of an example process for receiving and configuring heterogeneous states at a receiving platform.

[0024]

[0019] Like reference numbers and designations in the various drawings indicate like elements.

[0025] DETAILED DESCRIPTION

[0026]

[0020] FIG. 1 A is a diagram of an example environment 100 for transferring heterogeneous state between platforms. The environment 100 includes one or more systems and each system is associated with a platform. For example, the environment 100 can include a first system 101 and a second system 111. The first system 101 can be associated with the first platform 102, and the second system 111 can be associated with the second platform 112. In some implementations, one system can be associated with two or more platforms. For example, two platforms can be two services or two apps, and the two services or the two apps can be implemented in the same system. For example, the system 101 and the system 111 can be a single system.

[0027]

[0021] Each system 101 or 111 is an example of a system implemented as computer programs on one or more computers in one or more locations, in which the systems, components, and techniques described in this specification are implemented. The one or more computers for the systems 101 and 111 can include personal computers, mobile communication devices, servers, and other devices that can send and receive data over a network. The network (not shown), such as a local area network (“LAN”), wide area network (“WAN”), the Internet, or a combination thereof, connects the one or more computers that implements the systems 101 and 111. The systems 101 and 111 can use a single computer or multiple computers operating in conjunction with one another, including, for example, a set of remote computers deployed as a cloud computing service. Attorney Docket No. 56113-0795WO1

[0028]

[0022] A platform can be a device, an application, or a service implemented on one or more computers. In some implementations, the first platform 102 can be a first device that is implemented on one or more computers and the second platform 112 can be a second device that is implemented on one or more computers. For example, the first platform can be an old mobile device and the second platform can be a new mobile device. When a user receives a new mobile device, the user may want to transfer settings on the old mobile device to the new mobile device. The settings can include network settings, notification settings, battery settings, display settings, and so on.

[0029]

[0023] In some implementations, the first platform 102 can be a first software application that is implemented on one or more computers and the second platform 112 can be a second software application that is implemented on one or more computers. For example, the first platform can be a first ride-share app that allows users to request and pay for rides through their smartphones. The second platform can be a second ride-share app that has similar functionalities. An app is a computer program or software application designed to run on a mobile device, such as a phone, tablet, or watch. In some implementations, the first software application and the second software application can be implemented in the same system on the same device. In some implementations, the first software application and the second software application can be implemented in two different systems at two different devices.

[0030]

[0024] In some implementations, the first platform 102 can be a first service that is implemented on one or more computers and the second platform 112 can be a second service that is implemented on one or more computers. For example, the first platform can be a first music streaming service, and the second platform can be a second music streaming service. In some implementations, the first service and the second service can be implemented in the same system on the same device. In some implementations, the first service and the second service can be implemented in two different systems at two different devices.

[0031]

[0025] A platform can have a state. The state can be a status, one or more settings, user preferences, configurations, or knowledge data. For example, a state of a platform can include device settings or application settings. As another example, the platform can implement a generative model that performs a task in response to a prompt input, and the state of the platform can include relevant background information for the generative model, such as data describing how to perform a given task. As another example, the platform can be a map software application, and the state of the map software application can include user preference, e.g., a preference for a particular type of restaurant. As another example, the Attorney Docket No. 56113-0795WO1 platform can be a gaming device, and the state of the gaming device can include shortcut key settings.

[0032]

[0026] Parameters can be used to describe the state of the platform. For example, parameters 106 can describe the state of the first platform 102. Parameters 116 can describe the state of the second platform 112. For example, the parameters 116 can include a name and a password for a Wi-Fi network that is included in network settings, a time range and a ring tone that is included in notification settings, or a home screen image that is included in display settings.

[0033]

[0027] People sometimes need to change their platforms. For example, users who switch from a first gaming device to a second gaming device may want to transfer their shortcut key settings. Some platforms can have heterogeneous state format, and thus the state information being transferred cannot be directly mapped between different platforms.

[0034]

[0028] For example, the systems 101 and 111 for different devices can have different operating systems, different versions of the same operating sy stem, or two different customizations of the same underlying operating system. The systems 101 and 111 can have some differences in their notification settings, including how notifications are grouped, how users interact with the notification settings, and how the notification settings are displayed. To access notification settings for applications on the first system 101, users can perform the following steps: (1) Open the Settings App; (2) Tap Notifications; (3) Tap app notifications under Manage; (4) Choose Alerting or Silent for a selected app in the list of apps; and (5) Tap on the app to go to another page for additional settings. To access notification settings for applications on the second system, users can perform the following steps: (1) Go to Settings; (2) Tap Notifications; (3) Select an app under Notification Sty le; (4) Turn on or turn off Allow Notifications; and (5) If Allow Notifications is turned on, choose the alert style under Alerts.

[0035]

[0029] Therefore, it can be time consuming to manually transfer state information between platforms with heterogeneous state formats. Rule-based software programs cannot transfer such divergent settings without specifying correspondence between labels or categories of settings between the platforms, which may not be feasible in practice.

[0036]

[0030] The systems 101 and 1 11 use generative models that are associated with the platforms to assist the transferring of the heterogeneous states between the platforms.

[0037]

[0031] A platform can be associated with a generative model. A user of the platform can interact with the platform through the generative model. The generative model can be a generative agent that interacts with the user through natural language conversations and helps Attorney Docket No. 56113-0795WO1 the user to navigate and use the platform. For example, the first platform 102 can be associated with a sender generative model 104. The second platform 112 can be associated with a receiver generative model 114.

[0038]

[0032] A generative model is a machine learning (ML) model that generates content, including text, images, audio, or other synthetic data, based on an input. During inference, the generative model can generate a generative output, e.g.. a content of a certain type, in response to a query input. In some implementations, the generative model can generate multimodal outputs, such as an image and a corresponding text describing the image.

[0039]

[0033] In some implementations, the generative model can be configured to process an input sequence of tokens to generate an output sequence of tokens. The tokens can represent any appropriate type of content, e.g.. text, image, video, audio, or some combination of the above. For example, the generative model can be a large language model (LLM) and can be configured to process an input sequence of tokens from a vocabulary of text tokens to generate an output sequence of tokens from the vocabulary.

[0040]

[0034] More generally, the generative model can be any appropriate neural network that receives an input sequence that includes text tokens and auto-regressively generates an output sequence that includes text tokens. For example, the generative model can be a Transformerbased language model neural network or a recurrent neural netw ork-based language model neural netw ork.

[0041]

[0035] In some situations, the generative model can be referred to as an auto-regressive neural network when the neural network used to implement the language model auto- regressively generates an output sequence of tokens. More specifically, the auto-regressively generated output is created by generating each particular token in the output sequence conditioned on a current input sequence that includes any tokens that precede the particular token in the output sequence, i.e., the tokens that have already been generated for any previous positions in the output sequence that precede the particular position of the particular token, and a context input that provides context for the output sequence.

[0042]

[0036] For example, the current input sequence when generating a token at any given position in the output sequence can include the input sequence and the tokens at any preceding positions that precede the given position in the output sequence. As a particular example, the current input sequence can include the input sequence follow ed by the tokens at any preceding positions that precede the given position in the output sequence. Optionally, the input and the current output sequence can be separated by one or more predetermined tokens within the current input sequence. Attorney Docket No. 56113-0795WO1

[0043]

[0037] More specifically, to generate a particular token at a particular position within an output sequence, the generative model can process the current input sequence to generate a score distribution (e.g., a probability distribution) that assigns a respective score, e.g., a respective probability, to each token in a vocabulary of tokens. The language model neural network can then select, as the particular token, a token from the vocabulary7using the score distribution. For example, the neural network of the language model can greedily select the highest-scoring token or can sample, e.g., using nucleus sampling or another sampling technique, a token from the distribution.

[0044]

[0038] As a particular example, the generative model can be an auto-regressive Transformerbased neural network that includes (i) a plurality7of attention blocks that each apply a selfattention operation and (ii) an output subnetwork that processes an output of the last attention block to generate the score distribution.

[0045]

[0039] The generative model can have any of a variety of Transformer-based neural network architectures. Examples of such architectures include those described in J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A.

[0046] Hendricks. J. WelbL A. Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203. 15556, 2022; J.W. Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, H. F. Song, J. Aslanides, S. Henderson, R. Ring, S. Young, E. Rutherford, T. Hennigan, J. Menick, A. Cassirer, R. Powell, G. van den Driessche, L. A. Hendricks, M. Rauh, P. Huang, A. Glaese, J. Welbl, S. Dathathri, S. Huang. J. Uesato, J. Mellor, I. Higgins, A. Creswell, N. McAleese, A.Wu, E. Eisen, S. M. Jayakumar, E. Buchatskaya, D. Budden, E. Sutherland, K. Simonyan, M. Paganini, L. Sifre, L. Martens, X. L. Li, A. Kuncoro, A. Nematzadeh, E. Gribovskaya, D. Donato, A. Lazaridou, A. Mensch, J. Lespiau, M. Tsimpoukelli. N. Grigorev, D. Fritz, T. Sottiaux, M. Pajarskas, T. Pohlen, Z. Gong, D. Toyama, C. de Masson d’Autume, Y. Li, T. Terzi, V. Mikulik, I. Babuschkin, A. Clark, D. de Las Casas, A. Guy, C. Jones, J. Bradbury, M. Johnson, B. A. Hechtman, L. Weidinger, I. Gabriel, W. S. Isaac, E. Lockhart, S. Osindero, L. Rimell, C. Dyer, O. Vinyals, K. Ayoub, J. Stanway, L. Bennett, D. Hassabis, K. Kavukcuoglu, and G. Irving. Scaling language models: Methods, analysis & insights from training gopher. CoRR, abs / 2112. 11446, 2021; Cohn Raffel, Noam Shazeer. Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv: 1910.10683, 2019; Daniel Adiwardana, Minh-Thang Luong, David R. So. Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. Towards a human-like open-domain chatbot. CoRR, Attomey Docket No. 56113-0795WO1 abs / 2001.09977, 2020; and Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020.

[0047]

[0040] In some implementations, the generative model can use a decoder-only architecture that includes many decoder blocks, and without using an encoder. Each decoder block can include a self-attention layer and a feed forward neural network. The transformer-based generative model is an example of a generative model that the systems and techniques herein can be applicable.

[0048]

[0041] More generally, the systems and techniques described herein are applicable to other types of generative models. One example of the generative model can be a diffusion model. As another example, the generative model 110 can be an auto-regressive generative model that auto-regressively generates tokens representing audio, video, images, or other data. As yet another example, the generative model 110 can be a masked token generative model that sequentially unmasks tokens that represent text, video, audio, images, or other data during generation.

[0049]

[0042] In some implementations, the platform can include a generative model that runs on the one or more computers that implements the system for the platform. For example, a device can have a large language model (LLM) that runs on the computer that implements the system for the device. As another example, a software application can have a specialized LLM that can provide help for users of the software application. In some implementations, multiple platforms can share the same generative model. For example, a device can have the same LLM that controls settings for different applications or sendees.

[0050]

[0043] In some implementations, the sender generative model 104 and the receiver generative model 114 can be different instances of the same generative model (e.g., with the same parameters) deployed at different systems 101 and 111. For example, a general-purpose generative model can be used at two different platforms. As another example, the same generative model can be deployed in a system that implements two different services or two different software applications.

[0051]

[0044] In some implementations, the sender generative model 104 and the receiver generative model 114 can be different generative models. For example, the sender generative model 104 can be a first ty pe of a general -purpose generative model, and the receiver generative model 114 can be a second type of a general-purpose generative model. A general- purpose generative model is trained on many types of data sets and can perform a wide range of tasks. As another example, the sender generative model 104 can be a first type of a special- Attorney Docket No. 56113-0795WO1 purpose generative model, and the receiver generative model 114 can be a second type of a special-purpose generative model. A special-purpose generative model is trained to perform a specific task.

[0052]

[0045] The system 101 of the first platform 102 receives a request for transferring the state of the first platform 102 to the second platform 112. The state of the first platform 102 is to be used to configure the state of the second platform 112. In some implementations, the system 101 can receive a request for transferring one setting of the first platform 102 to the second platform 112. In some implementations, the system 101 can receive a request for transferring all settings of the first platform 102 to the second platform 112.

[0053]

[0046] For example, a user who is activating a new mobile device may want to transfer notification settings for all the apps on the old mobile device to the new mobile device. For example, in the old mobile device, the notification for app A is silenced and the user wants to silence the notification for app A on the new mobile device as well. The old mobile device and the new mobile device can have heterogeneous format for the notification settings because they have different operating systems or different manufactures. The user can submit a request for transferring the notification settings from their old mobile device to the new mobile device. For example, the user can send a prompt input to the sender generative model 104 of the system 101. The prompt input can include: “please transfer notification settings for all apps to my new mobile device.” The system 101 can receive the request for transferring the notification setting for all the apps from the old mobile device to the new mobile device.

[0054]

[0047] In some implementations, the system 101 can automatically generate a prompt input for the sender generative model 104 in response to a request from a user. The request from a user can include a user interface action performed by a user, such as selecting an user interface element or clicking an item in a menu.

[0055]

[0048] The system 101 can identify the parameters 106 of the state of the first platform 102. In some implementations, the sender generative model can include a visual language model, and the system 101 can identify the parameters 106 using the visual language model, such as a screen understanding model or a document understanding model. The visual language model can be trained on a large amount of training data that includes images of screens, documents, or both. Once trained, the visual language model can generate natural language description of an input image of a screen or a document, e.g., text data describing the parameters in the screen. For example, a screen understanding model can be trained on training images of screens displaying settings of different platforms. In some implementations, the visual language model can be trained in an unsupervised or self Attorney Docket No. 56113-0795WO1 supervised approach, in which the visual language model can be trained to predict masked settings from the screen or the document. In some implementations, the visual language model can be trained on annotated training data and the annotation can be obtained with underlying data from the operating system. For example, some operating systems make the settings available via one or more application programming interfaces (API). The system 101 can train the visual language model to predict the settings from images of screens or documents.

[0056]

[0049] In some implementations, the system 101 can identify the parameters 106 using the reasoning capability of the sender generative model 104. For example, system 101 can use chain of thought (CoT) prompting techniques and can request the sender generative model 104 to generate a step-by-step explanation for predicting the meaning of the settings, thus improving the performance of the sender generative model 104. In some implementations, the system 101 can identify the parameters 106 using API calls or tool calls. For example, the system 101 can query a specific API on the operating system of the first platform to fetch a particular value, e.g., a setting for the system language of the first platform.

[0057]

[0050] The system 101 of the first platform 102 generates, using the sender generative model 104, a natural language description 108 of parameters 106 of the state of the first platform 102. The system 101 can use the sender generative model 104 to generate the natural language description 108 of the parameters 106.

[0058]

[0051] For example, the system 101 can receive a request to transfer notification settings for all apps from an old mobile device to a new mobile device. The request can be a user instruction in natural language. The system 101 can use the sender generative model 104 to process the request and to generate one or more actions to be performed by the system 101. The system 101 can perform the one or more actions, such as browsing the settings app in the old device, reading the user manual to understand the meaning of the settings, and making a list of parameters 106 for the notification settings for each app installed in the old mobile device. The system 101 can use the sender generative model 104 to generate the natural language description 108 of the parameters 106 for the notification settings for each app installed in the old mobile device.

[0059]

[0052] The natural language description 108 for each parameter can include an explanation of the parameter and a value of the parameter. In some implementations, the sender generative model 104 can obtain the explanation of the parameter based on a system prompt. The system prompt can include: (i) a request asking the sender generative model 104 to generate the explanation of the parameter, and (ii) information for the first platform 102 and Attorney Docket No. 56113-0795WO1 its configuration settings. The system 101 can provide the system prompt as part of the input to the sender generative model 104. For example, the system prompt can include “please generate a description of the settings for app A.” The system prompt can include “this is a Brand A phone version x.x,” or “here is a webpage that explains the notification settings for this device.” In some implementations, the system prompt can include other information for the first platform, such as a user manual of the first platform, or an image of a screenshot of the user interface of the device showing the settings of the first platform.

[0060]

[0053] The system 101 of the first platform 102 sends the natural language description 108 of the parameters 106 of the state of the first platform 102 to the second platform 112. For example, the first platform 102 and the second platform 112 can be connected through a network. The system 101 of the first platform 102 can send the natural language description 108 of the parameters 106 of the state of the first platform 102 to the second platform 112 over the network.

[0061]

[0054] The system 111 of the second platform 112 receives the natural language description 108 of the parameters 106 of the state of the first platform 102. For example, the system 111 of the new mobile device can receive, from the old mobile device, the natural language description 108 of the parameters 106 for the notification settings for each app installed in the old mobile device.

[0062]

[0055] The system 111 of the second platform 112 determines, using the receiver generative model 114. relevant parameters 116 of the state of the second platform 112 based on the natural language description 108 of the parameters 106 of the state of the first platform 102. In some implementations, the system 111 can use similar techniques described above (e.g., how the sender generative model 104 identifies the parameters 106 of the first platform 102) to determine relevant parameters 116 of the second platform 112. For example, similar to how the sender generative model can be trained to understand parameters of the state of the first platform (e.g., via settings screens or documents), the system 111 can train the receiver generative model 114 to identify relevant parameters of the state of the second platform. In some implementations, if the system 111 does not correctly identify’ the relevant parameters and a user makes a correction, the system can generate training data based on the user correction and can train the receiver generative model 114 on the training data to improve the performance of the receiver generative model 114.

[0063]

[0056] In some implementations, the system 111 of the second platform 112 can generate a prompt input including the natural language description 108 of the parameters 106 of the state of the first platform 102. The system 111 can process the prompt input using the receiver Attorney Docket No. 56113-0795WO1 generative model 114 to obtain data describing candidate parameters of the state of the second platform 112. The system 111 can determine the relevant parameters 116 of the state of the second platform 112 based on the data describing the candidate parameters of the state of the second platform 112.

[0064]

[0057] In some implementations, the system 111 of the second platform 112 can generate a prompt input including the natural language description 108 of the parameters 106 of the state of the first platform 102. For example, the prompt input can include the natural language description 108 of the parameters 106 for the notification settings for each app installed in the old mobile device.

[0065]

[0058] The system 111 can process the prompt input using the receiver generative model 114 to obtain data describing candidate parameters of the state of the second platform 112. For example, the receiver generative model 114 can generate a natural language description of one or more actions to be performed by the system 111. The one or more actions can include browsing the settings app on the second platform, and obtaining information for all parameters in the settings app that can be used to configure the state of the second platform. The system 111 can perform the one or more actions. The system 1 11 can browse the settings app on the second platform 112. The system 111 can obtain data describing the candidate parameters in the settings app on the second platform.

[0066]

[0059] In some implementations, the receiver generative model 114 can obtain data describing the candidate parameters on the second platform based on a system prompt, which can be provided as an input to the receiver generative model 1 14. For example, the system prompt can be “this is a Brand B phone version x.x,” or “here is a webpage that explains the notification settings for this device.’' In some implementations, the system prompt can include other information for the second platform, such as a user manual of the second platform.

[0067]

[0060] The system 111 can determine the relevant parameters 116 of the state of the second platform 112 based on the data describing the candidate parameters of the state of the second platform 112. For each parameter of the parameters 106 of the first platform, the receiver generative model 114 can be prompted to find a corresponding one or more parameters for the second platform based on the data describing the parameters of the state of the second platform 112. The prompt input for the receiver generative model 114 can include the natural language description 108 for the parameter of the first platform and the data describing the parameters of the second platform. Attorney Docket No. 56113-0795WO1

[0068]

[0061] For example, the old mobile device can have an “Allow Notifications” parameter. Based on the prompt input that explains the “Allow Notifications” parameter of the old mobile device and explains the notification settings in the new mobile device, the receiver generative model 114 can determine that on the new mobile device, the relevant parameter for the “Allow Notifications” parameter is the “Alerting or Silence” parameter of the new mobile device.

[0069]

[0062] The system 111 of the second platform 112 configures the relevant parameters 116 of the state of the second platform 112 using the parameters 106 of the state of the first platform 102. In some implementations, the system 111 can obtain, using the receiver generative model 114, a value of a parameter 106 based on the natural language description of the parameter 106. The receiver generative model 114 can determine a corresponding value for the relevant parameter 116 based on the data describing the parameter 106 and the parameter 116. The receiver generative model 114 can generate a natural language description of an action to be performed by the system 111. The action can result in setting the parameter 116 to the determined corresponding value.

[0070]

[0063] For example, the old mobile device can have an “Allow Notifications” parameter with value “True.” The receiver generative model 114 can obtain data describing the “Allow Notifications” parameter based on the natural language description 108. The data describing the “Allow Notifications” parameter can describe that notifications are allowed when the “Allow Notifications” is set to “True,” and that notifications are not allowed when the “Allow Notifications” is set to “False.” The receiver generative model 1 14 can obtain data describing the “Alerting or Silence” parameter of the new mobile device. The data describing the “Alerting or Silence” parameter of the new mobile device can describe that the system will send an alert when the “Alerting or Silence” parameter is set to “1.” and the system will not send an alert when the “Alerting or Silence” parameter is set to “0.” Based on the data describing the “Allow Notifications” parameter of the old mobile device and the “Alerting or Silence” parameter of the new mobile device, the receiver generative model can determine that the value for the “Alerting or Silence” parameter should be “1”. The receiver generative model 114 can generate a natural language descnption of one or more actions to be performed by the system 111 to set the value for the “Alerting or Silence” parameter to “1”.

[0071]

[0064] In some implementations, the system 111 can configure the relevant parameters 116 of the state of the second platform 112 using the parameters 106 of the state of the first platform 102 by performing the one or more user interface (UI) actions. The one or more Attorney Docket No. 56113-0795WO1 actions can include clicking buttons or entering one or more keyboard inputs on the UI, similar to what a user would do manually on the UI.

[0072]

[0065] In some implementations, the system 111 can configure the relevant parameters 116 of the state of the second platform 112 using the parameters 106 of the state of the first platform 102 by performing an application programming interface (API) call. Programs can interact with another through APIs. API calls are the medium by which they interact. The programs for the second platform 112 can expose or provide access to an API for configuring parameters 1 16 and the system 111 can submit an API call to configure the parameters 116 accordingly.

[0073]

[0066] In some implementations, the receiver generative model 114 can configure the relevant parameters 116 of the state of the second platform 112 using the parameters 106 of the state of the first platform 102 by performing an API call. The receiver generative model 114 can make the API call through function calling. Function calling allows a generative model to determine, from the user prompt, the correct function to execute from a set of available functions and the correct parameters to pass to that function.

[0074]

[0067] In some implementations, the systems 101 and 111 can be configured to perform tests to verify whether the heterogeneous state has been correctly transferred. In some implementations, the system 101 can generate, using the sender generative model 104, text data 120 describing an integration test for verifying whether the second platform 112 is correctly configured. The system 101 can send the text data 120 describing the integration test to the second platform 1 12.

[0075]

[0068] For example, the system 101 can prompt the sender generative model 104 to generate text data 120 describing an integration test to verify whether the notification setting for app A is correctly configured in the second platform. The "‘Allow Notifications” parameter of the old mobile device is set to ‘‘True,” meaning that notifications from app A are allowed. Thus, if the notification setting for app A in the new mobile device is set correctly, the system 111 will issue a notification when the app A receives a message. If the notification setting for app A in the new mobile device is not set correctly, the system 111 will not issue a notification when the app A receives a message.

[0076]

[0069] The text data 120 describing the integration test can describe a test environment 122, an action 124 to be performed, and an expected result 126 in response to the action if the second platform 112 is correctly configured. The test environment 122 can describe the setup needed for the integration test, such as date and / or time of the system 111 and the software applications currently running on the system for the integration test. The action 124 describes Attorney Docket No. 56113-0795WO1 what action the system 111 or the receiver generative model 114 needs to perform for the integration test.

[0077]

[0070] For example, the text data 120 describing the integration test to verify whether the notification setting for app A is correctly configured in the second platform can include a test environment 122, an action 124 to be performed, and an expected result 126 in response to the action if the new mobile device is correctly configured. The test environment 122 can include running the app A on the new mobile device. The action 124 to be performed can include an action that results in the app A receiving a message. The expected result 126 in response to the app A receiving the message is that the system 111 issues a notification for the message, indicating that the notification setting for the app A is correctly configured on the new mobile device.

[0078]

[0071] The system 111 of the second platform 112 can receive the text data 120 describing the integration test for verifying whether the second platform 112 is correctly configured. After configuring the relevant parameters 116 of the state of the second platform 112, the system 111 of the second platform 112 can perform, using the receiver generative model 114, the integration test based on the text data 120 describing the integration test to verify whether the second platform 112 has been correctly configured.

[0079]

[0072] In some implementations, the system 111 of the second platform 112 can set the test environment of the second platform using the text data 120 describing the test environment 122. The system 111 can perform the action 124 described in the text data 120. The system 1 1 1 can obtain a result generated in response to the action 124. The system 1 11 can determine an integration test result by comparing the result with the expected result 126 described in the text data 120.

[0080]

[0073] For example, after the system 111 sets the value for the “Alerting or Silence” parameter 116 of the new mobile device to “1”, the system 111 can perform, using the receiver generative model 114, the integration test. The receiver generative model 114 can be prompted to process the text data 120 describing the integration test to determine the test environment 122, the action 124 to be performed, and the expected result 126 in response to the action if the new mobile device is correctly configured. The system 111 can set up the environment according to the test environment 122 described in the text data 120, e g., running the app A on the new mobile device. The system 111 can perform the action 124 as described in the text data 120, e.g., triggering the app A to receive a message. The system 111 can obtain a result generated in response to the action 124. Attorney Docket No. 56113-0795WO1

[0081]

[0074] If the integration test result indicates that the integration test is successful, the system 111 can determine that the second platform 112 has been correctly configured. For example, the system 111 can determine that a notification is issued for the app A after the app A receives the message. The system 111 can determine that the integration test is passed because the actual result of issuing a notification satisfies (e.g., matches) the expected result 126. Thus, the system 111 can determine that the notification setting for the app A is correctly configured on the new mobile device.

[0082]

[0075] In some implementations, the system 101 can receive a request for transferring a list of parameters of the state of first platform 102 to the second platform 112. Each parameter on the list of parameters can have a corresponding integration test. The system 101 can send the list of parameters, the natural language description 108 of the parameters, and their integration tests to the system 111.

[0083]

[0076] The system 111 can display a progress bar 118 to indicate the transfer progress. In some implementations, the system 101 of the first platform can display the progress bar. In some implementations, a system for both platforms can display the progress bar.

[0084]

[0077] For each parameter of list of parameters of the state of first platform 102. after configuring a relevant parameter of the second platform, the system 11 1 can perform the corresponding integration test for the parameter. If the integration test passes, the system 111 can increment the progress bar 118 shown to the user. This allows the user to understand the progress of the transfer of the list of the parameters.

[0085]

[0078] For example, if a user wants to transfer one hundred settings (e.g., all of the settings) from their old mobile device to their new mobile device, the sender generative model can send, to the receiver generative model, natural language descriptions related to how to map these settings. The new mobile device, the old mobile device, or both, can show the user a progress bar 118 of how many of the one hundred settings have already been correctly transferred.

[0086]

[0079] If the integration test result indicates that the integration test is not successful, the system 111 can determine that the second platform 112 has not been correctly configured. For example, the system 111 can determine that a notification is not issued for the app A after the app A receives the message. The system 111 can determine that the integration test failed because the actual result of not issuing a notification does not match the expected result 126. Thus, the system 111 can determine that the notification setting for the app A is not correctly configured on the new mobile device. Attorney Docket No. 56113-0795WO1

[0087]

[0080] In some implementations, if the integration test does not pass, the receiver generative model 114 can be prompted to generate text data 130 describing a failed integration test, a request for clarification, or both. The system 111 can send the text data 130 describing a failed integration test, a request for clarification, or both, to system 101 of the first platform 102.

[0088]

[0081] In some implementations, the system 111 of the second platform 112 can generate, using the receiver generative model, text data 130 describing the failed integration test. For example, the text data 130 describing the failed integration test can include a natural language description of the failed integration test, including details of the test environment, the message received by the app A, and the result indicating that there is no notification for the message received by the app A. The text data 130 describing the failed integration test can include what went wrong, why it is not easy to correctly configure the parameters, and so on.

[0089]

[0082] In some implementations, the system 111 of the second platform 112 can generate, using the receiver generative model, text data 130 describing the request for clarification. For example, the text data 130 describing the request for clarification can include a request for a different description of the heterogeneous state in the first platform, a different description of the integration test, a request for a new integration test, or a combination of these.

[0090]

[0083] The system 101 of the first platform 102 can receive the text data 130 describing the failed integration test, the request for clarification, or both. The system 101 can generate, using the sender generative model 104 and based on the text data 130 describing the failed integration test, the request for clarification, or both, an updated description. The updated description can include an updated natural language description of the parameters 106 of the state of the first platform 102, updated text data describing the same integration test, new text data describing a different integration test, or a combination of these.

[0091]

[0084] For example, the updated natural language description of the notification setting for app A in the old mobile device can include additional descriptions or more examples for the “Allow Notifications” parameter. The updated text data describing the same integration test can include a more accurate description of the test environment, such as including an additional condition indicating that the system time is not after a particular time of the day.

[0092]

[0085] The system 101 can send the updated natural language description to the second platform 112. The system 111 of the second platform 112 can receive the updated natural language description. The system 111 can configure the state of the second platform 112 based on the updated natural language description, perform the integration test based on the updated natural language description, or both. Attorney Docket No. 56113-0795WO1

[0093]

[0086] For example, after receiving the updated natural language description of the parameters 106 of the state of the first platform 102, the system 111 can more accurately determine, using the receiver generative model 114, the relevant parameters 116 of the state of the second platform 112 based on the updated natural language description of the parameters 106 of the state of the first platform 102. The system 111 can configure the relevant parameters 116 of the state of the second platform 112 using the parameters 106 of the state of the first platform 102.

[0094]

[0087] In some examples, after receiving an updated natural language description of the same or a different integration test, the system 111 can perform an integration test according to the updated natural language description of the same or a different integration test to verify whether the second platform 112 has been correctly configured.

[0095]

[0088] This exchange of text data in natural language between the sender and receiver generative models can repeat for several iterations. Thus, the sender generative model 104 associated with the first platform 102 and the receiver generative model 114 associated with the second platform 112 can correctly map the state of the first platform 102 to the state of the second platform 112.

[0096]

[0089] In some implementations, while the sender and the receiver generative models exchange text data to figure out the correct mapping for one parameter 106 of the first platform, the systems 101 and 111 can configure the other parameters 116 of the second platform 112 based on the other parameters 106 of the first platform 102. In some implementations, while the sender and the receiver generative models exchange text data to figure out the correct mapping for one parameter 106 of the first platform, the systems 101 and 111 can pause configuring the other parameters 116 of the second platform 112 based on the other parameters 106 of the first platform 102 until the one parameter 106 has been correctly mapped.

[0097]

[0090] In some implementations, the transfer of the state of the first platform can be stopped if several iterations have been performed without any progress. In some implementations, the transfer of the state of the first platform can be stopped if a predetermined time limit is reached before achieving a correct transfer.

[0098]

[0091] In some implementations, the system 111 can prompt the receiver generative model 114 to generate a natural language summary' of the results, summarizing what worked and what parameter(s) the system did not manage to transfer. A user of the system 111 can read the natural language summary to understand what parameter(s) the user might still want to configure on their own. Attorney Docket No. 56113-0795WO1

[0099]

[0092] In some implementations, the system 111 can save a natural language description of the steps for reverting the state of the second platform to a state that is before the transferring of the parameters, e.g., undoing the configuration of the parameters. For example, the system can determine that only a portion of the parameters 106 can be successfully transferred to the second platform 112 and the rest of the parameters 106 cannot be successfully transferred to the second platform 112. The system 111 can determine that partially transferring parameters of the state of the first platform is undesired. For example, the system can receive an input from a user indicating that the user may either want to transfer everything or nothing. To support this, the receiver generative model 114 can execute the herein described parameter transferring process while generating a natural language description of the steps for undoing the configuration of the parameters. If the system 101 and the system 111 (e.g., using the sender generative model 104 and the receiver generative model 114) cannot successfully transfer all of the parameters from the first platform to the second platform, the system 111 (e.g., using the receiver generative model 114) can then decide to undo the configuration, resulting in the same state as before the parameter transferring process was executed.

[0100]

[0093] FIG. IB is a diagram of an example environment for transferring settings from one mobile device 152 to another mobile device 162. The first mobile device 152 is associated with the first system 151. The second mobile device 162 is associated with the second system 161. The first system 151 receives a request for transferring one or more settings 156 of the first mobile device 152 to the second mobile device 162. The first system 151 generates, using a sender LLM 154, a natural language description 158 of the one or more settings 156 of the first mobile device 152. The first system 151 sends the natural language description 158 of the one or more settings 156 to the second mobile device 162.

[0101]

[0094] The system 161 of the second mobile device 162 receives the natural language description 158 of the one or more settings 156 of the first mobile device 152. The system 161 determines, using a receiver LLM 164, one or more relevant settings 166 of the second mobile device 162 based on the natural language description 158 of the one or more settings 156 of the first mobile device 152. The system 161 configures the one or more relevant settings 166 of the second mobile device 162 using the one or more settings 156 of the first mobile device 152.

[0102]

[0095] In some implementations, the first system 151 can generate, using the sender LLM 154, text data 170 describing an integration test for verifying whether the second mobile device 162 is correctly configured. The first system 151 can send the text data 170 describing the integration test to the second mobile device 161 . Attorney Docket No. 56113-0795WO1

[0103]

[0096] In some implementations, the second system 161 can receive the text data 170 describing the integration test for verifying whether the second mobile device 162 is correctly configured. After configuring the relevant settings of the state of the second mobile device 162, the second system 161 can perform, using the receiver LLM 164, the integration test based on the text data 170 describing the integration test to verify whether the second mobile device has been correctly configured. In some implementations, the second system 161 can determine that the second mobile device 162 has not been correctly configured. In response to determining that the second mobile device 162 has not been correctly configured, the second system 161 can generate, using the receiver LLM 164, text data 180 describing a failed integration test and a clarification request. The second system 161 can send the text data 180 describing the failed integration test and the clarification request to the first mobile device 152.

[0104]

[0097] In some implementations, the first system 151 can receive the text data 180 describing the failed integration test and the request for clarification. The first system 151 can generate, using the sender LLM 154 and based on the text data 180 describing the failed integration test, updated text data 170 of the integration test, an updated description 158 of the settings of the first mobile device, or both.

[0105]

[0098] FIG. 2 is a flow chart of an example process 200 for sending heterogeneous states from an originating platform. The process 200 will be described as being performed by an appropriately programmed computer system, such as the system 101.

[0106]

[0099] The system receives a request for transferring a state of a first platform to a second platform (202). The state of the first platform is to be used to configure a state of the second platform.

[0107]

[0100] The system generates, using a sender generative model, a natural language description of parameters of the state of the first platform (204).

[0108]

[0101] In some implementations, if the sender generative model does have access to the state of the first platform, the system can receive a user input from a user of the first platform and the user input can include one or more images or a natural language description of the parameters of the state of the first platform.

[0109]

[0102] For example, the sender generative model may not be able to automatically identify the current state of the first mobile device because the current state is related to a hardware setting or requires operating system level access that is not available to the sender generative model. A user can provide one or more images showing the settings of the first mobile device, or a natural language description of the settings. Attorney Docket No. 56113-0795WO1

[0110]

[0103] The system sends the natural language description of the parameters of the state of the first platform to the second platform (206). A receiver generative model of the second platform can be used to determine relevant parameters of the state of the second platform based on the natural language description of the parameters of the state of the first platform. The relevant parameters of the state of the second platform can be configured using the parameters of the state of the first platform.

[0111]

[0104] In some implementations, the system can generate an image of a screenshot of the state of the first platform. The system can send the image of the screenshot of the state of the first platform to the second platform. In some implementations, the system can send both the natural language description of the parameters of the state of the first platform and the image of the screenshot of the state of the first platform to the second platform. The subsequent processing by the receiver system can be based on the natural language description of the parameters of the state of the first platform, the image of the screenshot of the state of the first platform, or a combination of both.

[0112]

[0105] In some implementations, the system can generate, using the sender generative model, text data describing an integration test for verifying whether the second platform is correctly configured. The system can send the text data describing the integration test to the second platform. After the relevant parameters of the state of the second platform are configured, the receiver generative model can perform the integration test based on the text data describing the integration test to verify whether the second platform has been correctly configured.

[0113]

[0106] In some implementations, the text data describing the integration test can describe a test environment of the second platform, an action to be performed, and an expected result in response to the action if the second platform is correctly configured.

[0114]

[0107] In some implementations, the system can receive text data describing a failed integration test and / or a request for clarification. The system can generate, using the sender generative model and based on the text data describing the failed integration test and / or the request for clarification, an updated natural language description of the parameters of the state of the first platform. The system can send the updated natural language description of the parameters of the state of the first platform to the second platform. The second platform can configure the state of the second platform using the updated natural language description of the parameters of the state of the first platform.

[0115]

[0108] In some implementations, the system can generate, using the sender generative model and based on the text data describing the failed integration test and / or the request for clarification, an updated natural language description of the integration test. The system can Attorney Docket No. 56113-0795WO1 send the updated natural language description of the integration test to the second platform. The second platform can perform the integration test based on the updated natural language description of the integration test.

[0116]

[0109] FIG. 3 is a flow chart of an example process 300 for receiving and configuring heterogeneous states at a receiving platform. The process 300 will be described as being performed by an appropriately programmed computer system, such as the system 111.

[0117] [HO] The system receives a natural language description of parameters of a state of a first platform to be used to configure a state of a second platform (302). In some implementations, the first platform can be a first device, and the second platform can be a second device. In some implementations, the first platform can be a first software application, and the second platform can be a second software application. In some implementations, the first platform can be a first service, and the second platform can be a second service.

[0118] [Hl] In some implementations, the natural language description of the parameters of the state of the first platform can be generated by a sender generative model of the first platform in response to receiving a request for transferring the parameters of the state of the first platform to the second platform.

[0119]

[0112] The system determines, using a receiver generative model, relevant parameters of the state of the second platform based on the natural language description of the parameters of the state of the first platform (304). In some implementations, the system can generate a prompt input including the natural language description of the parameters of the state of the first platform. The system can process the prompt input using the receiver generative model to obtain data describing candidate parameters of the state of the second platform. The system can determine the relevant parameters of the state of the second platform based on the data describing the candidate parameters of the state of the second platform.

[0120]

[0113] In some implementations, if the receiver generative model does have access to the state of the second platform, the system can receive a user input from a user of the second platform and the user input can include one or more images or a natural language description of the relevant parameters of the state of the second platform.

[0121]

[0114] For example, the receiver generative model may not be able to determine the relevant parameters of the state of the second mobile device because the state is related to a hardware setting or requires operating system level access that is not available to the receiver generative model. A user can provide one or more images showing the settings of the second mobile device, or a natural language description of the settings. Attorney Docket No. 56113-0795WO1

[0122]

[0115] The system configures the relevant parameters of the state of the second platform using the parameters of the state of the first platform (306).

[0123]

[0116] In some implementations, if the receiver generative model cannot modify the relevant parameters of the state of the second platform, the receiver generative model can output an image or a natural language description of how to modify the relevant parameters of the state of the second platform such that a user of the second platform can make the modifications.

[0124]

[0117] For example, the receiver generative model may not be able to modify hardware settings of the second mobile phone, or the receiver generative model may not have operating system level access to modify the relevant setting. The receiver generative model can display an image or a natural language description of how to modify' the settings of the second mobile phone to a user of the second mobile phone. The user can perform the modifications following the instructions in the image or the natural language description.

[0125]

[0118] In some implementations, the system can receive text data describing an integration test for verify ing whether the second platform is correctly configured. After configuring the relevant parameters of the state of the second platform, the system can perform, using the receiver generative model, the integration test based on the text data describing the integration test to verify whether the second platform has been correctly configured (308).

[0126]

[0119] In some implementations, the text data describing the integration test can describe a test environment of the second platform, an action to be performed, and an expected result in response to the action if the second platform is correctly configured. In some implementations, the system can set the test environment of the second platform using the text data describing the test environment. The system can perform the action described in the text data. The system can obtain a result generated in response to the action. The system can determine an integration test result by comparing the result with the expected result described in the text data.

[0127]

[0120] In some implementations, in response to determining that the second platform has not been correctly configured, the system can generate, using the receiver generative model, text data describing the failed integration test and a clarification request. The system can send the text data describing the failed integration test and the clarification request to the first platform.

[0128]

[0121] In some implementations, the system can receive an updated natural language description of the parameters of the state of the first platform from the first platform. The system can configure the state of the second platform based on the updated natural language description of the parameters of the state of the first platform. Attorney Docket No. 56113-0795WO1

[0129]

[0122] In some implementations, the system can receive an updated natural language description of the integration test from the first platform. The system can perform an integration test based on the updated natural language description of the integration test.

[0130]

[0123] In some implementations, the transferring of the heterogeneous parameters can be stopped after several iterations have been performed without any progress. In some implementations, the transferring of the heterogeneous parameters can be stopped if some time limit has been reached.

[0131]

[0124] This specification uses the term “configured” in connection with systems and computer program components. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0132]

[0125] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non- transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium can be a machine- readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.

[0133]

[0126] The term “data processing apparatus” refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also be. or further include, off-the-shelf or custom-made parallel processing subsystems, e.g., a GPU or another kind of special-purpose processing subsystem. Attorney Docket No. 56113-0795WO1

[0134] The apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0135]

[0127] A computer program which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code) can be w ritten in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub-programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.

[0136]

[0128] As used in this specification, an ‘‘engine,” or “software engine,” refers to a software implemented input / output system that provides an output that is different from the input. An engine can be an encoded block of functionality, such as a library, a platform, a software development kit (“SDK”), or an object. Each engine can be implemented on any appropriate type of computing device, e.g., servers, mobile phones, tablet computers, notebook computers, music players, e-book readers, laptop or desktop computers, PDAs, smart phones, or other stationary or portable devices, that includes one or more processors and computer readable media. Additionally, two or more of the engines may be implemented on the same computing device, or on different computing devices.

[0137]

[0129] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flow s can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.

[0138]

[0130] Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Attorney Docket No. 56113-0795WO1

[0139] Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory’ or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to. or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0140]

[0131] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory’ devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0141]

[0132] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and pointing device, e.g.. a mouse, trackball, or a presence sensitive display or other surface by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g.. a smartphone, running a messaging application, and receiving responsive messages from the user in return.

[0142]

[0133] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context Attorney Docket No. 56113-0795WO1 of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0143]

[0134] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0144]

[0135] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

[0145]

[0136] What is claimed is:

Claims

Attorney Docket No. 56113-0795WO1CLAIMS1. A method performed by one or more computers, the method comprising: receiving a natural language description of parameters of a state of a first platform to be used to configure a state of a second platform; determining, using a receiver generative model, relevant parameters of the state of the second platform based on the natural language description of the parameters of the state of the first platform; and configuring the relevant parameters of the state of the second platform using the parameters of the state of the first platform.

2. The method of claim 1, further comprising: receiving text data describing an integration test for verifying whether the second platform is correctly configured; and after configuring the relevant parameters of the state of the second platform, performing, using the receiver generative model, the integration test based on the text data describing the integration test to verify whether the second platform has been correctly configured.

3. The method of claim 2, wherein the text data describing the integration test describes a test environment of the second platform, an action to be performed, and an expected result in response to the action if the second platform is correctly configured, and performing, using the receiver generative model, the integration test comprises: setting the test environment of the second platform using the text data describing the test environment; performing the action described in the text data; obtaining a result generated in response to the action; and determining an integration test result by comparing the result with the expected result described in the text data.

4. The method of any one of claims 2 or 3, further comprising: determining that the second platform has not been correctly configured;Attorney Docket No. 56113-0795WO1 in response to determining that the second platform has not been correctly configured, generating, using the receiver generative model, text data describing a failed integration test and a clarification request; and sending the text data describing the failed integration test and the clarification request to the first platform.

5. The method of any preceding claim, wherein the natural language description of the parameters of the state of the first platform is generated by a sender generative model of the first platform in response to receiving a request for transferring the parameters of the state of the first platform to the second platform.

6. The method of any preceding claim, where determining, using the receiver generative model, the relevant parameters of the state of the second platform comprises: generating a prompt input comprising the natural language description of the parameters of the state of the first platform; processing the prompt input using the receiver generative model to obtain data describing candidate parameters of the state of the second platform; and determining the relevant parameters of the state of the second platform based on the data describing the candidate parameters of the state of the second platform.

7. The method of any one of claims 1-6, wherein the first platform is a first device, and the second platform is a second device.

8. The method of any one of claims 1-6, wherein the first platform is a first software application, and the second platform is a second software application.

9. The method of any one of claims 1-6, wherein the first platform is a first service, and the second platform is a second service.

10. A method performed by one or more computers, the method comprising: receiving a request for transferring a state of a first platform to a second platform, wherein the state of the first platform is to be used to configure a state of the second platform; generating, using a sender generative model, a natural language description of parameters of the state of the first platform; andAttorney Docket No. 56113-0795WO1 sending the natural language description of the parameters of the state of the first platform to the second platform, wherein a receiver generative model of the second platform is used to determine relevant parameters of the state of the second platform based on the natural language description of the parameters of the state of the first platform, and the relevant parameters of the state of the second platform is configured using the parameters of the state of the first platform.

11. The method of claim 10, further comprising: generating, using the sender generative model, text data describing an integration test for verifying whether the second platform is correctly configured; and sending the text data describing the integration test to the second platform, wherein after the relevant parameters of the state of the second platform is configured, the receiver generative model performs the integration test based on the text data describing the integration test to verify whether the second platform has been correctly configured.

12. The method of claim 11. wherein the text data describing the integration test describes a test environment of the second platform, an action to be performed, and an expected result in response to the action if the second platform is correctly configured.

13. The method of any one of claims 11-12. further comprising: receiving text data describing a failed integration test and a request for clarification; generating, using the sender generative model and based on the text data describing the failed integration test, an updated natural language description of the parameters of the state of the first platform; and sending the updated natural language description of the parameters of the state of the first platform to the second platform.

14. The method of any one of claims 11-13, further comprising: receiving text data describing a failed integration test and a request for clarification; generating, using the sender generative model and based on the text data describing the failed integration test, an updated natural language description of the integration test; and sending the updated natural language description of the integration test to the second platform.Attorney Docket No. 56113-0795WO115. A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving a natural language description of parameters of a state of a first platform to be used to configure a state of a second platform; determining, using a receiver generative model, relevant parameters of the state of the second platform based on the natural language description of the parameters of the state of the first platform; and configuring the relevant parameters of the state of the second platform using the parameters of the state of the first platform.

16. The system of claim 15, the operations further comprise: receiving text data describing an integration test for verifying whether the second platform is correctly configured; and after configuring the relevant parameters of the state of the second platform, performing, using the receiver generative model, the integration test based on the text data describing the integration test to verify whether the second platform has been correctly configured.

17. The system of claim 16, wherein the text data describing the integration test describes a test environment of the second platform, an action to be performed, and an expected result in response to the action if the second platform is correctly configured, and performing, using the receiver generative model, the integration test comprises: setting the test environment of the second platform using the text data describing the test environment; performing the action described in the text data; obtaining a result generated in response to the action; and determining an integration test result by comparing the result with the expected result described in the text data.

18. The system of any one of claims 16-17, the operations further comprise: determining that the second platform has not been correctly configured;Attorney Docket No. 56113-0795WO1 in response to determining that the second platform has not been correctly configured, generating, using the receiver generative model, text data describing a failed integration test and a clarification request; and sending the text data describing the failed integration test and the clarification request to the first platform.

19. The system of any preceding claim, wherein the natural language description of the parameters of the state of the first platform is generated by a sender generative model of the first platform in response to receiving a request for transferring the parameters of the state of the first platform to the second platform.

20. One or more non-transitory storage media encoded with instructions that when executed by a computing device cause the computing device to perform operations comprising: receiving a natural language description of parameters of a state of a first platform to be used to configure a state of a second platform; determining, using a receiver generative model, relevant parameters of the state of the second platform based on the natural language description of the parameters of the state of the first platform; and configuring the relevant parameters of the state of the second platform using the parameters of the state of the first platform.

Images