Synchronizing access control between computing devices

Abstract

The present disclosure relates to synchronizing access controls between computing devices. Synchronization of access controls between computing devices is provided. A system receives a request from a first device. The system performs a session handoff to a second device in response to determining an incompatibility. The system modifies a parameter in an access control database. The system receives a request from a third device. The system provides a digital component to the third device.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese invention patent application 201980002194.X, filed on February 13, 2019.

[0003] Cross-references to related applications

[0004] This application claims priority to U.S. Provisional Patent Application No. 62 / 668,218, filed May 7, 2018, pursuant to 35 U.S. SC § 119, which is incorporated herein by reference in its entirety. Technical Field

[0005] This disclosure relates to synchronous access control between computing devices. Background Technology

[0006] Different computing devices may have different capabilities, such as different input or output interfaces or computing performance. Summary of the Invention

[0007] This disclosure generally relates to synchronizing access control between computing devices. Accessing certain types of digital components or modifying access control parameters of such digital components from certain types of devices can be challenging or impossible due to limited input or output interfaces or other processing capabilities available on different devices. The system and method of this technical solution allow a session to access or modify a digital component from a first device, and then transfer the session to a second device to complete the access or modification. Access control parameters can be modified in a digital access control database that can be used by multiple devices to access the digital component.

[0008] At least one aspect relates to a system for synchronizing access control between computing devices. The system may include a data processing system having one or more processors. The data processing system may execute an interactive data exchange component, a compatibility checker component, a session transfer component, and an access controller component. The interactive data exchange component may receive data packets via an interface of the data processing system, the data packets including input audio signals detected by sensors of a first client device linked to an account identifier. The interactive data exchange component may parse the data packets including the input audio signals based on natural language processing techniques to identify a request for a digital component and one or more keywords corresponding to the request. The interactive data exchange component may establish a session with the first client device. The interactive data exchange component may generate an input prompt based on policies for the digital component stored in a digital access control database, the input prompt having fields in response to the request for the digital component from the first client device linked to the account identifier. The compatibility checker component may determine that the input prompt is incompatible with the first client device. In response to determining that the input prompt is incompatible with the first client device, the session transfer component may identify and connect to the state of a second client device linked to the account identifier linked to the first client device. The compatibility checker can determine that the input prompt is compatible with the second client device based on the state identified by the session transfer component. The session transfer component can transfer the session to the second client device and terminate a portion of the session with the first client device. In response to the transfer, the interactive data exchange component can transmit the input prompt to the second client device. The interactive data exchange component can receive data packets input via the second client device in response to the input prompt. The access controller component can modify the parameters of the digital component stored in the digital access control database for the account identifier based on the data packets input via the second client device. The interactive data exchange component can receive a request to execute the digital component on a third client device linked to the account identifier. In response to the access controller component's verification of the parameters of the digital component stored in the digital access control database for the account identifier, and the compatibility checker component's determination that the digital component is compatible with the third client device, the data processing system can provide the digital component for the third client device to execute.

[0009] At least one aspect relates to a method for synchronizing access control between computing devices. The method can be performed by a data processing system having one or more processors and memory. The method may include: the data processing system receiving a data packet, the data packet including an input audio signal detected by a sensor of a first client device linked to an account identifier. The method may include: the data processing system parsing the data packet with the input audio signal based on natural language processing techniques to identify a request for a digital component and one or more keywords corresponding to the request. The method may include: the data processing system establishing a session with the first client device. The method may include: the data processing system generating an input prompt based on a policy for the digital component stored in a digital access control database, the input prompt having fields in response to the request for the digital component by the first client device linked to the account identifier. The method may include: the data processing system determining, via the data processing system, that the input prompt is incompatible with the first client device. The method may include: in response to determining that the input prompt is incompatible with the first client device, the data processing system identifying the state of a second client device linked to the account identifier linked to the first client device. The method may include: the data processing system determining, based on the state identified by the session transfer component, that the input prompt is compatible with the second client device. The method may include: the data processing system performing a portion of transferring the session to the second client device and terminating the session with the first client device. The method may include: in response to the transfer, the data processing system transmitting the input prompt to the second client device. The method may include: the data processing system receiving a data packet input via the second client device in response to the input prompt. The method may include: the data processing system modifying parameters of the digital component stored in the digital access control database for the account identifier based on the data packet input via the second client device. The method may include: the data processing system receiving a request to execute the digital component on a third client device linked to the account identifier. The method may include: in response to verification of the parameters of the digital component stored in the digital access control database for the account identifier by the access controller component, and determination of compatibility of the digital component with the third client device by the compatibility checker component, the data processing system can provide the digital component for execution by the third client device.

[0010] Each aspect may optionally include one or more of the following features: Identifying multiple client devices linked to the account identifier; identifying a subset of the multiple client devices compatible with the digital component; and providing the digital component for execution on the subset of the multiple client devices in response to modification of the parameters. Converting the format of the digital component based on the state of the third client device. Receiving a request to execute the second digital component from the third client device; determining that the third client device is not authorized to execute the second digital component based on parameters stored in the access control database for the second digital component; transmitting an input prompt for the second digital component to the second client device; and authorizing the third client device in response to input provided via the second client device. Storing one or more rules for the digital component in the access control database. Modifying the parameters to provide authorization for the third client device to execute the digital component. Determining the type of the parameters based on the type of the digital component; and modifying the parameters based on the type. Identifying multiple fields in the input prompt; generating an audio query for each of the multiple fields; and receiving one or more input audio signals in response to the audio query generated for each of the multiple fields. The session may include an access control session, which can be initiated in response to the input audio signal detected via the first client device; the access control session on the first client device is suspended; the access control session is transferred from the first client device to the second client device; and the access control session is terminated in response to the data packet entered via the second client device in response to the input prompt. The digital component may be provided by a third-party computing device. The data processing system may instruct the third-party computing device to provide the digital component for execution by the third client device. The first client device may execute a first surface or user interface type, the second client device may execute a second surface or user interface type, and the third client device may execute a third surface or user interface type different from the first and second surfaces. The data processing system may be configured to provide the digital component for execution by the third and second surfaces. Parameters of the digital component on the digital access control database for the account identifier are provided for storage in the memory of the third client device; the parameters of the digital component stored in the memory of the third client device are updated in response to the data packet entered via the second client device in response to the input prompt. Identify devices within a predetermined distance of the first digital assistant device; and wherein the second device is selected based on the devices identified within the predetermined distance of the first digital assistant device.

[0011] These and other aspects, as well as implementation methods, are discussed in detail below. The foregoing information and the following detailed description include illustrative examples of various aspects and implementation methods, and provide an overview or framework for understanding the nature and characteristics of the claimed aspects and implementation methods. The accompanying drawings provide illustration and further understanding of the various aspects and implementation methods, and are incorporated in and constitute a part of this specification. Attached Figure Description

[0012] The accompanying drawings are not intended to be drawn to scale. In the various drawings, the same reference numerals and names denote the same elements. For clarity, not every component will be labeled in every drawing. In the drawings:

[0013] Figure 1 This is a diagram of an example system used for synchronizing access control between computing devices;

[0014] Figure 2 This is a diagram illustrating an example operation of a system used to synchronize access control between computing devices;

[0015] Figure 3 This is a diagram illustrating an example method for synchronizing access control between computing devices; and

[0016] Figure 4 This is a block diagram illustrating the overall architecture of a computer system that can be used to implement elements of the systems and methods described and illustrated herein, including, for example... Figure 1 and 2 The system described in Figure 3 The method described in the text. Detailed Implementation

[0017] The following is a more detailed description of various concepts related to methods, apparatuses, and systems for routing packetization operations via computer networks, as well as various implementations thereof. The various concepts introduced above and discussed in more detail below can be implemented in any of many ways.

[0018] This disclosure generally relates to synchronizing access control between computing devices. Accessing certain types of digital components or modifying access control parameters of such digital components from certain types of devices can be challenging or impossible due to limited input or output interfaces or other processing capabilities available on different devices. The system and method of this solution allow a session to access or modify a digital component from a first device, and then transfer the session to a second device to complete the access or modification. Access control parameters can be modified in a digital access control database that can be used by multiple devices to access the digital component.

[0019] By providing a data processing system that automatically determines that a first digital assistant device is missing or not optimally configured to handle certain types of commands or instructions, and then transfers the session to a second digital assistant device, the data processing system provides a guided interaction process utilizing the attributes of both the first and second digital assistant devices. The data processing system can reduce unnecessary processing performed by the first digital assistant device while providing additional technical capabilities for adjusting or modifying access controls by seamlessly transferring the session to the second digital assistant. Furthermore, the data processing system can maintain an access control database, which may include parameters or rules for digital components. The access control database may be a centrally located access control database. By maintaining or managing the access control database, the data processing system can facilitate the provision of digital components to devices authorized to execute the digital components. This solution can reduce resource consumption, processor utilization, battery consumption, bandwidth utilization, audio file size, or the amount of time consumed by the first digital assistant device, which may not include input or output ports, interfaces, or components for approving or authenticating commands to modify access controls of digital components.

[0020] For example, digital components can be requested via a digital assistant human-machine interface (referred to herein as a surface) such as a conversational application programming interface (API). This can include requests for in-app items, subscriptions, or paid applications, and can be provided by third-party developers in their applications where digital assistant components or surfaces can be used. When a digital component (e.g., audiobooks, digital currency, electronic documents, or digital music) is accessed, the data processing system can provide this digital component for use by some or all digital assistant surfaces or applications linked to an account. For example, an audiobook can be requested on a first digital assistant device, which may be a speaker, and then subsequently executed on a smartphone.

[0021] The data processing system can record requests or access control parameters for digital components in a central database, which captures access control parameters adjusted by the user across different digital assistant surfaces. When using a specific digital assistant surface or application, the digital assistant surface can request the account's digital component access control parameters from the central database. If access control adjustments cannot be completed on a particular device, device handover can be performed (e.g., transferring the session to a smartphone to confirm an electronic transaction). For example, if authentication is required but is not provided in the first digital assistant device, the data processing system can transfer the session to a second digital assistant device (e.g., a smartphone) for authentication. Once authentication is complete, a device linked to the account and compatible with the digital component can use the digital component. Therefore, the system and method of this technical solution can allow digital components to be obtained via a voice interface across any digital assistant device, surface, or application, and then provided for execution on any digital assistant device, surface, or application configured to perform the digital component.

[0022] Figure 1 An example system 100 for synchronizing access control between computing devices is illustrated. System 100 may include a content selection infrastructure. System 100 may include a data processing system 102. The data processing system 102 may communicate via a network 105 with one or more of a content provider computing device 154, a third-party device 156, or a local computing device 140. Network 105 may include: computer networks such as the Internet, local area networks, wide area networks, metropolitan area networks or other domain networks, intranets, satellite networks, and other communication networks such as voice or data mobile phone networks. Network 105 may be used to access information resources, such as web pages, websites, domain names, or Uniform Resource Locators, which may be presented, output, rendered, or displayed on at least one local computing device 140 (e.g., a laptop computer, desktop computer, tablet computer, digital assistant device, smartphone, portable computer, or speaker). For example, via network 105, a user of local computing device 140 may access information or data provided by content provider 154. For example, computing device 140 may or may not include a display; for example, computing device may include limited types of user interfaces, such as microphones and speakers. In some cases, the main user interface of computing device 140 may be a microphone and speaker, or a voice interface.

[0023] Local computing device 140 can refer to a computing device or client device located in a public place—such as a hotel, office, restaurant, retail store, shopping mall, park—or a private place—such as a residence. The term "local" can also mean that the computing device is located in a location where a user can interact with it using voice input or other input. The local computing device can be located away from a remote server, such as data processing system 102. Therefore, local computing device 140 can be located in a hotel room, shopping mall, cubicle, or other building or residence where a user can interact with it using voice input, while data processing system 102 can be remotely located in a data center. Local computing device 140 can be referred to as a digital assistant device.

[0024] Network 105 may include or constitute a display network, such as a subset of information resources available on the Internet that are associated with content placement or search engine results systems, or a subset of information resources eligible to include third-party digital components as part of a digital component placement campaign. Data processing system 102 may use network 105 to access information resources, such as web pages, websites, domain names, or Uniform Resource Locators that can be presented, output, rendered, or displayed via local client computing device 140. For example, via network 105, a user of local client computing device 140 may access information or data provided by content provider computing device 154 or service provider computing device 108.

[0025] Network 105 can be any type or form of network and can include any of the following: point-to-point network, broadcast network, wide area network, local area network, telecommunications network, data communication network, computer network, ATM (Asynchronous Transfer Mode) network, SONET (Synchronous Optical Network) network, SDH (Synchronous Digital Hierarchy) network, wireless network, and wired network. Network 105 may include wireless links, such as infrared channels or satellite bands. The topology of network 105 may include bus, star, or ring network topologies. The network may include a mobile phone network that uses any one or more protocols for communication between mobile devices, including Advanced Mobile Phone Protocol (“AMPS”), Time Division Multiple Access (“TDMA”), Code Division Multiple Access (“CDMA”), Global System for Mobile Communications (“GSM”), General Packet Radio Service (“GPRS”), or Universal Mobile Telecommunications System (“UMTS”). Different types of data may be transmitted via different protocols, or the same type of data may be transmitted via different protocols.

[0026] System 100 may include at least one data processing system 102. Data processing system 102 may include at least one logical device, such as a computing device having a processor to communicate via network 105 with, for example, computing device 140, content provider computing device 154 (content provider 154), or third-party device 156 (or third party 156). Data processing system 102 may include at least one computing resource, server, processor, or memory. For example, data processing system 102 may include multiple computing resources or servers located in at least one data center. Data processing system 102 may include multiple logically grouped servers and facilitates distributed computing technologies. The logical group of servers may be referred to as a data center, server cluster, or machine cluster. Servers may also be geographically distributed. A data center or machine cluster may be managed as a single entity, or the machine cluster may include multiple machine clusters. Servers in each machine cluster may be heterogeneous, and one or more of the servers or machines may operate according to one or more types of operating system platforms.

[0027] Servers in a cluster can be stored alongside associated storage systems in a high-density rack system within an enterprise data center. For example, consolidating servers in this way by positioning them on a localized, high-performance network can improve system manageability, data security, physical security, and system performance. Centralizing all or some components of the data processing system 102, including servers and storage systems, and coupling them with advanced system management tools, allows for more efficient use of server resources, thereby saving power and processing demands and reducing bandwidth usage.

[0028] System 100 may include, access, or otherwise interact with at least one third-party device 156. Third-party device 156 may include at least one logical device, such as a computing device having a processor to communicate via network 105 with, for example, computing device 140, data processing system 102, or content provider 154. Third-party device 156 may include at least one computing resource, server, processor, or memory. For example, third-party device 156 may include multiple computing resources or servers located in at least one data center.

[0029] Content provider device 154 may provide audio-based digital components for display as audio output digital components by local computing device 140. The digital components may include a proposal of goods or services, such as a voice-based message stating, "Would you like me to order you a taxi?" For example, content provider computing device 154 may include memory for storing a series of audio digital components that can be provided in response to a voice-based query. Content provider computing device 154 may also provide audio-based digital components (or other digital components) to data processing system 102, wherein the digital components may be stored in data repository 122. Data processing system 102 may select and provide (or instruct content provider computing device 154 to provide) audio digital components to local client computing device 140. Audio-based digital components may be audio only or may be combined with text, image, or video data.

[0030] Third-party device 156 may include, be coupled to, or otherwise communicate with data processing system 102. Third-party device 156 may include, be coupled to, or otherwise communicate with local computing device 140. Third-party device 156 may include, be coupled to, or otherwise communicate with computing device 140, which may be a mobile computing device. Third-party device 156 may include, be coupled to, or otherwise communicate with content provider device 154. For example, third-party device 156 may provide digital components to local computing device 140 for execution by local computing device 140. Third-party device 156 may provide digital components to data processing system 102 for storage by data processing system 102. Third-party device 156 may provide rules or parameters related to digital components to data processing system 102 for storage in access control database 132.

[0031] Local computing device 140 may include at least one sensor 144, transducer 146, audio driver 148, or preprocessor 150, coupled to or otherwise communicating with thereto. Local computing device 140 may include a light source 142 or display device 152, such as a light indicator, light-emitting diode (“LED”), organic light-emitting diode (“OLED”), or other visual indicator configured to provide visual or light output. Sensor 144 may include, for example, an ambient light sensor, proximity sensor, temperature sensor, accelerometer, gyroscope, motion detector, GPS sensor, position sensor, microphone, or touch sensor. Transducer 146 may include a speaker or microphone. Audio driver 148 may provide a software interface to hardware transducer 146. Audio driver may execute audio files or other instructions provided by data processing system 102 to control transducer 146 to generate corresponding sound waves or audio waves. Preprocessor 150 may include a processing unit with hardware configured to detect keywords and perform actions based on those keywords. Before transmitting one or more terms to the data processing system 102 for further processing, the preprocessor 150 may filter or modify terms. The preprocessor 150 may convert analog audio signals detected by a microphone into digital audio signals and transmit one or more data packets carrying the digital audio signals to the data processing system 102 via the network 105. In some cases, the processor 150 may transmit data packets carrying some or all of the input audio signals in response to detecting an instruction to perform such transmission. Instructions may include, for example, trigger keywords or other keywords or approvals, to transmit data packets including the input audio signals to the data processing system 102.

[0032] A local client computing device 140 may be associated with an end user who inputs a voice query as audio input (via sensor 144) into the local client computing device 140 and receives audio output in the form of computer-generated speech. This audio output may be provided to the local client computing device 140 from data processing system 102 (or content provider computing device 154 or service provider computing device 108) and output from transducer 146 (e.g., a speaker). The computer-generated speech may include recordings from real people or computer-generated language.

[0033] Data repository 122 may include one or more local or distributed databases and may include a database management system. Data repository 122 may include computer data storage or storage and may store one or more signature and account information 124, linked device information 126, content data 128, template data 130, and access control database 132, as well as other data. Access control database 132 may include or store parameters 134, rules 136, and digital components 138 (or identifiers or reference identifiers of digital components 138). Signature and account information 124 may include acoustic or voice signatures that can be used by speaker identification component 112 to identify an account corresponding to a user of local computing device 140. Signature 124 may include acoustic or audio signatures. Signature 124 may refer to an acoustic fingerprint (which may include a compressed digital profile), a fingerprint deterministically generated from an audio signal—the fingerprint can be used to identify an audio sample or quickly locate similar items in an audio database. Signature 124 may include data that facilitates the identification of profiles by speaker identification component 112.

[0034] Account information may include or reference profile information, preferences, electronic account information, or an account corresponding to access control database 132. Linked device information 126 may include or refer to device identifiers associated with account 124 (e.g., local computing device 140, first digital assistant device 280, second digital assistant device 282, or third digital assistant device 284). Linked device information 126 may include devices already provided by data processing system 102 for use with the account. Linked device information 126 may include or reference devices authorized or authenticated to access digital component 138 according to access control database 132. Content data 128 may include digital components or associated metadata for audio output, as well as input audio messages, which may be part of one or more communication sessions with local client computing device 140. Content data 128 may include, for example, content campaign information, content groups, content selection criteria, digital component objects, or other information provided by content provider 154 or obtained or determined by data processing system to facilitate content selection. Content data 128 may include, for example, historical performance of content campaigns.

[0035] Template data 130 may include or refer to templates used to generate input prompts or other prompts, notifications, or alerts. Template 130 may be stored, for example, based on available user interfaces, input interfaces, or output interfaces available on local computing devices 140, to provide different templates configured for different types of local computing devices 140. Template 130 may be created by a third-party device 156 and provided to the data processing system 102 for storage and use, or template 130 may be created by the administrator of the data processing system 102 or generated based on rules or policies.

[0036] Access control database (“DB”) 132 may include information about digital component 138. Digital component 138 may refer to or include electronic data or data files that can be executed by local computing device 140 or a linked device (e.g., a third digital assistant device 284). Digital component 138 may refer to or include audiobooks, digital music files, digital video files, multimedia files, video games, programs, scripts, application programming interfaces, or digital currencies. Digital component 138 may be stored in access control database 132, or references or identifiers of digital component 138 may be stored in access control database 132. For example, digital component 138 (or its data files) may be stored on third-party device 156 (e.g., the provider or developer of digital component 138), and access control database 132 may store deep links, Uniform Resource Locators (URLs), or other reference identifiers pointing to digital component 138. Access control database 132 may store parameters 134. Parameter 134 may indicate the type of access or access control to digital component 138. Parameter 134 may include, for example, a threshold, distance, time interval, duration, score, or weight. Parameters may include values. These values ​​may be numbers, binary, alphanumeric, or symbols. For example, a parameter may indicate whether the local computing device 140 associated with the account can access the digital component 138 or not. Parameter 134 may be [Authorized], and its value may be [Yes] or [No]. Parameter 134 may be [subscription_type], and its value may be [one_time_use], [number_of_uses], [unlimited_use], or [time_interval], where the time interval may be weekly, monthly, or yearly. Other parameters 134 may be established by the third-party device 156 or the developer or provider of the digital component 138. Rule 136 may refer to or include policies or characteristics of the digital component that can be used to establish parameter 134 for the digital component 138. For example, the rule could be: if the digital component type is an audiobook, the parameter value could be [one_time_use], [number_of_uses], [unlimited_use], or [time_interval]. The rule could be: if the digital component is cryptocurrency, the parameter value could be [exchangeable]. The rule could be: if the digital component is streaming a multimedia file, the parameter could be [downloading_prohibited] or [monthly subscription].Therefore, by allowing different rules 136 and parameters 134 to be configured for different digital components 138, the access control database 132 can maintain or manage different types of digital components and their access controls within the central data processing system 102. The data processing system can store one or more rules in the access control database of a digital component.

[0037] Data processing system 102 may include a content placement system having at least one computing resource or server. Data processing system 102 may include at least one interface 106, coupled to or otherwise communicating with it. Data processing system 102 may include at least one natural language processor component 108, coupled to or otherwise communicating with it. Data processing system 102 may include at least one direct action application programming interface (“API”) 110, coupled to or otherwise communicating with it. Interface 106, natural language processing component 108, and direct action API 110 may form an interactive data exchange component 104. Data processing system 102 may include at least one interactive data exchange component 104, coupled to or otherwise communicating with it. Interactive data exchange component 104 may include or reference dialogue APIs or digital assistant functions. Interactive data exchange component 104 may communicate or be coupled to one or more voice-based interfaces or various digital assistant devices or surfaces to provide or receive data or perform other functions.

[0038] Data processing system 102 may include at least one content selector component 120, coupled to or otherwise communicating with it. Data processing system 102 may include at least one speaker recognition component 112, coupled to or otherwise communicating with it. Data processing system 102 may include at least one data repository 122, coupled to or otherwise communicating with it. Data processing system 102 may include at least one access controller component 114, coupled to or otherwise communicating with it. Data processing system 102 may include at least one compatibility checker component 116, coupled to or otherwise communicating with it. Data processing system 102 may include at least one session transfer component 118, coupled to or otherwise communicating with it.

[0039] Interactive data exchange component 104, interface 106, natural language processor component 108, direct action API 110, speaker recognition component 112, access controller component 114, compatibility checker component 116, session transfer component 118, or content selector component 120 may each include at least one processing unit or other logic device, such as a programmable logic array engine, or a module configured to communicate with database repository 122 or a database. Interactive data exchange component 104, interface 106, natural language processor component 108, direct action API 110, speaker recognition component 112, access controller component 114, compatibility checker component 116, session transfer component 118, or content selector component 120 and data repository 122 may be separate components, single components, or part of data processing system 102. System 100 and its components—such as data processing system 102—may include hardware elements, such as one or more processors, logic devices, or circuitry.

[0040] Data processing system 102 can obtain anonymous computer network activity information associated with multiple local computing devices 140 (or computing devices or digital assistant devices). Users of local computing devices 140 or mobile computing devices can authorize data processing system 102 to obtain network activity information corresponding to local computing devices 140 or mobile computing devices. For example, data processing system 102 can prompt users of computing devices 140 to consent to obtaining one or more types of network activity information. Local computing devices 140 may include mobile computing devices, such as smartphones, tablets, smartwatches, or wearable devices. The identity of users of local computing devices 140 can remain anonymous, and computing devices 140 can be associated with unique identifiers (e.g., unique identifiers of users or computing devices provided by users of the data processing system or computing devices). The data processing system can associate each observation with a corresponding unique identifier.

[0041] Content provider 154 can establish electronic content campaigns. Electronic content campaigns can be stored as content data 128 in data repository 122. An electronic content campaign can refer to one or more content groups corresponding to a public topic. The content campaign can include a hierarchical data structure comprising content groups, digital component data objects, and content selection criteria. To create a content campaign, content provider 154 can specify values ​​for campaign hierarchy parameters. Campaign hierarchy parameters can include, for example, a campaign name, a preferred content network for placing digital component objects, a value for the resources used in the content campaign, a start and end date for the content campaign, the duration of the content campaign, the timing of digital component object placement, language, geographic location, and the type of computing device on which the digital component objects are provided. In some cases, an impression can refer to when a digital component object is retrieved from its source (e.g., data processing system 102 or content provider 154), and the impressions are countable. In some cases, bot activity can be filtered and excluded as an impression due to the possibility of click fraud. Therefore, in some cases, a flash can refer to a measurement of the web server's response to a page request from a browser, filtered from bot activity and error codes, and recorded at a point as close as possible to the opportunity to render the digital component object for display on computing device 140. In some cases, a flash can refer to a visible or audible flash; for example, the digital component object is at least partially visible (e.g., 20%, 30%, 30%, 40%, 50%, 60%, 70% or more) on the display device 152 of the local client computing device 140, or audible via the speaker 136 of computing device 140. A click or selection can refer to user interaction with the digital component object, such as a voice response to an audible flash, a mouse click, touch interaction, gesture, shake, audio interaction, or keyboard click. A conversion can refer to a user taking a desired action in response to a digital component objection; for example, purchasing a product or service, completing a group survey, visiting a physical store corresponding to the digital component, or completing an electronic transaction.

[0042] Content providers 154 can further establish one or more content groups for content campaigns. Content groups include one or more digital component objects and corresponding content selection criteria, such as keywords, terms, phrases, geographic location, computing device type, time of day, interests, topics, or vertical elements. Content groups under the same content campaign can share the same campaign-level parameters, but can have customized specifications for specific content group-level parameters, such as keywords, negative keywords (e.g., preventing digital component placement if negative keywords are present in the main content), keyword bidding, or parameters associated with bidding or content campaigns.

[0043] To create a new content group, a content provider can provide values ​​for the content group-level parameters. Content group-level parameters include, for example, the content group name or content group title, and bids for different content placement opportunities (e.g., automatic or managed placement) or effects (e.g., clicks, flashes, or conversions). The content group name or content group title can be one or more terms that the content provider 154 can use to capture the topic or theme for which to select digital component objects for display within the content group. For example, a car dealership can create different content groups for each brand of vehicles it carries, and further, different content groups for each model of vehicle it carries. Examples of content group titles that a car dealership could use include, for example, “Make Asportscar,” “Make Bsportscar,” “Make C Sedan,” “Make C Truck,” “Make C Hybrid,” or “Make D Hybrid.” An example content campaign title could be “hybrid,” and include content groups such as “Make C Hybrid” and “Make D Hybrid.”

[0044] Content provider 154 may provide one or more keywords and digital component objects to each content group. Keywords may include terms related to or identified by a digital component object. Keywords may include one or more terms or phrases. For example, a car dealer may include "sports car," "V-6 engine," "four-wheel drive," and "fuel efficiency" as keywords for a content group or content campaign. In some cases, content providers may specify negative keywords to avoid, prevent, block, or disable the placement of certain terms or keywords in content. Content providers may specify the matching type used to select digital component objects, such as exact match, phrase match, or broad match.

[0045] Content provider 154 may provide one or more keywords for data processing system 102 to select digital component objects provided by content provider 154. Content provider 154 may identify one or more keywords to be bid on and further provide the bid amount for each keyword. Content provider 154 may provide additional content selection criteria for data processing system 102 to select digital component objects. Multiple content providers 154 may bid on the same or different keywords, and data processing system 102 may run a content selection process or ad auction in response to keyword instructions received via electronic message.

[0046] Content provider 154 may provide one or more digital component objects for data processing system 102 to select. Data processing system 102 (e.g., via content selector component 120) may select a content placement opportunity when it becomes available, matching it with resource allocation, content timing, maximum bid, keywords, and other selection criteria specified for the content group. The content group may include different types of digital component objects, such as voice digital components, audio digital components, text digital components, image digital components, video digital components, multimedia digital components, or digital component links. Digital component objects (or digital components) may include, for example, content items, online documents, audio, images, video, multimedia content, or sponsored content. After selecting a digital component, data processing system 102 may transmit the digital component object for rendering on computing device 140 or its display device 152. Rendering may include displaying the digital component on the display device or playing the digital component via speakers of computing device 140. Data processing system 102 may provide instructions to computing device 140 to render the digital component object. The data processing system 102 can instruct the computing device 140 or the audio driver 148 of the computing device 140 to generate audio signals or sound waves.

[0047] Data processing system 102 may include an interface 106 (or interface component) designed, configured, constructed, or operated to receive and transmit information using, for example, data packets. Interface 106 may use one or more protocols (e.g., network protocols) to receive and transmit information. Interface 106 may include a hardware interface, a software interface, a wired interface, or a wireless interface. Interface 106 may facilitate the conversion or reformatting of data from one format to another. For example, interface 106 may include an application programming interface (API) that includes definitions for communication between various components, such as software components. Interface 106 may communicate via network 105 with one or more of a local computing device 140, a content provider device 154, or a third-party device 156.

[0048] The data processing system 102 can be coupled to an application, script, or program (e.g., an application) installed on the local client computing device 140 to transmit input audio signals to the interface 106 of the data processing system 102 and drive components of the local client computing device to render output audio signals. The data processing system 102 can receive data packets or other signals that include or identify the audio input signals.

[0049] Data processing system 102 or interactive data exchange component 104 may include a natural language processor ("NLP") component 108. For example, data processing system 102 may execute or run NLP component 108 to receive or acquire audio signals and parse them. For example, NLP component 108 may provide human-computer interaction. NLP component 108 may be configured with techniques for understanding natural language and allowing data processing system 102 to derive meaning from human or natural language input. NLP component 108 may include or be configured with machine learning techniques based on, for example, statistical machine learning. NLP component 108 may utilize decision trees, statistical models, or probabilistic models to parse input audio signals. NLP component 108 can perform functions such as: named entity recognition (e.g., given a text stream, determining which items in the text map to proper names of, for example, people or places, and the type of each such name, such as person, location, or organization), natural language generation (e.g., translating information or semantic intent from a computer database into understandable human language), natural language understanding (e.g., translating text into a more formal representation, such as a first-order logical structure that a computer module can manipulate), machine translation (e.g., automatically translating text from one human language to another), lexical segmentation (e.g., separating words into individual morphemes and identifying the categories of morphemes, which can be challenging based on the lexical or structural complexity of the words in the language under consideration), question answering (e.g., determining answers to human language questions, which can be specific or open-ended), and semantic processing (e.g., processing that can be performed after recognizing words and encoding their meanings so that the recognized words are associated with other words having similar meanings).

[0050] By comparing the audio input signal with a stored set of representative audio waveforms (e.g., stored in data repository 122) and selecting the closest match, NLP component 108 can convert the input signal into recognizable text. The audio waveform set may be stored in data repository 122 or another database accessible to data processing system 102. Representative waveforms are generated across a large user set and can then be augmented using speech samples from users. After the audio signal is converted into recognizable text, NLP component 108 matches the text with words that are associated with actions that data processing system 102 can serve, for example, through cross-user training or by manually specifying such associations. Aspects or functions of NLP component 108 may be performed by data processing system 102 or local computing device 140. For example, a local NLP component may execute on local computing device 140 to perform aspects of converting the input audio signal into text and transmitting the text to data processing system 102 via data packets for further natural language processing.

[0051] The audio input signal can be detected by sensor 144 or transducer 146 (e.g., microphone) of local client computing device 140. The audio input signal can be provided to data processing system 102 (e.g., via network 105) via transducer 146, audio driver 148, or other components of local client computing device 140, where the audio input signal can be received (e.g., via interface 106) and provided to NLP component 108 or stored in data repository 122.

[0052] Local computing device 140 may include an audio driver 148, a transducer 146, a sensor 144, and a preprocessor component 150. Sensor 144 may receive or detect input audio signals (e.g., voice input). Preprocessor component 150 may be coupled to the audio driver, transducer, and sensor. Preprocessor component 150 may filter the input audio signal to create a filtered input audio signal (e.g., by removing certain frequencies or suppressing noise). Preprocessor component 150 may (e.g., using a software or hardware digital-to-analog converter) convert the filtered input audio signal into data packets. In some cases, preprocessor component 150 may convert an unfiltered input audio signal into data packets and transmit the data packets to data processing system 102. Preprocessor component 150 may transmit data packets to data processing system 102, which includes one or more processors and memories executing natural language processing components, interfaces, speaker recognition components, and direct action application programming interfaces.

[0053] Data processing system 102 can receive data packets from a preprocessor component via an interface, the data packets including filtered (or unfiltered) input audio signals detected by sensors. Data processing system 102 can identify acoustic signatures from the input audio signals. Data processing system 102 can identify electronic accounts corresponding to the acoustic signatures based on lookups in a data repository (e.g., querying a database). In response to the identification of electronic accounts, data processing system 102 can establish a session and an account for the session. Accounts may include profiles with one or more policies. Data processing system 102 can parse the input audio signals to identify requests and trigger keywords corresponding to the requests.

[0054] The data processing system 102 can provide status information to the preprocessor component of the local computing device 140. The local computing device 140 can receive indications of the status. The audio driver can receive indications of the profile status and generate an output signal based on the indications. The audio driver can convert the indications into an output signal, such as a sound signal or an acoustic output signal. The audio driver can drive the transducer 146 (e.g., a speaker) to produce sound based on the output signal generated by the audio driver.

[0055] In some cases, the local computing device 140 may include a light source. The light source may include one or more LEDs, lamps, displays, or other components or devices configured to provide optical or visual output. The preprocessor component may enable the light source to provide visual indications corresponding to its status. For example, visual indications may be: an on status indicator, a color change of the lamp, a lamp pattern with one or more colors, or a visual display of text or images.

[0056] NLP component 108 can acquire an input audio signal. Based on the input audio signal, NLP component 108 can identify at least one request or at least one trigger keyword corresponding to the request. The request may indicate the intent or topic of the input audio signal. The trigger keyword may indicate the type of action that may be taken. The trigger keyword may be a wake-up signal or hot word that instructs local computing device 140 to convert subsequent audio input into text and transmit the text to data processing system 102 for further processing. For example, NLP component 108 can parse the input audio signal to identify at least one request to leave home for dinner or to see a movie in the evening. The trigger keyword may include at least one word, phrase, root word or part of a word, or a derived word indicating the action to be taken. For example, the trigger keyword “go” or “togo to” from the input audio signal may indicate a need for transportation. In this example, the input audio signal (or the identified request) does not directly express an intent to transport, but the trigger keyword indicates that transportation is an auxiliary action to at least one other action indicated by the request.

[0057] NLP component 108 can parse the input audio signal to identify, determine, retrieve, or otherwise obtain request and trigger keywords. For example, NLP component 108 can apply semantic processing techniques to the input audio signal to identify trigger keywords or requests. NLP component 108 can apply semantic processing techniques to the input audio signal to identify trigger phrases that include one or more trigger keywords (e.g., a first trigger keyword and a second trigger keyword). For example, the input audio signal may include the statement "I want to purchase an audiobook". NLP component 108 can apply semantic processing techniques or other natural language processing techniques to data groups including said statement to identify the trigger phrases "want to purchase" and "audiobook". NLP component 108 can further identify multiple trigger keywords, such as purchase and audiobook. For example, NLP component 108 can determine that the trigger phrase includes a trigger keyword and a second trigger keyword.

[0058] NLP component 108 can filter the input audio signal to identify trigger keywords. For example, a data packet carrying the input audio signal might include "It would be great if I could get someone that could help me go to the airport." In this case, NLP component 108 can filter out one or more terms, such as: "it," "would," "be," "great," "if," "I," "could," "get," "someone," "that," "could," or "help." By filtering out these terms, NLP component 108 can more accurately and reliably identify trigger keywords, such as "go to the airport," and determine that this is a request for a taxi or ride-sharing service.

[0059] In some cases, the NLP component can determine that the data packet carrying the input audio signal includes one or more requests. For example, the input audio signal may include the statement "I want to purchase an audiobook and a monthly subscription to movies." NLP component 108 can determine that this is a request for audiobooks and streaming multimedia services. NLP component 108 can determine whether this is a single request or multiple requests. NLP component 108 can determine that this is two requests: a first request to a service provider offering audiobooks, and a second request to a service provider offering movie streaming services. In some cases, NLP component 108 can combine multiple determined requests into a single request and transmit said single request to third-party device 156. In some cases, NLP component 108 can transmit individual requests to another service provider device, or transmit two requests separately to the same third-party device 156.

[0060] Data processing system 102 may include a Direct Action API 110, which is designed and configured to generate action data structures in response to requests based on trigger keywords. A processor of data processing system 102 may invoke Direct Action API 110 to execute scripts that generate data structures to provide to third-party device 156 or other service providers to obtain digital components, order services, or products, such as a car from a car-sharing service or an audiobook. Direct Action API 110 may obtain data from data repository 122, as well as data received from local client computing device 140 with end-user consent, to determine location, time, user account, logistics, or other information, allowing third-party device 156 to perform actions such as booking a car from a car-sharing service. Using Direct Action API 110, data processing system 102 may also communicate with third-party device 156 to complete a conversion in this example by making a car-sharing pick-up reservation.

[0061] The Direct Action API 110 can execute specified actions to satisfy the intent of the end user, as determined by the data processing system 102. Depending on the actions specified in its input and the parameters or rules in the data repository 122, the Direct Action API 110 can execute code or dialogue scripts that identify the parameters required to satisfy the user's request. Such code may, for example, look up additional information in the data repository 122 such as the name of a home automation service or a third-party service, or it may provide audio output to be rendered at the local client computing device 140 to ask the end user questions, such as the intended destination of a requested taxi. The Direct Action API 110 can determine the parameters and can package the information into an action data structure, which can then be sent to another component—such as the content selector component 120—or to the service provider computing device 108 for execution.

[0062] The Direct Action API 110 can receive instructions or commands from the NLP component 108 or other components of the data processing system 102 to generate or construct action data structures. The Direct Action API 110 can determine the type of action to select a template from the template repository 122 stored in the data repository 122. Action types can include, for example, services, products, appointments, tickets, multimedia content, audiobooks, managing subscriptions, adjusting subscriptions, transferring digital currency, making purchases, or music. Action types can further include the type of service or product. For example, service types can include car-sharing services, food delivery services, laundry services, domestic services, repair services, housekeeping services, equipment automation services, or media streaming services. Product types can include, for example, clothing, shoes, toys, electronics, computers, books, or jewelry. Appointment types can include, for example, dinner appointments or hair salon appointments. Ticket types can include, for example, movie tickets, sports stadium tickets, or airline tickets. In some cases, the type of service, product, appointment, or ticket can be categorized based on price, location, transportation type, availability, or other attributes.

[0063] NLP component 108 can parse the input audio signal to identify a request and a trigger keyword corresponding to the request, and provide the request and trigger keyword to Direct Action API 110, so that Direct Action API 110 generates a first action data structure in response to the request based on the trigger keyword. After identifying the type of request, Direct Action API 110 can access a corresponding template from template repository 122. The template may include fields from a structured dataset, which can be populated by Direct Action API 110 to advance the operation requested by the input audio detected by local computing device 140 via third-party device 156 (e.g., dispatching a taxi to pick up an end user at a pick-up location and take them to a destination location). Direct Action API 110 can perform a lookup in template repository 122 to select a template that matches one or more features of the trigger keyword and request. For example, if the request corresponds to a request for a car or a ride to a destination, data processing system 102 can select a car-sharing service template. The car-sharing service template may include one or more of the following fields: device identifier, pick-up location, destination location, number of passengers, or service type. Direct Action API 110 can populate the fields with values. To populate fields with values, the Direct Action API 110 can ping, poll, or otherwise obtain information from one or more sensors 144 of the computing device 140 or from the user interface of the device 140. For example, the Direct Action API 110 can detect the source location using a location sensor (e.g., a GPS sensor). The Direct Action API 110 can obtain further information by submitting surveys, prompts, or queries to the user interface of the computing device 140. The Direct Action API can submit surveys, prompts, or queries via the interface 106 of the data processing system 102 and the user interface of the computing device 140 (e.g., an audio interface, a voice-based user interface, a display, or a touchscreen). Therefore, the Direct Action API 110 can select a template for an action data structure based on trigger keywords or requests, populate one or more fields in the template with information detected by one or more sensors 144 or obtained via the user interface, and generate, create, or otherwise construct the action data structure to facilitate the performance of an operation by the third-party device 156.

[0064] To construct or generate action data structures, the data processing system 102 can identify one or more fields in the selected template to populate with values. These fields can be populated with numeric values, strings, Unicode values, Boolean logic, binary values, hexadecimal values, identifiers, location coordinates, geographic regions, timestamps, or other values. The fields or data structures themselves can be encrypted or masked to maintain data security.

[0065] After determining the fields in the template, data processing system 102 can identify the values ​​of the fields to populate the fields of the template, thereby creating an action data structure. Data processing system 102 can obtain, retrieve, determine, or otherwise identify the values ​​of the fields by performing lookups or other query operations on data repository 122.

[0066] In some cases, data processing system 102 may determine that information or values ​​for fields are missing in data repository 122. Data processing system 102 may determine that information or values ​​stored in data repository 122 are outdated, obsolete, or otherwise unsuitable for constructing action data structures that respond to trigger keywords and requests identified by NLP component 108 (e.g., the location of local client computing device 140 may be an old location instead of the current location; the account may have expired; the target restaurant may have moved to a new location; sports activity information; or mode of transportation).

[0067] If data processing system 102 determines that it cannot currently access the value or information of a template field in its memory, it can retrieve the value or information. Data processing system 102 can retrieve or obtain the information by querying or polling one or more available sensors of local client computing device 140, prompting the end user of local client computing device 140 regarding the information, or accessing online web-based resources using the HTTP protocol. For example, data processing system 102 may determine that it does not have the current location of local client computing device 140, which may be a required field of the template. Data processing system 102 can query location information from local client computing device 140. Data processing system 102 can request local client computing device 140 to provide location information using one or more location sensors 144 (e.g., GPS sensors, Wi-Fi triangulation, cell tower triangulation, Bluetooth beacons, IP addresses, or other location sensing technologies).

[0068] In some cases, data processing system 102 may use a second profile to generate an action data structure. Data processing system 102 may then determine whether the action data structure generated using the second profile is compatible with the first profile. For example, the first profile may include a strategy to block a type of action data structure, such as purchasing a product from an online electronic retailer via local computing device 140. The input audio detected by local computing device 140 may have included a request to purchase a product from an online electronic retailer. Data processing system 102 may have used the second profile to identify account information associated with the online electronic retailer and then generated the action data structure for purchasing the product. The action data structure may include an account identifier corresponding to an electronic account associated with an acoustic signature identified by speaker recognition component 112.

[0069] Data processing system 102 may include speaker identification component 112, which is designed, constructed, and operated to receive audio input signals detected by local computing device 140, identify acoustic signatures, and identify electronic accounts corresponding to the acoustic signatures. Speaker identification component 112 may receive data packets including input audio signals detected by sensors 144 or transducers 146 of local computing device 140 via interface 106. Speaker identification component 112 may identify acoustic signatures from the input audio signals. Speaker identification component 112 may identify electronic accounts corresponding to acoustic signatures based on lookups in data repository 122.

[0070] Speaker identification component 112 can receive data packets with a payload carrying an input audio signal. Speaker identification component 112 can perform pre-filtering or preprocessing on the input audio signal to remove certain frequencies of the audio. Pre-filtering can include filters such as low-pass filters, high-pass filters, or band-pass filters. Filters can be applied in the frequency domain. Digital signal processing techniques can be used to apply the filters. Filters can be configured to preserve frequencies consistent with human speech or human voice while eliminating frequencies falling outside the typical frequencies of human voice. For example, a band-pass filter can be configured to remove frequencies below a first threshold (e.g., 70Hz, 75Hz, 80Hz, 85Hz, 90Hz, 95Hz, 100Hz, or 105Hz) and above a second threshold (e.g., 200Hz, 205Hz, 210Hz, 225Hz, 235Hz, 245Hz, or 255Hz). Applying band-pass filters can reduce computational resource utilization in downstream processing. In some cases, the preprocessor 150 on the local computing device 140 can apply a bandpass filter before transmitting the input audio signal to the data processing system 102, thereby reducing network bandwidth utilization. However, given the computing resources available to the local computing device 140, providing the input audio signal to the data processing system 102 to allow the data processing system 102 to perform filtering may be more efficient.

[0071] Speaker identification component 112 can apply additional preprocessing or pre-filtering techniques to facilitate the identification of electronic accounts corresponding to speakers. For example, speaker identification component 112 can apply noise reduction techniques to reduce the level of ambient noise that may interfere with speaker identification. Noise reduction techniques can improve the accuracy and speed of speaker identification, thereby improving the efficiency of data processing system 102 in identifying electronic accounts.

[0072] Speaker identification component 112 can access signature 124 stored in data repository 122. Speaker identification component 112 can analyze filtered input audio signal to identify signature and use said signature to identify electronic account. Therefore, speaker identification component 112 can perform speaker identification or speech recognition to identify electronic account corresponding to signature of input audio signal.

[0073] Speaker identification component 112 may be configured with one or more speaker identification techniques, such as pattern recognition. Speaker identification component 112 may be configured with a text-independent speaker identification process. In the text-independent speaker identification process, the text used to establish the electronic account may differ from the text subsequently used to identify the speaker.

[0074] For example, speaker identification component 112 can identify distinct acoustic features among input speech sources in the input audio signal. These acoustic features can reflect physical patterns or learned patterns that can correspond to a unique input speech source. Acoustic features may include, for example, speech tone or speaking style. Techniques for identifying, processing, and storing signatures may include frequency estimation (e.g., instantaneous fundamental frequency or discrete energy separation algorithms), hidden Markov models (e.g., stochastic models for modeling randomly varying systems where future states depend on current states and where the modeled system has unobserved states), Gaussian mixture models (e.g., parametric probability density functions represented as a weighted sum of Gaussian component densities), pattern matching algorithms, neural networks, matrix representations, vector quantization (e.g., quantization techniques in signal processing that allow modeling of probability density functions through the distribution of prototype vectors), or decision trees. Other techniques may include speaker de-speaker techniques such as cohort models and world models. Speaker identification component 112 may be configured with machine learning models to facilitate pattern recognition or adapt speaker characteristics.

[0075] After recognizing the acoustic signature in the input audio signal, the speaker identification component 112 can perform a lookup in the data repository 122 or a signature data structure (e.g., signature and account 124). The signatures stored in the signature data structure can be mapped to identifiers of electronic accounts. The speaker identification component 112 can use the signature recognized in the input audio signal to perform a lookup in the signature data structure to retrieve the electronic account corresponding to the signature of the input audio.

[0076] In some cases, speaker identification component 112 may determine, based on a lookup, that no electronic account corresponding to the acoustic signature exists. For example, the signature may not be stored in signature data structure 124, or an electronic account for the signature may not yet have been created. The lookup may return null values ​​or an empty set. In response to determining that no signature or electronic account exists, speaker identification component 112 may create an electronic account in a data repository. Speaker identification component 112 may update signature data structure 124 to include a new signature constructed based on the input audio signal and its corresponding electronic account. Speaker identification component 112 may map the electronic account to the new acoustic signature.

[0077] Therefore, the data processing system 102 may include an interactive data exchange component 104 that can receive data packets via an interface 106 of the data processing system 102. These data packets include input audio signals detected by a sensor 144 of a local computing device 140 (e.g., a first client device or a first digital assistant device) linked to an account identifier. The interactive data exchange component 104 may parse the data packets including the input audio signals based on natural language processing techniques (e.g., via NLP component 108) to identify requests for digital components and one or more keywords corresponding to those requests. The interactive data exchange component 104 may establish a session with the local computing device 140 and generate input prompts based on policies or rules 136 stored in a digital access control database 132, wherein the input prompts have fields responding to requests for digital components from the first client device linked to the account identifier.

[0078] For example, the interactive data exchange component 104 may communicate with the access control database 132 or the access controller component 114 to determine if additional input may be required to fulfill a request. Additional input may be user credentials, electronic account information, payment information, transaction information, or authentication information. The interactive data exchange component 104, communicating with the data repository 122 or the access controller component 114, may use template 130 to generate input prompts. Template 130 may be a value for parameter 134, determined by rule 136 established for the requested digital component 138. For example, if the request is to listen to an audiobook, the data processing system 102 may generate input prompts to obtain information about how the user wishes to pay for the audio if the user wants to stream or download the audiobook, if the user wants to establish a monthly subscription to an audiobook provider, or if the user wants to establish a one-time streaming-only configuration, or any other type of configuration made available or provided by a third-party device 156 providing the audiobook service. Therefore, input prompts may include one or more fields with one or more input text boxes, drop-down menus, buttons, or other graphical user interface widgets. Data processing system 102 may transform the input text boxes for a conversational computing environment. Data processing system 102 may determine that an input text box is unsuitable for a voice-based conversational computing environment; for example, data processing system 102 may determine that transforming the input text box may be cumbersome, inefficient, insecure, processor-intensive, or cause transaction delays.

[0079] Data processing system 102 may include a compatibility checker component 116, coupled to or otherwise accessed by the compatibility checker component 116, which is designed, constructed, and operated to determine whether an input prompt is compatible with local computing device 140. Data processing system 102 may determine that the input prompt is incompatible with a first client device linked to a user account. Data processing system 102 may determine that the input prompt is compatible with a second client device linked to a user account.

[0080] The compatibility checker component 116 can determine the type of input or output associated with the generated input prompt. The compatibility checker component 116 can receive device information associated with the local computing device 140. The compatibility checker component 116 can compare the type of input or output associated with the generated input prompt with the input and output interfaces available on the local computing device 140. If the compatibility checker component 116 determines that the input type required by the generated input prompt is not available on the local computing device 140, then the compatibility checker component 116 can determine that the local computing device 140 is incompatible with the generated input prompt. For example, the input prompt may request an electronic transaction account number, but the local computing device 140 may not include a keyboard, mouse, or touchscreen. Therefore, the local computing device 140 may not provide a mechanism for entering the electronic account number. Although it is possible to provide an audio signal or speak a number, the data processing system 102 can determine that providing electronic account information by voice may not be secure. Therefore, the data processing system 102 can determine incompatibility.

[0081] The compatibility checker component 116 can determine that the local computing device 140 is a public or insecure computing device, insufficiently secure to receive account information or authentication information, and therefore incompatible. The compatibility checker component 116 can use various strategies or techniques to determine incompatibility based on resource consumption, device configuration, interface availability, preferences, or security. The compatibility checker component 116 can determine a strategy based on rules for the digital component requested by the user. For example, the rule for the requested digital component could be biometric authentication, such as fingerprint as a preferred authentication technology, but the local computing device 140 may not include a biometric sensor. Therefore, the compatibility checker component 116 can determine that the local computing device 140 is incompatible with the generated input prompts.

[0082] When it is determined that the local computing device 140 is incompatible with the generated input prompt, the data processing system 102 can identify a second computing device. For example, the data processing system 102 may include a session transfer component 118, a state coupled to or otherwise accessed by the session transfer component 118, which is designed, constructed, and operated to identify a second client device in response to determining that the input prompt is incompatible with the first client device, and the second client device is linked to an account identifier associated with the first client device. The first and second client devices may include one or more components or functions of the local computing device 140. The second client device may be compatible with the input prompt because it may include the required input interface, output interface, application, program, computing resources, or other functionality.

[0083] To identify a second client device, data processing system 102 (e.g., via session transfer component 118) may poll one or more devices linked to the account, as indicated in the linked device data structure 126. Data processing system 102 may perform a lookup in the linked device data structure 126 using the account identifier to determine the device identifier of the linked device, and then poll such devices. Data processing system 102 may use other polling techniques to identify nearby devices or devices connected to the first client device (e.g., via a wireless connection). Session transfer component 116 may receive device information from the second client device to determine the status of the client device. The status may indicate the availability of input interfaces, output interfaces, computing resources, or other functions used to fill, satisfy, or respond to generated input prompts. For example, session transfer component 118 may poll the second client device for status information regarding the availability of a keyboard input or touchscreen input interface, and the status information may indicate that the touchscreen interface is available.

[0084] The compatibility checker component 116 can determine that the input prompt is compatible with the second client device based on the state identified by the session transfer component. The compatibility checker component 116 can then instruct the session transfer component 118 to transfer the session. The session transfer component 118 can transfer the session to the second client device and terminate a portion of the session with the first client device. Transferring a session may refer to or include transferring the session with the first device to the second device. Transferring a session may refer to or include initiating a session to perform a requested electronic transaction with the second device. Transferring a session may refer to or include terminating the session with the first client device and then continuing that session with the second client device. Transferring a session may refer to or include initiating a second session with the second client device. Transferring a session may refer to or include providing information associated with a portion of the session conducted via the first client device to the second client device. Transferring a session may refer to or include transmitting or forwarding the generated input prompt to the second client device.

[0085] A session may include or refer to an access control session. An access control session may refer to or include one or more data communications or connections associated with obtaining, requesting, authorizing, or approving access to a digital component. Data processing system 102 may initiate an access control session in response to an input audio signal detected via a first client device (e.g., local computing device 140). In response to determining that a generated input prompt is incompatible with local computing device 140 or the first client device, data processing system 102 may suspend the access control session on the first client device. Data processing system 102 may transfer the access control session from the first client device to a second client device. In response to a data packet entered via the second client device in response to the input prompt, data processing system 102 may terminate the access control session.

[0086] After a session is transferred or handed over, the interactive data exchange component 104 may transmit input prompts to the second client device in response to the handover. The interactive data exchange component 104 may also receive data packets input via the second client device in response to the input prompts.

[0087] Data processing system 102 may include, be coupled to, or otherwise accessed by, an access controller component 114, which is designed, constructed, and operated to modify parameters of digital components stored in a digital access control database for account identifiers based on data packets input via a second client device. Access controller component 114 may maintain access control database 132. Access controller component 114 may modify, adjust, manipulate, or update access control database 132. For example, access controller component 114 may update subscription service parameters for digital components stored in access control database 132 to indicate monthly subscription, one-time viewing, annual subscription, or other purchase types. For example, access controller component 114 may update parameter values ​​for digital components for an account to indicate that devices linked to the account are authorized to perform, render, process, or otherwise present the digital component.

[0088] Access controller component 114 can maintain subscription, purchase, or access information for any type of digital component in access control database 132. Access controller component 114 can be configured to parse or process rule 136 corresponding to a type of digital component 138 to update the value of parameter 134. Access controller component 114 can receive updated or new rules from third-party device 156 or other providers of digital components and store the updated or new rules in access control database 132. Therefore, access controller component 114 can maintain, modify, add, remove, delete, or otherwise manipulate or adjust access control to digital components on behalf of third-party device 156 or other entities besides third-party device 156.

[0089] Then, data processing system 102 can provide digital components for execution by client devices. For example, data processing system 102 can receive a request to execute a digital component on a third client device linked to an account identifier. Data processing system 102 (e.g., via access controller component 114) can verify that the second client device has the right to access the digital component based on parameter information stored in a digital access control database for the account identifier. Data processing system 102 (e.g., via compatibility checker component 116) can determine that the digital component is compatible with the third client device. For example, the type of application required to execute, present, or render the digital component is available on the third client device (e.g., application version, operating system version); computing resources (e.g., processor speed, memory, bandwidth, storage) are sufficient to process or execute the digital component; or the interface used to present the digital component (e.g., display, input, keyboard, touchscreen, sensor, accelerometer, GPS, or speaker) is available on the client device. Compatibility checker component 114 can poll the third client device to obtain status information to determine compatibility. Therefore, in response to verifying access and compatibility of the digital component, data processing system 102 can provide the digital component for execution by the third client device.

[0090] The compatibility checker component 116 can determine, based on the device's state, that a requested digital component is incompatible for the device to execute. The data processing system 102 can convert the format of the digital component based on the state of a third client device. The data processing system 102 can convert the digital component from a first format to a second format in response to compatibility or state information. For example, the device may include a display with a first resolution; then, the data processing system 102 can adjust the resolution of the digital component to be compatible with the display. The device may be configured with a graphics driver capable of supporting two-dimensional images and video rather than three-dimensional video; the data processing system 102 can convert the digital component for two-dimensional display.

[0091] Data processing system 102 (e.g., compatibility checker 116) can identify multiple client devices linked to an account identifier. Compatibility checker 116 can identify a subset of the multiple client devices compatible with the digital component. Compatibility checker 116 can provide the digital component for execution on the subset of multiple client devices in response to parameter modifications. Therefore, access controller component 114 can modify the parameters by providing approval for execution of the digital component on a third client device. Access controller component 114 can determine the type of the parameter based on the type of the digital component and modify the parameter based on said type.

[0092] For example, data processing system 102 may include a content selector component 120 designed, constructed, configured, or operated for selecting digital component objects. To select digital components for display in a voice-based environment, data processing system 102 (e.g., via NLP component 108) may parse the input audio signal to identify keywords (e.g., trigger keywords) and use the keywords to select matching digital components based on broad matching, exact matching, or phrase matching. For example, content selector component 120 may analyze, parse, or otherwise process the subject of candidate digital components to determine whether the subject of the candidate digital component corresponds to the subject of a keyword or phrase in the input audio signal detected by the microphone of local client device 140. Content selector component 120 may use image processing techniques, character recognition techniques, natural language processing techniques, or database lookups to identify, analyze, or recognize the speech, audio, terms, characters, text, symbols, or images of candidate digital components. Candidate digital components may include metadata indicating the subject of the candidate digital component, in which case content selector component 120 may process the metadata to determine whether the subject of the candidate digital component corresponds to the input audio signal.

[0093] Content selector component 120 can further utilize information provided in the profile stack to perform content selection. Content selector component 120 can utilize a second profile layer, which may correspond to an electronic account associated with a speaker providing an input audio signal detected by local computing device 140. Content selector 120 can also use information associated with a first profile layer, which may correspond to a default profile layer provided by third-party device 156. Content promotional activities provided by content provider 154 may include content selection criteria, which data processing system 102 can match against criteria indicated in the second or first profile layer.

[0094] Content provider 154 can provide additional indicators when establishing content campaigns that include digital components. Content provider 154 can provide information at the content campaign or content group level, which content selector component 120 can identify by using information about candidate digital components for lookup. For example, candidate digital components can include unique identifiers that can be mapped to content groups, content campaigns, or content providers. Content selector component 120 can determine information about content provider 154 based on information stored in the content campaign data structure in data repository 122.

[0095] Data processing system 102 can receive requests for content to be presented on computing device 140 via a computer network. Data processing system 102 can identify the request by processing input audio signals detected by the microphone of the local client computing device 140. The request may include selection criteria such as device type, location, and keywords associated with the request. The request may include an action data structure or an action data structure.

[0096] In response to a request, data processing system 102 may select a digital component object from data repository 122 or a database associated with content provider 154 and provide the digital component via network 105 for presentation via computing device 140. The digital component object may be provided by content provider device 154, which is different from third-party device 156. The digital component may correspond to a service type different from the service type of the action data structure (e.g., taxi service versus food delivery service). Computing device 140 may interact with the digital component object. Computing device 140 may receive audio responses to the digital component. Computing device 140 may receive instructions to select a hyperlink or other button associated with the digital component object, which enables or allows computing device 140 to identify content provider device 154 or third-party device 156, request a service from content provider device 154 or third-party device 156, instruct content provider device 154 or third-party device 156 to perform a service, transmit information to content provider device 154 or third-party device 156, or otherwise query content provider device 154 or third-party device 156.

[0097] Data processing system 102 (e.g., via interface 106 and network 105) can transmit data packets including output signals in response to input audio signals or their requests or queries. The output signals can cause the audio driver component 138 of local computing device 140, or the audio driver component 138 executed by local computing device 140, to drive the speakers (e.g., transducers 146) of local computing device 140 to generate sound waves corresponding to the output signals.

[0098] Figure 2 This is a diagram illustrating the operation of a system 200 for synchronizing access control between computing devices. System 200 may include... Figure 1 The system 100 or as depicted in Figure 4The system 400 depicted may include one or more components. For example, system 200 may include data processing system 102. System 200 may include, or be coupled to or communicate with, a first digital assistant device 280, a second digital assistant device 282, or a third digital assistant device 284. The first digital assistant device 280, the second digital assistant device 282, and the third digital assistant device 284 may include one or more components or functions of local computing device 140. The first digital assistant device 280, the second digital assistant device 282, and the third digital assistant device 284 may include functions or interfaces different from each other. For example, the first digital assistant device 280 may be a speaker digital assistant that does not include a display device 152; the second digital assistant device 282 may be a mobile computing device that includes a touchscreen; and the third digital assistant device 284 may be a laptop computer.

[0099] In action 202, the first digital assistant device 280 receives an input audio signal. The input audio signal may include a request, one or more keywords, or a trigger phrase or keyword. The input audio signal may include a query for accessing content. For example, the input audio signal could be “I would like to sign up for a monthly subscription to Company_Astreaming service”. The input audio signal could be “I want to purchase AudioBook_B”. The input audio signal could be “I want to purchase Album_C using digital currency_D”. The input audio signal could be “I want to get a free trial version of VideoGame_E”.

[0100] In action 204, the first digital assistant device 280 can transmit an input audio signal (or its text version or data packets) to the data processing system 102 via network 105. The data processing system 102 can receive the data packets. In action 206, the data processing system 102 can identify a request. For example, the data processing system 102 can use the NLP component of its interactive data exchange component to identify the request.

[0101] At action 208, data processing system 102 may execute a decision box regarding whether to modify access control. The decision box may refer to or include: the data processing system determining whether to modify access control parameters to provide the digital component. The decision box may refer to or include: the data processing system 102 determining whether the first digital assistant device 280 has been authorized to access the requested digital component. If, at the decision box corresponding to action 208, the data processing system 102 determines that it is not necessary to modify the access control parameters of the requested digital component for the account identifier corresponding to the first digital assistant device 280, the data processing system may proceed to decision box 210 to determine whether the digital component is compatible for execution by the first digital assistant device 280. If, at decision box 210, the data processing system 102 determines that the digital component is compatible with the first digital assistant device 280 (e.g., based on the status or device information of the first digital assistant device 280), then at action 214, the data processing system may transfer the digital component to the first digital assistant device 280 for execution.

[0102] However, if at decision box 210 the data processing system 102 determines that the digital component is incompatible with the first digital assistant device 280, then at action 212, the data processing system 102 may determine to reformat the digital component to be compatible with the first digital assistant device 280. If the data processing system 102 can reformat the digital component at action 212, the data processing system 102 proceeds to reformatting and at action 214 transmits the reformatted digital component to the first digital assistant device 280 for execution. If the data processing system 102 cannot reformat the digital component to a compatible format, then at action 242, the data processing system 102 proceeds to session handover, or may exit the loop. Session handover at action 242 may include: identifying a compatible digital assistant device linked to the account, and transmitting the digital component for execution on the linked compatible digital assistant device.

[0103] If, at decision box 208, data processing system 102 determines to modify access control parameters, then in action 216, the data processing system can proceed to generating an input prompt. Data processing system 102 can generate the input prompt based on the type of the requested digital component and the rules associated with that type. Data processing system 102 can generate an input prompt with one or more fields in which values ​​can be entered. In action 218, data processing system 102 can determine whether the generated input prompt is compatible with the first digital assistant device 280. If the generated input prompt is compatible with the first digital assistant device, then in action 220, data processing system 102 can transmit the generated input prompt to the first digital assistant device. In action 22, data processing system 102 can receive input data in response to the input prompt. For example, data processing system 102 can receive account information, selections, payment information, time interval information, preferences, or other information. In action 224, data processing system 102 can use the received input data from the first digital assistant device 280 to update the access control database.

[0104] However, if at action 218 the data processing system 102 determines that the generated input prompt is incompatible with the first digital assistant device 280, then at action 226, the data processing system can poll for a compatible second device. The data processing system 102 can broadcast polling or probing to identify devices within the range of the first digital assistant device 280. For example, the data processing system 102 can instruct the first digital assistant device 280 to use a communication protocol or broadcast protocol to broadcast polling, probing, or beacons to identify other digital assistant devices 282. The data processing system 102 can perform a lookup in the linked device data structure to identify other devices linked to an account linked to the first digital assistant device 280.

[0105] Upon identifying the second digital assistant device 282, in action 228, the data processing system can perform a session handover. The data processing system 102 can suspend, maintain, or terminate a session previously established with or initiated via the first digital assistant device 280. The data processing system 102 can hand over the session because it can determine that the first digital assistant device 280 is incompatible or inoperable to perform or adjust access controls to perform further processing or functions associated with digital components.

[0106] In action 230, and in response to the session handover at action 228, data processing system 102 may transmit the generated input prompt to second digital assistant device 282. In response to receiving the generated input prompt at action 230, in action 232, second digital assistant device 282 may provide input data. In action 224, data processing system 102 may update the access control database to adjust, update, modify, or add parameters and values ​​for digital components for that account.

[0107] In action 234, data processing system 102 may receive a request for a digital component from third digital assistant device 284. In decision box 236, data processing system 102 may determine whether third digital assistant device 284 is authorized to execute, access, receive, or otherwise use the digital component. If at decision box 236 the data processing system determines that third digital assistant device 284 is authorized to access the requested digital component (e.g., based on a lookup of update parameters in access control database 132), then in action 238, data processing system 102 may instruct third-party device 156 to provide the digital component to third digital assistant device 284. In action 240, third-party device 256 may provide the digital component to third digital assistant device 284 for execution.

[0108] However, if at action 236 the data processing system 102 determines that the third digital assistant device 284 is not authorized to access the digital component (e.g., parameters only allow access by the second digital assistant device or the access time interval or geographical boundary has expired), the data processing system 102 can proceed to action 208 to modify the access control parameters of the digital component.

[0109] Therefore, data processing system 102 can centrally manage access control for different types of digital components and across different types of digital assistant devices, eliminating the need for third-party device 156 or digital component providers to manage this. Third-party device 156 can provide digital components in response to instructions from data processing system 102, while data processing system 102 performs functions to determine whether a digital assistant device is authorized to receive digital components and to allow the digital assistant device to purchase digital components. Data processing system 102 can execute electronic transactions using digital currency that can be obtained from a first digital assistant device and subsequently used on a second digital assistant device. For example, a second digital assistant device can receive digital currency, but a user can use a third digital assistant device to spend or exchange digital currency with another digital component.

[0110] The first digital assistant device 280, the second digital assistant device 282, and the third digital assistant device 284 may be different physical devices, or they may be applications or surfaces performed or provided by one or more devices.

[0111] The data processing system can receive a request from a third client device to execute the second digital component. The data processing system can determine, based on parameters stored in the access control database for the second digital component, that the third client device is not authorized to execute the second digital component (e.g., Figure 2 The decision box 236 depicted in the figure. The data processing system can use input prompts for the second digital component (e.g., Figure 2 The actions described in section 208, 216, 218, 226, 228, and 230) are transmitted to the second client device. The data processing system can approve the third client device in response to input provided via the second client device.

[0112] The data processing system can identify multiple fields in an input prompt. The data processing system can generate an audio query for each of the multiple fields. The data processing system can receive one or more input audio signals in response to the audio query generated for each of the multiple fields. For example, a rule for a digital component can instruct the generation of an input prompt with fields configured to be converted to audio and the reception of audio input. The data processing system can determine whether to provide the input prompt to a first digital assistant device (e.g., Figure 2 Actions depicted in 218 and 220).

[0113] The first client device (or digital assistant device 280) can execute a first surface or user interface type, the second client device (or digital assistant device 282) can execute a second surface or user interface type, and the third client device (or digital assistant device 284) can execute a third surface or user interface type different from the first and second surfaces. The data processing system can provide digital components for the third and second surfaces to execute.

[0114] One or more client devices may store account information in local storage or memory. For example, the data processing system may provide parameters of a digital component on a digital access control database for an account identifier, which may be stored in the memory of a third client device. In response to a data packet entered via a second client device in response to an input prompt, the data processing system may update the parameters of the digital component stored in the memory of the third client device.

[0115] Figure 3 This is a diagram illustrating an example method for synchronizing access control between computing devices. Method 300 can be derived from... Figure 1 The system 100 depicted in the text Figure 2 The system 200 or as depicted in Figure 4The system 400 described herein may be performed by one or more components, systems, or elements. For example, method 300 may be performed by a data processing system. In action 302, the data processing system may receive a request from a first device. The data processing system may receive a data packet including an input audio signal detected by a sensor of a first client device linked to an account identifier. The data processing system may parse the data packet including the input audio signal based on natural language processing techniques to identify a request for a digital component and one or more keywords corresponding to the request. The data processing system may establish a session with the first client device. The data processing system may generate an input prompt based on policies (or rules) for digital components stored in a digital access control database, the input prompt having fields in response to a request for a digital component from the first client device linked to the account identifier.

[0116] In action 304, the data processing system may transfer the session to a second device. For example, the data processing system may determine that the input prompt is incompatible with the first client device. In response to determining that the input prompt is incompatible with the first client device, the data processing system may identify the state of the second client device linked to the account identifier linked to the first client device. The data processing system may determine that the input prompt is compatible with the second client device based on the state identified by the session transfer component. In response to the determination, the data processing system may perform a session transfer to the second client device and terminate a portion of the session with the first client device. In response to the transfer, the data processing system may transmit the input prompt to the second client device.

[0117] In action 306, the data processing system may modify the access control database. For example, the data processing system may receive a data packet input via a second client device in response to an input prompt. The data processing system may modify the parameters of digital components stored in the digital access control database for account identifiers based on the data packet input via the second client device.

[0118] In action 308, the data processing system may receive a request from a third device. The data processing system may receive a request to execute a digital component on a third client device linked to an account identifier. In action 310, the data processing system may provide the digital component to the third device. In response to verification by the access controller component of the parameters of the digital component stored in the digital access control database for the account identifier, and determination by the compatibility checker component that the digital component is compatible with the third client device, the data processing system may provide the digital component for execution by the third client device.

[0119] Figure 4This is a block diagram of an example computer system 400. The computer system or computing device 400 may include or be used to implement system 100 or its components, such as data processing system 102, or system 600 or its components, such as state engine 634. Data processing system 102 may include an intelligent personal assistant or a voice-based digital assistant. Computing system 400 includes a bus 405 or other communication components for transmitting information, and a processor 410 or processing circuitry coupled to the bus 405 for processing information. Computing system 400 may also include one or more processors 410 or processing circuitry coupled to the bus for processing information. Computing system 400 also includes main memory 415, such as random access memory (RAM) or other dynamic storage devices, coupled to the bus 405 for storing information and instructions to be executed by processor 410. Main memory 415 may be or include a data repository 145. Main memory 415 may also be used to store location information, temporary variables, or other intermediate information during the execution of instructions by processor 410. The computing system 400 may also include a read-only memory (ROM) 420, or other static storage device, coupled to the bus 405 for storing static information and instructions of the processor 410. Storage device 425 (e.g., a solid-state device, disk, or optical disk) may be coupled to the bus 405 for persistent storage of information and instructions. Storage device 425 may include or be part of data storage device 145.

[0120] The computing system 400 can be coupled to a display 435, such as a liquid crystal display or an active matrix display, via a bus 405 to display information to a user. An input device 430, such as a keyboard including alphanumeric and other keys, can be coupled to the bus 405 to transmit information and command selections to the processor 410. The input device 430 may include a touchscreen display 435. The input device 430 may also include cursor control, such as a mouse, trackball, or arrow keys, to transmit directional information and command selections to the processor 410 and to control cursor movement on the display 435. For example, the display 435 may be... Figure 1 It is part of the data processing system 102, the client computing device 140, or other components.

[0121] The processes, systems, and methods described herein can be implemented by computing system 400 in response to processor 410 executing an instruction set contained in main memory 415. These instructions may be read into main memory 415 from another computer-readable medium, such as storage device 425. Execution of the instruction set contained in main memory 415 enables computing system 400 to perform the illustrative processes described herein. One or more processors in a multiprocessor configuration may also be used to execute the instructions contained in main memory 415. Hardwired circuitry may be used in place of or in combination with software instructions in conjunction with the systems and methods described herein. The systems and methods described herein are not limited to any particular combination of hardware circuitry and software.

[0122] Despite Figure 4 Example computing systems have been described, but the subject matter of the operations described herein can be implemented in other types of digital electronic circuits or in computer software, firmware or hardware, including the structures disclosed herein and their equivalents, or combinations thereof.

[0123] In situations where systems discussed in this paper collect or may utilize personal information about users, users can be given the opportunity to control whether a program or feature can collect personal information (e.g., information about a user's social networks, social actions or activities, user preferences, or user location), or to control whether or how content more relevant to the user is received from a content server or other data processing system. Additionally, some data can be anonymized in one or more ways before storage or use to remove personally identifiable information when generating parameters. For example, a user's identity can be anonymized so that no personally identifiable information about the user can be determined, or the user's geographic location can be generalized (e.g., to a city, zip code, or state level) if location information is available, making it impossible to determine the user's specific location. Therefore, users can control how information about them is collected and how the content server uses their information.

[0124] The subject matter and operations described herein can be implemented in other types of digital electronic circuits or in computer software, firmware, or hardware—including the structures disclosed herein and their equivalents, or combinations thereof. The subject matter described herein can be implemented as one or more computer programs—for example, one or more circuits of computer program instructions encoded on one or more computer storage media for execution by or control of the operation of a data processing device. Alternatively or additionally, the program instructions can be encoded on artificially generated propagating signals, such as machine-generated electrical, optical, or electromagnetic signals, which generate the signals to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium can be or includes computer-readable storage devices, computer-readable storage substrates, random or serial access memory arrays or devices, or combinations thereof. While the computer storage medium is not a propagating signal, it can be a source or destination of computer program instructions encoded in artificially generated propagating signals. The computer storage medium can also be or be contained in one or more separate components or media (e.g., multiple CDs, disks, or other storage devices). The operations described in this specification can be implemented as operations performed by a data processing device on data stored on one or more computer-readable storage devices or on data received from other sources.

[0125] The terms “data processing system,” “computing device,” “component,” or “data processing apparatus” encompass a variety of devices, apparatuses, and machines for processing data, including, for example, programmable processors, computers, systems-on-a-chip, or a combination thereof. The apparatus may include special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, cross-platform runtime environments, virtual machines, or combinations thereof. The apparatus and execution environment can implement various different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures. For example, Direct Action API 110, Content Selector Component 120, or NLP Component 108, and other data processing system components 102 may include or share one or more data processing apparatuses, systems, computing devices, or processors.

[0126] Computer programs (also known as programs, software, software applications, applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, declarative languages, or procedural languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, objects, or other units suitable for use in a computing environment. A computer program may correspond to a file in a file system. A computer program may be stored as a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file storing portions of one or more modules, subroutines, or code). Computer programs can be deployed to execute on a single computer or on multiple computers located at one site or distributed across multiple sites and interconnected through a communication network.

[0127] The processes and logical flows described herein can be executed by one or more programmable processors that execute one or more computer programs (e.g., components of data processing system 102) to perform actions by manipulating input data and generating outputs. The processes and logical flows can also be executed by dedicated logic circuitry, and the apparatus can be implemented as dedicated logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). Suitable devices for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROMs, EEPROMs, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. Processors and memory can be supplemented by or incorporated into dedicated logic circuitry.

[0128] The subject matter described herein can be implemented in a computing system that includes backend components (e.g., as a data server), middleware components (e.g., an application server), or frontend components (e.g., a client computer with a graphical user interface or web browser through which a user interacts with embodiments of the subject matter described herein), or a combination of one or more such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), interconnected networks (e.g., the Internet) and peer-to-peer networks (e.g., point-to-point peer-to-peer networks).

[0129] Computing systems such as System 100 or System 400 may include clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network (e.g., Network 105). The relationship between the client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. In some implementations, the server transmits data (e.g., data packets representing digital components) to the client device (e.g., for the purpose of displaying data to a user interacting with the client device and receiving user input from the user). Data generated on the client device (e.g., the result of user interaction) may be received at the server from the client device (e.g., received by data processing system 102 from local computing device 140, content provider computing device 154, or third-party device 156).

[0130] Although the operations are depicted in a specific order in the accompanying drawings, it is not necessary to perform such operations in the specific order shown or in sequential order, nor is it necessary to perform all the operations shown. The actions described herein can be performed in different orders.

[0131] Separation of system components is not required in all implementations, and the described program components may be contained within a single hardware or software product. For example, NLP component 108 or content selector component 120 may be a single component, application, or program, or a logic device with one or more processing circuits, or part of one or more servers of data processing system 102.

[0132] Some illustrative embodiments have now been described. It should be understood that the foregoing is illustrative and not limiting, and has been presented by way of example. Specifically, while many of the examples presented herein involve specific combinations of method actions or system elements, those actions and elements can be combined in other ways to achieve the same objective. The actions, elements, and features discussed in connection with one embodiment are not intended to exclude similar functions in other embodiments.

[0133] The wording and terminology used herein are for descriptive purposes and should not be considered limiting. The use of “comprising,” “having,” “including,” “involving,” “characterized in,” and variations thereof is intended to cover the items listed thereafter, their equivalents, and other items, as well as alternative implementations consisting only of the items listed thereafter. In one implementation, the systems and methods described herein consist of one, each combination of, or all of the described elements, actions, or components.

[0134] Any singular reference to any implementation, element, or action of the systems and methods herein may also cover implementations that include multiple such elements, and any plural reference to any implementation, element, or action herein may also cover implementations that include only a single element. References in either the singular or plural form are not intended to limit the currently disclosed systems or methods, their components, actions, or elements to a single or multiple configurations. A reference to any action or element based on any information, action, or element may include implementations where said action or element is at least partially based on any information, action, or element.

[0135] Any implementation disclosed herein may be combined with any other implementation or embodiment, and references to “implementation,” “some implementations,” “one implementation,” etc., are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described in connection with the implementation may be included in at least one implementation or embodiment. Such terms as used herein do not necessarily refer to all the same implementation. Any implementation may be incorporated into or exclusively combined with any other implementation in any manner consistent with the aspects and implementations disclosed herein.

[0136] A reference to "or" can be interpreted as inclusive, so any term described using "or" can refer to a single term, more than one term, or any one of all the terms described. For example, a reference to "at least one of 'A' and 'B'" can include only 'A', only 'B', or both 'A' and 'B'. Such references used in conjunction with "include" or other open-ended terms can include additional terms.

[0137] Where reference numerals follow technical features in the drawings, detailed embodiments, or any claims, reference numerals are already included to enhance the comprehensibility of the drawings, detailed embodiments, and claims. Therefore, the presence or absence of reference numerals does not limit the scope of any claim element.

[0138] The systems and methods described herein may be embodied in other specific forms without departing from their characteristics. The foregoing embodiments are illustrative and not limiting of the systems and methods described. Therefore, the scope of the systems and methods described herein is indicated by the appended claims rather than the foregoing description, and modifications within the equivalent meaning and scope of the claims are included therein.

Claims

1. A system for synchronizing access control between devices to process operations, comprising: A data processing system comprising one or more processors and a memory coupled to the one or more processors, the data processing system being configured to: Data packets are received via the interface of the data processing system, the data packets including input audio signals detected by sensors of a first client device linked to an account identifier; The data packets, including the input audio signal, are parsed using natural language processing techniques to identify requests for digital components and one or more keywords corresponding to the requests. as well as A session is established with the first client device, and an input prompt is generated based on the policy for the digital component stored in the digital access control database. The input prompt has fields that respond to the request for the digital component from the first client device linked to the account identifier. Based on the incompatibility of the input prompt with the first user device, the session is transferred to a second client device linked to the account identifier linked to the first client device; In response to the handover, the input prompt is transmitted to the second client device; Receive data packets input via the second client device in response to the input prompt; The parameters of the digital component stored in the digital access control database for the account identifier are modified based on the data packets input via the second client device. as well as In response to the modification of the parameters, the digital component is provided for execution on a subset of a plurality of client devices that are compatible with the digital component and linked to the account identifier.

2. The system according to claim 1, wherein, The data processing system is further configured to: It is determined that the input prompt is incompatible with the first client device; and In response to the determination that the input prompt is incompatible with the first client device, the session is transferred to the second client device.

3. The system according to claim 1, wherein, The data processing system is further configured to: The session is transferred to the second client device based on the status of the second client device and the indication that the input prompt is compatible with the second client device.

4. The system according to claim 1, wherein, The data processing system is further configured to: terminate a portion of the session with the first client device in response to the session being transferred to the second client device.

5. The system according to claim 1, wherein, The data processing system is further configured to: Receive a request to execute the digital component on a third client device linked to the account identifier; It is determined that the digital component is compatible with the third client device; as well as The digital component is provided for execution by the third client device in response to the determination that the digital component is compatible with the third client device.

6. The system according to claim 5, wherein, The data processing system is further configured to: provide the digital component for execution by the third client device in response to verification of the parameters of the digital component stored in the digital access control database for the account identifier.

7. The system according to claim 1, wherein, The data processing system is further configured to: Identify multiple client devices linked to the account identifier; Identify a subset of the plurality of client devices that are compatible with the digital components; as well as In response to the modification of the parameters, the digital components are provided for execution on the subset of the plurality of client devices.

8. The system according to claim 1, wherein, The data processing system is further configured to: Generate one or more audio queries for one or more fields in the input prompt; and Receive one or more input audio signals in response to the one or more audio queries.

9. The system according to any one of claims 1 to 8, wherein, The data processing system is further configured to: Receive a request from the third client device to execute the second digital component; Based on the parameters for the second digital component stored in the digital access control database, it is determined that the third client device has not been authorized to execute the second digital component; Transmit input prompts for the second digital component to the second client device; as well as The third client device is authorized in response to input provided via the second client device.

10. A method for synchronizing access control between devices to process operations, the method being performed by a data processing system including a memory coupled to one or more processors and comprising: Data packets are received via an interface, the data packets including input audio signals detected by sensors of a first client device linked to an account identifier; The data packets, including the input audio signal, are parsed using natural language processing techniques to identify requests for digital components and one or more keywords corresponding to the requests. as well as A session is established with the first client device, and an input prompt is generated based on the policy for the digital component stored in the digital access control database. The input prompt has fields that respond to the request for the digital component from the first client device linked to the account identifier. Based on the incompatibility of the input prompt with the first user device, the session is transferred to a second client device linked to the account identifier linked to the first client device; In response to the handover, the input prompt is transmitted to the second client device; Receive data packets input via the second client device in response to the input prompt; The parameters of the digital component stored in the digital access control database for the account identifier are modified based on the data packets input via the second client device. as well as In response to the modification of the parameters, the digital component is provided for execution on a subset of a plurality of client devices that are compatible with the digital component and linked to the account identifier.

11. The method of claim 10, further comprising: It is determined that the input prompt is incompatible with the first client device; as well as In response to the determination that the input prompt is incompatible with the first client device, the session is transferred to the second client device.

12. The method of claim 10, further comprising: The session is transferred to the second client device based on the status of the second client device and the indication that the input prompt is compatible with the second client device.

13. The method of claim 10, further comprising: In response to the transfer of the session to the second client device, a portion of the session with the first client device is terminated.

14. The method according to any one of claims 10 to 13, further comprising: Receive a request to execute the digital component on a third client device linked to the account identifier; It is determined that the digital component is compatible with the third client device; as well as In response to determining that the digital component is compatible with the third client device, the digital component is provided for execution by the third client device.

Images