Systems and methods for detecting improper rendering of content items by applications.

Abstract

This disclosure relates to systems and methods for detecting improper implementation of content item presentation by an application. One method may include: receiving a content package comprising a first content item, a second content item, and an interaction confirmation script, the interaction confirmation script being configured to execute within an application and cause the application to: detect a first interaction event with the first content item in response to the presentation of the first content item; present the second content item after detecting the first interaction event on the first content item; detect a second interaction event with the second content item; transmit confirmation of the first interaction event after detecting the second interaction event; and transmit a request to access an information resource linked to the second content item. The method may further include: presenting the information resource in response to the second interaction event on the second content item.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese invention patent application 201780002022.3, filed on March 3, 2017. Technical Field

[0003] This disclosure relates to systems and methods for detecting improper rendering of content items by an application running on a client device. Background Technology

[0004] In a networked computer environment (such as the Internet), data from a server can be transmitted to a client device in response to a request for content. The client device receives data from the server. This data may include content used to render content corresponding to data displayed on the client device. The content publisher may specify instructions corresponding to rendering parameters for content items (e.g., local content items). Summary of the Invention

[0005] At least one aspect relates to a method for detecting improper presentation of content items by an application running on a client device. The method can detect unintentional interactions with local content items performed within an application running on the client device due to improper presentation of content items by the application. The method may include: receiving, from a first server, a content package comprising a first content item, a second content item, and an interaction confirmation script by an application running on a client device including one or more data processors, the interaction confirmation script being configured to execute within the application running on the client device and causing the application to: detect a first interaction event with the first content item in response to the first content item presented within the application on the client device; present the second content item within the application after detecting the first interaction event on the first content item; detect a second interaction event with the second content item; transmit an acknowledgment of the first interaction event to the first server after detecting the second interaction event; and transmit a request to the second server to access an information resource linked to the second content item. The method may further include: presenting an information resource by the client device in response to the second interaction event on the second content item.

[0006] In some implementations, the first content item includes a local content item configured to match the display parameters of other content items in the application. In some implementations, the second content item includes an interstitial content item configured to occupy a major portion of the application's display area when displayed. In some implementations, the second content item includes a script containing instructions for the application to detect a second interaction event and transmit confirmation of the first interaction event. In some implementations, the first content item includes the same text as the text included in the second content item.

[0007] In some embodiments, the interactive confirmation script further includes computer-executable instructions that, when executed by an application, cause the application to transmit an indication of the first interactive event to a first server after detecting the first interactive event. In some embodiments, the first server is configured to maintain a content restriction list in a database, the content restriction list including multiple application identifiers corresponding to multiple applications, the content restriction list being used to restrict the selection of a first content item for transmission in response to content requests from multiple applications corresponding to application identifiers on the list. In some embodiments, the first server is further configured to: add an application identifier of an application to the content restriction list in response to failure to receive confirmation from an application; and determine, in response to receiving a subsequent content request, to restrict the selection of the first content item for transmission to the application corresponding to the application identifier on the content restriction list. In some embodiments, the confirmation of the first interactive event includes an application identifier for identifying the application.

[0008] At least one aspect relates to a system for detecting improper presentation of content items by an application executing on a client device. The system can detect unintentional interactions with local content items performed by an application executing on the client device due to improper presentation of content items by the application. The system may include: a processor; and a memory coupled to the processor. The memory may store computer-executable instructions that, when executed by the processor, cause the processor to: receive a content package including a first content item, a second content item, and an interaction confirmation script from a first server by the application executing on the client device. The interaction confirmation script may be configured to execute within the application executing on the client device and, in response to the first content item presented within the application on the client device, cause the application to detect a first interaction event with the first content item; after detecting the first interaction event on the first content item, present the second content item within the application; detect a second interaction event with the second content item; after detecting the second interaction event, transmit an acknowledgment of the first interaction event to the first server; and transmit a request to the second server to access an information resource linked to the second content item. The computer-executable instructions may further cause the processor to: present an information resource by the client device in response to the second interaction event on the second content item.

[0009] In some implementations, the first content item includes a local content item configured to match the display parameters of other content items in the application. In some implementations, the second content item includes an interstitial content item configured to occupy a major portion of the application's display area when displayed. In some implementations, the second content item includes a script containing instructions for the application to detect a second interaction event and transmit confirmation of the first interaction event. In some implementations, the first content item includes the same text as the text included in the second content item.

[0010] In some embodiments, the interactive confirmation script further includes computer-executable instructions that, when executed by an application, cause the application to transmit an indication of the first interactive event to a first server after detecting the first interactive event. In some embodiments, the first server is configured to maintain a content restriction list in a database, the content restriction list including multiple application identifiers corresponding to multiple applications, the content restriction list being used to restrict the selection of a first content item for transmission in response to content requests from multiple applications corresponding to application identifiers on the list. In some embodiments, the first server is further configured to: add an application identifier of an application to the content restriction list in response to failure to receive confirmation from an application; and determine, in response to receiving a subsequent content request, to restrict the selection of the first content item for transmission to the application corresponding to the application identifier on the content restriction list. In some embodiments, the confirmation of the first interactive event includes an application identifier for identifying the application.

[0011] At least one aspect relates to a method for detecting inappropriate rendering of content items in an application running on a client device. The method can detect unintentional interaction with local content items within an application running on a client device due to inappropriate rendering of content items by the application. The method includes: receiving content requests from a plurality of client devices by a data processing system comprising one or more processors, each content request being generated by an application running on a corresponding client device among the plurality of client devices. The method may further include: in response to a content request from a client device, transmitting a content package comprising a first content item, a second content item, and an interaction confirmation script to each client device in a first subset of the plurality of client devices, the interaction confirmation script being configured to cause the application to: configure the first content item to be rendered according to a first rendering format; render the first content item according to the first rendering format; display the second content item in response to a first interaction event of detecting the first content item; and transmit an confirmation to the data processing system in response to a second interaction event of detecting the second content item. The method may further include: in response to a content request from a client device, transmitting the first content item to each client device in a second subset of the plurality of client devices. The method may further include: a data processing system determining a first performance metric for a first content packet based on acknowledgments received from a first subset of multiple client devices. The method may further include: a data processing system determining a second performance metric for a first content item based on acknowledgments received from a second subset of client devices; the method may further include: a data processing system comparing the first performance metric with the second performance metric. The method may further include: a data processing system adding an application identifier to a content restriction list based on the comparison, the content restriction list being used to restrict the selection of the first content item for transmission in response to subsequent content requests identifying the application.

[0012] In some implementations, the first content item includes a local content item configured to match the display parameters of other content items in the application.

[0013] These and other aspects, as well as implementation methods, are discussed in detail below. The foregoing information and the following detailed description include illustrative examples of various aspects and implementation methods, and provide an overview or framework for understanding the nature and characteristics of the claimed aspects and implementation methods. The accompanying drawings provide illustration and further understanding of the various aspects and implementation methods, and are incorporated into and constitute a part of this specification. It should be understood that aspects and implementation methods can be combined, and the features described in one aspect or implementation method can be implemented in other contexts. Attached Figure Description

[0014] The accompanying drawings are not intended to be drawn to scale. The same reference numerals and designations in the various drawings indicate the same elements. For clarity, not every component may be labeled in every drawing.

[0015] In the attached diagram:

[0016] Figure 1 It is a block diagram depicting an environment for detecting inappropriate presentation of content items by an application running on a client device, according to some implementations.

[0017] Figure 2 This is a block diagram depicting a system for detecting inappropriate presentation of content items by an application running on a client device, according to some implementations.

[0018] Figure 3 This is a signaling diagram illustrating communications within an environment used to detect inappropriate presentation of content items by an application running on a client device, according to some implementations.

[0019] Figure 4 This is a flowchart depicting a method for detecting unintentional interactions with local content items according to some implementations.

[0020] Figure 5 It describes, according to some embodiments, a method for detecting and in relation to Figure 4 A flowchart describing a method for inappropriately presenting content items in a data processing system for application communication executed on a client device.

[0021] Figure 6 It is a block diagram depicting a general architecture for a computer system of elements that can be employed to implement the systems and methods described and illustrated herein, according to some embodiments. Detailed Implementation

[0022] The following is a more detailed description of various concepts related to methods, apparatus, and systems for detecting inappropriate presentation of content items by applications running on client devices, and implementations of such methods, apparatus, and systems. The various concepts introduced above and discussed in more detail below can be implemented in any of a variety of ways, as the described concepts are not limited to any particular implementation.

[0023] This disclosure relates to systems and methods for detecting improper presentation of content items by an application running on a client device. Specifically, this disclosure relates to systems and methods for detecting erroneous interactions with content items performed by an application running on a mobile computing device.

[0024] Within mobile applications, dynamically loaded content items (such as images, videos, or other displayable content) can be embedded into the application's visual experience in either a native or interstitial format. In the native format, independent of the mobile application that ultimately displays the content item, the content publisher has the freedom to customize the content item's characteristics, such as the font, color, position, or other details. In the interstitial format, content items are typically displayed at natural transition points or logical breaks in the application's flow and can overlay the application's regular user interface (e.g., interstitial content items can be displayed full-screen).

[0025] For natively formatted content items, serving native content items typically requires providing independent components (e.g., titles, images, descriptions, etc.) that the content publisher can freely implement or display when they deem appropriate. However, publishers often display native content items inappropriately, leading to poor user experiences due to unintentional user interactions. For example, consider enabling a "slide to unlock" feature on a mobile device to unlock it by swiping across the lock screen. On the lock screen, the mobile device may also display content items that are also activated by swiping. These content items might be incorrectly displayed on or very close to the "slide to unlock" area. Similarly, when a user swipes to unlock the mobile device, the swipe may not only be recognized as a swipe to unlock, but also additionally (and incorrectly) as an interaction with the content item, causing the mobile device to perform unwanted actions, such as opening links in the browser associated with the unintentionally swiped content item.

[0026] There are technical problems related to content items that are incorrectly presented within an application on a client device (e.g., content items that are improperly rendered within the application). For example, causing the client device to execute computer-executable instructions to process unintentional clicks on incorrectly displayed or presented content items may result in increased battery consumption, wasted network resources when providing data associated with unintentionally clicked content items, and additional computing resources that may adversely affect the performance of the client device. Furthermore, unintentional interaction with content items may lead to a poor user experience.

[0027] Figure 1This is a block diagram depicting an environment for detecting inappropriate presentation of content items by an application running on a client device, according to some embodiments. Environment 100 includes at least one data processing system 110. The data processing system 110 may include at least one processor (or processing circuitry) and memory. The memory stores processor-executable instructions that, when executed by the processor, cause the processor to perform one or more of the operations described herein. The processor may include: a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc., or combinations thereof. The memory may include, but is not limited to: electronic storage devices, optical storage devices, magnetic storage devices, or any other storage or transmission device capable of providing program instructions to a processor. The memory may further include: a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ASIC, FPGA, read-only memory (ROM), random access memory (RAM), electrically erasable ROM (EEPROM), erasable programmable ROM (EPROM), flash memory, optical media, or any other suitable memory whose instructions can be read by its processor. The instructions may include code from any suitable computer programming language. The data processing system 110 may include one or more computing devices or servers capable of executing various functions.

[0028] Network 105 may include: computer networks (such as the Internet), local area networks, wide area networks, metropolitan area networks or other regional networks, intranets, satellite networks, other computer networks (such as voice or data mobile phone communication networks), and combinations thereof. The data processing system 110 of environment 100 may communicate via network 105 with, for example, at least one content provider computing device 115, at least one content publisher computing device 120, or at least one client device 125. Network 105 may be any form of computer network that relays information between client device 125, data processing system 110, and one or more content sources (e.g., web servers, content servers, etc.). For example, network 105 may include: the Internet and / or other types of data networks, such as local area networks (LANs), wide area networks (WANs), cellular networks, satellite networks, or other types of data networks. Network 105 may also include any number of computing devices (e.g., computers, servers, routers, network switches, etc.) configured to receive and / or transmit data within network 105. Network 105 may further include any number of hardwired connections and / or wireless connections. For example, client device 125 may wirelessly communicate with transceivers of other computing devices in network 105 via hardwired (e.g., via fiber optic cable, CAT5 cable, etc.) connections to the transceivers of other computing devices in network 105 (e.g., via WiFi, cellular, radio, etc.).

[0029] Content provider computing device 115 may include a server or other computing device operated by the content provider entity to provide one or more content items to be displayed on an information resource at a client device. The content provided by content provider computing device 115 may include third-party content items to be displayed on an information resource (such as a website or webpage containing primary content, for example, content provided by content publisher computing device 120). Content items may also be displayed on search results webpages. For example, content provider computing device 115 may provide or be the source of one or more content items to be displayed in a content slot on a content webpage (such as a company's webpage, where the primary content of the webpage is provided by the company) or on a search results landing page provided by a search engine. Content items associated with content provider computing device 115 may be displayed on information resources other than webpages, such as content displayed as part of the execution of an application (such as a Global Positioning System (GPS) or map application, or other type of application) on a smartphone or other client device 125.

[0030] Content publisher computing device 120 may include a server or other computing device operated by a content publishing entity to provide primary content to be displayed via network 105. For example, content publisher computing device 120 may include a webpage operator providing primary content to be displayed on a webpage. The primary content may include content other than that provided by content publisher computing device 120, and the webpage may include content slots configured to display third-party content items from content provider computing device 115. For example, content publisher computing device 120 may operate a company's website and may provide content about the company for display on webpages of the website. The webpage may include content slots configured to display third-party content items from content provider computing device 115. In some embodiments, content publisher computing device 120 includes a search engine computing device (e.g., a server) that operates a search engine operator of a search engine website. The primary content of a search engine webpage (e.g., a results or landing page) may include search results and third-party content items (such as content items from content provider computing device 115) displayed in content slots. In some embodiments, content publisher computing device 120 may include a server for serving video content.

[0031] Client device 125 may include a computing device configured to communicate via network 105 to display data (such as content provided by content publisher computing device 120 (e.g., main web page content or other information resources) and content provided by content provider computing device 115 (e.g., third-party content items configured to be displayed in content slots on a web page or in an application)). For example, client device 125 may be configured to run an application created or published by content publisher computing device 120, and the application may be configured to display content items (e.g., local content items) therein. Client device 125, content provider computing device 115, and content publisher computing device 120 may include: desktop computers, laptop computers, tablet computers, smartphones, personal digital assistants, mobile devices, consumer computing devices, servers, clients, digital video recorders, television set-top boxes, video game consoles, or any other computing device configured to communicate via network 105. Client device 125 may be a communication device through which an end user can submit a request to receive content. The request may be a request to a search engine, and the request may include a search query. In some implementations, a request may include a request to access a webpage.

[0032] The content provider computing device 115, the content publisher computing device 120, and the client device 125 may include a processor and memory, i.e., processing circuitry. The memory stores machine instructions that, when executed by the processor, cause the processor to perform one or more of the operations described herein. The processor may include: a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination thereof. The memory may include, but is not limited to: electronic storage devices, optical storage devices, magnetic storage devices, or any other storage or transmission device capable of providing program instructions to the processor. The memory may further include: floppy disks, CD-ROMs, DVDs, magnetic disks, memory chips, ASICs, FPGAs, read-only memory (ROM), random access memory (RAM), electrically erasable ROM (EEPROM), erasable programmable ROM (EPROM), flash memory, optical media, or any other suitable memory whose instructions can be read by its processor. Instructions may include code from any suitable computer programming language.

[0033] The content provider computing device 115, the content publisher computing device 120, and the client device 125 may further include one or more user interface devices. Typically, a user interface device refers to any electronic device (e.g., keyboard, mouse, pointing device, touchscreen display, microphone, etc.) that transmits data to a user by generating sensory information (e.g., visualization on a display, one or more sounds, etc.) and / or converts sensory information received from the user into electronic signals. According to various embodiments, the one or more user interface devices may be inside the housing of the content provider computing device 115, the content publisher computing device 120, and the client device 125 (e.g., a built-in display, microphone, etc.) or outside the housing of the content provider computing device 115, the content publisher computing device 120, and the client device 125 (e.g., a monitor connected to the user computing device 115, a speaker connected to the user computing device 115, etc.). For example, the content provider computing device 115, the content publisher computing device 120, and the client device 125 may include an electronic display that visually displays web pages using web page data received from one or more content sources and / or from the data processing system 110 via the network 105. In some embodiments, a content placement activity manager or a third-party content provider may communicate with the data processing system 110 via the content provider computing device 115. In some embodiments, the content placement manager or the third-party content provider may communicate with the data processing system 110 via a user interface displayed on a user interface device of the content provider computing device 115.

[0034] Data processing system 110 may include at least one server. For example, data processing system 110 may include multiple servers located in at least one data center or server farm. In some embodiments, data processing system 110 may include a third-party content delivery system, such as a content server. Data processing system 110 may include at least one content request module 130, at least one content selection module 135, at least one script provider module 140, and at least one database 145. Content request module 130, content selection module 135, and script provider module 140 may each include at least one processing unit, server, virtual server, circuit, engine, agent, appliance, or other logical device, such as a programmable logic array, configured to communicate via network 105 with database 145 and other computing devices (e.g., content provider computing device 115, content publisher computing device 120, or client device 125).

[0035] Content request module 130, content selection module 135, and script provider module 140 may include or execute at least one computer program or at least one script. Content request module 130, content selection module 135, and script provider module 140 may be separate components, a single component, or part of data processing system 110. Content request module 130, content selection module 135, and script provider module 140 may include a combination of software and hardware, such as one or more processors configured to execute one or more scripts.

[0036] The data processing system 110 may also include one or more content repositories or databases 145. The database 145 may be local to the data processing system 110. In some embodiments, the database 145 may be remote to the data processing system 110, but may communicate with the data processing system 110 via network 105. The database 145 may include web pages, portions of web pages, third-party content items, and asynchronous click-through scripts, etc., to serve client devices 125. In some embodiments, the database 145 is configured to store a content restriction list 150. This content restriction list 150 can identify applications that execute on client devices 125, improperly render local content, and therefore should be restricted from receiving local content (e.g., as identified by the content restriction list 150). Further disclosure regarding the content restriction list 150 is presented below.

[0037] The content request module 130 can receive content requests from the client device 125. The content request may include a request for an information resource, a request for one or more third-party content items, a request for a script, or a combination thereof. In some embodiments, the content request may include a request for third-party content. In some embodiments, the request for third-party content may include the address or identifier of an information resource on which the third-party content is to be displayed. The request for third-party content may also include or identify one or more parameters that the data processing system 110 may use to determine the content to be provided in response to the content request. For example, the parameters may identify the size of the content slot into which the requested content is inserted. The parameters may identify the type of content associated with the information resource, the type of the requested third-party content (e.g., text, image, video, etc.), client device information, size information of the requested third-party content item, or a combination thereof.

[0038] The content selection module 135 can be configured to determine the content to be transmitted to the client device 125 in response to a received request for content. The content selection module 135 can determine the content to be sent to the client device 125 based on information included in the content request. For example, after receiving a request for an information resource, the content selection module 135 can use the address or identifier of the information resource in the content request to determine the content to be sent to the client device. When receiving a request for one or more third-party content items, the content selection module 135 can select the third-party content item based on the address or identifier of the information resource on which the third-party content item will be presented, the content type information of the information resource (e.g., sports, news, music, movies, travel, etc.), the size information of the slot in which the third-party content item will be displayed, and client device information (e.g., device type, device identifier, device location, etc.). In some embodiments, the content selection module 135 can access the database 145 and retrieve content for transmission to the client device 125.

[0039] In some implementations, the content selection module 135 may be configured to determine, in response to a request for content, whether to transmit a script to the client device 125. The content selection module 135 may determine whether to send the script to the client device 125 based on the type of content request (e.g., a request for local content), an information resource identifier in the content request, or a combination thereof. In some implementations, the content selection module 135 may determine whether to send the script to the client device 125 by classifying the received content request using an interactive confirmation protocol. After determining that the script should be sent to the client device 125, the content selection module 135 may forward the request for content or its indication to the script provider module 140. In some implementations, the content selection module 135 may select third-party content items to be sent to the client device 125 along with the script (e.g., an interactively confirmed script).

[0040] The script provider module 140 can be configured to retrieve an interactive confirmation script from the database 145 and transmit the retrieved script to the client device 125 (or another entity for sending to the client device 125). In some embodiments, the interactive confirmation script can be transmitted to the client device 125 alone or together with a third-party content item. For example, the script provider module 140, the content selection module 135, or another entity of the data processing system 110 can include the interactive confirmation script in a third-party content item before sending the third-party content item to the client device 125. In some embodiments, the script provider module 140, the content selection module 135, or another entity of the data processing system 110 can send the interactive confirmation script together with the requested information resource to the client device 125.

[0041] The interactive confirmation script may include computer-executable instructions. These computer-executable instructions may include scripts such as Hypertext Markup Language (HTML), Extensible Hypertext Markup Language (XHTML), Extensible Markup Language (XML), Cascading Style Sheets (CSS), and JavaScript. The computer-executable instructions may be executed within an application on client device 125 (such as an application that causes client device 125 to transmit a content request received by content request module 130). The application may include, for example, an internet browser, a mobile application, a game application, a GPS application, or any other computer program capable of reading and executing computer-executable instructions.

[0042] In short, when executed by the processor of client device 125, the computer-executable instructions enable an application of the client device to: (a) detect a first interaction event with a first content item presented within the application on the client device, (b) present a second content item within the application after detecting the first interaction event with the first content item, (c) detect a second interaction event with the second content item, (d) transmit an acknowledgment of the first interaction event to a first server after detecting the second interaction event, and (e) transmit a request to a second server to access an information resource linked to the second content item. Figure 2 , Figure 3 , Figure 4 and Figure 5 Further details regarding the interactive confirmation script are provided.

[0043] Figure 2 This is a block diagram depicting a system for detecting inappropriate presentation of content items by an application running on a client device, according to some embodiments. The interactive confirmation system 200 may include: an application 205, an information resource 210, an interactive event detection module 220, a content item presentation module 225, a confirmation generation module 230, and a request generation module 235. In some embodiments, the interactive event detection module 220, the content item presentation module 225, the confirmation generation module 230, and the request generation module 235 may be modules within the interactive confirmation script 215. The interactive confirmation script system 200 may be powered by one or more processors of a computing device (such as, hereinafter referred to as...). Figure 6 The processors described herein may run or execute. In some implementations, any one of the interactive event detection module 220, content item presentation module 225, confirmation generation module 230, and request generation module 235 may be part of application 205, information resource 210, or interactive confirmation script 215.

[0044] In some embodiments, the interactive confirmation script 215 is received by the client device 125 via an application 205 running on the client device 125. The data processing system 110 may send the interactive confirmation script 215 to the client device 125 in response to a content request sent by the client device 125 to the data processing system 110. In some embodiments, the content request sent by the client device 125 includes a request for local content. In some embodiments, the data processing system 110 transmits a content package 240 including the interactive confirmation script 215. The content package 240 may further include a first content item 245 and a second content item 250 for rendering and presentation by the application 205. In some embodiments, the first content item 245 includes a local content item, and the second content item 250 includes an interstitial content item.

[0045] In some implementations, in response to a content request received from client device 125, data processing system 110 may classify the received request as a request to package and transmit to client device 125 a triggering interactive confirmation script 215 along with selected local content items (e.g., first content item 245) and similar interstitial content items (e.g., second content item 250). Further disclosure regarding content package 240, first content item 245, and second content item 250 is presented below.

[0046] Application 205 may include, for example, an internet browser, a mobile application, a game application, a GPS application, or any other computer program capable of executing or invoking computer-executable instructions (such as computer-executable instructions included in information resource 210, interactive confirmation script 215, interactive event detection module 220, content item presentation module 225, confirmation generation module 230, and request generation module 235). Application 205 may obtain or receive information resource 210 from a server hosting information resource 210 (such as the server of content provider 120, the server of content provider 115, or the server of data processing system 110).

[0047] Figure 3 This is a signaling diagram illustrating communications within an environment used to detect inappropriate presentation of content items by an application running on a client device, according to some implementations. Reference is now also made to... Figures 1 to 3Client device 125 (e.g., via application 205) may transmit a content request to data processing system 110 at signal 302. In some embodiments, the content request is a request for a local content item. In some embodiments, the local content item may appear to have the same look and feel as content generated by content publisher 120 within the environment of application 205. For example, the local content item may include visual characteristics that match or substantially match the surrounding environment of application 205 in which the local content item is to be displayed. In some embodiments, data processing system 110 sends the local content item to client device 125 in the form of multiple content components that make up the local content item, and application 205 renders the content components at client device 125. For example, content components may include, but are not limited to: headings, images, body text, calls to actions, logos, location, click-through URLs, etc.

[0048] In some implementations, at block 303, in response to the content request module 130 receiving a content request from the client device 125, the content selection module 135 may classify the received request as a package that triggers the interactive confirmation script 125 and transmits it to the client device 125 along with selected local content items and similar interstitial content items. In some implementations, the content request includes an application identifier for identifying application 205. Therefore, the data processing system 110 may maintain a list of candidate applications (e.g., in database 145) for confirmation regarding whether native content is appropriately displayed. In some implementations, the application identifier identifies application 205 and a specific version of the identified application 205. In some implementations, the application identifier identifies the type of application 205 (e.g., based on the type of client device 125 on which application 205 is installed). For example, the application identifier may identify a specific operating system that the client device 125 operates on and is compatible with application 205.

[0049] Similarly, in some implementations, the content request module 130 performs a lookup in the database 125 to determine whether an application identifier received in a content request (e.g., an identification request sent from application 205) matches an identifier stored for identifying candidate applications used to trigger a determination of whether local content is to be displayed appropriately.

[0050] In some implementations, once the content request module 130 identifies application 205 as an application to be evaluated for local content rendering performance, the content request module 130 determines whether the client device 125 sending the content request will receive the interactive confirmation script 215 and the selected local content item. For example, the database 145 may also store a predetermined percentage assigned to each of the candidate application identifiers in the candidate application identifiers, or be associated with each of the candidate application identifiers stored to trigger local content rendering performance evaluation relative to applications identified by those candidate application identifiers. The predetermined percentage may identify the percentage of content requests to be classified as script requests so that the interactive confirmation script 215 is sent in response to content requests classified as script requests.

[0051] As an example, if a predetermined percentage of 35% is assigned to the application identifier of application 205, the content request module 130 can classify 35% of the content requests received from application 205 (e.g., on multiple client devices 125) as script requests, thereby causing an interactive confirmation script 215 to be sent to client device 125, which then sends those content requests classified as script requests. In some implementations, application 205 corresponds to an application instance installed on a single client device. In some implementations, application 205 corresponds to a specific version of a specific application installed on multiple client devices. In some implementations, application 205 corresponds to any version of application 205 on multiple client devices 125. Therefore, the remaining 65% of content requests can be classified as non-script requests, causing client device 125 that the data processing system 110 has classified as a non-script content request to not receive the interactive confirmation script 215, but instead receives the requested local content item. In some implementations, the content request module 130 randomly categorizes content requests (e.g., at a rate indicated by an assigned predetermined percentage) into scripted content requests or non-scripted content requests. In some implementations, the content request module 130 categorizes content requests into scripted content requests based on a predetermined pattern (e.g., when the predetermined percentage is 25%, the fourth out of every four content requests is categorized as a scripted content request).

[0052] In addition to sending an interactive confirmation script 215 in response to a content request categorized as a script content request, the content selection module 135 can also select and provide local content items (e.g., as initially requested) and interstitial content items, thereby enabling the data processing system 110 to provide a content package at signal 304 that includes the interactive confirmation script 215, the requested local content items, and the interstitial content items. Interstitial content items may include content items that are presented and displayed to take over a large portion of the display area of ​​application 205. For example, interstitial content items may cover the display of application 205. In some embodiments, the substance of an interstitial content item is substantially the same as that of a local content item included in a content package. In some embodiments, interstitial content items include inline HTML code sent along with local content items.

[0053] In some implementations, application 205 may render local content and present it at both frame 305 and signal 306 for display within the application 205's environment. In this implementation, a local content item comprising one or more content components is transmitted from data processing system 110 to client device 125, where application 205 at client device 125 aggregates the content components of the content item according to predetermined parameters of application 205. Once application 205 renders the local content using the content components, it presents the rendered local content for display within the application 205's environment, such that the rendered content appears to have the same look and feel as the content generated by content publisher 120 within application 205. In some implementations, content publisher 120 may establish predetermined parameters for application 205 to aggregate and present local content items. However, because these parameters vary from application to application and depend on the independent content publisher 120 to establish them, some content publishers 120 may implement the parameters incorrectly, resulting in local content items being incorrectly presented on the client device 125, which may cause unintentional interaction with incorrectly presented local content items.

[0054] For example, content publisher 120 may establish parameters for display relative to its application. The parameters may indicate the appearance of local content components received from data processing system 110. As an example, the received local content components may include elements of local content items, such as, but not limited to, headlines, images, body text, calls to action, logos, location, click-through URLs, etc. The established parameters may indicate the appearance of the local content components, such as their position within the local content item, color, font, font size, type of interaction (e.g., tap or swipe for interaction), etc. Similarly, the parameters provide instructions at the application level for rendering and presenting the received local content components within the application. In some implementations, content publisher 120 provides local content parameters within its own application.

[0055] However, because the content parameters are established and implemented by the respective content publisher 120, the way native content components are displayed within the application can vary significantly. Furthermore, the content publisher 120 may improperly set its parameters, causing native content components to be rendered and displayed incorrectly within the application. This could lead to unintentional user interaction with native content items that are improperly displayed within the application (e.g., when a native content item improperly covers a separate link that the user intends to click). Because these native content display parameters can vary from application to application, from version of the same application (e.g., compared to version 1.0 versus 2.0), or from different types of the same application (e.g., compared to an application configured for one operating system versus one configured for another), the parameters set by the content publisher 120 can similarly vary relative to different applications, different versions of applications, and different types of applications. Therefore, native content items may be improperly displayed relative to different versions or types of the same application.

[0056] In some implementations, at signal 308, the interaction event detection module 220, executed on client device 125, can enable client device 125 to detect interaction events relative to presented local content items within application 205. For example, interaction events may include: clicks, swipes, taps, cursor hovers, or any other interaction with presented local content items.

[0057] In some implementations, at block 311, in response to detecting an interaction event at signal 308, the content item rendering module 225 executing on client device 125 can cause client device 125 to render the interstitial content item received in the content package at signal 304. In some implementations, application 205 executing on client device 125 can cause client device 125 to detect whether an interstitial content item corresponding to the rendered local content item has been received (e.g., including the same or similar information rendered in the local content item). For example, application 205 can cause client device 125 to detect that the content package at block 304 includes more than the requested local content item. In some implementations, application 205 causes client device 125 to detect that the content package includes an interstitial content item by determining whether HTML inline code representing an interstitial content item exists in the content package. In some implementations, the content package includes a flag or any other indicator indicating that the content package includes interstitial content items in addition to local content items, and application 205 can cause client device 125 to detect the presence of the flag in the content package.

[0058] In some implementations, in response to detecting the presence of an interstitial content item in a received content packet and in response to detecting an interactive event at signal 308, the content item rendering module 225, executed on client device 125, can cause client device 125 to render the interstitial content item received in the content packet at signal 304. In some implementations, the content item rendering module 225 causes client device 125 to parse inline HTML code received in the content packet to obtain instructions for rendering the interstitial content item, and then render the interstitial content item based on the instructions. Once the interstitial content item is rendered, the content item rendering module 225 can cause client device 125 to render the interstitial content item within application 205. For example, the interstitial content item can be rendered in information resource 210 (e.g., by occupying a large portion of the space in information resource 210).

[0059] In some implementations, at signal 312, the acknowledgment generation module 230, executed on client device 125, can cause client device 125 to detect interaction events relative to presented interstitial content items within application 205. For example, interaction events may include: clicks, swipes, taps, cursor hovers, or any other interaction with the presented interstitial content items. In response to detecting an interaction event relative to an interstitial content item, acknowledgment generation module 230 can cause client device 125 to send an acknowledgment of the interaction event relative to local content items to data processing system 110.

[0060] In some implementations, the data processing system 110 receives confirmations of interaction events with respect to local content items and can perform performance evaluations based on the received confirmations of local content interaction events on multiple client devices 125. For example, as described above, the data processing system 110 may select a predetermined percentage of client devices 125 that request local content items via application 205 as client devices 125 that receive content packages including local content items, interstitial content items, and interaction confirmation scripts (e.g., a subset of content packages from all client devices 125 that made local content item requests to application 205), while the remaining percentage of client devices 125 receive local content items without receiving interstitial content items and interaction confirmation scripts (e.g., a non-content package subset from all client devices 125 that made local content item requests to application 205). Therefore, in some implementations, when determining whether the content publisher 120 correctly renders and displays local content items (e.g., whether the first interaction event at signal 308 relative to the local content item was intentional or unintentional due to improper presentation of the local content item), the data processing system 110 compares and analyzes performance metrics of the content package subset and the non-content package subset.

[0061] In some implementations, the data processing system 110 determines a performance metric relative to a subset of content packets from all client devices 125 that have requested local content items from application 205. For example, the performance metric may include a percentage or ratio representing the number of acknowledgments received by the data processing system 110 (e.g., at signal 314) compared to the number of client devices 125 that sent content packets (e.g., at signal 304). Thus, the performance metric for the content packet subset may represent the percentage of users intentionally interacting with local content items relative to the number of individual content packets sent to the subset of content packets from all client devices 125 that have requested local content items from application 205. Similarly, in some implementations, the data processing system 110 determines a performance metric relative to a subset of non-content packet content from all client devices 125 that have requested local content items from application 205. For example, the performance metric may include a percentage or ratio representing the number of indications of interaction events for local content items received by the data processing system 110 compared to the number of client devices 125 that sent requested local content items. Therefore, the performance metric for the non-content package subset can represent the percentage of the number of independent local content items that a user interacts with (e.g., intentionally and unintentionally) relative to the total number of local content items sent to the entire client device 125 that makes a request for local content items to application 205.

[0062] In some implementations, the data processing system compares performance metrics of a content package subset with performance metrics of a non-content package subset when determining whether a local content item is being improperly displayed in application 205. For example, data processing system 110 may compare the difference between the performance metrics of the content package subset and the non-content package subset with a predetermined threshold. In some implementations, if the difference between the two performance metrics exceeds the predetermined threshold, data processing system 110 identifies application 205 as improperly displaying local content items. As an example, when the performance metric of the non-content package subset is at least a predetermined threshold greater than the performance metric of the content package subset, data processing system 110 may identify application 205 (e.g., a specific version of application 205, multiple versions of application 205, this specific instance of application 205, etc.) as improperly displaying local content items. In some implementations, a predetermined threshold is input at data processing system 110 and stored in database 145, and the predetermined threshold can be any suitable value used to determine the difference (e.g., expressed as a percentage) between performance metrics that determine if application 205 is improperly displaying local content items (e.g., the value of the predetermined threshold could be 5%, 10%, 15%, etc.).

[0063] In some implementations, the data processing system 110 determines that application 205 is improperly displaying local content items based on performance metrics of a subset of content packages rather than on performance metrics of non-content package subsets. The data processing system 110 may compare the performance metrics of the content package subset itself with a predetermined threshold and make a determination based on that comparison. For example, performance metrics may include a percentage or ratio representing the number of acknowledgments received by the data processing system 110 relative to the number of client devices 125 that received the content package. The data processing system 110 may compare the percentage performance metric with a threshold percentage, and if the performance metric is below the threshold percentage, the data processing system 110 may identify application 205 as improperly displaying local content.

[0064] In some implementations, in response to identifying application 205 as improperly displaying local content, data processing system 110 may perform one or more restriction processes relative to application 205. For example, data processing system 110 may add application 205 to content restriction list 150. This content restriction list 150 may be stored in database 145 and may include an application identifier that identifies application 205, allowing data processing system 110 to cross-reference this list when it receives an incoming subsequent content request that includes the application identifier of application 205. If data processing system 110 matches the application identifier of the requesting application with an application identifier on content restriction list 150, data processing system 110 may restrict itself from transmitting local content items in response to a request (e.g., until content publisher 120 fixes the display performance of local content items at application 205). In some implementations, data processing system 110 receives an alert from content publisher 120 of application 205 indicating that the improper presentation of local content has been corrected (e.g., in response to a corresponding alert sent to content publisher 120 indicating that their application 205 improperly displays local content). In addition to acknowledging the interaction event sent by application 205 for local content items, in some embodiments, and in response to detecting an interaction event at signal 312, a request generation module 235 executed on client device 125 can cause client device 125 to transmit a request for accessing information resources linked to interstitial content items. This request can be sent to the server of content provider 115, which provides both local and interstitial content items displayed in application 205.

[0065] Figure 4 This is a flowchart 400 depicting a method for detecting inappropriate presentation of content items by an application running on a client device, according to some implementations.

[0066] In some implementations, method 400 includes client device 125 transmitting a content request for a content item (block 405), and data processing system 110 receiving the content request for the content item (block 410). Method 400 further includes data processing system 110 transmitting a content packet including an interactive confirmation script (block 415), and client device 125 receiving the content packet including the interactive confirmation script (block 420). Method 400 further includes client device 125 presenting a first content item (block 425). Method 400 further includes client device 125 detecting a first interaction event with the presented first content item (block 430). Method 400 further includes client device 125 presenting a second content item (block 435). Method 400 further includes client device 125 detecting a second interaction event with the presented second content item (block 440). Method 400 further includes client device 125 transmitting confirmation of the first interaction event (block 445). Method 400 further includes the data processing system 110 receiving an acknowledgment of the first interaction event (box 450). Method 400 further includes the client device 125 transmitting a request for information resources of the second content item (box 455). Method 400 further includes the client device 125 presenting information resources of the second content item (box 460).

[0067] Specifically, in some embodiments, method 400 includes client device 125 transmitting a content request for a content item (block 405), and data processing system 110 receiving the content request for the content item (block 410). Client device 125 may send the content request to data processing system 110. In some embodiments, the content request is a request for a local content item. In some embodiments, the local content item may appear to have the same look and feel as content generated by content publisher 120 within the application's environment. For example, a local content item may include visual characteristics that match or substantially match the surrounding environment of the application in which the local content item is displayed. In some embodiments, data processing system 110 sends the local content item to client device 125 in the form of multiple content components that make up the local content item, and the application renders the content components at client device 125. For example, content components may include, but are not limited to: a main title, an image, body text, an invocation of an action, a logo, a location, a click-through URL, etc.

[0068] In addition to sending an interactive confirmation script 215 in response to a content request categorized as a script content request, the content selection module 135 can also select and provide local content items (e.g., as initially requested) and interstitial content items, thereby enabling the data processing system 110 to provide a content package at signal 304 that includes the interactive confirmation script 215, the requested local content items, and the interstitial content items. Interstitial content items may include content items that are presented and displayed to take over a large portion of the display area of ​​application 205. For example, interstitial content items may cover the display of application 205. In some embodiments, the substance of an interstitial content item is substantially the same as that of a local content item included in a content package. In some embodiments, interstitial content items include inline HTML code sent along with local content items.

[0069] Method 400 further includes data processing system 110 transmitting a content packet including an interactive confirmation script (block 415), and client device 125 receiving the content packet including the interactive confirmation script (block 420). In addition to sending the interactive confirmation script in response to a content request, data processing system 110 may also select and provide local content items (e.g., as initially requested) and interstitial content items, thereby enabling data processing system 110 to provide a content packet including the interactive confirmation script, the requested local content items, and the interstitial content items. Interstitial content items may include content items that are presented and displayed to take over a large portion of the application's display area. For example, interstitial content items may cover the application's display. In some embodiments, the substance of the interstitial content item is substantially the same as that of the local content item included in the content packet. In some embodiments, the interstitial content item includes inline HTML code sent along with the local content item.

[0070] Method 400 further includes client device 125 (box 425) presenting the first content item. In some embodiments, the application can render local content and can present the local content for display within the application's context. In this embodiment, a local content item comprising one or more content components is transmitted from data processing system 110 to client device 125, wherein the application at client device 125 aggregates the content components of the content item according to predetermined parameters of the application. Once the application uses the content components to render the local content, the application presents the rendered local content for display within the application's context, such that the rendered content appears to have the same look and feel as content generated within the application by content publisher 120. In some embodiments, content publisher 120 can establish predetermined parameters that the application uses as instructions for a manner of aggregating and presenting local content items. However, because these parameters vary from application to application and depend on the individual content publisher 120 for establishment, some content publishers 120 may incorrectly implement the parameters, resulting in the local content item being incorrectly presented at client device 125, which may cause unintentional interaction with the incorrectly presented local content item.

[0071] For example, content publisher 120 may establish parameters for display relative to its application. The parameters may indicate the appearance of local content components received from data processing system 110. As an example, the received local content components may include elements of local content items, such as, but not limited to, headings, images, body text, calls to actions, logos, location, clicks via URLs, etc. The established parameters may indicate the appearance of the local content components, such as their position within the local content item, color, font, font size, type of interaction (e.g., tap or swipe for interaction), etc. Similarly, the parameters provide instructions at the application level for rendering and presenting the received local content components within the application. In some implementations, content publisher 120 provides local content parameters within its own application.

[0072] However, because the content parameters are established and implemented by the respective content publisher 120, the way native content components are displayed within the application can vary significantly. Furthermore, the content publisher 120 may incorrectly set its parameters, causing native content components to be rendered and displayed incorrectly within the application. This could lead to unintentional user interaction with native content items that are improperly displayed within the application (e.g., when a native content item inappropriately covers a separate link that the user wants to click). Because these native content display parameters can vary from application to application, from version of the same application (e.g., compared to version 1.0 versus 2.0), or from different types of the same application (e.g., compared to an application configured for one operating system versus one configured for another), the parameters set by the content publisher 120 can similarly vary relative to different applications, different versions of applications, and different types of applications. Therefore, native content items may be improperly displayed relative to different versions or types of the same application.

[0073] Method 400 further includes the client device 125 detecting a first interaction event with the first content item being presented (box 430). In some implementations, the client device 125 detects the interaction event within the application relative to the presented local content item. For example, the interaction event may include a click, swipe, tap, cursor hover, or any other interaction with the presented local content item.

[0074] Method 400 further includes client device 125 presenting a second content item (box 435). In some embodiments, in response to detecting an interaction event, client device 125 renders the interstitial content item received in the content package. In some embodiments, an application executing on client device 125 can cause client device 125 to detect whether an interstitial content item corresponding to the presented content item has been received (e.g., including the same or similar information presented in the local content item). For example, the application can cause client device 125 to detect that the content package includes more than the requested local content item. In some embodiments, the application causes client device 125 to detect that the content package includes an interstitial content item by determining whether HTML inline code representing an interstitial content item exists in the content package. In some embodiments, the content package includes a flag or any other indicator indicating that the content package includes an interstitial content item in addition to the local content item, and the application can cause client device 125 to detect the presence of the flag in the content package.

[0075] In some implementations, in response to detecting the presence of an interstitial content item in a received content packet and in response to detecting an interaction event, an application executing on client device 125 can cause client device 125 to render the interstitial content item received in the content packet. In some implementations, the application causes client device 125 to parse inline HTML code received in the content packet to obtain instructions for rendering the interstitial content item, and then renders the interstitial content item based on the instructions. Once the interstitial content item is rendered, the application can cause client device 125 to present the interstitial content item within the application.

[0076] Method 400 further includes the client device 125 detecting a second interaction event performed on a presented second content item (block 440). In some embodiments, an application running on the client device 125 may enable the client device 125 to detect interaction events relative to the presented interstitial content item within the application 205. For example, interaction events may include: clicks, swipes, taps, cursor hovers, or any other interaction with the presented interstitial content item.

[0077] Method 400 further includes the client device 125 transmitting an acknowledgment of the first interaction event (block 445). Method 400 further includes the data processing system 110 receiving an acknowledgment of the first interaction event (block 450). In response to detecting an interaction event relative to an interstitial content item, the application may cause the client device 125 to send an acknowledgment of the interaction event relative to the local content item to the data processing system 110.

[0078] Method 400 further includes client device 125 transmitting a request for information resources of the second content item (box 455). Method 400 further includes client device 125 presenting information resources of the second content item (box 460). The request may be sent to the server of content provider 115, which provides local content items and interstitial content items to be displayed in application 205.

[0079] Figure 5 It describes, according to some embodiments, a method for detecting and in relation to Figure 4A flowchart of a method 500 for improperly presenting content items by a data processing system performing application communication on a client device. In some embodiments, method 500 includes a data processing system 110 receiving multiple content requests from multiple client devices (box 505). Method 500 further includes the data processing system 110 transmitting content packets to a first subset of the multiple client devices (box 510). Method 500 further includes the data processing system 110 transmitting first content items to a second subset of the multiple client devices (box 515). Method 500 further includes the data processing system 110 determining a first performance metric relative to the transmitted content packets (box 520). Method 500 further includes the data processing system 110 determining a second performance metric relative to the first content item (box 525). Method 500 further includes the data processing system 110 comparing the first performance metric with the second performance metric (box 530). Method 500 further includes adding an application identifier to a content restriction list based on the comparison (box 535).

[0080] Specifically, in some embodiments, method 500 includes data processing system 110 receiving multiple content requests from multiple client devices (block 505). In some embodiments, in response to receiving a content request from a client device, data processing system 110 may classify the received request as a request to package and transmit along with a selected local content item and similar interstitial content items that will trigger an interactive confirmation script. In some embodiments, the content request includes an application identifier for identifying an application. Therefore, data processing system 110 may maintain a list of applications as candidate applications, in which a determination will be made regarding whether local content is appropriately displayed. In some embodiments, the application identifier identifies the application and a specific version of the identified application. In some embodiments, the application identifier identifies the type of application (e.g., based on the type of client device on which the application is installed). For example, the application identifier may identify a specific operating system on which the client device operates and is compatible with the application.

[0081] Similarly, in some implementations, the data processing system 110 performs a lookup in a database to determine whether an application identifier received in a content request (e.g., an application identifier that identifies the request as being sent from an application) matches an identifier stored in a database of application identifiers used to identify candidate applications that trigger a determination of whether local content is being displayed appropriately.

[0082] Method 500 further includes data processing system 110 transmitting content packets to a first subset of a plurality of client devices (block 510). Method 500 further includes data processing system 110 transmitting first content items to a second subset of the plurality of client devices (block 515). In some embodiments, once data processing system 110 identifies an application as one to be evaluated for local content rendering performance, data processing system 110 determines whether the client device 125 sending the content request will receive an interactive confirmation script 215 with selected local content items. For example, what may be stored in the database is a predetermined percentage assigned to each of the candidate application identifiers in the candidate application identifiers, or associated with each of the candidate application identifiers, which is stored to trigger local content rendering performance evaluation relative to applications identified by those candidate application identifiers. The predetermined percentage may identify the percentage of content requests to be classified as script requests so that an interactive confirmation script is sent in response to a content request classified as a script request.

[0083] As an example, if a predetermined percentage of 35% is assigned to the application identifier of an application, the data processing system 110 can classify 35% of the content requests received from the application (e.g., on multiple client devices) as script requests, thereby causing an interactive confirmation script to be sent to the client device, which then sends those content requests classified as script requests. In some implementations, the application corresponds to an application instance installed on a single client device. In some implementations, the application corresponds to a specific version of a particular application installed on multiple client devices. In some implementations, the application corresponds to any version of an application on multiple client devices. Therefore, the remaining 65% of content requests can be classified as non-script requests, causing the client device that sent the content requests that the data processing system 110 classified as non-script content requests to not receive the interactive confirmation script 215, but instead receive the requested local content item. In some implementations, the data processing system 110 randomly classifies content requests (e.g., at a rate indicated by the assigned predetermined percentage) as script content requests or non-script content requests. In some implementations, the data processing system 110 classifies content requests as script content requests based on a predetermined pattern (e.g., when the predetermined percentage is 25%, every fourth content request is classified as a script content request).

[0084] In addition to sending an interactive confirmation script in response to a content request categorized as a script content request, the data processing system 110 may also select and provide local content items (e.g., as initially requested) and interstitial content items, thereby enabling the data processing system 110 to provide a content package including the interactive confirmation script 215, the requested local content items, and the interstitial content items. Interstitial content items may include content items that are presented and displayed to take over a major portion of the application's display area. For example, interstitial content items may cover the application's display. In some embodiments, the substance of an interstitial content item is substantially the same as that of a local content item included in the content package. In some embodiments, interstitial content items include inline HTML code sent along with local content items.

[0085] Method 500 further includes the data processing system 110 determining a first performance metric relative to the transmitted content packets (block 520). In some embodiments, the data processing system 110 receives acknowledgments of interaction events with respect to local content items and may perform performance evaluation based on the received acknowledgments of local content interaction events on multiple client devices. For example, as described above, the data processing system 110 may select a predetermined percentage of client devices that request local content items via an application as client devices receiving content packets including local content items, interstitial content items, and interaction acknowledgment scripts (e.g., a subset of content packets from all client devices that requested local content items to the application), while the remaining percentage of client devices receive local content items without receiving interstitial content items and interaction acknowledgment scripts (e.g., a non-content packet subset from all client devices that requested local content items to the application). Therefore, in some implementations, the data processing system 110 compares and analyzes performance metrics of the content package subset and non-content package subset when determining whether the content publisher correctly renders and displays local content items (e.g., whether the first interaction event at signal 308 relative to the local content item is intentional or unintentional due to improper rendering of the local content item).

[0086] In some implementations, the data processing system 110 determines a performance metric relative to a subset of content packets from all client devices that have requested local content items from the application. For example, the performance metric may include a percentage or ratio representing the number of acknowledgments received by the data processing system 110 compared to the number of client devices that sent content packets. Therefore, the performance metric for the content packet subset may represent the percentage of users intentionally interacting with local content items relative to the number of individual content packets sent to the subset of content packets from all client devices that have requested local content items from the application.

[0087] Method 500 further includes the data processing system 110 determining a second performance metric relative to the first content item (block 525). Similarly, the data processing system 110 determines a performance metric relative to a subset of non-content packets from all client devices that have made local content item requests to the application. For example, the performance metric may include a percentage or ratio representing the number of indications of interaction events relative to local content items received by the data processing system 110 to the number of client devices that have sent requests for local content items. Thus, the performance metric for the subset of non-content packets may represent a percentage of the number of users interacting with local content items (e.g., both intentionally and unintentionally) relative to the number of individual local content items sent to the subset of non-content packets from all client devices that have made local content item requests to the application.

[0088] Method 500 further includes the data processing system 110 comparing a first performance metric with a second performance metric (block 530). In some embodiments, the data processing system 110 compares a performance metric of a subset of content packages with a performance metric of a subset of non-content packages when determining whether a local content item is being improperly displayed in the application. For example, the data processing system 110 may compare the difference between the performance metric of the subset of content packages and the performance metric of the subset of non-content packages with a predetermined threshold. In some embodiments, if the difference between the two performance metrics exceeds the predetermined threshold, the data processing system 110 identifies the application as improperly displaying local content items. As an example, when the performance metric of the subset of non-content packages is at least a predetermined threshold greater than the performance metric of the subset of content packages, the data processing system 110 may identify the application (e.g., a particular version of the application, multiple versions of the application, this particular instance of application 205, etc.) as improperly displaying local content items. In some implementations, a predetermined threshold is input at the data processing system 110 and stored in a database, and the predetermined threshold can be any suitable value used to determine the difference (e.g., expressed as a percentage) between performance metrics that determine if an application is improperly displaying local content items (e.g., the value of the predetermined threshold could be 5%, 10%, 15%, etc.).

[0089] In some implementations, the data processing system 110 determines that application 205 is improperly displaying local content items based on performance metrics of a subset of content packages rather than on performance metrics of non-content package subsets. The data processing system 110 may compare the performance metrics of the content package subset itself with a predetermined threshold and make a determination based on that comparison. For example, performance metrics may include a percentage or ratio representing the number of acknowledgments received by the data processing system 110 relative to the number of client devices 125 that received the content package. The data processing system 110 may compare the percentage performance metric with a threshold percentage, and if the performance metric is below the threshold percentage, the data processing system 110 may identify the application as improperly displaying local content.

[0090] Method 500 further includes data processing system 110 adding an application identifier to a content restriction list based on comparison (block 535). In some embodiments, in response to identifying an application as improperly displaying local content, data processing system 110 may perform one or more restriction processes relative to the application. For example, data processing system 110 may add the application to a content restriction list. This content restriction list may be stored in database 145 and may include an application identifier that identifies the application, allowing data processing system 110 to cross-reference the list when it receives an incoming subsequent content request that includes the application identifier of the application. If data processing system 110 matches the application identifier of the requesting application with an application identifier on the content restriction list, data processing system 110 may restrict itself from transmitting local content items in response to the request (e.g., until content publisher 120 fixes the display performance of local content items at the application). In some embodiments, data processing system 110 receives an alert from the application's content publisher indicating that the improper presentation of local content has been corrected (e.g., in response to a corresponding alert sent to the content publisher indicating that their application improperly displays local content).

[0091] Figure 6 This is a block diagram depicting the general architecture of a computer system 600, illustrating elements that can be employed to implement the systems described and illustrated herein (including system 110 and its components, such as content request module 130, content selection module 135, and script provider module 140), according to some embodiments. The computer system 600 can be used to provide information for display via network 105. Figure 6The computer system 600 includes one or more processors 620 communicatively coupled to memory 625, one or more communication interfaces 605, one or more output devices 610 (e.g., one or more display units), and one or more input devices 615. The processors 620 may be included in the data processing system 110 or other components of the system 110 (such as content request module 130 and content selection module 135).

[0092] exist Figure 6 In the computer system 600, the memory 625 may include any computer-readable storage medium and may store computer instructions (such as processor-executable instructions for the respective system to implement the various functions described herein) and any data associated with them, any data generated therefrom, or any data received (if present) via a communication interface or input device. See again Figure 1 The system 110, data processing system 110, may include a memory 625 to store information related to the inventory of one or more content units, the availability of reserved one or more content units, etc. The memory 625 may include a database 145. Figure 6 The processor 620 shown can be used to execute instructions stored in memory 625, and in doing so can also read from or write to memory various information processed and / or generated according to the execution of instructions.

[0093] exist Figure 6 The processor 620 of the computer system 600 shown can also be communicatively coupled to or control a communication interface 605 to transmit or receive various information according to the execution of instructions. For example, the communication interface 605 can be coupled to a wired or wireless network, a bus, or other communication device, and therefore, can allow the computer system 600 to transmit information to or receive information from other devices (e.g., other computer systems). Although in Figure 1 While not explicitly shown in the system, one or more communication interfaces facilitate the flow of information between components of system 600. In some embodiments, the communication interface may be configured (e.g., via various hardware or software components) as a website providing access to at least some aspects of computer system 600. Examples of communication interface 605 include a user interface (e.g., a webpage) through which a user can communicate with data processing system 110.

[0094] For example, it can be provided in Figure 6The output device 610 of the computer system 600 shown herein allows viewing or otherwise perceiving various information relating to the execution of instructions. For example, an input device 615 may be provided to allow a user to manually adjust, select, enter data, or interact with the processor in any manner during instruction execution. This document further provides additional information relating to general computer system architectures that the various systems discussed herein may employ.

[0095] The embodiments and operations of the subject matter described herein can be implemented in digital electronic circuit systems, or in computer software, firmware, or hardware (including the structures disclosed herein and their equivalents) contained on tangible media, or in one or more combinations thereof. Embodiments of the subject matter described herein can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a computer storage medium for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. The program instructions can be encoded on artificially generated propagating signals, such as machine-generated electrical, optical, or electromagnetic signals generated to encode information for transmission to a suitable receiver device for execution by the data processing apparatus. The computer storage medium can be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination thereof, or included therein. Furthermore, when the computer storage medium is not a propagating signal, it can include a source or destination of computer program instructions encoded in artificially generated propagating signals. The computer storage medium can also be one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices), or included therein.

[0096] The features disclosed herein can be implemented on a smart TV module (or a connected TV module, hybrid TV module, etc.), which may include a processing module configured to integrate internet connectivity with more traditional television program sources (e.g., received via cable, satellite, wireless, or other signals). The smart TV module may be physically incorporated into a television set, or it may include a separate device such as a set-top box, Blu-ray or other digital media player, game console, hotel TV system, and other companion devices. The smart TV module can be configured to allow viewers to search and find videos, movies, photos, and other content on the web, on local cable TV channels, on satellite TV channels, or stored on a local hard drive. A set-top box (STB) or set-top unit (STU) may include an information appliance that may include a tuner and connect to the television set and external signal sources, converting the signal into content and then displaying that content on a television screen or other display device. The smart TV module can be configured to provide a main screen or top-level screen with icons for multiple different applications (such as web browsers and various streaming media services, connected cable or satellite media sources, other network "channels," etc.). The smart TV module can be further configured to provide users with an electronic program guide. Companion applications for the smart TV module can operate on mobile computing devices to provide users with additional information about available programming, allowing users to control the smart TV module, etc. In alternative implementations, this feature can be implemented on laptops or other personal computers, smartphones, other mobile phones, handheld computers, tablet PCs, or other computing devices.

[0097] The operations described in this specification can be implemented as operations performed by a data processing device on data stored in one or more computer-readable storage devices or received from other sources.

[0098] The terms "data processing apparatus," "data processing system," "user equipment," or "computing device" encompass all kinds of apparatus, devices, and machines for processing data, including, for example, programmable processors, computers, systems-on-a-chip, or a combination thereof. The apparatus may include special-purpose logic circuit systems, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program under consideration, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, cross-platform runtime environments, virtual machines, or combinations thereof. The apparatus and execution environment can implement various computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures. Content request module 130 and content selection module 135 may include or share one or more data processing apparatuses, computing devices, or processors.

[0099] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, declarative languages, or procedural languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, objects, or other units suitable for use in a computing environment. Computer programs may, but are not required to, correspond to files in a file system. Programs can be stored as a part of a file containing other programs or data (e.g., in one or more scripts in a markup language document), in a single file dedicated to the program in question, or in multiple collaborating files (e.g., in a file storing one or more modules, subroutines, or portions of code). Computer programs can be deployed to execute on a single computer or on multiple computers located at a single site or distributed across multiple sites and interconnected via a communication network.

[0100] The processes and logic flows described herein can be performed by one or more programmable processors, which execute one or more computer programs to perform actions by manipulating input data and generating outputs. The processes and logic flows can also be performed by a dedicated logic circuit system (e.g., an FPGA (Field Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit)), and the device can also be implemented as a dedicated logic circuit system (e.g., an FPGA or an ASIC).

[0101] Processors suitable for executing computer programs include, for example, general-purpose microprocessors, special-purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor receives instructions and data from read-only memory or random access memory, or both. The essential components of a computer are: a processor for performing actions according to instructions, and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices (e.g., disks, magneto-optical disks, or optical disks) for storing data, or the computer may be operatively coupled to receive data from or transfer data to or both of such mass storage devices. However, a computer does not necessarily need to have such devices. Furthermore, a computer may be embedded in another device, such as a mobile phone, personal digital assistant (PDA), mobile audio or video player, game console, GPS receiver, or portable storage device (e.g., a Universal Serial Bus (USB) flash drive). Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, CD-ROMs, and DVD-ROMs. The processor and memory may be supplemented by or incorporated into a dedicated logic circuit system.

[0102] To provide interaction with the user, embodiments of the subject matter described herein can be implemented on a computer having: a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor; and a keyboard and pointing device, such as a mouse or trackball, through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can include any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form (including acoustic input, voice input, or tactile input). Additionally, the computer can interact with the user by sending documents to and receiving documents from the device used (e.g., sending a webpage to a web browser on the user's client device in response to a request received from a web browser).

[0103] Implementations of the subject matter described herein can be carried out in a computing system that includes back-end components (e.g., as a data server), or middleware components (e.g., an application server), or front-end components (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with implementations of the subject matter described herein), or any combination of one or more such back-end components, middleware components, or front-end components. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., a peer-to-peer network).

[0104] Computing systems (such as system 600 or system 110) may include clients and servers. For example, data processing system 110 may include one or more servers in one or more data centers or server farms. Clients and servers are generally geographically distant from each other and typically interact via a communication network. The client-server relationship is created by computer programs running on respective computers that have a client-server relationship with each other. In some implementations, the server transmits data (e.g., HTML pages) to the client device (e.g., to display data to a user interacting with the client device and to receive user input from that user). Data generated at the client device (e.g., the result of user interaction) may be received at the server from the client device.

[0105] While this specification contains numerous details of specific implementations, these details should not be construed as limiting the scope of this disclosure or any claims that may be made, but rather as descriptions of features specific to embodiments of the systems and methods described herein. Certain features described in this specification within the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, while features may be described above as functioning in certain combinations and initially even claimed in this way, in some cases, one or more features from the claimed combination may be removed from the combination. And the claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0106] Similarly, although the operations are shown in a specific order in the accompanying drawings, this should not be construed as requiring them to be performed in that specific order or in a sequential order, or as requiring all the illustrated operations to achieve the desired result. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired result.

[0107] In some cases, multitasking and parallel processing can be advantageous. Furthermore, the separation of the various system components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products. For example, content request module 130 and content selection module 135 may be part of data processing system 110, a single module, a logical device with one or more processing modules, one or more servers, or part of a search engine.

[0108] Now that some illustrative embodiments have been described, it is apparent that the foregoing is illustrative and not restrictive, and is presented by way of example. Specifically, while many of the examples presented herein relate to specific combinations of method actions or system elements, these actions and elements can be combined in other ways to achieve the same purpose. Actions, elements, and features discussed in connection with only one embodiment are not intended to exclude similar effects from other embodiments or embodiments.

[0109] In embodiments of this disclosure, the content or content item can be any suitable content. In some applications, the content or content item can be an advertisement (“ad”), but this disclosure is not limited thereto. When the content or content item is an advertisement, the content server 205 can be an ad server and / or a third-party content provider can be an advertiser.

[0110] In some implementations, a request for third-party content may also include a minimum revenue amount (or minimum bid value) for displaying the third-party content item on the information resource. In some implementations, the content selection module 135 may communicate with an auction system (e.g., an advertising auction system) and provide auction system information included in the request for third-party content received from the client device. The content selection module 135 may also receive one or more third-party content items from the auction system in response to providing the information included in the request for third-party content.

[0111] In some embodiments, the data processing system 110 may include an auction system configured to host an auction, such as an advertising auction system. In some embodiments, the data processing system 110 does not include an auction system, but is configured to communicate with an auction system via network 105.

[0112] In some implementations, the landing page may include additional product information for the product advertised in the content item. Conversions related to the content item can occur at the landing page; for example, a user may purchase a product or service advertised through the content item via the landing page. Conversions may occur when a user clicks on a content item and then takes an action (such as purchasing, registering, or enrolling) via the landing page.

[0113] The phrases and terms used herein are for descriptive purposes and should not be construed as limiting. The use of “including,” “comprising,” “having,” “containing,” “involving,” “characterized by,” “characterized in that,” and variations thereof herein refers to encompassing the items listed thereafter, their equivalents, and additional items, as well as alternative embodiments consisting of items listed separately thereafter. In one embodiment, the systems and methods described herein consist of one of the described elements, actions, or components, various combinations of more than one of the described elements, actions, or components, or all of the described elements, actions, or components.

[0114] Any reference to an embodiment, element, or action of a system or method expressed in the singular herein may also include embodiments that include a plurality of such elements, and any reference to any embodiment, element, or action expressed in the plural herein may also include embodiments that include only a single element. References in the singular or plural form are not intended to limit the currently disclosed systems or methods, their components, actions, or elements to a single or multiple configurations. References to any action or element based on any information, action, or element may include actions or elements based at least in part on embodiments of any information, action, or element.

[0115] Any implementation disclosed herein may be combined with any other implementation, and references to “implementation,” “some implementations,” “alternative implementations,” “various implementations,” “an implementation,” etc., are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described in connection with an implementation may be included in at least one implementation. Such terms as used herein do not necessarily refer to the same implementation. Any implementation may be combined with any other implementation in any manner consistent with the aspects and implementations disclosed herein, either inclusively or exclusively.

[0116] A reference to “or” can be interpreted as inclusive, such that any term described using “or” can refer to a single described term, more than one described term, or any of all described terms.

[0117] Where technical features in the drawings, detailed descriptions, or any claims are accompanied by reference numerals, these reference numerals are included to increase the comprehensibility of the drawings, detailed descriptions, and claims. Therefore, the presence or absence of reference numerals does not limit the scope of any claim element.

[0118] The systems and methods described herein may be embodied in other specific forms without departing from the characteristics of the systems and methods described herein. Although the examples provided herein relate to the display of content of controlled information resources, the systems and methods described herein can be applied to other environments. The above embodiments are illustrative and not limiting of the systems and methods described herein. Therefore, the scope of the systems and methods described herein is indicated by the appended claims rather than the foregoing description, and variations in the equivalent meaning and scope of the claims are included therein.

Claims

1. A method comprising: An application executing on a client device having one or more processors and memory receives a content package from a first server. The content package includes a first content item having a first actionable object and a content interaction script responding to a content request from the application. The content interaction script includes instructions executed within the application, causing the application to perform the following actions: Detect and display the first interaction of the application with the first content item having the first actionable object; In response to detecting the first interaction, a second content item having multiple content components is received, at least one of the multiple content components including a second actionable object, and at least one of the multiple content components including a video; Using the content parameters defined in the content interaction script, the second content item is aggregated for presentation using the multiple content components; In response to aggregating the second content item, the second content item is presented within the application such that the second content item covers a portion of the application's interface; Detect a second interaction with the second actionable object of the second content item; In response to detecting the second interaction, an acknowledgment of the first interaction is transmitted to the first server; as well as A request is transmitted to the second server to access information resources identified by the second actionable object of the second content item; and In response to the second interaction with the second actionable object of the second content item, the client device presents the information resource identified in the request.

2. The method according to claim 1, wherein, The content interaction script includes instructions, which, when executed, further cause the application to perform the following operations: The content package is determined to include inline HTML code; and A third content item is rendered in response to determining that the content package includes inline HTML code.

3. The method according to claim 1, wherein, The application is a web browser and is configured to render the first content item as part of a webpage displayed on the client device in response to an initial request for an initial information resource.

4. The method according to claim 1, wherein, The content interaction script includes instructions, which, when executed, further cause the application to detect the swipe interaction as the first interaction.

5. The method according to claim 1, wherein, The content interaction script includes instructions that, when executed, further cause the application to transmit confirmation of the presentation of the second content item to the first server when presenting the second content item.

6. The method according to claim 1, wherein, The first content item is configured to match the display parameters of other content items in the application.

7. The method according to claim 1, wherein, The second content item is received in the content package, and the application running on the client device is configured to aggregate the second content item included in the content package in response to detecting that the second content item has been received.

8. The method according to claim 1, wherein, The plurality of content components received by the application include at least one of images, calls to actions, clicks via URLs, or videos.

9. A system comprising: A client device having one or more processors, the one or more processors being configured by machine-readable instructions to execute an application, the client device being configured to: A content package is received from a first server, the content package including a first content item having a first actionable object and a content interaction script responding to a content request from the application, the content interaction script including instructions that execute within the application, the instructions causing the application to perform the following operations: Detect and display the first interaction of the application with the first content item having the first actionable object; In response to detecting the first interaction, a second content item having multiple content components is received, at least one of the multiple content components including a second actionable object, and at least one of the multiple content components including a video; Using the content parameters defined in the content interaction script, the second content item is aggregated for presentation using the multiple content components; In response to aggregating the second content item, the second content item is presented within the application such that the second content item covers a portion of the application's interface; Detect a second interaction with the second actionable object of the second content item; In response to detecting the second interaction, an acknowledgment of the first interaction is transmitted to the first server; as well as A request is transmitted to the second server to access information resources identified by the second actionable object of the second content item; and In response to the second interaction with the second actionable object of the second content item, the information resource identified in the request is presented.

10. The system according to claim 9, wherein, The content interaction script includes instructions, which, when executed, further cause the application to perform the following operations: The content package is determined to include inline HTML code; and A third content item is rendered in response to determining that the content package includes inline HTML code.

11. The system according to claim 9, wherein, The application is a web browser and is configured to render the first content item as part of a webpage displayed on the client device in response to an initial request for an initial information resource.

12. The system according to claim 9, wherein, The content interaction script includes instructions, which, when executed, further cause the application to detect the swipe interaction as the first interaction.

13. The system according to claim 9, wherein, The content interaction script includes instructions that, when executed, further cause the application to transmit confirmation of the presentation of the second content item to the first server when presenting the second content item.

14. The system according to claim 9, wherein, The first content item is configured to match the display parameters of other content items in the application.

15. The system according to claim 9, wherein, The second content item is received in the content package, and the application is configured to aggregate the second content item included in the content package in response to detecting that the second content item has been received.

16. The system according to claim 9, wherein, The plurality of content components received by the application include at least one of images, calls to actions, clicks via URLs, or videos.

17. A non-transitory computer-readable storage medium having instructions embodied thereon, the instructions being executable by one or more processors of a client device to perform a method comprising: An application running on the client device receives a content package from a first server. The content package includes a first content item having a first actionable object and a content interaction script responding to a content request from the application. The content interaction script includes instructions that execute within the application, causing the application to perform the following operations: Detect and display the first interaction of the application with the first content item having the first actionable object; In response to detecting the first interaction, a second content item having multiple content components is received, at least one of the multiple content components including a second actionable object, and at least one of the multiple content components including a video; Using the content parameters defined in the content interaction script, the second content item is aggregated for presentation using the multiple content components; In response to aggregating the second content item, the second content item is presented within the application such that the second content item covers a portion of the application's interface; Detect a second interaction with the second actionable object of the second content item; In response to detecting the second interaction, an acknowledgment of the first interaction is transmitted to the first server; as well as A request is transmitted to the second server to access information resources identified by the second actionable object of the second content item; and In response to the second interaction with the second actionable object of the second content item, the information resource identified in the request is presented.

18. The computer-readable storage medium according to claim 17, wherein, The content interaction script includes instructions, which, when executed, further cause the application to perform the following operations: The content package is determined to include inline HTML code; and A third content item is rendered in response to determining that the content package includes inline HTML code.

19. The computer-readable storage medium according to claim 17, wherein, The application is a web browser and is configured to render the first content item as part of a webpage displayed on the client device in response to an initial request for an initial information resource.

20. The computer-readable storage medium of claim 17, wherein, The content interaction script includes instructions, which, when executed, further cause the application to detect the swipe interaction as the first interaction.

Images