Using crowdsourcing to combat misinformation

Abstract

Methods, systems, and apparatus are disclosed, including a computer program encoded on a computer storage medium, for combating false advertising using crowdsourcing. In one aspect, the method includes receiving a false digital component alert indicating that a digital component presented at a client device includes false information, verifying the false digital component alert based on a public key to verify that a digital signature included in the false digital component alert matches a public key of a stored proof token specifying presentation and interaction data for the digital component. In response, the false digital component alert is added to an aggregate report. Based on a false alarm ratio, it is determined that the strength of the verified false digital component alerts in the aggregate report meets a threshold, and triggering a false digital component mitigation response, including providing a false information warning along with the false digital component to one or more client devices or digital component providers.

Description

Technical Field

[0001] The present disclosure generally relates to computer-implemented methods and systems for verifying and reporting the distribution of false digital components (eg, video clips, audio clips, multimedia clips, images, text, or other content units) using crowdsourcing. Background Art

[0002] The distribution of false information severely deteriorates users’ online experience. Reporting of false information by users may compromise user privacy, leading to under-reporting or causing privacy / security issues for reporting users. Furthermore, erroneous reporting of false information may hinder the distribution of content by content providers. Summary of the invention

[0003] This specification describes techniques involving the use of crowdsourcing to combat false advertising to protect user privacy and prevent fraudulent reporting of false advertising.

[0004] In general, an innovative aspect of the subject matter described in this specification can be specifically implemented in a method including an action of receiving a false digital component alert through one or more servers, the false digital component alert indicating that a digital component presented at a client device includes false information. The false digital component alert is verified based on a public key to verify that a digital signature included in the false digital component alert matches a public key of a certification token that specifies storage of presentation and interaction data for the digital component, and in response to verifying the false digital component alert, the false digital component alert is added to an aggregate report for the digital component. Based on a false alarm ratio, it is determined that the magnitude of the verified false digital component alert in the aggregate report for the digital component meets a threshold, and based on the magnitude of the verified false digital component alert meeting the threshold, a false digital component mitigation response is triggered, and a false information warning is provided together with the false digital component to one or more client devices or digital component providers. Other embodiments of this aspect include corresponding systems, devices, and computer programs, which are configured to perform the actions of the method encoded on a computer storage device.

[0005] These and other embodiments can each optionally include one or more of the following features. In some implementations, the method further includes providing a false information warning along with the false digital component to one or more client devices or digital component providers.

[0006] In some embodiments, the method further includes reducing or stopping the distribution of the false digital component based at least in part on the false digital component mitigation response. The reduction or cessation of the distribution of the false digital component can be performed by the digital component provider. The reduction or cessation of the distribution of the false digital component can be performed by a browser, application, or operating system of a client device.

[0007] In some embodiments, the method further includes receiving the false digital component mitigation response by another client device, and reducing or stopping the distribution of the false digital component through a browser, application, or operating system of the other client device. Reducing or stopping the distribution of the false digital component may include preventing the false digital component from being output on the client device. Reducing or stopping the distribution of the false digital component may include preventing the false digital component from being output on the other client device.

[0008] In some embodiments, the proof tokens designated for storage of presentation and interaction data of a digital component include multiple impression proof tokens for multiple presentations of the digital component, and multiple click proof tokens (each click proof token corresponding to a specific impression proof token among the multiple impression proof tokens), and / or multiple conversion tokens (each conversion token corresponding to a specific conversion proof token among the multiple conversion proof tokens).

[0009] In some implementations, each impression-proof token of the plurality of impression-proof tokens includes i) a campaign ID, ii) a date / time of presentation, and iii) a uniform resource locator for a landing page linked to by the particular digital component.

[0010] In some implementations, the impression proof token may include i) a campaign ID, ii) a date / time of presentation, and iii) an advertiser's phone number (e.g., for click-to-call ads) or a geo-fence of the advertiser's store location (e.g., for in-store visit ads).

[0011] In some implementations, each click proof token of the plurality of click proof tokens includes a conversion confirmation of a user action performed in response to presentation of the digital component.

[0012] In some implementations, validating the false digital component alert includes determining that the false digital component alert includes a specific impression proof token and a corresponding specific click proof token and / or other relevant conversion proof token for the digital component.

[0013] In some implementations, providing a false information warning with the false digital component to one or more client devices includes providing a visual signal for presentation on the client device along with the false digital component. The visual signal may include presenting an option on the client device to the user not to proceed with an associated action defined by the digital component, such as not to proceed with a login page, not to proceed with dialing the advertiser's phone number, locating the advertiser's store location (e.g., opening a map application and accessing a geo-fence of the store location), etc. Providing a false information warning with the false digital component to one or more client devices may include providing a verified false digital component report to a publisher of the digital component.

[0014] Other embodiments of this aspect include corresponding systems, apparatus, and computer programs configured to perform the actions of the method encoded on a computer storage device.

[0015] Specific embodiments of the subject matter described in this specification can be implemented to achieve one or more of the following advantages. Using server-side verification and aggregation of false digital component alerts from multiple users can protect individual user privacy while allowing consumers to report false digital components in a verified manner. Using proof tokens to verify false digital component alerts protects the interests of publishers of digital components by preventing malicious or erroneous reporting of digital components (e.g., by malicious entities). False advertising reports reported by users rather than by specific companies or stakeholders can be viewed as more neutral, and therefore more trustworthy.

[0016] The subject matter can be used to quickly identify false information distributed through online systems using crowdsourcing. By independently verifying that reporting users have viewed and interacted with the content, bad actors can be prevented from falsely reporting information as false. By ensuring that the false digital content alert being processed is valid before processing the alert, the processing, memory, and resources used to provide content to end users are reduced because resources are not used to store, distribute, or present false information. In addition, the use of proof tokens can help classify digital components as false in a more reliable and efficient manner, reducing the resources required to store and distribute misleading information. For example, the proof token is configured to ensure that the alarm of false information is issued by a user who is actually exposed to and / or interacts with the digital component containing false information, while still maintaining the privacy of the user. Therefore, it can be confirmed that the false information alarm is legitimate and not issued by a malicious actor (e.g., an entity attempting to harm a specific content distributor). Therefore, resources are not wasted on processing illegal false information alarms, and, since each false information alarm can be verified as legitimate or illegitimate using proof tokens, the result measurement and / or report providing information about the digital component containing false information becomes more accurate and more efficiently created.

[0017] The details of one or more embodiments of the subject matter described in this specification are set forth in the following drawings and description. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a block diagram of an environment in which a digital component system distributes digital components.

[0019] Figure 2 is a block diagram of an environment in which a false digital component is reported.

[0020] Figure 3 is a flow chart illustrating an example process for crowdsourcing false advertising reporting.

[0021] Figure 4 is a block diagram of an example computer system. DETAILED DESCRIPTION

[0022] Overview

[0023] In general, this document relates to computer-implemented methods and systems for verifying and reporting the distribution of false digital components (e.g., video clips, audio clips, multimedia clips, images, text, or other content units) using crowdsourcing. User alerts of false digital components are verified (e.g., submitted for identification) using content request proofs, subsequent impression proofs and click proofs in response to or as a result of content requests proved by the content request proofs, and aggregated by a server to protect the user privacy of the user who issued the alert. Digital components that are reported as false digital components by a threshold portion of users who view / interact with the digital component are marked as false digital components for future presentation of the digital component to users. Reports of false digital components can be provided to content providers, publishers, and end users. As discussed throughout this document, the distribution of digital components that present false information can be restricted, modified, or stopped.

[0024] An operating system (OS), application, or browser may provide a user interface that enables users who have been exposed to false digital components to report the false digital components in a privacy-preserving manner. Specifically, the OS / browser may send false digital component alerts from users to (multiple) central aggregation servers that implement privacy-preserving aggregation metrics. In addition, in order to safeguard the interests of content publishers, measures are taken to prevent fraudulent reporting to verify each false digital component alert using encrypted content request proof tokens, impression proof tokens, and click proof tokens. Verifying the entire causal chain of events (e.g., content request, user's impression of the content, and user's interaction with the content) via proof tokens may more effectively detect forged reports. The user interface may alert users who have viewed a digital component that has been deemed to contain false information in future viewings, such as through a visual alert in the user interface, as described below with reference to Figure 2 and 3 described in further detail.

[0025] Example operating environment

[0026] Figure 1 is a block diagram of an environment 100 in which a digital component distribution system 150 distributes digital components. Example environment 100 includes a data communications network 105, such as a local area network (LAN), a wide area network (WAN), the Internet, a mobile network, or a combination thereof. Network 105 connects client devices 110, publishers 130, websites 140, digital component distribution system 150, detection system 170, aggregation system 180, and reporting system 190. Example environment 100 may include many different client devices 110, publishers 130, and websites 140. In some implementations, environment 100 may also include multiple digital component distribution systems 150.

[0027] A website 140 is one or more resources 145 associated with a domain name and hosted by one or more servers. An example website is a collection of web pages formatted in HTML that may contain text, images, multimedia content, and programming elements (such as scripts). Each website 140 is maintained by a publisher 130, which is an entity that controls, manages, and / or owns the website 140.

[0028] Resource 145 is any data that can be provided via network 105. Resource 145 is identified by a resource address (e.g., a universal resource locator (URL)) associated with resource 145. Resources include, to name a few, HTML pages, word processing documents, portable document format (PDF) documents, images, videos, and feed sources. Resources can include content, such as words, phrases, images, and sounds, which can include embedded information (such as meta information in a hyperlink) and / or embedded instructions (such as scripts).

[0029] Client device 110 is an electronic device capable of communicating over network 105. Example client devices 110 include personal computers, mobile communication devices (eg, smart phones), and other devices that can send and receive data over network 105.

[0030] The client device 110 typically includes applications 112 such as a web browser and / or native applications to facilitate sending and receiving data over the network 105. Native applications are applications developed for a specific platform or a specific device. Publishers 130 can develop and provide (e.g., make available for download) native applications to the client device 110. In some embodiments, the client device 110 is a digital media device, for example, a streaming device that plugs into a television or other display to stream video to the television. The digital media device may also include a web browser and / or other applications that stream video and / or present resources.

[0031] For example, in response to a user of client device 110 entering a resource address of resource 145 in an address bar of a web browser or selecting a link referencing the resource address, the web browser may request resource 145 from a web server hosting website 140 of publisher 130. Similarly, a local application may request application content from a remote server of publisher 130.

[0032] Some resources 145, application pages or other application content may include a digital component slot for presenting a digital component using a resource 145 or application page. As used throughout this document, the phrase "digital component" refers to a discrete unit of a digital component or digital information (e.g., a video clip, an audio clip, a multimedia clip, an image, a text, or another content unit). A digital component may be stored electronically in a physical memory device as a single file or a collection of files, and the digital component may take the form of a video file, an audio file, a multimedia file, an image file, or a text file, and include advertising information so that an advertisement is a type of digital component. For example, a digital component may be content intended to supplement the content of a web page or other resource presented by an application 112. More specifically, a digital component may include digital content related to resource content (e.g., a digital component may be related to the same theme or related theme as the web page content). Thus, providing a digital component by the digital component distribution system 150 may supplement and generally enhance web page or application content.

[0033] When application 112 loads resource 145 (or application content) that includes one or more digital component slots, application 112 may request a digital component for each slot from digital component distribution system 150. Digital component distribution system 150 may in turn request the digital component from digital component provider 160. Digital component provider 160 is an entity that provides digital components for presentations that use resource 145.

[0034] In some cases, the digital component distribution system 150 may also request digital components from one or more digital component partners 157. The digital component partners 157 are agents that select digital components 129 ( Figure 1 Entity not shown).

[0035] Digital component distribution system 150 may select a digital component for each digital component slot based on various criteria. For example, digital component distribution system 150 may select a digital component from among the digital components received from digital component provider 160 and / or digital component partner 157 based on relevance to resource 145 (or application content), performance of the digital component (e.g., rate at which a user interacts with the digital component), etc. Digital component distribution system 150 may then provide the selected digital component(s) to client device 110 for presentation using resource 145 or other application content.

[0036] When application 112 presents a digital component, application 112 (which may be a browser) may store impression data 114 for the presentation of the digital component. Impression data 114 for a particular presentation of a digital component may include a URL or domain for a landing page for the digital component (e.g., a page linked by the digital component and presented to the user by the application / browser when the user clicks on or interacts with the digital component), one or more identifiers for the digital component, event level data associated with the impression, an expiration time specifying when the impression data is deleted from client device 110, and / or a reporting URL or domain to which conversion reports for the digital component are to be sent. This data may be provided by the digital component, for example, as metadata for the digital component or an anchor tag for the digital component. As described in more detail below, instead of storing raw impression data for each impression of the digital component, or in addition to storing raw impression data for each impression of the digital component, application 112 may store impression data blindly signed by detection system 170 to prove the authenticity of the impression event and the impression data.

[0037] The application 112 may also store conversion data 116 in response to detecting a conversion. A conversion for a digital component is the completion of a specified user action after the digital component is presented to the user and / or the user interacts with it (e.g., clicks or taps). Because some conversions may have multiple types, the conversion data 116 for the conversion may include data indicating the type of conversion. The type of conversion may specify a subsequent operation after the conversion is completed. For example, a conversion may be to add an item to an online shopping cart. In this example, the first type of conversion may be to add an item without checking out (e.g., not completing the purchase), and the second type of conversion may be to check out. As described in more detail below, the conversion data for the conversion may be a set of one or more bits indicating the type of conversion, and the application may store the conversion data blindly signed by the digital component distribution system 150 (or another appropriate system) to prove the authenticity of the conversion event and the conversion data.

[0038] The detection system 170 can evaluate fraud signals received from the client device 110 to determine whether an impression or conversion is valid or fraudulent. The fraud signal can be application dependent and can vary in different implementations. The application 112 (or the operating system on the client device 110) can include an API that enables the digital component to call the application 112 (or the operating system on the client device 110) to collect the fraud detection signal and provide the fraud detection signal to the detection system 170. In addition to the fraud detection signal, the application 112 can also send blinded impression data for the impression to the detection system 170. If the detection system 170 determines that the impression is valid, the detection system 170 can sign the blind impression data and provide the signed blind impression data to the application 112 (or the operating system on the client device 110). Example techniques for generating blind impression data and signing the blind impression data are described below.

[0039] Aggregation system 180 can generate aggregated network measurements based on data received from client devices 110. In the following description, techniques for generating and providing measurement data elements are described as being performed by application 112, which can be a web browser or a native application. However, in some implementations, an operating system of client device 110 can generate and send measurement data elements. In such implementations, the web browser(s) and application(s) on client device 110 can be configured to report impressions and conversions to the operating system. The operating system can perform each of the operations described below as being performed by application 112 for reporting impressions and conversions.

[0040] Applications 112 on client devices 110 may provide measurement data elements including encrypted data representing network data to aggregation system 180. The network data may include impression data and / or conversion data for each conversion. For example, application 112 may generate and send to aggregation system 180 measurement data elements for each impression, user interaction, and / or conversion that occurred at client device 110. For each of one or more digital components, the aggregated network measurements may include a total number of impressions, user interactions, and / or conversions for the digital component across multiple client devices 110 or applications 112.

[0041] In some embodiments, a secure, privacy-preserving aggregation system may be implemented as follows: the application 112 may generate data in a measurement data element using a (t, n) threshold scheme. In some embodiments, when the application 112 detects a transformation or receives transformation data for a transformation, the application 112 generates a group key (e.g., a polynomial function) based on the impression data and the transformation data for the transformation. The application may then generate a group member key that represents a portion of the group key and may be used to regenerate the group key only when a sufficient number of group member keys for the same impression and transformation pair are received. In this example, the measurement data element for the transformation may include a group member key generated by the application and a label corresponding to the impression and transformation pair. Each unique impression and transformation pair may have a corresponding unique label so that the aggregation system 180 may aggregate the measurement data element for each impression and transformation pair using the label for that impression and transformation pair.

[0042] In a (t, n)-threshold encryption scheme, the aggregation system 180 will need to receive at least t group member keys for the same impression and conversion pair in order to be able to decrypt the impression and conversion data. If fewer than t group member keys are received, the aggregation system 180 cannot decrypt the impression and conversion data. Once at least t measurement data elements for the same impression and conversion pair are received from the client device 110, the aggregation system 180 can determine the group key from the at least t group member keys and obtain the impression and conversion data from the group key.

[0043] Aggregation system 180 may determine a conversion amount for an impression and conversion pair based on a number of received measurement data elements that include impression data and conversion data for the impression and conversion pair. For example, after obtaining the impression and conversion data using at least t group member keys, aggregation system 180 may determine a count of the number of group member keys received for the impression and conversion pair as the conversion amount. Aggregation system 180 may report the impression data, conversion data, and conversion amount to reporting system 190, which may be implemented as a reporting domain corresponding to a reporting URL for a digital component that, in turn, corresponds to the impression and conversion data. In some embodiments, another secure, privacy-preserving aggregation system may implement multi-party computation (MPC).

[0044] False digital component reporting and verification

[0045] The digital component distribution system 150 can distribute digital components from various digital component providers 160 to the client device 110. The digital components may be marked as false, such as spam, fake, or misleading, by the user of the client device 110. To protect the privacy of the reporting end user and to protect the interests of the digital component providers 160, the alerts generated for false digital components may be verified and aggregated. Figure 2 2 is a block diagram of an environment 200 in which fake digital components are verified and reported. A fake digital component refers to a digital component that contains false or misleading information. A fake digital component reporting server 202 is configured to receive as input a fake digital component alert 204 in response to a digital component 205a, b provided by a digital component provider 207 from a client device 206 and / or a browser 208, and to provide a fake information response 222 to the client device 206 and / or the browser 208 as output.

[0046] The false digital component reporting server 202 may include a verification engine 212, an aggregation engine 214, and a response generator 216. In some implementations, the processing performed by the verification engine 212, the aggregation engine 214, and the response generator 216 may be performed by more or fewer modules. The verification engine 212 is configured to receive the alert 204 from the operating system 209 of the client device 206 and / or to receive the alert 204 from the website browser 208. In addition, the verification engine receives a proof token 218 from the corresponding operating system 209 and / or browser 208.

[0047] In some embodiments, the proof token 218 includes an impression proof token, for example, Figure 1The impression data 114 described above. An impression token generated by presenting a digital component 205a, b on a client device or web browser may include one or more of campaign identification information, the date and time the digital component was presented, a uniform resource locator (URL) of a landing page linked to by the digital component, and a digital signature created by the operating system or web browser. In some implementations, the impression token may include a phone number for a click-to-call advertisement, or a physical location for an in-store visit advertisement.

[0048] In some embodiments, the attestation token 218 may carry an identification of an application 112 (e.g., "Application A") or a domain that a user is browsing (e.g., example.com). In some embodiments, the attestation token 218 may carry additional parameters describing an impression, user interaction, or conversion. In some embodiments, the impression token may carry a device integrity token or browser trust token created by the detection system 170 to prove that the device / browser is trustworthy and / or that the user's actions on the device or in the browser are authentic. The device integrity token / browser trust token may carry a level of trust / authenticity, a timestamp when the device integrity token / browser trust token was generated, a public key of the device / browser or a cryptographic hash thereof that binds the device integrity token / browser trust token to the device / browser, and a digital signature of the detection system 170 so that anyone can verify the authenticity of the device integrity token / browser trust token.

[0049] In some embodiments, the proof token 218 includes a click proof token, for example, Figure 1 The conversion data 116 described. Each click proof corresponds to a specific impression token and is generated by a user interacting with a presented digital component. The click proof token may include a digital signature from the client device where the click occurred. In some embodiments, the click proof token includes a conversion confirmation of a user action performed in response to the presentation of the digital component. For example, the user action may include a user clicking a hyperlink and following it to the linked landing page. In another example, the user action may be a user scrolling through at least a portion of a news article (or other content). In yet another example, the user action may be a user providing feedback in response to the presentation of the digital component, such as a "like", "dislike" or "share" button. In some embodiments, the click token may include the source of the interaction, such as whether the interaction is from a stylus, mouse, keyboard, trackball, touch screen, or programmatically injected by the application 112, a virtual keyboard on the screen. In some embodiments, the click token may include metadata indicating which subset of the information included in the token is self-declared by the application 112 and which subset is verified by the operating system of the client device 110.

[0050] The attestation token 218 may be verified and aggregated by the verification engine 212 and the aggregation engine 214, for example, Figure 1 2. The verification engine 212 can receive the alert 204 and the proof token 218 and verify the alert 204. The verification of the alert 204 can include verifying the impression or presentation of the digital component, verifying the interaction of the user with the digital component, and verifying the alert generated in response to a report of the same human user viewing and interacting with the digital component. Figure 3 Describes further details of the verification process.

[0051] The verification alert 220 is provided as an output from the verification engine 212 to the aggregation engine 214. The aggregation engine 214 is configured to receive the verified alert 220 as an input and determine from the aggregated verified alert 220 and the proof token 218 that the strength of the verified alert 220 meets a threshold. The strength of the verified alert 220 can be a ratio of the verified alert 220 in response to the digital component 205a,b to the total number of presentations of the digital component 205a,b to the user, or the total number of users exposed to the digital component 205, or the total number of proof tokens 218 for the digital component. For example, for 1000 users who viewed and interacted with the digital component 205a, 100 of the users generated a false digital component alert 204 in response to the digital component 205a, meeting the threshold of 10% of the total views / interactions that resulted in the alert 204.

[0052] In some implementations, the strength of the verified alerts 220 may be the number of alerts for the digital component 205a, b from the start of activity, e.g., the first impression of the digital component on the client device 206 and / or website browser 108. The false digital component reporting server 202 may have an aggregation threshold that is the total number of verified alerts that may be allowed before triggering a response to the digital component 205a, b. For example, once the aggregation engine 214 has logged a total of 100 verified alerts 220 from the start of activity for the digital component 205a, the threshold strength of the verified alerts 220 is met.

[0053] In response to determining that the threshold of the validated alert 220 has been met, the aggregation engine 214 can provide a response trigger 221 as an output to a response generator 216. The response generator 216 receives the response trigger 221 as an input and provides a false digital component mitigation response 222 as an output. The false digital component mitigation response 222 can be provided to one or more client devices 206 and / or website browsers 208. In one example, a false information warning 224b is provided by the web browser 208 in a web page 226 together with the digital component 205b. In another example, a false information warning 224a is provided in an application environment 228 together with the digital component 205a.

[0054] In some implementations, the browser or operating system may query the server 202 for a false information response 222, such as a report, for a particular digital component 205 before requesting the digital component from the digital component provider 207. For example, before selecting a digital component from a particular digital component provider 207, the browser may check whether the digital component has a false digital component alert to determine whether to serve the digital component in an active window.

[0055] In some implementations, a browser or operating system may query a false information response 222 for a current digital component 205 being rendered on a client device, for example, to provide a false information warning 224a along with a digital component 224b.

[0056] like Figure 2 As depicted in , false information warnings 224a, 224b may include information to the user related to the report of the digital component being a false digital component. In some embodiments, the false information warning 224a is an overlay display on the current view in the application environment, such as a pop-up window, which includes information to warn the user of false digital content.

[0057] In addition, the false information warning 224a may include active buttons 225a, b for the user to interact with the false information warning 224a. Figure 2 As depicted, the action buttons 225a,b include "Yes" and "No," allowing the user to proceed to a login page or other action associated with selection of the digital component 205a, or not proceed with the process or action associated with selection of the digital component 205a, respectively.

[0058] In some implementations, the user's selection of the active buttons 225a, 225b can provide feedback to the server 202. The server 202 can receive the user selection of the active button 225a (the user selection of the active button 225a causes the action for the digital component 205a to continue, for example, to continue to the login page) as part of the click proof and used to re-evaluate the threshold of the digital component 205a. In one example, for the strength of the verified selection of the active button 225a (e.g., "yes" or "continue" that causes the user interacting with the digital component 205a to continue to the login page of the digital component 205a), the server 202 can re-evaluate the label of the digital component 205a as a fake digital component 205a. The re-evaluation process can involve a review by a human expert.

[0059] In some embodiments, Figure 2 , the false information warning 224b is an overlay, bounding box, or sticky note displayed with the digital component 205b to warn the user of the false digital component 205b. The false information warning 224b may include an active button 225b that allows the user to provide feedback in response to the false information warning 224b. The active button 225b may allow the user to indicate that the false information warning 224b is incorrect, for example, a news article is incorrectly marked as false news. The selection of the active button 225b may be provided to the server 202 along with the click proof token to trigger a re-evaluation of the false digital component mitigation response 222.

[0060] In some implementations, a false digital component mitigation response 222 is provided to the digital component provider 207. The false digital component mitigation response 222 can be, for example, a report notifying the digital component provider 207 that the verified alert 220 has been received by the false digital component reporting server 202 in response to the presentation of the digital component 205a. In the example of providing the report to the digital component provider 207, information about the specific reporting user of the client device and / or browser 208 can be anonymized or removed before providing the report to the digital component provider 207.

[0061] In some implementations, the digital component provider 207 can provide an appeal 227 to the false digital component reporting server 202, e.g., in response to receiving the false digital component mitigation response 222. In one example, the digital component provider 207 can appeal 227 the false digital component mitigation response 222 that flagged the digital component 205a, b. The appeal 227 can trigger a human expert review of the digital component 205a, b in question to determine whether the mitigation response 222 is accurate (e.g., whether the digital component 205a, b was incorrectly flagged as a false digital component 205a, b).

[0062] In some implementations, the validation and aggregation of alerts 204 in response to the presentation of digital components 205 is performed by digital component provider 207. Digital component provider 207 can determine to trigger human expert review of digital components 205a, b based on the strength of the validated alerts 220 satisfying a threshold. For example, an online news platform can validate and aggregate alerts 204 in response to news stories on its platform, for example, to determine which news stories are marked as false news by readers.

[0063] In some implementations, fake digital component reporting server 202 may determine that a particular source (e.g., a particular digital component provider 207) is the source of multiple fake digital components. Server 202 may generate response 222 that includes, when a given digital component from digital component provider 207 is presented on a client device, alerting a user of the client device that digital component provider 207 is known to be the source of fake digital components. For example, a particular website domain may be associated with multiple verified fake content, and server 202 may generate fake digital component mitigation response 222, e.g., a report alerting digital component provider 207 to be aware that digital components from that domain may not be trustworthy.

[0064] In some implementations, the server 202 may determine that a particular digital component provider 207 (e.g., a website domain or application) is associated with multiple verified fake digital components and provide a report 222 along with some or all of the digital components 205a, b from the particular digital component provider 207. For example, a particular domain may be associated with multiple fake news stories, such that when the news stories are presented to a user, the server 202 will preemptively provide a warning, e.g., "This website has been reported for fake news."

[0065] In some implementations, the digital component provider 207 does not call the browser 208 application programming interface (API) to generate a proof token 218 for the digital component 205b. The web browser 208 can, for example, use the browser history to generate the proof token 218. In one example, the browser 208 can take a screenshot of the digital component 205b and / or verify a login page for the digital component 205b.

[0066] In some implementations, the aggregation engine 214 can cluster similar digital components 205, e.g., similar in visual appearance, linked to the same landing page, etc., but may have different activity identification information. The aggregation engine 214 can aggregate verified alerts 220 from the clustered digital components 205, which can reduce the number of verified alerts 220 required to trigger a response 221 from each of the digital components in the cluster.

[0067] In some embodiments, the methods and systems described herein can be used to verify and aggregate the distribution of other forms of digital components, for example, for malware identification, fake news distribution, social media, inappropriate content, etc.

[0068] Figure 3 is a flow chart illustrating an example process for crowdsourcing false digital component reporting. A false digital component alert (302) is received indicating that a digital component presented at a client device includes false information. Figure 2 As depicted in FIG. 2 , a fake digital component alert 204 is received by a verification engine 212. The fake digital component alert 204 stores a corresponding proof token 218, which includes an impression proof token (e.g., impression data 114) and / or a click proof token (e.g., conversion data 116). Each proof token 218 includes information for presenting the digital component to the user and (optionally) the user's interaction with the digital component. A corresponding proof token 218 can be generated for each instantiation of the digital component 205 on the client device, or for each user interaction with the digital component 205.

[0069] In some embodiments, the stored proof token 218 for a particular instantiation of the digital component 205 includes an impression proof token and / or a click proof token. The impression proof token may be generated as follows: the operating system 209 (or another trusted application 228 on the client device 206 or web browser 208) receives a request from the application 228 or web browser 208 to generate a digital signature token, which may be partially or fully encrypted. The digital signature token may be used to verify that the particular digital component 205 provided on the content page within the application 228 (or in the web page 226 of the web browser 208) and displayed at a particular portion of the display of the client device 206 and visible to the user is the particular digital component 205 provided by the digital component provider 207.

[0070] In some cases, the operating system 209 of the client device 206 (or another trusted application on the client device or web browser) determines whether a digital watermark is embedded in a particular digital component 205 displayed on a particular portion of a display (e.g., a touch screen for mobile devices and an LCD / LED display for desktop computers). As used in this specification, a trusted application is an application that operates in a secure environment on the device 206 and performs certain core device services (similar to device driver services performed by privileged code within the operating system). If the operating system 209 (or web browser 208) does not recognize the digital watermark in the digital component 205, it concludes that the digital component 205 provided by the digital component provider 207 (which would include such a watermark) is not visible on the client device 206. This may indicate an error in the content page, content platform, application, or content provider. On the other hand, if the operating system 209 (or web browser 208) recognizes a digital watermark in a particular digital component 205, it decodes the digital watermark to obtain data for multiple attributes (e.g., content item identifier, content type, content description, content creator, etc.) that describe the particular digital component 205 being presented at the client device.

[0071] In some embodiments, a hash message authentication code (HMAC) algorithm can be used to protect the metadata extracted from the watermark by the operating system. The extracted watermark can be provided to a potentially untrusted application and then to a trusted application, which will generate an impression proof token that includes the watermark content in the payload. Utilizing the HMAC method enables the trusted application to distinguish between forged watermark content and watermark content that has been extracted by the operating system.

[0072] Using the decoded data, the operating system 209 (or another trusted application 228 on the client device or web browser 208) generates a digitally signed token that includes data for a plurality of attributes in a payload of the digitally signed token. In some embodiments, the digitally signed token is an impression proof token that includes (1) a unique identifier of the client device (e.g., a public key of the device), (2) a payload, (3) a device integrity token that can be used to assess the authenticity of the client device, and / or (4) a digital signature for the device identifier, payload, and device integrity token of the client device generated using a private key of the client device. Digitally signed tokens, and particularly impression proof tokens, ensure that a recipient of the digitally signed token is able to identify whether any data included in the payload (or otherwise) of the token has been tampered with or forged. In the case where the web browser 208 generates the digitally signed token, the browser's native code generates the token. The browser native code has privileges that scripts running on web pages rendered by the browser do not have. The impression proof token can be stored at the server 202 along with the stored proof token 218 for use in verifying the false digital component alert 204. For example, information in the impression proof token may be used to verify that the digital component identified in the digital component alert was actually presented to the user who submitted the false digital component alert 204 so that the integrity of the false digital component alert 204 may be ensured.

[0073] The trusted application (which may be part of or separate from the operating system of the client device or web browser) may also receive a request from the application or web page to generate a click-proof token indicating that a particular digital component was interacted with by a user. Before generating this click-proof token, the trusted application determines whether certain data included with the request identified by the application as obtained from the operating system 209 (or web browser 208) is indeed generated or obtained from the operating system (or web browser). The data may include data for a first set of parameters identified by the application as describing a first interaction, and a first key-hash message authentication code (HMAC) identified by the application as generated using the data for the first set of parameters. The trusted application provides this data to the operating system (or web browser), and the operating system (or web browser) generates a second HMAC using the data for the first set of parameters included with the request.

[0074] If the second HMAC is not the same as the first HMAC, the operating system (or web browser) determines that the first interaction is not an interaction by an actual human user, and there may be an error in the application (and / or its SDK) and / or the content platform. More specifically, for example, if the second HMAC result obtained by applying the HMAC algorithm to the first set of parameters identified by the application as describing the first interaction and a key that the operating system (trusted application) keeps secret is different from the first HMAC result.

[0075] On the other hand, if the second HMAC is the same as the first HMAC, the operating system (or web browser) concludes that the first interaction is that of an actual human user. In this case, the operating system (or web browser) also indicates to the trusted application that the first HMAC is the same as the second HMAC, and therefore, the data included in the request is authentic (not forged / compromised).

[0076] In response to determining that the first HMAC is the same as the second HMAC, the trusted application generates a click proof token using data for the first set of parameters included in the request. In generating the click proof token, the trusted application includes in a payload of the click proof token (1) data describing the first set of parameters for the first interaction and / or (2) data identifying the specific digital component 205. The operating system (or another trusted application on the client device or web browser) sends the impression proof token to the server 202 (or another appropriate content verification system), where the server 202 can store, for example, the stored proof token 218 for use in verifying the false digital component alert 204. For example, the information in the click proof token can be used to verify that the user who submitted the false digital component alert 204 actually clicked (or tapped) the digital component identified in the digital component alert. Therefore, the system can conclude that the user was actually presented with information on the page to which the digital component links, which puts the user in a position to judge the authenticity or accuracy of the information presented in the digital component and on the page to which the digital component links.

[0077] In some embodiments, the proof token designated for storage of presentation and interaction data of the digital component also includes a plurality of conversion tokens, for example, as referenced Figure 1The conversion data 116. Each of the conversion tokens included in the stored proof tokens corresponds to a specific conversion proof token in the plurality of conversion proof tokens. For example, a conversion proof token can be created for purchasing digital content, calling an advertiser, or actually visiting a physical location. The conversion proof token also provides support for reporting that a user is qualified to evaluate whether a digital component is a fake digital component. Each conversion proof token may include information about the conversion (e.g., a conversion event, an identifier of a client device used to perform the conversion event, the time of the conversion event, etc.) and may be protected (e.g., digitally signed and / or encrypted) to prevent manipulation of the information carried by the conversion proof token.

[0078] Reference Figure 3 , verifying the false digital component alert based on a digital signature included in the false digital component alert, the corresponding public key of the digital signature matches a public key in a certification token designated for storage of presentation and interaction data for the digital component (304).

[0079] The verification engine 212 can compare the public key from the operating system 209 or browser 208 for the alert 204 with the public key of the stored attestation token 218 for the specific instantiation of the digital component 205 and determine that the received alert is in response to an authentic impression / transformation of the digital component 205. In addition, the verification engine 212 can verify from the device integrity token / browser trust token and its digital signature that the client device / browser is authentic.

[0080] In some embodiments, verification of the proof token may be performed by server 202 as follows: server 202 verifies whether the digital signature token (e.g., impression proof token) is authentic by verifying the digital signature. If not authentic, server 202 concludes that digital component 205 may not be displayed on the client device and that there may be an error in the content page, content platform, or application. In the event that the impression proof token cannot be verified for a particular impression of a digital component, server 202 may ignore (or discard) a false digital component alert 204 corresponding to that particular impression.

[0081] If the server 202 determines that the digital signature token is authentic, it next determines whether the data for the attributes included in the payload matches the data for the attributes of the digital component provided by the digital component provider. For example, in order to prove that the digital component is provided by the digital component provider, the digital component provider can digitally sign the digital component, where the digital signature can verify the authenticity of the digital component.

[0082] If server 202 determines that the data for the attributes matches, server 202 concludes that the particular digital component 205 that was provided for display on the particular portion of the device's display and visible to the user was indeed provided by the digital component provider. In these cases, server 202 can conclude that false digital component alert 204 was submitted by a user who was actually presented with the particular digital component 205. If not, server 202 concludes that the digital component is / was not actually displayed on the client device and that there may be an error in the content page, content platform, application, or digital component provider. In the event that server 202 concludes that the particular digital component was not actually displayed, server 202 can ignore (or discard) the false digital component alert 204 corresponding to the impression identified in the false digital component alert 204.

[0083] The corresponding click proof token for the impression proof token can be used to verify whether the interaction is an interaction performed by an actual human user, or whether the interaction has been erroneously generated and / or forged. Therefore, the server 202 can use the click proof token to further ensure that the user who submitted the false digital component alert 204 actually interacted with the specific digital component, which will expose the user to more information related to the specific digital component.

[0084] As part of this click verification, the server 202 verifies whether the click proof token is authentic. If not, the server 202 concludes that the interaction was not an interaction by an actual human user and that there may be a bug in the application (and / or its SDK) and / or the content platform. In the event that the click proof token is not authentic, the server 202 may ignore (or discard) the fake digital component alert 204 corresponding to the click (or impression) identified in the fake digital component alert 204.

[0085] If the server 202 determines that the click proof token is authentic, the server 202 uses a rule engine or another statistical model (e.g., a supervised or unsupervised machine learning model) to determine whether the data for the first set of parameters indicates that the interaction is an interaction performed by an actual human user. Based on this process, the server 202 can conclude that the interaction is or is not an interaction performed by an actual human user.

[0086] Server 202 can conclude from the verification of the impression proof token and the click proof token whether alert 204 is authentic and represents a real impression and interaction of a human user with the digital component. Thus, using the impression and click proof tokens, the false digital component alert 204 can be verified as authentic and coming from a user who was actually presented with and / or actually interacted with the specific digital component.

[0087] In some implementations, validating the false digital component alert includes determining that the false digital component alert includes other relevant conversion proof tokens for the digital component. For example, validating the false digital component alert may include determining that the alert includes conversion data associated with an in-app purchase, click-to-call, or visit to a physical store location by the receiving user.

[0088] In response to validating the false digital component alarm, the false digital component alarm is added to an aggregate report for the digital component (306). The validated alarm 220 can be provided to the aggregation engine 214 at the server 202. The aggregate report for a particular digital component can track multiple validated alarms 220 for the particular digital component 205. In some implementations, the aggregate report can track other information, such as the regionality of the validated alarms, the time dependency of the validated alarms, or other statistical information related to the validated alarms of the digital component.

[0089] Based on the false alarm ratio, it is determined that the intensity of verified false digital component alarms in the aggregate report of the digital component meets the threshold (308). The aggregation engine 214 can compare the verified alarms 220 with the stored certification tokens 218 of the specific digital component 205. For example, the server 202 can compare the ratio of verified alarms 220 relative to the total number of verified impressions / conversions of the digital component 205a. In this example, the threshold is met when the ratio of verified alarms exceeds a certain amount or range (e.g., greater than 10% of the total impressions / conversions of the digital component reported as false digital components).

[0090] Based on the strength of the verified false digital component alert meeting the threshold, a false digital component mitigation response (310) is triggered. In response to determining that the threshold is met, the aggregation engine 214 will trigger a false digital component mitigation response 222 at the response generator 216. The response generator 216 can determine the type of false information warning to be provided with the new presentation of the digital component based on, for example, the classification of the digital component. The classification of the digital component can include, for example, advertisements or news or other information articles. In one example, a digital component 205b that is a news article can trigger a false information warning 224b, which warns the user of false news by presenting the digital component 205b with a border or overlap.

[0091] In some implementations, the response generator may determine the type of false information warning to provide based on the type of client device presenting the digital component. For example, a digital component 205b to be presented in an application environment 228 may trigger a false information warning 224a including a pop-up warning 224a including one or more action buttons 225a for the user to select after interacting with the digital component 205a.

[0092] In some embodiments, the false information alert 224 is a false digital component report including reporting statistics (e.g., the region of the alert), which is anonymized to protect the reporting user's privacy. The false digital component report can include information to be used by the digital component provider 207, web browser, or application provider to better understand the digital component reported as a false digital component.

[0093] A false information warning is provided with the false digital component to one or more client devices (312). The false information warning 224 can be provided to the digital component provider 207, the client device 206, and / or the website browser 208. Before providing the digital component 205 for presentation to the user, the false information warning 224 can be queried from the server 202 by the digital component provider 207 so that the false information warning 224 can be provided with the digital component 205. In one example, the web browser 208 can query the server 202 for the false information warning 224 of a particular news article to be provided in a web page for presentation to the user.

[0094] In some embodiments, the distribution of fake digital components can be reduced or stopped based in part on the fake digital component mitigation response. The distribution of fake digital components can be reduced or stopped by digital component provider 207. In one example, digital component provider 207 can determine to stop or reduce the distribution of a particular digital component 205 that has been marked as fake by a threshold strength of users in a particular region (e.g., in a particular country). In some embodiments, the distribution of fake digital components can be reduced or stopped by browser 208, application 228, or operating system 209 in response to a fake digital component mitigation response (e.g., a report generated by server 202 and queried by browser 208, application 228, or operating system 209 before publishing the digital component).

[0095] In some implementations, in response to the false digital component mitigation response, distribution of the false digital component may be reduced or stopped by preventing the false digital component from being output on the client device 206. Thus, a reduction in computing resources is achieved by not wasting resources on processing and rendering false information within the false digital component.

[0096] Figure 44 is a block diagram of an example computer system 400 that can be used to perform the above operations. System 400 includes a processor 410, a memory 420, a storage device 430, and an input / output device 440. Each of components 410, 420, 430, and 440 can be interconnected, for example, using a system bus 450. Processor 410 is capable of processing instructions for operation within system 400. In some embodiments, processor 410 is a single-threaded processor. In another embodiment, processor 410 is a multi-threaded processor. Processor 410 is capable of processing instructions stored on memory 420 or storage device 430.

[0097] Memory 420 stores information within system 400. In one implementation, memory 420 is a computer readable medium. In some implementations, memory 420 is a volatile memory unit. In another implementation, memory 420 is a non-volatile memory unit.

[0098] The storage device 430 can provide mass storage for the system 400. In some embodiments, the storage device 430 is a computer-readable medium. In various embodiments, the storage device 430 may include, for example, a hard disk device, an optical disk device, a storage device shared by multiple computing devices over a network (e.g., a cloud storage device), or some other large-capacity storage device.

[0099] The input / output device 440 provides input / output operations for the system 400. In some embodiments, the input / output device 440 may include one or more of a network interface device (e.g., an Ethernet card, a serial communication device (e.g., and an RS-232 port), and / or a wireless interface device (e.g., and an 802.11 card)). In another embodiment, the input / output device may include a driver device 460 (e.g., a keyboard, a printer, and a display device) configured to receive input data and send output data to an external device. However, other embodiments may also be used, such as a mobile computing device, a mobile communication device, a set-top box television client device, etc.

[0100] Despite Figure 4 An example processing system is described in the specification, but the subject matter and implementation of the functional operations described in this specification may be implemented in other types of digital electronic circuits including the structures disclosed in this specification and their structural equivalents, or in computer software, firmware or hardware, or a combination of one or more of them.

[0101] The subject matter and the embodiments of the operation described in this specification may be implemented in a digital electronic circuit including the structures disclosed in this specification and their structural equivalents, in computer software, firmware or hardware, or in a combination of one or more of them. The embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more computer program instruction modules, encoded on a computer storage medium (or medium) for operation by a data processing device or controlling the operation of a data processing device. Alternatively, or in addition, the program instructions may be encoded on an artificially generated propagation signal (e.g., a machine-generated electrical signal, optical signal, or electromagnetic signal), which is generated to encode information for transmission to a suitable receiver device for operation by a data processing device. The computer storage medium may be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more thereof or may be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more thereof. Moreover, although a computer storage medium is not a propagation signal, a computer storage medium may be a source or destination of computer program instructions encoded in an artificially generated propagation signal. The computer storage media may also be or be included in one or more separate physical components or media (eg, multiple CDs, disks, or other storage devices).

[0102] The operations described in this specification can be implemented as operations performed by a data processing device on data stored on one or more computer-readable storage devices or received from other sources.

[0103] The term "data processing apparatus" includes various apparatuses, devices and machines for processing data, including, for example, a programmable processor, a computer, a system on a chip, or a plurality or combination of the foregoing. The apparatus may include dedicated logic circuits, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the apparatus may also include code that creates an operating environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and operating environment may implement a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0104] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). A computer program may be deployed to run on a single computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communications network.

[0105] The processes and logic flows described in this specification can be performed by one or more programmable processors running one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits), and the apparatus can also be implemented as special purpose logic circuits such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits).

[0106] For example, processors suitable for running computer programs include general and special microprocessors. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for performing actions according to instructions and one or more memories for storing instructions and data. Typically, a computer will also include or be operatively coupled to receive data from one or more mass storage devices or transfer data to one or more mass storage devices or both, and the one or more mass storage devices are used to store data, such as magnetic, magneto-optical disks or optical disks. However, a computer does not necessarily have such a device. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), etc. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0107] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, as well as a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, voice, or tactile input. In addition, the computer may interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.

[0108] Embodiments of the subject matter described in this specification may be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described in this specification), or any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), interconnections (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0109] A computing system may include a client and a server. The client and the server are usually remote from each other and usually interact through a communication network. The relationship between the client and the server is due to the computer programs running on the respective computers and the client-server relationship with each other. In some embodiments, the server sends data (e.g., an HTML page) to the client device (e.g., for the purpose of displaying data to a user interacting with the client device and receiving user input from the user interacting with the client device). Data generated on the client device (e.g., the result of the user interaction) can be received from the client device at the server.

[0110] A user may be provided with controls that allow the user to make choices to determine whether and when the systems, programs, or features described herein can collect user information (e.g., information about the user's social network, social behavior or activities, occupation, the user's preferences, or the user's current location), and whether the user is sent personalized content or communications from a server. In addition, before certain data is stored or used, it may be processed in one or more ways to remove personally identifiable information. For example, the user's identity may be processed so that personally identifiable information cannot be determined for the user, or the user's geographic location may be summarized where location information is obtained (such as to a city, zip code, or state level) so that the user's specific location cannot be determined. Thus, the user can control what information is collected about the user, how the information is used, the information retention policy, and what information is provided to the user.

[0111] Although this specification contains many specific implementation details, these should not be interpreted as limitations on the scope of any invention or content that may be claimed, but rather as descriptions of specific features of specific embodiments of specific inventions. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Moreover, although features may be described above as appearing in certain combinations and even requested to be protected as such at the outset, one or more features from the combination requested for protection can be deleted from the combination in some cases, and the combination requested for protection may focus on sub-combinations or changes in sub-combinations.

[0112] Similarly, although the operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring the execution of the operations in the particular order shown or in a sequential order, or requiring the execution of all the operations shown, in order to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated in a single software product or packaged in multiple software products.

[0113] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing may be advantageous.

Claims

1. A method performed by a data processing device, the method comprising: receiving, by one or more servers, a false digital component alert indicating that a digital component presented at a client device includes false information; verifying the false digital component alert by comparing a public key from the client device for the false digital component alert to public keys of stored proof tokens specifying presentation and interaction data of the digital component, wherein the stored proof tokens include a plurality of impression proof tokens and a plurality of click proof tokens for a plurality of presentations of the digital component, each click proof token corresponding to a particular impression proof token of the plurality of impression proof tokens; responsive to verifying the false digital component alarm, adding the false digital component alarm to an aggregate report of the digital component; generating a false alarm ratio based on a number of verified false digital component alarms collected from a plurality of different client devices and included in an aggregate report of the digital component relative to a total number of stored attestation tokens for the digital component; determining, based on the false alarm ratio exceeding a threshold, that an intensity of verified false digital component alarms in the aggregate report of the digital component satisfies the threshold; as well as Based on the strength of the verified false digital component alert meeting a threshold, a false digital component mitigation response is triggered.

2. The method of claim 1, further comprising providing a false information warning along with the false digital component to one or more client devices or digital component providers.

3. The method of any preceding claim, further comprising reducing or ceasing distribution of the spurious digital component based at least in part on the spurious digital component mitigation response.

4. The method of claim 3, wherein reducing or stopping the distribution of the false digital component is performed by a digital component provider.

5. The method of claim 3, wherein reducing or stopping the distribution of the false digital component is performed by a browser, application, or operating system of the client device.

6. The method according to claim 1, further comprising: receiving, by another client device, the spoofed digital component mitigation response; as well as Distribution of the false digital component is reduced or stopped by a browser, application or operating system of another client device.

7. The method of claim 5, wherein reducing or stopping distribution of the false digital component comprises preventing output of the false digital component on a client device.

8. The method of claim 6, wherein reducing or stopping distribution of the false digital component comprises preventing output of the false digital component on another client device.

9. A method according to claim 1, wherein each of the plurality of impression proof tokens includes i) an activity ID, ii) a date / time of presentation, and iii) a uniform resource locator for a landing page linked to by the particular digital component, a telephone number of a publisher of the particular digital component, or an address or geo-fence associated with a store location of the publisher of the particular digital component.

10. The method of claim 1, wherein each of the plurality of click-proof tokens comprises a conversion confirmation of a user action performed in response to presentation of the digital component.

11. The method of claim 1 , wherein validating the false digital component alert comprises determining that the false digital component alert includes a specific impression proof token and a corresponding specific click proof token and / or other related conversion proof token for the digital component.

12. The method according to claim 2, wherein: Providing the false information alert along with the false digital component to one or more client devices includes providing a visual signal for presentation along with the false digital component on a client device.

13. The method according to claim 12, wherein: The visual signal includes presenting to a user on a client device an option to not proceed to a login page defined by the digital component, not proceed to dial a phone number defined by the digital component, or not proceed to locate a location defined by the digital component.

14. The method according to claim 2, wherein: Providing the false information warning along with the false digital component to one or more client devices includes providing a verified false digital component report to a publisher of the digital component.

15. A computer storage medium encoded with a computer program, the program comprising instructions which, when executed by a data processing apparatus, cause the data processing apparatus to perform operations comprising: receiving, by one or more servers, a false digital component alert indicating that a digital component presented at a client device includes false information; verifying the false digital component alert by comparing a public key from the client device for the false digital component alert to public keys of stored proof tokens specifying presentation and interaction data of the digital component, wherein the stored proof tokens include a plurality of impression proof tokens and a plurality of click proof tokens for a plurality of presentations of the digital component, each click proof token corresponding to a particular impression proof token of the plurality of impression proof tokens; responsive to verifying the false digital component alarm, adding the false digital component alarm to an aggregate report of the digital component; generating a false alarm ratio based on a number of verified false digital component alarms collected from a plurality of different client devices and included in an aggregate report of the digital component relative to a total number of stored attestation tokens for the digital component; determining, based on the false alarm ratio exceeding a threshold, that an intensity of verified false digital component alarms in the aggregate report of the digital component satisfies the threshold; as well as Based on the strength of the verified false digital component alert meeting a threshold, a false digital component mitigation response is triggered.

16. A system comprising: Client devices; as well as One or more servers operable to interact with the device and perform operations, the operations comprising: receiving, by the one or more servers, a false digital component alert indicating that a digital component presented at the client device includes false information; verifying the false digital component alert by comparing a public key from the client device for the false digital component alert to public keys of stored proof tokens specifying presentation and interaction data of the digital component, wherein the stored proof tokens include a plurality of impression proof tokens and a plurality of click proof tokens for a plurality of presentations of the digital component, each click proof token corresponding to a particular impression proof token of the plurality of impression proof tokens; responsive to verifying the false digital component alarm, adding the false digital component alarm to an aggregate report of the digital component; generating a false alarm ratio based on a number of verified false digital component alarms collected from a plurality of different client devices and included in an aggregate report of the digital component relative to a total number of stored attestation tokens for the digital component; Based on the false alarm ratio exceeding a threshold, determining that the intensity of the verified false digital component alarms in the aggregated report of the digital component meets the threshold; and Based on the strength of the verified false digital component alert meeting a threshold, a false digital component mitigation response is triggered.

Images