Systems and methods for the ethical collection of data
An automated system using machine learning and continuous monitoring addresses the challenge of ensuring ethical data sourcing for MSPs by continuously verifying data sources, reducing costs and delays, and maintaining compliance with privacy policies, thus improving data quality and reliability.
Patent Information
- Application Number
- JP2021559277
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-08-07
- Filing Date
- 2020-06-12
- Publication Date
- 2024-11-14
- Estimated Expiration
- 2040-06-12
AI Technical Summary
Marketing service providers (MSPs) face challenges in ensuring that data sourced from multiple partners is ethically collected and compliant with privacy policies, which is time-consuming, costly, and prone to delays due to manual checks that are ineffective against dynamic policy changes, leading to potential use of unethically sourced data.
An automated system using software and machine learning to continuously monitor and analyze data source URLs for ethical compliance, applying keyword sets and scoring to ensure ethical data sourcing, and maintaining a database of approved sources, reducing the need for manual checks and ensuring timely compliance.
The system ensures ethical data sourcing by continuously verifying data sources, reducing labor costs, improving data freshness, and maintaining compliance with privacy policies, thereby enhancing the quality and reliability of marketing data.
Smart Images

Figure 0007583737000001 
Figure 0007583737000002 
Figure 0007583737000003
Abstract
Description
[Technical field]
[0001] Marketing services providers (MSPs) exist to assist their clients with many aspects of marketing campaigns, including multi-channel marketing campaigns. [Background technology]
[0002] Channels may include, for example, email, web, direct mail, and digital television. In providing these services, the MSP may perform, for example, customer data consolidation, data augmentation, data hygiene and quality improvement, deduplication, database marketing, prospecting, marketing campaign management, data analytics, and related services. All of these services are driven by data maintained by the MSP that is related to the customer. To access the customer data that drives the services offered by the MSP, many MSPs enter into partnership agreements with customer data brokers. Some MSPs may have partnership agreements with numerous different data brokers in order to provide the most comprehensive suite of solutions for their customers.
[0003] MSPs must take steps to ensure that data provided to them by data partners is ethically sourced. This process is extremely time-consuming and expensive. A single data partner may be collecting data from tens of thousands of websites associated with a particular uniform resource locator (URL). Considering that there may be numerous data partners, each collecting data from a large number of URLs, it is not feasible for an MSP to manually check each URL individually to ensure that the data collected there is ethically sourced and with an appropriate privacy policy. By the time this kind of manual check is completed, the data may no longer be fresh enough to be useful. The above problem is compounded when you consider that the privacy policies at any of these URLs are not immutable and may change at any time, necessitating periodic rechecks. By the time any manual check is completed, it is quite possible that some of the privacy policies of the relevant URLs have changed, rendering the effort wasted.
[0004] Nevertheless, the process that MSPs use to ensure their data is ethically sourced has traditionally been performed manually due to a lack of alternatives. After each data partner fills out a form listing the URLs for their data sources, MSP personnel check each of those URLs individually. This is very costly for the MSP due to the significant number of employee hours that must be allocated to this task. The delays caused by manually checking URLs mean that the MSP must wait before using the latest and potentially most accurate source of data, degrading the quality of the product that is ultimately delivered to the MSP's customer. This creates particular difficulties for data that must be acted upon quickly, such as an indication that a consumer is currently searching for a particular product in the market. Finally, manual processes make it impossible to continually check data source URLs to see if any policies have changed. This degrades the quality of the process of checking data and increases the opportunities that unethically sourced data can make its way into the MSP's system. Only occasional periodic checks are feasible. In today's world, where data is most valuable when it is current, where labor costs continue to rise, and where penalties for using unethically sourced data (even by mistake) continue to grow, manual methods of checking data sources are simply no longer an option. What is desired is an automated system and method to ensure that data is ethically sourced, provide results quickly enough to immediately respond to fresh data, and continually check URLs to data provided by data partners to quickly identify any changes in privacy policies that may demonstrate that the data may no longer be collected and stored ethically.
[0005] The references mentioned in this Background section are not admitted to be prior art with respect to the present invention. Summary of the Invention [Means for solving the problem]
[0006] The present invention relates to an automated, software-implemented system and method for use by MSPs to ensure that data from data partners is ethically sourced. The software continuously surveys active data source URLs to ensure that the data can be used ethically in marketing campaigns. Source and privacy data is extracted and captured for each of the many, many URLs from which the data partner may source data, and this information is analyzed and stored in a specific structure for future use. In some implementations, a keyword set is used to analyze the privacy terms of the website associated with the URL. In some implementations, a keyword set database may be used to track the keywords for this purpose. URLs associated with websites that are ethically collecting data are stored in a URL database so that these URLs do not have to be checked every time a new data partner identifies them as a source of its data. Alternatively, all checked URLs may be maintained in a database along with additional information related to the policies attached to those URLs, such as a flag indicating ethical sourcing or a score reflecting the ethics used in sourcing the data. Also, in some implementations, the system may periodically re-check the URLs in the URL database to ensure that no changes have been made to the data collection practices of the corresponding websites that would indicate that the data is no longer ethically sourced. This check may be scheduled to run on a specific schedule, may be run when resources are available, or may even be re-run constantly to ensure that the most up-to-date privacy policies are available for each corresponding URL.
[0007] These and other features, objects and advantages of the present invention will be better understood from a consideration of the following detailed description taken in conjunction with the drawings. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 is a process flow diagram according to a first implementation of the present invention. [Diagram 2] FIG. 1 is a system architecture diagram according to an embodiment of the present invention. [Diagram 3] FIG. 1 is a process flow diagram according to a second implementation of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Before describing the present invention in further detail, it should be understood that the invention is not limited to the particular embodiments described, and the terminology used in describing particular embodiments is for the purpose of describing those particular embodiments only, and is not intended to be limiting, since the scope of the present invention will be limited only by the claims in any subsequent application to which priority is claimed.
[0010] Before the software-implemented automated portion of the process operates, the process begins by posing a series of questions to the potential data partner. The data partner can log into the software that manages the process and the questions can be posed in a sequential form. The questions asked of the potential data partner can vary depending on whether the potential data partner is a domestic or international entity. A non-limiting sample set of questions posed to domestic entities is as follows: Company and Contact Information date: Company Name: address: Company website: Contact person: Contact Email: Contact Phone Number: Years in business: Data information 1. Outline the data you plan to provide to MSPs. 2. Attach a record layout or data dictionary showing all data elements you have available. The layout should include the number and value of each element (or each value within an element). For example, for age ranges, please provide the estimated ranges you have and the number of items per range. 3. What sanitation processes (e.g., address cleaning, NCOA, opt-out suppression, etc.) do you perform on your files and how frequently? Data Procurement 1. Are you a data originator, an aggregator, or both? 2. What is the source of your data and how is it collected? (e.g. online, offline, call center, warranty, survey, retail transaction, etc.) 3. In what country or countries did your data originate? 4. Do you consult a privacy policy during data collection? Please provide the URL of your privacy policy. 5. Are individuals informed during the data collection process in your privacy policy or elsewhere about the transfer and use of their data by third parties? 6. Does your company (or the data collector) provide a mechanism by which individuals can specifically “opt-in”? If so, what is the individual opting in to (e.g., data transfers to third parties, mobile marketing campaigns, email marketing campaigns, etc.)? 7. Does your company (or data collector) provide a mechanism by which individuals can exercise the choice to "opt out" or prevent the transfer of their data to third parties? 8. Does the data collected include any information about children (i.e., those under 13 or between 13-17 years of age)? If your answer is yes, are you providing that data to the MSP? 9. If you are not the data originator, please specifically identify, if any, contractual restrictions between you and your data sources that affect your ability to supply data to the MSP. California Data Procurement 1. Are you providing to an MSP any “personal information” of a California resident as defined by the California Consumer Privacy Act (CCPA)? For guidance, please refer to the CCPA definition of “personal information” provided below. Please note that this definition may be changed by the California Legislature at any time. Personal Information Definition: Personal information is information that identifies, relates to, describes, is capable of being associated with, or could reasonably be linked, directly or indirectly, to a particular consumer (including, but not limited to, his or her real name, alias, signature, social security number, physical characteristics or description, postal address, physical address, unique personal identifier, online identifier, Internet protocol address, email address, account name, social security number, driver’s license number, passport number, telephone number, passport number, driver’s license or state identification card number, insurance policy number, education, employment, employment history, bank account number, credit card number, debit card number, or any other financial information, medical information, health insurance information, or other similar identifier) or (2) household. 2. Does your consumer privacy notice comply with CCPA requirements, if applicable? Online and / or mobile data procurement 1. Were any parts of the file sourced online or from a mobile device? If the answer is "no", skip to questions 2-4 in this section. 2. Outline how data is collected online, both directly from individuals and by automated means (e.g. data entry fields filled in by site visitors, cookies, mobile SDKs, API feeds, crawlers, web scraping, etc.). 3. How were individuals informed about collected data that they did not knowingly provide (e.g., through a posted privacy policy, in a pop-up box, etc.)? 4. If you are an aggregator and not the original collector of the data you are sharing, please provide the names of the top 100 URLs of data producers from which you source data or the top 10% of distinct data collector URLs by number of records, whichever is greater. Regulatory / Compliance Information 1. Has your company ever been involved in any government inquiry or investigation? If so, please explain.
[0011] A sample set of questions to be posed to international actors is as follows: Company and Contact Information date: Company Name: address: country: Does your company have any significant decision-making presence in the EU? Company website: Contact person: Contact Email: Contact Phone Number: Years in business: Data information 1. Please provide an overview of the data. 2. Attach to this document a record layout showing all data elements you have available. The layout should include the count and value of each element (or each value within an element). For example, for age ranges, please provide the estimated ranges you have and the counts per range. 3. What sanitation procedures (e.g., address cleaning, NCOA, opt-out suppression, etc.) do you perform on your files? 4. Is all data collected made available to MSPs? If not, what is not available? 5. If applicable, please specifically identify any contractual restrictions between you and your data sources that affect your ability to supply data to the MSP. 6. If you are not the data originator, specifically identify any contractual restrictions between you and your data sources that affect your ability to supply data to the MSP, if any. 7. Please list any self-regulatory organizations in which your company currently participates or maintains membership. 8. Are you supplying us with any of the following types of data, namely data deemed legally sensitive and / or data that falls under the MSP’s restricted data categories? If so, which and what additional precautions are taken to inform consumers at the point of data collection? 9. Indicate below the specific consumer privacy laws that are applicable to the data set presented. Data Procurement 11. Are you a data originator, an aggregator, or both? 12. What are the sources of your data and how is the data collected? (e.g. online, offline, call center, warranty, surveys, retail transactions, etc.) 13. In which country or countries did your data originate? (please list all countries) 14. If you are a data aggregator, please describe the measures you take, in addition to contractual clauses, to ensure that your data suppliers have taken appropriate measures to ensure compliance (e.g. due diligence, audits, checking certifications, etc.). 15.Does your company refer to a privacy policy during data collection? Yes / No. Please provide the URL of your company's privacy policy. 16. Please answer the following questions regarding notice and choice. a. Can users express their preferences through www.youronlinechoices.com? b. Is the data you are providing to MSPs collected with consumer / individual consent (opt-in)? c. If the answer to the above question is yes, what are individuals opted-in to (e.g., data transfers to third parties, mobile marketing campaigns, email marketing campaigns, etc.)? d. If the answer to question 16.a. above is "yes," is consent collected using the Transparency and Consent Framework (TCF)? e. Are records kept to document what individuals consented to, including what they said and when and how they consented? f. If the answer to the above question is yes, will such records be readily available to the MSP upon request? g. Since opt-in / consent is not the only legal basis for collecting data, is the data you are providing collected on the legal basis of legitimate interest? h. If you answer “yes” to the above question, please explain why your and our joint legitimate interests override those of the data subject. i. Are individuals informed about transfers and uses of their data by third parties? j. Does your company provide a mechanism by which individuals can exercise the choice to "opt out" or prevent the transfer of their data to third parties? k. Do your data sources (if applicable) offer mechanisms by which individuals can exercise the choice to "opt out" or otherwise prevent the transfer of their data to third parties? L. Does the data collected include any information about children (i.e., those under 13 or between 13 and 17 years of age)? m. If your answer is yes, do you provide that data to MSPs? 17. Is your company a party to any EU voluntary regulatory schemes? Please only list those that are valid at the time you complete this form. 18. Has your company been certified for your privacy and / or security practices? Please list only those that are valid at the time of completing this form. Online and / or mobile data procurement 19. Were any parts of the file sourced online and / or from a mobile device? If your answer is "no," skip the remainder of this section and proceed to Government Inquiries. If your answer is "yes," please indicate the percentage of files procured online or from a mobile device. 20. Please outline how data is collected online, both directly from individuals and by automated means (e.g. data entry fields filled in by site visitors, cookies, mobile SDKs, API feeds, crawlers, web scraping, etc.). 21. How were individuals informed about collected data that they did not knowingly provide (e.g., through a posted privacy policy, in a pop-up box, etc.)? 22. If your company provides location data, please answer the following questions: a. Please indicate the frequency of data collection (how often data is collected). b. Before their location data is collected, is a notice provided to users explaining that their location data will be shared / used with third parties for their marketing purposes? 23. If you are an aggregator and not the original collector of the data you are sharing, please provide the names of the top 100 URLs of data producers from which you source data or the top 10% of distinct data collector URLs by number of records, whichever is greater. Government Inquiry 20. Has your company ever been involved in any government inquiry or investigation? If so, please explain.
[0012] This information is subject to updates as conditions change, such as the introduction of new privacy regulations or changes to existing privacy regulations. Responses returned from the questionnaire are examined to identify any obvious and salient issues. If no such issues exist, the information is fed into a software-implemented machine learning system to ensure ethical data is being provided.
[0013] FIG. 2 illustrates a particular implementation of a system that performs this automated process. A marketing service provider (MSP) server 30 provides a central processing server for the marketing service provider. It communicates with a machine learning system 32, which provides artificial intelligence processing for the system as described below. The machine learning system 32 receives a variety of inputs, whether real or training inputs, and learns to identify privacy policies that are indicative of ethical data sourcing. The machine learning system 32 may also derive a score for the level of ethics associated with data collection at a particular website and / or create a flag that indicates sites that ethically source data. The MSP server 30 receives as input a list of URLs 38 and maintains a database 34 of URLs for approved (or, alternatively, reviewed and scored or flagged) websites. The machine learning system 32 utilizes a keyword set 36, which includes keywords and potentially key phrases, for privacy policy research. These are applied to the websites 40 that correspond to the URLs in the URL database 34, and the websites 40 are accessed through the MSP server 30 over a network such as the Internet 44. Each data provider 42 connects to one or more of the websites 40 because that is the means by which the data provider 42 collects its data. The data providers 42 also have connections (not shown) to the MSP server 30 to provide the data they have collected for use by the MSP, and also to provide the MSP server 30 with the URLs from which this data has been collected.
[0014] Referring now to Figure 1, a process for using the system of Figure 2 will be described. As can be appreciated, after step 10, in which data partners are selected, and step 12, in which URL data 38 is collected (by the form described above or otherwise), processing moves to step 14, during which the identified URLs are uploaded to the MSP server 30 for examination by a machine learning system 32 that trains the subsequent automated portions of the process.
[0015] The MSP's machine learning system 32 maintains a URL database 34 of URLs that it has already approved as ethical sources of data. In step 16, the machine learning system first checks, for each URL provided to the server by the potential partner being investigated, whether such URL has already been investigated and recognized by the system as an ethical data source according to the URL database 34. Alternatively, the URL database 34 may hold information for all URLs that have been checked, along with scores assigned to those URLs or a Boolean flag indicating whether the URL has been accepted for use. In that case, the check against the database 34 is to see whether an appropriate score (i.e., meets the threshold score) or flag has been presented to indicate that data from the URL in question is acceptable for ethical use. If so, further investigation of the URL is not undertaken at this point. This step removes duplication that would otherwise result in checking URLs, since many potential data source partners may be using overlapping sets of URLs to collect data, which would require a conventional manual system to recheck the same URLs multiple times.
[0016] In step 18, the machine learning system scans the material available at the website 40 associated with a given URL using predefined keywords to extract relevant privacy policy components from that URL, i.e., privacy policy components that indicate whether the data provided at that URL was ethically sourced. (As will be appreciated, while a single website 40 is illustrated, the system contemplates that a large number of data providers 42 and associated websites 40 may be used, and furthermore, that multiple websites 40 may be associated as data sources for a particular data provider 42. For example, there may be hundreds of data providers 42 and tens of thousands of websites 40.) It should again be noted that the URL for the website 40 is provided by the data provider 42, indicating that the data provider 42 uses data collected through the website 40. The machine learning system 32 may initially start with a predefined set of keywords stored in the keyword database 36, but over time the system may independently learn additional keywords useful in this process by inputting words from privacy policies that have been found to be associated with ethical data. This information may be stored in the keyword set database 36 by the machine learning system 32 on an ongoing basis. In a manner intended for use with a machine learning system, a training website may be used for this purpose. After the privacy policy components are extracted, in step 20, they are added to a dashboard graphical user display for review by the MSP. This review may initially be performed manually, but ultimately the review is performed by the machine learning system 32 and the graphical display may alternatively simply show the results of the process for a human reader.
[0017] In step 22, URLs are assigned a score for their privacy policy components. This review and scoring is performed by a machine learning system 32. The system improves its accuracy over time through regular human feedback on its results. Based on the score, URLs are accepted or rejected for use, with URLs that have a score equal to or greater than a threshold score being accepted. Then, in step 24, URLs that are accepted for use are written to a database 34 so that once these URLs are approved, they do not need to be checked again when new partners are being evaluated.
[0018] At step 26, already approved URLs are reviewed by the machine learning system 32 to determine if there have been any relevant changes with respect to the URL's privacy policy. These reviews may be performed periodically, or whenever the system has hardware resources available to perform these reviews, or there may even be a dedicated system within the machine learning system 32 to continually review each of the URLs in the URL database 34. If there are no changes, no changes are made to the information in the database 34. If there are changes, the process returns to step 22 to rescore or reassess the URL. If only URLs associated with websites that ethically source data are stored in the URL database 34, any URLs found to be associated with websites that no longer ethically source data may be removed from the database 34. If all URLs are retained but a Boolean flag is used to indicate ethical compliance, the flag may be changed. If all URLs are retained and a score indicating ethical compliance is maintained for the URLs, the old score may be overwritten with the new score.
[0019] At step 28, the machine learning system completes its processing. As will be appreciated, human input becomes less and less important as the machine learns from performing evaluations of potential partners and the URLs used by those potential partners to collect their data. Eventually, it is anticipated that it will be possible to completely eliminate human involvement at all steps of this process as shown in Figure 1.
[0020] Now referring to FIG. 3, a second process for implementing the present invention can be described. At step 50, user authentication is performed, in one implementation using single sign-on (SSO) technology. At step 52, one or more URLs are uploaded for evaluation of policies and terms. It should be noted here that there is a distinction between the policies and terms of the website associated with the URL. Policies are intended to cover specifically consumer privacy policies, including information collection, use, sharing, opt-outs, etc. Terms are intended to cover the terms and conditions of the website, covering appropriate use of the site (and information thereon) and user / use agreements. In some cases, the data being evaluated is harvested from the website, and in those cases the terms are examined to ensure that there is nothing preventing the intended use of the corresponding data. At step 54, URL identification / validation is performed. Specifically, the system examines the input for proper URL formatting, calls the website to get a response, and verifies that the domain is in fact active. Domains that cannot be verified as active are captured in an exception report for further investigation. Policies and terms are then downloaded for the verified domains in step 30. Once downloaded, they are classified as either privacy policies or terms and conditions and are associated with the data partner or partners for which the evaluation is being performed, respectively. This generates a timestamp and a new entry in the corresponding database for each document.
[0021] Step 58 is the periodic adjudication of procurement policies. The system recaptures all active policies and clauses under new control. If a change in the text from the previous policies and clauses is detected, the download and classification process restarts. If the text has not changed, the system creates a timestamp entry of the investigation, indicating that no change was detected. In either case, processing then proceeds to step 60, where natural language processing (NLP) techniques are applied to extract data points required to understand the context of the privacy policy and approve or reject a particular source. By creating data points that the neural network can understand as to why / how a particular policy was rejected or approved, the AI model learns the semantics and, over time, becomes able to automatically make reject or approve decisions without manual intervention.
[0022] At step 62, the policies and clauses are taxonomized. The system uses established key elements that must be present in the policies, and the process identifies, captures, and presents specific text within a predefined taxonomy that allows for both human and machine exploration. In this way, the process is standardized for the various texts and phrases that enable or prevent data usage within the policies. The taxonomized policies and clauses may then be presented through a user interface (UI) at step 64. Using the UI, a human reviewer may inspect the policies at step 66 for language that allows or prevents sharing of data with third parties. In this case, the system has done much of the preliminary work of surfacing relevant text in an easy to navigate taxonomy. These manual reviews form the basis for the training data set required to enable machine scoring. It should be understood that after some time the machine learning system has matured using the input from the manual reviews at step 66, the manual step may eventually be removed entirely.
[0023] After the system has matured to the point where manual review is no longer necessary, processing instead moves to step 90 where, instead of surfacing text for manual review through a UI, data points are extracted and run through a machine learning algorithm to generate a quantified output, or score, of how closely the policy meets the approved and bad policies contained in the training set, step 70. Whether manual or automated with machine learning, in decision block 72 it is determined whether the policy contains language that prevents or permits the sharing of consumer data with third parties and whether the clause allows for or prevents commercial use of the data. A pass indicates that the data may be included in the campaign, while a fail indicates that the data will be excluded.
[0024] The systems and methods described herein may be implemented in various embodiments by any combination of hardware and software. For example, in one embodiment, the systems and methods may be implemented by a computer system or collection of computer systems, each computer system including one or more processors that execute program instructions stored on a computer-readable storage medium coupled to the processor. The program instructions may implement the functions described herein. The various systems and displays illustrated in the drawings and described herein represent example implementations. The order of any method may be changed and various elements may be added, modified, or omitted.
[0025] The computing system or computing device described herein may implement the hardware portion of a cloud computing system or a non-cloud computing system as forming part of various implementations of the present invention. The computing system may be any of a variety of types of devices including, but not limited to, a commodity server, a personal computer system, a desktop computer, a laptop or notebook computer, a mainframe computer system, a handheld computer, a workstation, a network computer, a consumer device, an application server, a storage device, a telephone, a mobile phone, or generally any type of computing node, computing node, computing device, and / or computing device. The computing system includes one or more processors (any of which may include multiple processing cores and may be single-threaded or multi-threaded) coupled to a system memory via an input / output (I / O) interface. The computing system may further include a network interface coupled to the I / O interface.
[0026] In various embodiments, the computer system may be a single processor system including one processor or a multiprocessor system including multiple processors. The processors may be any suitable processor capable of executing computing instructions. For example, in various embodiments, they may be general purpose or embedded processors implementing any of a variety of instruction set architectures. In a multiprocessor system, each of the processors may, but need not, implement the same instruction set in common. The computer system also includes one or more network communication devices (e.g., network interfaces) for communicating with other systems and / or components over a communications network, such as a local area network, a wide area network, or the Internet. For example, a client application executing on the computing device may use the network interface to communicate with a server application executing on a single server or a cluster of servers implementing one or more of the components of the system described herein in a cloud computing or non-cloud computing environment implemented in various subsystems. In another example, an instance of a server application executing on a computer system may use the network interface to communicate with other instances of the application that may be implemented on other computer systems.
[0027] The computing device also includes one or more persistent storage devices and / or one or more I / O devices. In various embodiments, the persistent storage devices may correspond to disk drives, tape drives, solid state memory, other mass storage devices, or any other persistent storage device. A computer system (or a distributed application or operating system running thereon) may store instructions and / or data in the persistent storage devices as desired, and may retrieve the stored instructions and / or data as needed. For example, in some embodiments, a computer system may implement one or more nodes of a control plane or control system, and the persistent storage may include SSDs attached to the server nodes. Multiple computer systems may share the same persistent storage device, or may share a pool of persistent storage devices, with the devices in the pool representing the same or different storage technologies.
[0028] A computer system includes one or more system memories that may store code / instructions and data accessible by the processor. System memory may include, for example, multiple levels of memory and memory caches in systems designed to swap information in memory based on access speed. Interleaving and swapping may be extended to persistent storage in virtual memory implementations. Technologies used to implement memory may include, by way of example, static random-access memory (RAM), dynamic RAM, read-only memory (ROM), non-volatile memory, or flash memory. As with persistent storage, multiple computer systems may share the same system memory, or may share a pool of system memory. The system memory(s) may include program instructions executable by the processor to implement the routines described herein. In various embodiments, the program instructions may be encoded in binary, assembly language, any interpreted language such as Java, compiled languages such as C / C++, or any combination thereof. The specific languages given herein are merely examples. In some embodiments, the program instructions may implement multiple separate client, server nodes, and / or other components.
[0029] In some implementations, the program instructions may include instructions executable to implement an operating system (not shown), which may be any of a variety of operating systems, such as UNIX, LINUX, Solaris™, MacOS™, or Microsoft Windows™. Any or all of the program instructions may be provided as a computer program product, or software, which may include a non-transitory computer-readable storage medium having instructions stored thereon that may be used to program a computer system (or other electronic device) to perform a process according to a variety of implementations. A non-transitory computer-readable storage medium may include any mechanism for storing information in a form (e.g., software, processing application) readable by a machine (e.g., a computer). Generally speaking, a non-transitory computer-accessible medium may include a computer-readable storage medium or memory medium, such as a magnetic or optical medium, e.g., a disk or DVD / CD-ROM, coupled to a computer system via an I / O interface. A non-transitory computer-readable storage medium may also include any volatile or non-volatile medium, such as RAM or ROM, that may be included as system memory or other types of memory in some embodiments of a computer system. In other implementations, program instructions may be communicated using optical, acoustic or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.) transmitted over a communication medium, such as a network and / or wired or wireless link, as may be implemented via a network interface. A network interface may be used to interface with other devices, which may include other computer systems or any type of external electronic device.Generally, system memory, persistent storage, and / or remote storage accessible on other devices over a network may store data blocks, replicas of data blocks, metadata associated with the data blocks and / or their state, database configuration information, and / or any other information usable in implementing the routines described herein.
[0030] In certain implementations, the I / O interface may coordinate I / O traffic between the processor, system memory, and any peripheral devices in the system, including through a network interface or other peripheral interface. In some embodiments, the I / O interface may perform any necessary protocol, timing, or other data conversions to convert data signals from one component (e.g., system memory) into a format suitable for use by another component (e.g., processor). In some embodiments, the I / O interface may include support for devices attached through various types of peripheral buses, such as, for example, variants of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard. Also, in some embodiments, some or all of the functionality of the I / O interface, such as the interface to the system memory, may be incorporated directly into the processor.
[0031] The network interface may enable data to be exchanged between the computer system and other devices attached to the network, such as, for example, other computer systems (which may implement one or more storage system server nodes, primary nodes, read-only nodes, and / or clients of the database system described herein). Additionally, the I / O interface may enable communication between the computer system and various I / O devices and / or remote storage. The I / O devices may, in some embodiments, include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other device suitable for inputting or obtaining data by one or more computer systems. These may be directly connected to a particular computer system or may generally be connected to multiple computer systems in a cloud computing environment, grid computing environment, or other system involving multiple computer systems. Multiple I / O devices may be in communication with the computer system or may be distributed on various nodes of a distributed system that includes the computer system. The user interfaces described herein may be visible to a user using various types of display screens, which may include CRT displays, LCD displays, LED displays, and other display technologies. In some implementations, input may be received through a display using touch screen technology, while in other implementations, input may be received through a keyboard, mouse, touchpad, or other input technology, or any combination of these technologies.
[0032] In some embodiments, similar input / output devices may be separate from the computer system and may interact with one or more nodes of a distributed system that includes the computer system through a wired or wireless connection, such as via a network interface. The network interface may generally support one or more wireless networking protocols (e.g., Wi-Fi / IEEE 802.11, or other wireless networking standards). The network interface may support communication over any suitable wired or wireless general data network, such as, for example, other types of Ethernet networks. Additionally, the network interface may support communication over a telecommunications / telephone network, such as an analog voice network or a digital fiber communications network, communication over a storage area network, such as a Fibre Channel SAN, or communication over any other suitable type of network and / or protocol.
[0033] Any of the distributed system embodiments described herein, or any of their components, may be implemented as one or more network-based services in a cloud computing environment. For example, read-write and / or read-only nodes in a database tier of a database system may present database services and / or other types of data storage services using the distributed storage system described herein to clients as network-based services. In some embodiments, the network-based services may be implemented by software and / or hardware systems designed to support interoperable machine-to-machine interaction over a network. A web service may have an interface described in a machine-processable format, such as Web Services Description Language (WSDL). Other systems may interact with the network-based service in a manner specified by the description of the network-based service's interface. For example, the network-based service may define various operations that other systems may invoke and may define specific application programming interfaces (APIs) that other systems may be expected to follow when requesting various operations.
[0034] In various embodiments, a network-based service may be requested or invoked through the use of a message that includes parameters and / or data associated with a request for the network-based service. Such a message may be formatted according to a particular markup language, such as Extensible Markup Language (XML), and / or may be encapsulated using a protocol, such as Simple Object Access Protocol (SOAP). To perform a request for a network-based service, a client of the network-based service may assemble a message that includes the request and communicate the message to an addressable endpoint (e.g., a Uniform Resource Locator (URL)) corresponding to the web service using an Internet-based application-layer transport protocol, such as Hypertext Transfer Protocol (HTTP). In some embodiments, a network-based service may be implemented using Representational State Transfer (REST) technology rather than a message-based technology. For example, a network-based service implemented according to REST technology may be invoked through parameters included within an HTTP method, such as PUT, GET, or DELETE.
[0035] Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention, a limited number of exemplary methods and materials are described herein. Those skilled in the art will recognize that many more modifications are possible without departing from the inventive concepts herein.
[0036] All terms used herein should be interpreted in the broadest possible sense consistent with the context. In particular, the terms "comprise" and "comprise" refer to elements, components, or steps in a non-exclusive manner and should be interpreted to indicate that the referenced element, component, or step may be present, utilized, or combined with other elements, components, or steps not specifically referenced. When Markush-type groups or other groupings are used herein, it is intended that all individual members of the group and all possible combinations and subcombinations of the group are individually included in the disclosure. When ranges are mentioned herein, it is specifically intended that the disclosure includes all points within that range and all subranges within that range. All references cited herein are incorporated by reference to the extent that they are not inconsistent with the disclosure herein.
[0037] The present invention has been described with reference to certain preferred and alternative embodiments that are intended to be illustrative only and not limiting of the full scope of the invention. Although a single placeholder claim is presented herein, the invention is not so limited, as broader claims will be presented in subsequent nonprovisional patent applications.
Claims
1. 1. A computerized method for ensuring that data is collected ethically, comprising: a data provider uploading a plurality of uniform resource locators (URLs) to a marketing service provider (MSP) server, each of the plurality of URLs corresponding to a website from which data is being collected; the MSP server communicating with a URL database to search the URL database for a match with each of the plurality of URLs; utilizing a machine learning system by the MSP server to extract, for each URL of the plurality of URLs for which no match is found in the URL database, from the website corresponding to such URL of the plurality of URLs, a privacy policy component from the privacy policy that indicates whether data provided at the URL was ethically sourced, the machine learning system comprising a machine that receives a variety of inputs, either actual inputs or training inputs, and learns to identify privacy policy components that are indicative of ethical data sourcing, and if the data was collected by the website in accordance with the privacy policy component that is indicative of ethical data sourcing, the data provided by the URL is ethically sourced; For each URL of the plurality of URLs for which no match is found by the MSP server in the URL database, the MSP server analyzes the privacy policy components extracted from the corresponding website by searching the website for predefined keywords, the predefined keywords being retrieved from a keyword set database by communicating with the machine learning system, to determine whether the privacy policy complies with ethical data collection; for each URL of the plurality of URLs for which no match is found in the URL database, the MSP server adds the URL to the URL database if the privacy policy of the corresponding website complies with ethical data collection, or adds the URL to the URL database with an associated Boolean flag or calculated score indicative of the privacy policy; A computerized method comprising:
2. 2. The computerized method of claim 1, further comprising the step of the MSP server training the machine learning system by presenting a plurality of training websites to the machine learning system and identifying to the machine learning system which of the training websites are associated with ethical collection of data.
3. 3. The computerized method of claim 2, further comprising the step of the machine learning system extracting additional keywords from training websites associated with an ethical collection of data and updating the keyword set database by adding the extracted additional keywords to the keyword set database.
4. 2. The computerized method of claim 1, wherein for each URL of the plurality of URLs for which no match is found in the URL database, the step of the MSP server analyzing the privacy policy components extracted from the corresponding website to determine whether the privacy policy complies with ethical data collection comprises the MSP server scoring each URL.
5. 5. The computerized method of claim 4, wherein the step of scoring each URL comprises determining whether the URL's score meets or exceeds a threshold score.
6. 5. The computerized method of claim 4, wherein said step of adding said URL to said URL database further comprises the step of associating a score of said URL with said URL in said URL database.
7. 5. The computerized method of claim 4, wherein the step of adding the URL to the URL database further comprises the step of associating a flag with the URL in the URL database, the flag being a Boolean indicator of whether the corresponding website of such URL sources data ethically.
8. For each URL in the URL database: utilizing the machine learning system to extract privacy policy components from the websites corresponding to such URLs of the plurality of URLs; utilizing the machine learning system to examine the privacy policy components extracted from the corresponding websites to determine whether the privacy policies comply with ethical data collection; the MSP server retaining the URL in the URL database if the privacy policy of the corresponding website complies with ethical data collection, or the MSP server removing the URL from the URL database if the privacy policy of the corresponding website no longer complies with ethical data collection; The computerized method of claim 1 , further comprising:
9. 9. The computerized method of claim 8, wherein the MSP server periodically automatically performs the steps of utilizing the machine learning system to extract the privacy policy components from the websites corresponding to such URLs of the plurality of URLs; examining the privacy policy components extracted from the corresponding websites to determine whether the privacy policy is consistent with ethical data collection; and retaining the URL in the URL database if the privacy policy of the corresponding website is consistent with ethical data collection or removing the URL from the URL database if the privacy policy of the corresponding website is no longer consistent with ethical data collection.
10. 10. The computerized method of claim 9, further comprising the step of the MSP server constructing a human readable display that includes the change in the privacy policy for at least one of the plurality of URLs.
Citation Information
Patent Citations
Information providing method and system based on personal information
JP2004258872A
US08296819B1
Privacy policy rating system
US20160164915A1
Systems and methods for extraction of policy information
US20180081967A1
