Personalized Content Recommendation System Using Segmented Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems struggle to effectively collect, analyze, and process user behavior data from multiple sources to provide personalized content recommendations on websites, while ensuring real-time processing and compliance with data privacy regulations.
Innovation Solution
A system comprising trackers, Interia Web Analytics (IWA) with its Raw Data Stream (RDS) component, and an Information Assistant (IA) module, which processes user data in real-time, uses algorithms to create user profiles, and provides personalized content recommendations while adhering to GDPR requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If user behavior data is collected and processed in real-time from multiple sources, then personalized content recommendations can be provided, but system complexity and data processing requirements increase significantly
Solution Approach 1:
The system divides user behavior data processing into distinct segments: collection phase (trackers on web pages), transmission phase (data sent to server), storage phase (database storage), and analysis phase (recommendation generation). Each segment handles specific tasks independently, reducing overall system complexity while enabling personalized recommendations.
Solution Approach 2:
A server acts as an intermediary between trackers and the recommendation system. The server receives data from multiple trackers, processes it centrally, and generates recommendations. This intermediary architecture simplifies the system by centralizing complex processing logic in one location rather than distributing it across multiple components.
2Measurement precision
If comprehensive user behavior data is collected and analyzed, then accurate personalized recommendations are achieved, but data processing time and computational resources increase
Solution Approach 1:
User behavior data is collected and stored in a database continuously in the background before recommendation requests are made. This preliminary data collection and storage allows the recommendation system to retrieve pre-processed data quickly when generating recommendations, reducing real-time processing time while maintaining comprehensive data analysis for accuracy.
3Productivity
If real-time user behavior tracking is implemented, then dynamic content personalization is possible, but user privacy concerns and compliance requirements increase
Solution Approach 1:
The system extracts and stores only essential user behavior data (pages viewed, time spent, clicks) necessary for generating recommendations, rather than collecting comprehensive personal information. This selective data extraction enables content personalization while minimizing privacy concerns and compliance requirements.
Data Source
Figure 1
Figure 2
AI summary
pn. "A system for collecting, analyzing and processing of user behavior data from multiple sources in order to create a personalized user distribution scheme with contextual selection of content on the website." 1. Identifying and collecting information about the Internet user and his behavior in real time. 2. Processing of available data in different time intervals. 3. Providing information (real-time and historical) on the Internet user's traffic on the website. 4. Automatic contextual selection of materials on the website The prepared platform is an IT system designed to build 'content' websites focused on interaction with Internet users. The platform contains a number of logical modules aimed at increasing the activation of the Internet user. Modules can connect to each other using webservices or other defined APIs required in a specific situation. This enables easy connection of platform modules with external applications. The individual components have been additionally divided into virtual zones, only within which communication and data flow are possible. As part of building the prototype, we try to embed new projects on the PaaS platform, but it is necessary to ensure full compatibility with previous solutions, which is why an application development strategy was adopted in the microservices model. Flowchart of information Assistant (IA)) 1. Each internet user is automatically marked and assigned a unique identifier. 2. On each page or in our application there are IWA scripts that send information to the collector: 1. Describing the client environment, i.e. the type of application, operating system, browser information, IP addresses, adblock information, etc. 2. Optional (if completed) personal information such as demographic data such as age, gender, place of residence, e-mail hash, IDFA / GAID advertising identifiers. 3. Internet user's traffic on our websites, i.e. where he was, when and where did he come from and where he went, and what types of materials he viewed. 3. All the above data goes to our Big Data system, after appropriate transformation to the appropriate formats. 4. For all our products (internet services), a common content categorization taxonomy has been developed, which is used at subsequent stages to determine the interest groups of Internet users. 5. Based on the collected data, applications work in which we have developed a number of algorithms that are aimed at: 1. Preparation and calculation of an Internet user profile. 2. Preparation of tables describing thematic interests for each Internet user identified by us. 3. Reconstruction of Internet user IDs and implementation of the so-called cross-device identification. 6. On the basis of the collected data, another group of applications with recommendation and predictive algorithms works, the purpose of which is: 1. Preparation of a stream of information that is selected and arranged in such a way that the Internet user increases his engagement by making more views on our websites. 2. In the information stream, we inject additional individual supplementary information depending on the data resulting from the Internet user profile.