Data operation analysis and intelligent recommendation method

Abstract

The invention discloses a data operation analysis and intelligent recommendation method. Firstly, burying point data, business system business data and log data are reported to a data acquisition service through a set top box application; performing data processing and cleaning on the original data through a data processing service, converting the original data into data in a standard format, and storing the data in an OSS data storage; and analyzing TopN recommendation data through the data analysis service program watching data. Through decoupling design and data closed loop of the recommendation platform and the algorithm management platform, effective combination of personalized recommendation and operation guidance is realized, the recommendation effect and the operation and maintenance efficiency are improved, and support for broadcast and television characteristic services such as live broadcast and the like is enhanced. Moreover, the design considers the characteristics of the radio and television industry, such as high-concurrency user behavior data, supervision compliance, real-time performance and expandability. The whole architecture adopts a micro-service mode and is combined with a big data tool to ensure that the end-to-end process of data from acquisition, analysis to recommendation is efficient and reliable.

Description

Technical Field

[0001] This invention relates to the field of software management technology, specifically a data operation analysis and intelligent recommendation method. Background Technology

[0002] Currently, intelligent recommendations in the broadcasting industry typically support multiple scenarios (such as homepage, special topics, and detail pages), pushing recommendation results to set-top boxes via API. They can support multi-data source integration, blacklist / whitelist filtering on some platforms, and content heavily promoted by operations (such as holidays and major events). Real-time recommendations are usually applied on smart set-top boxes.

[0003] The main problem with this technology lies in the flexibility of scenario management and the burden of operation and maintenance. Because scenario management needs to support multiple data sources, filtering rules and A / B testing, the configuration is complex and operators need to make frequent adjustments (such as weights and rules), which may lead to configuration errors or poor results.

[0004] The bucketing logic in A / B testing may lead to statistical bias due to uneven user distribution, and dynamic configuration loading may cause service anomalies due to insufficient memory or hot update failure.

[0005] To address this issue, a data operation analysis and intelligent recommendation method is proposed. Summary of the Invention

[0006] The purpose of this invention is to provide a data operation analysis and intelligent recommendation method, which solves the problems of complex configuration, large A / B test deviation, and unstable hot update in the current intelligent recommendation system of the broadcasting industry. Through the decoupling design of the recommendation platform and the algorithm management platform and the data closed loop, it realizes the effective combination of personalized recommendation and operation guidance, improves the recommendation effect and operation and maintenance efficiency, and strengthens the support for broadcasting characteristic services such as live broadcasting.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a data operation analysis and intelligent recommendation method, comprising the following steps: Step 1: Data Collection: Data collection service for set-top box application embedded data, business system business data, log data reporting; Step 2: Data Processing: The data processing service processes and cleans the raw data, converts it into a standard format, and stores it in OSS data storage. Step 3: Data Analysis: Analyze the program viewing data to generate Top N recommendation data; Step 4: Data storage: The algorithm service uses user data, media asset data, user behavior data, etc., to train and generate algorithm-based recommendation data. The TopN recommendation data and algorithm recommendation data are stored in the Redis database. Step 5: Business module of the recommendation service: The recommendation service retrieves recommendation data from Redis; Step 6: Data Flow: Each application requests recommendation data from the recommendation service through the interface and presents it.

[0008] Preferably, in step 1, the system collects raw data in multiple ways: set-top box application tracking data: recording user behavior in front-end applications, such as clicks, browsing, viewing time, ratings, etc.; business system data: including structured data such as user information, media asset information, and program EPG data; log data: system operation logs, interface call logs, etc., used for monitoring and anomaly analysis; all data is uniformly received and initially verified through a data collection service to ensure data integrity and timeliness. This module adopts the "reporting interface / data collection module" in the recommendation platform. The reporting interface or data collection is to obtain user feedback or behavioral details on recommendation data. Using the reporting interface can provide higher real-time performance. Therefore, in scenarios that require the use of session-based, real-time, and sequence-sensitive algorithms, it is generally necessary to call the active reporting interface to provide feedback information. This module is responsible for receiving data from various front-end applications and business systems and performing initial verification to ensure data integrity and timeliness.

[0009] Preferably, in step 2, the data processing service performs operations such as cleaning, deduplication, format conversion, and field standardization on the raw data to eliminate noisy data. The processed data is converted into a unified standardized format (such as JSON, Avro) and injected into the OSS object storage system for subsequent batch or streaming analysis. This module adopts the "data management module" in the algorithm management platform. Through the data management module, data is extracted and transformed from external sources (such as data merging, filtering, and row and column transformation), and the data is saved in storage in a standard built-in format for use by various algorithms. This module is responsible for extracting and transforming data from external sources (such as data merging, filtering, and row and column transformation) and saving the data in a standard format for use by subsequent algorithms.

[0010] Preferably, in step 3, the data analysis service performs statistical analysis and aggregation calculations based on the cleaned data to generate statistical recommendation data such as "Top N Popular Programs" and "Channel Viewership Rankings"; at the same time, the algorithm service uses user historical behavior, media asset attributes, contextual information, etc., and trains machine learning models (such as collaborative filtering, deep learning, etc.) to generate personalized recommendation results.

[0011] Preferably, in step 3, the statistical analysis-based recommendation is the "data integration module" in the recommendation platform. This module integrates multiple data sources, acquiring recommendation data from various data sources defined in the recommendation scenario definition; supplementary data sources are acquired from supplementary data sources defined in the recommendation scenario. The supplementary data sources are no different from the main data sources, differing only in sorting and merging rules during data integration; whitelist filtering is performed, requiring maintenance in data management and timely changes; blacklist filtering is also performed, requiring maintenance in data management and timely changes; filtered for already-watched programs is provided as input parameters when a terminal requests the program, or actively recorded by the recommendation platform in the session or user context, or extracted from other application systems, and excluded from the recommendation results; and finally, the recommendation results are sorted. The ranking of recommendation results can utilize the ranking rules set within the recommendation scenario, or rely on online ranking algorithms and the "Scenario Management Module." Each recommendation slot can have its own set of recommendation scenarios, containing all the necessary configuration information for that slot. Multiple data source rule definitions support extracting recommendation data from multiple different data sources, including quantity and weight; filtering methods can be set, specifying different blacklists and whitelists; and in certain special scenarios, a "premium pool" can be defined; supplementary data source settings are used when the main data source cannot obtain a sufficient number of recommendations; integration methods are set, integrating recommendation data obtained from multiple data sources and supplementary data sources according to rules, such as deduplication and supplementing metadata; ranking methods can be set, such as tiered ranking, multiple hit priority, weighted mixed ranking, and ranking based on specified program data (e.g., release time) or operational data (e.g., ratings), or more personalized ranking can be achieved using online ranking algorithms; evaluation strategy settings (A / B testing) are also included. If A / B testing is to be enabled for a scenario, the target audience and proportions for the A / B test, the test comparison items, etc., need to be defined to collaboratively generate rule-based recommendation results.

[0012] Preferably, in step 3, the personalized algorithm recommendation is the "task management module" in the algorithm management platform. The task management module is responsible for preparing necessary data for the algorithm, starting the algorithm, tracking the algorithm execution status, allocating resources, managing the automatic execution of the algorithm, maintaining the execution results of the algorithm, and scheduling the algorithm execution to generate model-based recommendation results.

[0013] Preferably, in step 4, both the statistical analysis results and the algorithm recommendation results are written to a Redis high-speed cache database, utilizing its low-latency characteristics to support high-concurrency real-time queries. The data structures stored in Redis include user profiles, program feature vectors, real-time ranking results, etc., and a reasonable expiration policy is set to ensure timely data updates. This module adopts the "data management module" in the recommendation platform. It maintains offline data sources, which need to be pre-generated by the statistical module or algorithm prediction service and then imported into the recommendation platform. The offline data sources need to be updated regularly. The whitelist / bestseller list / blacklist are maintained and updated by the media asset or business operation system. The whitelist or bestseller list updates should pay attention to timeliness. Online data sources are configured, and online data sources provide real-time prediction services or intelligent ranking services. This module is responsible for maintaining offline / online data sources, whitelists / blacklists, bestseller lists, etc., and writing the recommendation results to Redis for real-time queries.

[0014] Preferably, in step 5, the recommendation service acts as a unified interface layer, receiving recommendation requests from various front-end applications (such as homepage, detail page, and topic page). Based on the user ID, scene identifier, context information, etc. carried in the request, the service retrieves the corresponding recommendation results from Redis and returns them after logical integration (such as deduplication, mixed sorting, and filtering). This responsible module adopts the "recommendation interface module" and "data integration module" in the recommendation platform. It receives requests and extracts context through the recommendation interface, and the data integration module is responsible for merging, filtering, and sorting multi-source data.

[0015] Preferably, in step 5, the "recommendation interface module" receives context from the client, including: user identifier, asset identifier, scene identifier, time, location, etc.; and extracts the context through the system, including: user attributes / tags, asset attributes / tags, etc.; a sequence of preceding actions, which is actively provided by the terminal; if not input by the client, the system can also record it by the server according to the session; an exclusion list, which is actively requested by the terminal, and the exclusion may be due to viewing, dislike, or already being recommended, etc. If not input by the client, the system can also supplement it according to the user context; and finally returns the recommendation result.

[0016] Preferably, in step 6, the business application obtains recommended content by calling the recommendation service interface and displays it on the interface in the form of recommendation slot cards, playlists, category rankings, etc.; user feedback behavior on the recommended content (such as clicks, plays, ratings) is again transmitted back through the data collection service to form a closed loop iteration. This responsible module adopts the "reporting interface / data collection module" of the front-end application and the recommendation platform. The front-end is responsible for displaying the recommendation results, and the reporting interface is responsible for collecting user feedback behavior to form a data closed loop.

[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention patent emphasizes the combination of guided recommendations (premium pool / whitelist) and personalized recommendations, grasping the boundaries of content. It not only enables the emergency removal of blacklisted content to ensure safety and compliance, but also incorporates appropriate guided recommendations and ranking strategies in conjunction with important events and specific times. The recommendation platform, while highlighting personalized and intelligent recommendations, achieves complete manageability and control over recommended content and rules. 2. This invention patent enhances the operation of live streaming content, leveraging user viewing behavior in algorithm training. It simultaneously recommends live and on-demand content, fully utilizing the advantages of intelligent recommendation in cable television. Furthermore, it integrates media assets across multiple channels and operates in multiple regions, incorporating various recommendation data sources. This allows for the fusion of multiple recommendation data sources, comprehensive ranking recommendations, efficiency analysis, and market segmentation, all with different approaches and practices. By combining the strengths of various sources, it finds the most suitable recommendations. 3. This invention patent takes into account the characteristics of the broadcasting industry, such as high-concurrency user behavior data, regulatory compliance, real-time performance, and scalability. The overall architecture adopts a microservice model, combined with big data tools, to ensure efficient and reliable end-to-end data flow from collection and analysis to recommendation. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall framework of the intelligent recommendation service of this invention; Figure 2 This is a schematic diagram of the recommended platform architecture for this invention. Detailed Implementation

[0019] The present invention will now be described in more detail by way of examples. These examples are merely illustrative and do not limit the scope of the present invention in any way.

[0020] This invention provides a technical solution: a data operation analysis and intelligent recommendation method, comprising the following steps: Step 1: Data Collection: Data collection service for set-top box application embedded data, business system business data, log data reporting; Step 2: Data Processing: The data processing service processes and cleans the raw data, converts it into a standard format, and stores it in OSS data storage. Step 3: Data Analysis: Analyze the program viewing data to generate Top N recommendation data; Step 4: Data storage: The algorithm service uses user data, media asset data, user behavior data, etc., to train and generate algorithm-based recommendation data. The TopN recommendation data and algorithm recommendation data are stored in the Redis database. Step 5: Business module of the recommendation service: The recommendation service retrieves recommendation data from Redis; Step 6: Data Flow: Each application requests recommendation data from the recommendation service through the interface and presents it.

[0021] Example 1: First, the set-top box application collects data, and the business system reports business data and log data to the data collection service. Then, the data processing service processes and cleans the raw data, converting it into a standard format and storing it in OSS data storage. Next, the data analysis service analyzes program viewing data to generate Top N recommendation data. Then, the algorithm service trains and generates algorithm-based recommendation data using user data, media asset data, user behavior data, etc., and stores the Top N recommendation data and algorithm recommendation data in a Redis database. Finally, the recommendation service retrieves the recommendation data from Redis. Finally, each application requests the recommendation data from the recommendation service through an interface and presents it.

[0022] Example 2: In Example 1, the following steps are added: In step 1, the system collects raw data through multiple methods: set-top box application tracking data: recording user behavior in the front-end application, such as clicks, browsing, viewing time, ratings, etc.; business system data: including structured data such as user information, media asset information, and program EPG data; log data: system operation logs, interface call logs, etc., used for monitoring and anomaly analysis. All data is uniformly received and initially verified through the data collection service to ensure data integrity and timeliness. This module adopts the "reporting interface / data collection module" in the recommendation platform. The reporting interface or data collection is to obtain user feedback or behavioral details on recommendation data. Using the reporting interface can provide higher real-time performance. Therefore, in scenarios that require the use of session-based, real-time, and sequence-sensitive algorithms, it is generally necessary to call the active reporting interface to provide feedback information. This module is responsible for receiving data from various front-end applications and business systems and performing initial verification to ensure data integrity and timeliness.

[0023] In step 2, the data processing service performs cleaning, deduplication, format conversion, and field standardization on the raw data to eliminate noise. The processed data is converted into a unified standardized format (such as JSON or Avro) and injected into the OSS object storage system for subsequent batch or streaming analysis. This module utilizes the "Data Management Module" within the algorithm management platform. This module extracts and transforms data from external sources (e.g., data merging, filtering, row and column transformations) and saves the data in a standard built-in format for use by various algorithms.

[0024] Firstly, the system collects data from set-top box applications, business systems, and log data via a data collection service. The system gathers raw data through various methods: set-top box application data records user behavior in the front-end application, such as clicks, browsing, viewing time, and ratings; business system data includes structured data such as user information, media asset information, and program EPG data; and log data includes system operation logs and API call logs for monitoring and anomaly analysis. All data is uniformly received and initially verified through the data collection service to ensure data integrity and timeliness. This module utilizes the "Reporting Interface / Data Collection Module" from the recommendation platform. Both the reporting interface and data collection are used to obtain user feedback or behavioral details regarding recommendation data. Using the reporting interface provides higher real-time performance; therefore, in scenarios requiring session-based, real-time, and sequence-sensitive algorithms, it is generally necessary to call the active reporting interface to provide feedback information. This module is responsible for receiving data from various front-end applications and business systems and performing initial verification to ensure data integrity and timeliness. The data processing service then processes and cleans the raw data, converting it into a standard format and storing it in OSS data storage. This process includes cleaning, deduplication, format conversion, and field standardization to eliminate noise. The processed data is then converted to a unified, standardized format (such as JSON or Avro) and injected into the OSS object storage system for subsequent batch or streaming analysis. This module utilizes the "Data Management Module" within the algorithm management platform. This module extracts and transforms data from external sources (e.g., data merging, filtering, row and column transformations), storing the data in a standard built-in format for use by various algorithms. The data analysis service then analyzes viewing data to generate Top N recommendation data. The algorithm service then uses user data, media asset data, and user behavior data to train and generate algorithm-based recommendation data. The Top N recommendation data and algorithm-based recommendation data are stored in a Redis database. The recommendation service then retrieves the recommendation data from Redis. Finally, each application requests and presents the recommendation data through an interface.

[0025] Example 3: In Example 2, the following additional steps are added: In step 3, the data analysis service performs statistical analysis and aggregation calculations based on the cleaned data to generate statistical recommendation data such as "Top N Popular Programs" and "Channel Viewership Ranking." Simultaneously, the algorithm service utilizes user historical behavior, media asset attributes, and contextual information to generate personalized recommendation results through machine learning models (such as collaborative filtering and deep learning). Statistical analysis-based recommendations are represented by the "data integration module" in the recommendation platform. This module integrates multiple data sources, acquiring recommendation data according to the various data sources defined in the recommendation scenario. Supplementary data sources acquire recommendation data according to the supplementary data sources defined in the recommendation scenario. These supplementary data sources are essentially the same as the main data source, differing only in their sorting and merging rules during data integration. Whitelist filtering requires maintenance in data management, and changes to the whitelist must be timely. Blacklist filtering also requires maintenance in data management, and changes to the blacklist must be timely. On-demand program filtering is provided as input parameters when a terminal requests the program, or actively recorded by the recommendation platform in the session or user context. The system extracts data from other application systems and excludes it from the recommendation results. Recommendation results are ranked using sorting rules set within the recommendation scenario, or by relying on online ranking algorithms and the "Scenario Management Module." Each recommendation slot can have its own defined scenario, containing all necessary configuration information for that slot. Multiple data source rule definitions support extracting recommendation data from multiple different data sources, specifying quantity, weight, etc. Filtering methods can be set, specifying different blacklists, whitelists, and, in certain special scenarios, defining a "premium pool," etc. Supplementary data source settings are used when the main data source cannot obtain sufficient recommendations. Integration methods are set, integrating recommendation data from multiple data sources and supplementary data sources according to rules, such as deduplication and supplementing metadata. Sorting methods can be set, such as tiered ranking, multiple hit priority, weighted mixed ranking, based on specified program data (e.g., release time) or operational data (e.g., ratings), or by relying on online ranking algorithms for more personalized sorting. Evaluation strategies (A / B testing) are also set. If A / B testing is to be initiated in a certain scenario, it is necessary to define the target audience and proportion of the A / B testing, the items to be compared, etc., and collaboratively generate rule-based recommendation results. Personalized algorithm recommendation is the "task management module" in the algorithm management platform. The task management module is responsible for preparing necessary data for the algorithm, starting the algorithm, tracking the algorithm execution status, allocating resources, managing the automatic execution of the algorithm, maintaining the execution results of the algorithm, scheduling the algorithm execution, and generating model-based recommendation results.

[0026] In step 4, both the statistical analysis results and the algorithm recommendation results are written to the Redis high-speed cache database, leveraging its low latency to support high-concurrency real-time queries. The data structures stored in Redis include user profiles, program feature vectors, real-time ranking results, etc., and a reasonable expiration policy is set to ensure timely data updates. This module adopts the "data management module" in the recommendation platform. It maintains offline data sources, which need to be pre-generated by the statistical module or algorithm prediction service and then imported into the recommendation platform. The offline data sources need to be updated regularly. The whitelist / premium list / blacklist are maintained and updated, all provided by the media asset or business operation system. The timeliness of whitelist or premium list updates is important. Online data sources are configured, which provide real-time prediction services or intelligent ranking services. This module is responsible for maintaining offline / online data sources, whitelists / blacklists, premium lists, etc., and writing the recommendation results to Redis for real-time queries.

[0027] Firstly, the system collects data from set-top box applications, business systems, and log data via a data collection service. The system gathers raw data through various methods: set-top box application data records user behavior in the front-end application, such as clicks, browsing, viewing time, and ratings; business system data includes structured data such as user information, media asset information, and program EPG data; and log data includes system operation logs and API call logs for monitoring and anomaly analysis. All data is uniformly received and initially verified through the data collection service to ensure data integrity and timeliness. This module utilizes the "Reporting Interface / Data Collection Module" from the recommendation platform. Both the reporting interface and data collection are used to obtain user feedback or behavioral details regarding recommendation data. Using the reporting interface provides higher real-time performance; therefore, in scenarios requiring session-based, real-time, and sequence-sensitive algorithms, it is generally necessary to call the active reporting interface to provide feedback information. This module is responsible for receiving data from various front-end applications and business systems and performing initial verification to ensure data integrity and timeliness. The data processing service then processes and cleans the raw data, transforming it into a standard format and storing it in OSS data storage. The service performs cleaning, deduplication, format conversion, and field standardization on the raw data to eliminate noise. The processed data is converted to a unified standardized format (such as JSON or Avro) and injected into the OSS object storage system for subsequent batch or streaming analysis. This module utilizes the "Data Management Module" within the algorithm management platform. This module extracts and transforms data from external sources (e.g., data merging, filtering, row and column transformations), saving the data in a standard built-in format for use by various algorithms. Next, the data analysis service analyzes program viewing data to generate Top N recommendation data. Based on the cleaned data, the data analysis service performs statistical analysis and aggregation calculations to generate statistical recommendation data such as "Top N Popular Programs" and "Channel Viewership Ranking". At the same time, the algorithm service uses user history behavior, media asset attributes, contextual information, etc., and trains machine learning models (such as collaborative filtering, deep learning, etc.) to generate personalized recommendation results. Statistical analysis recommendation is the "data integration module" in the recommendation platform. It integrates multiple data sources and obtains recommendation data according to multiple different data sources defined in the recommendation scenario definition. Supplementary data sources are used to obtain recommendation data according to the supplementary data sources defined in the recommendation scenario.The supplementary data source and the primary data source are essentially the same; the only difference lies in the sorting and merging rules during data integration. Whitelist filtering requires maintenance in data management, and changes to the whitelist must be timely. Blacklist filtering also requires maintenance in data management, and changes to the blacklist must be timely. On-demand program filtering is provided as input parameters when a terminal requests the program, or actively recorded by the recommendation platform in the session or user context, or extracted from other application systems. This excludes programs from the recommendation results. Recommendation result sorting can use the sorting rules set in the recommendation scenario. Simultaneously, relying on the online ranking algorithm and the "Scene Management Module," each recommendation slot can have its own set of recommendation scenarios. These scenarios include all the configuration information needed for that recommendation slot, multi-data source rule definitions, and support for extracting recommendation data from multiple different data sources, specifying quantities, weights, etc.; filtering method settings, specifying different blacklists and whitelists, and defining a premium pool for certain special scenarios; supplementary data source settings, using supplementary data sources when the main data source cannot obtain enough recommendations; integration method settings, integrating recommendation data obtained from multiple data sources and supplementary data sources according to rules, such as deduplication and supplementing metadata; and ranking method settings, such as tiered ranking, multiple hit priority, weighted mixed ranking, and ranking by specified program data such as release time, or operational data such as ratings. More personalized ranking can also be achieved using online ranking algorithms. Evaluation strategy settings (A / B testing) are crucial; if A / B testing is required for a particular scenario, the target audience and proportions, and the items being compared need to be defined and collaboratively generated into rule-based recommendation results. Personalized algorithm recommendations are handled by the "task management module" within the algorithm management platform. Task management is responsible for preparing necessary data for the algorithm, starting the algorithm, tracking its execution status, allocating resources, managing the automatic execution of the algorithm, maintaining the execution results, and scheduling the algorithm to generate model-based recommendation results. The algorithm service then uses user data, media asset data, and user behavior data to train and generate algorithm-based recommendation data. Top-N recommendation data and algorithm recommendation data are stored in a Redis database. Statistical analysis results and algorithm recommendation results are both written to a Redis cache database, leveraging its low latency to support high-concurrency real-time queries. The data structures stored in Redis include user profiles, program feature vectors, and real-time ranking results, with appropriate expiration policies to ensure timely data updates.This module utilizes the "Data Management Module" within the recommendation platform. It maintains offline data sources, which are pre-generated by the statistics module or algorithm prediction service and then imported into the recommendation platform. These offline data sources must be updated regularly. The module also maintains and updates whitelists / featured lists / blacklists, all provided by the media asset management or business operations system. Timeliness is crucial for whitelist and feature list updates. An online data source is configured, providing real-time prediction or intelligent ranking services. This module is responsible for maintaining offline / online data sources, whitelists / blacklists, and feature lists, and writes recommendation results to Redis for real-time querying. The recommendation service then retrieves recommendation data from Redis. Finally, each application requests recommendation data from the recommendation service via an API and presents it.

[0028] Example 4: In Example 3, the following steps are added: In step 5, the recommendation service, acting as a unified interface layer, receives recommendation requests from various front-end applications (such as the homepage, details page, and topic page). Based on the user ID, scene identifier, and context information carried in the request, the service retrieves the corresponding recommendation results from Redis and returns them after logical integration (such as deduplication, mixed sorting, and filtering). This responsible module utilizes the "Recommendation Interface Module" and "Data Integration Module" from the recommendation platform. It receives requests and extracts context through the recommendation interface, while the data integration module is responsible for merging, filtering, and sorting multi-source data. The "Recommendation Interface Module" receives context from the client, including user identifier, asset identifier, scene identifier, time, and location. It also extracts context through the system, including user attributes / tags, asset attributes / tags, etc. A sequence of preceding actions is provided by the terminal; if not input by the client, the system can record it on the server according to the session. An exclusion list is requested by the terminal; exclusions may be due to viewing, dislike, or already being recommended. If not input by the client, the system can supplement it according to the user context. Finally, the recommendation results are returned.

[0029] In step 6, the business application obtains recommended content by calling the recommendation service interface and displays it on the interface in the form of recommendation slot cards, playlists, category rankings, etc. User feedback on the recommended content (such as clicks, plays, ratings) is then transmitted back through the data collection service to form a closed-loop iteration. This responsible module adopts the "reporting interface / data collection module" of the front-end application and the recommendation platform. The front-end is responsible for displaying the recommendation results, and the reporting interface is responsible for collecting user feedback behavior to form a data closed loop.

[0030] Firstly, the system collects data from set-top box applications, business systems, and log data via a data collection service. The system gathers raw data through various methods: set-top box application data records user behavior in the front-end application, such as clicks, browsing, viewing time, and ratings; business system data includes structured data such as user information, media asset information, and program EPG data; and log data includes system operation logs and API call logs for monitoring and anomaly analysis. All data is uniformly received and initially verified through the data collection service to ensure data integrity and timeliness. This module utilizes the "Reporting Interface / Data Collection Module" from the recommendation platform. Both the reporting interface and data collection are used to obtain user feedback or behavioral details regarding recommendation data. Using the reporting interface provides higher real-time performance; therefore, in scenarios requiring session-based, real-time, and sequence-sensitive algorithms, it is generally necessary to call the active reporting interface to provide feedback information. This module is responsible for receiving data from various front-end applications and business systems and performing initial verification to ensure data integrity and timeliness. The data processing service then processes and cleans the raw data, transforming it into a standard format and storing it in OSS data storage. The service performs cleaning, deduplication, format conversion, and field standardization on the raw data to eliminate noise. The processed data is converted to a unified standardized format (such as JSON or Avro) and injected into the OSS object storage system for subsequent batch or streaming analysis. This module utilizes the "Data Management Module" within the algorithm management platform. This module extracts and transforms data from external sources (e.g., data merging, filtering, row and column transformations), saving the data in a standard built-in format for use by various algorithms. Next, the data analysis service analyzes program viewing data to generate Top N recommendation data. Based on the cleaned data, the data analysis service performs statistical analysis and aggregation calculations to generate statistical recommendation data such as "Top N Popular Programs" and "Channel Viewership Ranking". At the same time, the algorithm service uses user history behavior, media asset attributes, contextual information, etc., and trains machine learning models (such as collaborative filtering, deep learning, etc.) to generate personalized recommendation results. Statistical analysis recommendation is the "data integration module" in the recommendation platform. It integrates multiple data sources and obtains recommendation data according to multiple different data sources defined in the recommendation scenario definition. Supplementary data sources are used to obtain recommendation data according to the supplementary data sources defined in the recommendation scenario.The supplementary data source and the primary data source are essentially the same; the only difference lies in the sorting and merging rules during data integration. Whitelist filtering requires maintenance in data management, and changes to the whitelist must be timely. Blacklist filtering also requires maintenance in data management, and changes to the blacklist must be timely. On-demand program filtering is provided as input parameters when a terminal requests the program, or actively recorded by the recommendation platform in the session or user context, or extracted from other application systems. This excludes programs from the recommendation results. Recommendation result sorting can use the sorting rules set in the recommendation scenario. Simultaneously, relying on the online ranking algorithm and the "Scene Management Module," each recommendation slot can have its own set of recommendation scenarios. These scenarios include all the configuration information needed for that recommendation slot, multi-data source rule definitions, and support for extracting recommendation data from multiple different data sources, specifying quantities, weights, etc.; filtering method settings, specifying different blacklists and whitelists, and defining a premium pool for certain special scenarios; supplementary data source settings, using supplementary data sources when the main data source cannot obtain enough recommendations; integration method settings, integrating recommendation data obtained from multiple data sources and supplementary data sources according to rules, such as deduplication and supplementing metadata; and ranking method settings, such as tiered ranking, multiple hit priority, weighted mixed ranking, and ranking by specified program data such as release time, or operational data such as ratings. More personalized ranking can also be achieved using online ranking algorithms. Evaluation strategy settings (A / B testing) are crucial; if A / B testing is required for a particular scenario, the target audience and proportions, and the items being compared need to be defined and collaboratively generated into rule-based recommendation results. Personalized algorithm recommendations are handled by the "task management module" within the algorithm management platform. Task management is responsible for preparing necessary data for the algorithm, starting the algorithm, tracking its execution status, allocating resources, managing the automatic execution of the algorithm, maintaining the execution results, and scheduling the algorithm to generate model-based recommendation results. The algorithm service then uses user data, media asset data, and user behavior data to train and generate algorithm-based recommendation data. Top-N recommendation data and algorithm recommendation data are stored in a Redis database. Statistical analysis results and algorithm recommendation results are both written to a Redis cache database, leveraging its low latency to support high-concurrency real-time queries. The data structures stored in Redis include user profiles, program feature vectors, and real-time ranking results, with appropriate expiration policies to ensure timely data updates.This module utilizes the "Data Management Module" within the recommendation platform. It maintains offline data sources, which are pre-generated by the statistics module or algorithm prediction service and then imported into the recommendation platform. These offline data sources must be updated regularly. The module also maintains and updates whitelists / featured lists / blacklists, all provided by the media asset management or business operations system. Timeliness is crucial for whitelist and feature list updates. An online data source is configured, providing real-time prediction or intelligent ranking services. This module is responsible for maintaining offline / online data sources, whitelists / blacklists, and feature lists, and writes recommendation results to Redis for real-time querying. The subsequent recommendation service retrieves recommendation data from Redis. As a unified interface layer, the recommendation service receives recommendation requests from various front-end applications (such as the homepage, details page, and topic page). Based on the user ID, scene identifier, and context information carried in the request, the service retrieves the corresponding recommendation results from Redis and returns them after logical integration (such as deduplication, mixed sorting, and filtering). This module utilizes the "Recommendation Interface Module" and "Data Integration Module" from the recommendation platform. It receives requests and extracts context through the recommendation interface, while the data integration module handles the merging, filtering, and sorting of multi-source data. The "Recommendation Interface Module" receives context from the client, including user identifiers, asset identifiers, scene identifiers, time, and location. It then extracts context from the system, including user attributes / tags and asset attributes / tags. Preceding action sequences are provided by the terminal; if not input by the client, the system can record them on the server according to the session. An exclusion list is provided by the terminal, indicating exclusions might be due to viewing, dislike, or prior recommendations. If not input by the client, the system can supplement this list based on the user's context. Finally, the module returns the recommendation results. Each application requests recommendation data from the recommendation service via the interface and presents it. Business applications obtain recommended content by calling the recommendation service interface and display it on the interface in the form of recommendation slot cards, playlists, and category rankings. User feedback on recommended content (such as clicks, plays, and ratings) is again transmitted back through the data collection service, forming a closed-loop iteration. This module adopts the "reporting interface / data collection module" of the front-end application and recommendation platform. The front-end is responsible for displaying recommendation results, and the reporting interface is responsible for collecting user feedback behavior, forming a data closed loop.

[0031] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A data operation analysis and intelligent recommendation method, characterized in that: Includes the following steps: Step 1: Data Collection: Data collection service for set-top box application embedded data, business system business data, log data reporting; Step 2: Data Processing: The data processing service processes and cleans the raw data, converts it into a standard format, and stores it in OSS data storage. Step 3: Data Analysis: Analyze the program viewing data to generate Top N recommendation data; Step 4: Data storage: The algorithm service uses user data, media asset data, user behavior data, etc., to train and generate algorithm-based recommendation data. The TopN recommendation data and algorithm recommendation data are stored in the Redis database. Step 5: Business module of the recommendation service: The recommendation service retrieves recommendation data from Redis; Step 6: Data Flow: Each application requests recommendation data from the recommendation service through the interface and presents it.

2. The data operation analysis and intelligent recommendation method according to claim 1, characterized in that: In step 1, the system collects raw data through multiple methods: set-top box application tracking data, which records user behavior in front-end applications, such as clicks, browsing, viewing time, ratings, etc.; business system data, including structured data such as user information, media asset information, and program EPG data; and log data, such as system operation logs and interface call logs, used for monitoring and anomaly analysis. All data is uniformly received and initially verified through a data collection service to ensure data integrity and timeliness. This module adopts the "reporting interface / data collection module" in the recommendation platform. Both the reporting interface and data collection are used to obtain user feedback or behavioral details on recommendation data. Using the reporting interface can provide higher real-time performance. Therefore, in scenarios that require the use of session-based, real-time, and sequence-sensitive algorithms, it is generally necessary to call the active reporting interface to provide feedback information. This module is responsible for receiving data from various front-end applications and business systems and performing initial verification to ensure data integrity and timeliness.

3. The data operation analysis and intelligent recommendation method according to claim 1, characterized in that: In step 2, the data processing service performs operations such as cleaning, deduplication, format conversion, and field standardization on the raw data to eliminate noise. The processed data is converted into a unified standardized format (such as JSON or Avro) and injected into the OSS object storage system for subsequent batch or streaming analysis. This module adopts the "data management module" in the algorithm management platform. The data management module extracts and transforms data from external sources (such as data merging, filtering, and row / column transformation) and saves the data in a standard built-in format for use by various algorithms. This module is responsible for extracting and transforming data from external sources (such as data merging, filtering, and row / column transformation) and saving the data in a standard format for use by subsequent algorithms.

4. The data operation analysis and intelligent recommendation method according to claim 1, characterized in that: In step 3, the data analysis service performs statistical analysis and aggregation calculations based on the cleaned data to generate statistical recommendation data such as "Top N Popular Programs" and "Channel Viewership Ranking". At the same time, the algorithm service uses user historical behavior, media asset attributes, contextual information, etc., and trains machine learning models (such as collaborative filtering, deep learning, etc.) to generate personalized recommendation results.

5. The data operation analysis and intelligent recommendation method according to claim 1, characterized in that: In step 3, the statistical analysis-based recommendation is the "data integration module" in the recommendation platform. It integrates multiple data sources, acquiring recommendation data from several different data sources defined in the recommendation scenario definition. Supplementary data sources acquire recommendation data from supplementary data sources defined in the recommendation scenario. These supplementary data sources are no different from the main data source, differing only in their sorting and merging rules during data integration. Whitelist filtering requires maintenance in data management, and changes to the whitelist must be timely. Blacklist filtering also requires maintenance in data management, and changes to the blacklist must be timely. On-demand program filtering is provided as input parameters when requested by the terminal, or actively recorded by the recommendation platform in the session or user context, or extracted from other application systems, and excluded from the recommendation results. Recommendation result sorting can use sorting rules set in the recommendation scenario, or rely on online ranking algorithms and the "Scenario Management Module." Each recommendation slot can have its own recommendation scenario set, containing all the configuration information needed for that slot. Multiple data source rule definitions support extracting recommendation data from multiple different data sources, including quantity and weight. Filtering method settings include specifying different blacklists, whitelists, and defining a "premium pool" for certain special scenarios. Supplementary data source settings are used when the main data source cannot obtain enough recommendations. Integration method settings integrate recommendation data obtained from multiple data sources and supplementary data sources according to rules, such as deduplication and supplementing metadata. Sorting method settings include tiered ranking, multiple hit priority, weighted mixed ranking, and sorting based on specified program data such as release time or operational data such as ratings. More personalized sorting can also be achieved using online ranking algorithms. Evaluation strategy settings (A / B testing) are also included. If A / B testing is to be initiated in a certain scenario, it is necessary to define the target audience and proportion of the A / B testing, the items to be tested and compared, etc., and collaboratively generate rule-based recommendation results.

6. The data operation analysis and intelligent recommendation method according to claim 1, characterized in that: In step 3, the personalized algorithm recommendation is the "task management module" in the algorithm management platform. The task management module is responsible for preparing the necessary data for the algorithm, starting the algorithm, tracking the algorithm execution status, allocating resources, managing the automatic execution of the algorithm, maintaining the execution results of the algorithm, and scheduling the algorithm execution to generate model-based recommendation results.

7. The data operation analysis and intelligent recommendation method according to claim 1, characterized in that: In step 4, both the statistical analysis results and the algorithm recommendation results are written to the Redis high-speed cache database, which utilizes its low latency characteristics to support high-concurrency real-time queries. The data structures stored in Redis include user profiles, program feature vectors, real-time ranking results, etc., and a reasonable expiration policy is set to ensure timely data updates. This responsible module adopts the "data management module" in the recommendation platform, which maintains offline data sources. The offline data sources need to be generated in advance by the statistical module or algorithm prediction service and then imported into the recommendation platform. The offline data sources need to be updated regularly. The maintenance and updates of the whitelist / featured list / blacklist are all provided by the media asset or business operation system. Attention should be paid to the timeliness of whitelist or feature list updates. Configure an online data source, which provides real-time prediction or intelligent sorting services; This module is responsible for maintaining offline / online data sources, whitelists / blacklists, and a curated selection database, and writes the recommendation results to Redis for real-time querying.

8. The data operation analysis and intelligent recommendation method according to claim 1, characterized in that: In step 5, the recommendation service, as a unified interface layer, receives recommendation requests from various front-end applications (such as homepage, detail page, and topic page). Based on the user ID, scene identifier, context information, etc. carried in the request, the service retrieves the corresponding recommendation results from Redis and returns them after logical integration (such as deduplication, mixed sorting, and filtering). This responsible module adopts the "recommendation interface module" and "data integration module" in the recommendation platform. It receives requests and extracts context through the recommendation interface, and the data integration module is responsible for merging, filtering, and sorting multi-source data.

9. The data operation analysis and intelligent recommendation method according to claim 1, characterized in that: In step 5, the "recommendation interface module" receives the context through the client, which includes: user identifier, asset identifier, scene identifier, time, location, etc. The system extracts context, including user attributes / tags, asset attributes / tags, etc. The preceding action sequence is provided by the terminal. If it is not input by the client, the system can also record it by the server according to the session. The exclusion list is the exclusion actively requested by the terminal. The exclusion may be due to viewing, dislike, or recommendation. If it is not input by the client, the system can also supplement it according to the user context. Finally, the recommendation result is returned.

10. The data operation analysis and intelligent recommendation method according to claim 1, characterized in that: In step 6, the business application obtains recommended content by calling the recommendation service interface and displays it on the interface in the form of recommendation slot cards, playlists, category rankings, etc. User feedback on the recommended content (such as clicks, plays, ratings) is then transmitted back through the data collection service to form a closed-loop iteration. This responsible module adopts the "reporting interface / data collection module" of the front-end application and the recommendation platform. The front-end is responsible for displaying the recommendation results, and the reporting interface is responsible for collecting user feedback behavior to form a data closed loop.

Images