Market influence course effect evaluation and optimization system based on big data

Abstract

The application discloses a market influence course effect evaluation and optimization system based on big data, which comprises a hardware layer and a function layer, and the hardware layer and the function layer realize bidirectional communication through a TCP / IP protocol data bus; through OAuth2.0 authentication, data desensitization, containerized deployment and other technologies, multi-terminal adaptation and load balancing are realized under the premise of guaranteeing data security compliance, the system has strong stability and scalability, and can meet the large-scale application requirements of small and medium-sized institutions to large education groups.

Description

Technical Field

[0001] This invention relates to the field of course effectiveness evaluation and optimization technology, and in particular to a market influence course effectiveness evaluation and optimization system based on big data. Background Technology

[0002] Driven by both the digital education strategy and the "Artificial Intelligence+" action plan, the education industry is accelerating its transformation from large-scale supply to high-quality, personalized services. As the core carrier of educational services, the accurate assessment and dynamic optimization of the market influence of courses have become key indicators of the core competitiveness of educational institutions. Whether it's the development of first-class courses in higher education, the creation of job-oriented courses in vocational education, or the market-oriented operation of online education courses, all urgently require scientific evaluation tools to support the improvement of course quality and the maximization of market value.

[0003] However, current practices in course effectiveness evaluation and optimization still face many technical bottlenecks, making it difficult to meet the practical needs of digital transformation in education. These bottlenecks are mainly reflected in the following four aspects:

[0004] 1. The one-sidedness and fragmentation of data collection: Existing assessment systems mostly focus on internal course data (such as exam scores and online time), with insufficient coverage of external market data, especially lacking the ability to integrate data on key dimensions such as industry sentiment, competitor dynamics, and commercial conversion. Furthermore, due to the heterogeneous formats of different data sources and the lack of unified data association standards, a "data silo" phenomenon is formed, making it difficult for course assessments to simultaneously consider the dual attributes of "teaching effectiveness" and "market performance," resulting in a significantly limited assessment perspective.

[0005] 2. Subjectivity and Staticity of Evaluation Indicator Systems: Traditional evaluations often rely on qualitative scoring by experts, with limited quantitative evaluation dimensions and a lack of data support for indicator weighting, making it difficult to objectively reflect the market value of courses. While some systems have constructed quantitative indicators, they are mostly limited to basic dimensions such as learning outcomes, failing to form a multi-level indicator framework covering market penetration, word-of-mouth dissemination, and commercial conversion. Furthermore, the indicator system is rigid and cannot be dynamically adjusted according to industry changes and course types, resulting in insufficient adaptability.

[0006] 3. Insufficient accuracy and timeliness of evaluation models: Existing evaluation tools mostly employ simple statistical methods, exhibiting weak capabilities in processing unstructured data (such as student comments and public opinion texts), making it difficult to uncover the semantic value and sentiment behind the data. Furthermore, model training relies on static datasets and lacks a regular update mechanism. As the market environment changes and courses iterate, the bias in evaluation results gradually increases, failing to provide reliable support for decision-making. Practice at Zhengzhou University shows that the accuracy rate of AI evaluation models without continuous iteration is only 63%, far from meeting practical needs.

[0007] 4. Failure of the closed-loop mechanism for evaluation and optimization: Most systems stop at outputting evaluation results, lacking the ability to target and optimize weak links, and failing to establish a closed-loop mechanism of "evaluation-strategy-verification-re-evaluation". There is a disconnect between evaluation results and optimization actions, and optimization strategies often rely on human experience to formulate, lacking data-driven scientific rigor and the rigorousness of A / B testing verification, resulting in low efficiency of course improvement and difficulty in achieving sustained improvement in market influence.

[0008] Therefore, the key to solving the current challenge of evaluating the market influence of courses lies in how to utilize big data and artificial intelligence technologies to build an integrated system that covers multi-source data collection, scientific indicator evaluation, intelligent model calculation, and closed-loop optimization and iteration. Summary of the Invention

[0009] In order to overcome the shortcomings of the existing technology, one of the objectives of this invention is to provide a market influence course effectiveness evaluation and optimization system based on big data.

[0010] One of the objectives of this invention is achieved through the following technical solution:

[0011] A market influence course effectiveness evaluation and optimization system based on big data includes a hardware layer and a functional layer, wherein the hardware layer and the functional layer achieve bidirectional communication through a TCP / IP protocol data bus; the hardware layer includes:

[0012] Cloud server: configured with ≥256GB of memory, ≥16 cores of CPU and ≥10TB of distributed storage, used to store historical course data for the past 5 years (including course videos, exercises, student files) and industry market benchmark database (including annual market size, growth rate and data of leading institutions).

[0013] Edge computing nodes: Deployed on local servers in the course platform's computer room, configured with ≥32GB of memory and ≥8 CPU cores, used to process high-frequency collected data of ≥1000 records per second in real time, with latency controlled within 50ms;

[0014] Multi-source data interfaces: including RESTful API interfaces, direct database connection interfaces (supporting MySQL and MongoDB), and third-party platform SDK interfaces. The interfaces are configured with API adaptation modules (which can automatically adapt to different platform data formats) and permission verification units (based on the OAuth2.0 protocol to implement interface access authentication).

[0015] The functional layer includes a data acquisition module, a data preprocessing module, a market influence assessment module, and a closed-loop optimization module that are connected in sequence. Each module is deployed in a containerized manner on cloud servers and edge computing nodes to achieve load balancing.

[0016] The data acquisition module synchronously acquires course lifecycle data (including development, launch, and iteration phases) and external market data through multi-source data interfaces.

[0017] The data preprocessing module cleans, fuses, and performs feature engineering on the collected data, outputting a standardized feature dataset.

[0018] The market influence assessment module, based on a preset multi-dimensional indicator system and a trained machine learning model, performs quantitative assessment of a standardized feature dataset and generates a market influence assessment report.

[0019] The closed-loop optimization module automatically generates course optimization strategies based on the deviation indicators in the evaluation report and pushes them to the course management terminal (PC / mobile) via HTTPS protocol.

[0020] As a further improvement to the above technical solution:

[0021] The data acquisition module includes an internal data acquisition unit and an external market data acquisition unit, and is configured with a data compression subunit (using the LZ77 algorithm for a compression rate of ≥60%). The internal data acquisition unit obtains the following data through the course platform interface:

[0022] Learning behavior data includes: cumulative online time (accurate to the second), real-time interaction frequency (including the number of bullet comments sent, the number of questions asked, and the number of posts in the discussion area, with a statistical period of hourly), chapter completion rate (calculated as: number of completed chapters / total number of chapters × 100%), and active feedback error rate ratio (calculated as: number of errors actively marked by students / total learning time × 100%).

[0023] Learning outcome data includes: homework grades (recorded on a percentage basis, rounded to one decimal place), exam pass rate (calculated as: number of students who passed the exam / number of students who took the exam × 100%), and skills certification pass rate (calculated as: number of students who obtained certification / number of students who registered for certification × 100%).

[0024] Course operation data includes: number of students enrolled (differentiating between new and returning students), dropout rate (calculated as: number of dropouts / total number of enrollees × 100%), and repurchase rate (calculated as: number of students who purchased the same course series ≥ 2 times / total number of purchasers × 100%).

[0025] The external market data acquisition unit obtains the following data through a multi-source data interface:

[0026] Industry public opinion data includes social media comments (from Weibo, Xiaohongshu, and Zhihu, including text content, posting time, and number of likes) and education platform ratings;

[0027] Competitive benchmarking data includes: pricing of similar courses (differentiating between basic / advanced / flagship classes), update frequency (number of updated class hours per month), and market share (calculated as: number of users of competitor courses / total number of users in the industry × 100%).

[0028] Business conversion data includes: salary increase for students upon employment (calculated as: first month's salary after employment - pre-enrollment salary / pre-enrollment salary × 100%) and enterprise cooperation renewal rate (calculated as: number of renewing enterprises / total number of cooperating enterprises × 100%).

[0029] Policy environment data: including industry regulatory documents (from the official websites of the Ministry of Education and the State Administration for Market Regulation, including the release date and effective date) and support policies (including subsidy amounts and applicable conditions).

[0030] The data preprocessing module includes:

[0031] Data cleaning unit: Uses Z-score method (threshold set to |Z|>3) to identify and remove outliers; uses KNN algorithm (K value set to 5) to fill missing values, using weighted average for continuous data and mode for discrete data; desensitizes public opinion text (replaces phone numbers and emails with "", and blurs names to "Li" and "Zhang");

[0032] Data fusion unit: Based on the course's unique ID (consisting of 8 digits + 2 letters) and timestamp (accurate to milliseconds), an associated index is built, and internal and external data are spatiotemporally aligned on a weekly basis (e.g., the learning behavior data of week 1 is associated with the public opinion data of the same period), generating a unified data view containing 23 data fields;

[0033] Feature engineering unit: For structured data (such as scores and duration), min-max normalization is applied (mapped to the [0, 1] interval); for unstructured text data (such as comments), the BERT-base-chinese model is used for word segmentation (word-level granularity) and semantic embedding (generating a 768-dimensional vector). The Top 20 keyword features are extracted using the TextRank algorithm, and sentiment tendency features (positive / negative / neutral, quantized as scores in the [-1, 1] interval) are extracted using the VADER sentiment analysis tool; finally, a 128-dimensional standardized feature vector is generated (containing 64-dimensional structured features + 64-dimensional unstructured features).

[0034] The market influence assessment module includes an indicator system configuration unit, a model calculation unit, and a result output unit.

[0035] The indicator system configuration unit presets three levels of evaluation indicators:

[0036] Primary indicator: Market influence comprehensive index (out of 100 points);

[0037] Secondary indicators: Market penetration (weight 35%), word-of-mouth marketing (weight 30%), and commercial conversion rate (weight 35%).

[0038] The third-level indicators include 12 specific indicators such as course exposure (daily average UV), student referral rate (calculated as: number of active referrals / total number of students × 100%), and enterprise purchase volume (annual purchase of course hours). The weight of each indicator is determined by a combination of the analytic hierarchy process (inviting 5-7 industry experts to score) and the entropy weight method (calculating information entropy based on data from the past 3 years) (combined weight = 0.6 × expert weight + 0.4 × entropy weight).

[0039] The model computation unit adopts a dual-model architecture:

[0040] The first model is a public opinion sentiment classification model based on MOOC-BERT (the input is a 768-dimensional text embedding vector, the output is a word-of-mouth communication power score, and the accuracy is ≥85%).

[0041] The second model is an improved random forest regression model (containing 500 decision trees, with a maximum depth of 15, the input being the remaining 112-dimensional feature vectors, and the output being market penetration and commercial conversion scores, with a mean squared error MSE ≤ 5).

[0042] The final score is obtained by weighted summation (overall index = penetration × 35% + reputation × 30% + conversion rate × 35%).

[0043] The result output unit generates a visual evaluation report, including: a radar chart of scores for each indicator, a bar chart of deviation from the industry average (deviation rate = (score of this course - industry average) / industry average × 100%), and a list of the top 3 strength indicators and the top 3 weaknesses indicators (weak indicators are defined as indicators with scores < 80% of the industry average).

[0044] The closed-loop optimization module includes a strategy generation unit, a strategy verification unit, and a dynamic adjustment unit.

[0045] The strategy generation unit generates targeted optimization strategies for weak indicators:

[0046] If market penetration is insufficient (score < 60 points), generate a promotion channel adjustment plan (including ROI analysis of each channel, recommendations for new channels and budget allocation suggestions, such as reducing the budget of inefficient channels by 20% and transferring it to short video platforms).

[0047] If word-of-mouth marketing is low (score < 60 points), generate content iteration suggestions (including a case update list, requiring the addition of ≥ 3 new industry cases per month; interactive design optimization plans, such as adding an instant reward mechanism for chapter quizzes).

[0048] If the business conversion rate is weak (score <60 points), generate a pricing and service package optimization plan (such as launching a "basic class + 1-on-1 consultation" package with a premium of 15%-20%).

[0049] The strategy verification unit adopts an A / B testing mechanism: 10%-20% of the user group is randomly selected as the experimental group (experience optimization strategy), and the rest are used as the control group. The verification period is set according to the strategy type (7 days for promotion channels and 30 days for content iteration). The evaluation index is the difference in index scores between the experimental group and the control group (the significance standard for difference is p<0.05).

[0050] The dynamic adjustment unit updates the strategy parameters based on the verification results (e.g., increasing the channel budget by 10% for channels with a conversion rate increase of ≥5%), and feeds back the optimization effect (e.g., changes in indicator scores) to the market influence assessment module, triggering the next round of assessment, thus achieving a closed-loop iteration at least once per quarter.

[0051] The model computation unit also includes a model update unit: each quarter, based on newly added course data (≥100,000 records) and market data (≥50,000 records), the improved random forest regression model is retrained using the sliding window method (window size is 12 months); an early stop mechanism is set during training (training stops when the validation set MSE increases for 5 consecutive rounds); seamless switching between old and new models is achieved through model version management tools (such as MLflow), and the system can still provide evaluation services based on the old model during the update process, with a switching interruption time of ≤10 seconds.

[0052] The external market data collection unit is equipped with a real-time monitoring subunit: industry public opinion data is collected at a high frequency of 5 minutes (the platform's new content is scanned once every 5 minutes), competitor benchmarking data is collected at a daily level (fully updated at 3:00 AM every day), and commercial conversion data is collected at a weekly level (the data for the week is summarized at 24:00 every Sunday). The collection frequency can be manually adjusted through the management terminal (customized periods of 1 minute to 30 days are supported), and the adjustment will take effect within 30 seconds.

[0053] It also includes a user interaction module: providing a web-based visual interface, including:

[0054] Customizable evaluation metrics interface: Allows course operators to add evaluation dimensions (such as "geographical distribution of students") and set custom weights via sliders (the total weight is forced to 100%).

[0055] Optimization strategy manual intervention entry: Users can edit system-generated strategies (such as modifying budget allocation ratios), pause or terminate execution, and record the reasons for intervention;

[0056] Historical data backtracking function: Supports querying evaluation reports, optimization strategies and execution results at any point in time in the past 3 years, and provides a data export interface.

[0057] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0058] 1. Deep fusion of multi-source data, breaking through the limitations of evaluation perspective:

[0059] This invention enables comprehensive collection of internal data (learning behavior, results, operations) and external market data (public opinion, competitors, conversion, policies) throughout the entire course lifecycle by configuring RESTful APIs, direct database connections, and third-party SDKs. The data coverage dimensions are more than 3 times higher than those of traditional systems.

[0060] By leveraging the association indexing technology of unique course IDs and timestamps, the spatiotemporal alignment and unified view construction of heterogeneous data are achieved, effectively solving the problem of "data silos". This enables the assessment to reflect both the micro-level effectiveness of teaching and the macro-level performance of the market, significantly improving the comprehensiveness and accuracy of the assessment.

[0061] 2. Combining scientific indicator systems with intelligent models enhances the professionalism of the assessment:

[0062] A three-level evaluation framework of "comprehensive index - secondary dimension - tertiary indicator" is constructed, covering 12 core indicators. By combining the analytic hierarchy process and the entropy weight method, the organic combination of "expert experience" and "objective data laws" is achieved, and the credibility of indicator weights is improved by 40% compared with a single method.

[0063] The system employs a dual-model architecture combining MOOC-BERT sentiment classification and improved random forest regression, with optimized processing for both text and structured data. The accuracy of sentiment analysis is ≥85%, and the mean squared error (MSE) is ≤5, which is more than 60% higher than that of traditional statistical methods, reaching the advanced level in the industry.

[0064] 3. Dynamic iteration and real-time response ensure timely evaluation:

[0065] The deployment of edge computing nodes enables high-frequency data processing latency to be controlled within 50ms. Combined with a hierarchical collection mechanism that includes 5-minute-level public opinion monitoring and daily competitor updates, it achieves near real-time updates of evaluation data, solving the pain point of "lagging feedback" in traditional systems.

[0066] A quarterly model retraining and sliding window update mechanism was established, combined with an early stop strategy and seamless switching technology, to ensure that the model is always adapted to the latest market data. The evaluation results are dynamically adjusted with course iteration and market changes, significantly enhancing timeliness and reliability.

[0067] 4. The closed-loop optimization mechanism is implemented to realize the transformation of assessment value:

[0068] Targeted optimization strategies are generated for weak indicators, such as channel adjustment plans when penetration is insufficient and content iteration suggestions when reputation is low. The strategies are 80% more targeted than manual solutions, effectively shortening the course improvement cycle.

[0069] By introducing A / B testing and dynamic adjustment modules, the effectiveness of optimization strategies is ensured through quantitative comparison between the experimental and control groups (p<0.05 is the significance standard). This forms a closed-loop iteration of "evaluation-optimization-verification-re-evaluation", enabling the course's market influence to be precisely improved at least once per quarter, and the commercial conversion efficiency to increase by an average of 15%-20%.

[0070] 5. Flexible adaptation and security compliance enhance system usability:

[0071] It offers interactive features such as customizable indicators, manual intervention in strategies, and historical data backtracking, supporting personalized assessment needs for different types of courses (vocational education, higher education, online courses), and its adaptability is far superior to fixed template systems.

[0072] By employing technologies such as OAuth2.0 authentication, data anonymization, and containerized deployment, the system achieves multi-terminal adaptation and load balancing while ensuring data security and compliance. It boasts strong system stability and scalability, meeting the large-scale application needs of organizations ranging from small and medium-sized institutions to large education groups.

[0073] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0074] Figure 1 This is the overall system flowchart for this embodiment;

[0075] Figure 2 This is a flowchart of the data acquisition process in this embodiment;

[0076] Figure 3 This is a flowchart of the data preprocessing process in this embodiment;

[0077] Figure 4 This is a flowchart of the market influence assessment process in this embodiment. Detailed Implementation

[0078] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0079] It should be noted that when a component is described as "fixed to" another component, it can be directly on the other component or may have a component in between. When a component is considered "connected to" another component, it can be directly connected to the other component or may have a component in between. When a component is considered "set on" another component, it can be directly set on the other component or may have a component in between. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0080] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. Specific Implementation

[0081] This embodiment takes the "Python Data Analysis in Practice" online course of a vocational education institution as an example to explain in detail the specific implementation method of the market influence course effect evaluation and optimization system based on big data.

[0082] (a) System hardware layer deployment

[0083] 1. Cloud Server Configuration: An Alibaba Cloud ECS g7e instance is used, configured with 320GB of memory, 24-core CPU, and 15TB of distributed storage (OSS service), deployed with CentOS 8.5 operating system and MySQL 8.0 database. The storage module is divided into two levels: Level 1 storage is used for real-time storage of nearly 6 months of course operation data (including 2 million learning behavior records and 500,000 public opinion data), and Level 2 storage uses archive mode to store nearly 5 years of historical course data (including complete datasets of 120 similar courses).

[0084] 2. Edge computing node deployment: Two Huawei FusionServerPro2488HV5 servers are deployed as edge nodes in the organization's Beijing data center. Each server is configured with 64GB of memory and 16-core CPU. Data acquisition and real-time processing services are deployed through Docker containerization. The servers are connected to the cloud server via a dedicated line, with a stable latency of 35-45ms. This can support the parallel processing of 1,500 high-frequency data points per second.

[0085] 3. Implementation of multi-source data interfaces:

[0086] RESTful API interface: Developed based on the Spring Boot framework, it supports API integration with course platforms (NetEase Cloud Classroom, Tencent Classroom), transmits data in JSON format, and has an interface response time of ≤200ms;

[0087] Direct database connection: A JDBC driver is used to implement direct connections with MySQL (course data) and MongoDB (public opinion text data), and connection pool parameters are configured (maximum number of connections 50, idle timeout 300s).

[0088] Third-party SDK interfaces: integrates Douyin Open Platform SDK (version 2.10.0), Qichacha API SDK (version 3.5.0) and WeChat Official Accounts Platform SDK, completes authentication through OAuth2.0 protocol, and automatically updates interface tokens at 2:00 AM every day.

[0089] (II) Functional Layer Module Operation Flow

[0090] 1. Data acquisition module operation:

[0091] Internal data collection: Learning behavior data is synchronized every 10 minutes through the course platform interface (cumulative online time accurate to the second, and interaction frequency is counted by the hour). Learning outcome data (homework scores are retained to one decimal place) and operational data (differentiating between the proportion of new and old students selecting courses) are fully synchronized at 4:00 AM every day.

[0092] External data collection: Real-time monitoring sub-units enable 5-minute-level public opinion scanning (covering 6 platforms including Weibo and Zhihu), daily updates of competitor data (including course pricing and update frequency of 5 leading institutions) at 3:00 AM, weekly summaries of commercial conversion data (student salary increase statistics accurate to yuan) at 12:00 AM every Sunday, and monthly policy environment data (from the official websites of the Ministry of Education and the Ministry of Human Resources and Social Security) on the 1st of each month.

[0093] Data compression: The LZ77 algorithm is used to compress the collected data, achieving a compression rate of 68% for text data and 62% for structured data, effectively reducing the bandwidth usage.

[0094] 2. Data preprocessing module operation:

[0095] Data cleaning: The Z-score method (with a threshold of |Z|>3) was used to identify and remove outliers (such as records with a single online duration of more than 24 hours, accounting for about 0.3%). The KNN algorithm (K=5) was used to fill in missing values ​​(the missing data rate of the learned data was about 1.2%, and the data integrity reached 99.8% after filling). 12,000 pieces of content containing personal information in the public opinion text were anonymized.

[0096] Data fusion: Based on the unique course ID (formatted as “PYD20240301”, 8 digits + 2 letters) and millisecond-level timestamps, an associated index is built. Internal learning data is aligned with external public opinion data on a weekly basis to generate a unified data view containing 23 fields (fields include “course ID, weekly online time, public opinion sentiment score, competitor pricing”, etc.).

[0097] Feature engineering: Structured data is normalized to the [0,1] interval using min-max normalization. The public opinion text is segmented (word-level granularity) and semantically embedded (generating a 768-dimensional vector) using the BERT-base-chinese model. The Top 20 keywords (such as "data visualization" and "practical case") are extracted using the TextRank algorithm. Sentiment scores (range [-1,1]) are generated using the VADER tool. Finally, the features are fused into a 128-dimensional feature vector (64-dimensional structured + 64-dimensional unstructured).

[0098] 3. Market influence assessment module operation:

[0099] The indicator system is configured as follows: The primary indicator, "Comprehensive Market Influence Index," comprises three secondary indicators (market penetration 35%, word-of-mouth dissemination 30%, and commercial conversion 35%) and twelve tertiary indicators. Six experts specializing in vocational education research (three professors and three senior operations experts) were invited to score the indicators. Combined with entropy weighting calculations based on industry data from the past three years, a combined weighting was obtained (e.g., student referral rate weight 0.08, enterprise purchase volume weight 0.12).

[0100] Model calculation: The MOOC-BERT sentiment classification model takes text embedding vectors as input and outputs a word-of-mouth spread score (accuracy 86.2%); the improved random forest regression model (500 decision trees, maximum depth 15) takes the remaining 112 features as input and outputs market penetration and commercial conversion scores (MSE 4.1 and 4.5 respectively); a comprehensive index (out of 100) is obtained by weighted summation.

[0101] Output results: Generate a visual report, including an indicator radar chart (showing the score distribution of 12 indicators), a deviation bar chart (compared with the industry average, such as the course reputation score being 12% higher than the industry average), and a list of strengths and weaknesses indicators (TOP3 strengths: repurchase rate 82 points, skills certification pass rate 85 points; TOP3 weaknesses: exposure 58 points, corporate purchase volume 55 points).

[0102] 4. Closed-loop optimization module operation:

[0103] Strategy Generation: Solutions were generated for weak metrics: Insufficient market penetration (exposure score 58), it is recommended to transfer 25% of the budget from inefficient channels (Baidu Tieba promotion) to the Douyin short video platform, and add 3 new practical skills short videos per week; Weak commercial conversion (corporate purchase volume score 55), a "basic course + 1 in-house training" package was launched with a 18% premium.

[0104] Strategy Validation: A / B testing was used, randomly selecting 15% of users (5320 people) as the experimental group and the remainder (47880 people) as the control group. The validation period was 30 days. The experimental group implemented channel adjustments and package optimizations, while the control group maintained the original plan.

[0105] Dynamic adjustment: After 30 days, the exposure of the experimental group increased by 42% and the enterprise's purchase volume increased by 35%, confirming the effectiveness of the strategy. The budget allocation ratio was solidified as the system default parameter, and the optimization effect was fed back to the evaluation module, triggering a new round of evaluation in July.

[0106] User interaction module operation: Provides a web-based operation interface (developed based on Vue 3.0). Course operators can add the "student geographic distribution breadth" indicator (weighted at 5%) through a custom interface, adjust the package premium to 16% through the intervention entry, and export the evaluation report (CSV format) for March 2024 through the retrospective function for quarterly review.

[0107] II. Experimental Records

[0108] (I) Experimental Objective

[0109] This study verifies the performance of the system in three dimensions: data collection completeness, evaluation model accuracy, and closed-loop optimization effectiveness, and compares its advantages with traditional evaluation methods (manual scoring + simple statistics).

[0110] (II) Experimental Subjects and Environment

[0111] 1. Experimental subjects: Three similar online courses from this institution were selected ("Python Data Analysis in Practice", "Advanced Excel Applications", and "SQL Database Development"). Each course had 15,000-20,000 students and 5-8 partner companies.

[0112] 2. Experimental environment: The system deployment environment is as described in the above example. The traditional evaluation method adopts the "5-person expert team + Excel statistics" model, and the evaluation cycle is once a month.

[0113] (III) Experimental Design and Procedure

[0114] Experiment 1: Verification of Data Acquisition Integrity

[0115] 1. Experimental method: Record the coverage dimensions and data volume of internal data (learning behavior, results, operations) and external data (public opinion, competitors, conversion, policies) collected by the system for 30 consecutive days, and calculate the collection coverage rate of each type of data (actual collection dimension / planned collection dimension × 100%).

[0116] 2. Experimental steps:

[0117] Days 1-5: Configure the data source and acquisition frequency of the data acquisition module, and complete the interface integration test;

[0118] Days 6-35: Start fully automated data collection and generate daily data collection logs;

[0119] Day 36: Analyze the coverage dimensions and data volume of each data source, and calculate the coverage rate.

[0120] Experiment 2: Evaluation of Model Accuracy Validation

[0121] 1. Experimental method: Based on the evaluation results of the joint expert scoring (10-person expert group, using the Delphi method), the evaluation deviation rate between this system and the traditional method was calculated (|system score - expert score| / expert score × 100%). At the same time, the accuracy of the model (classification task) and MSE (regression task) were statistically analyzed.

[0122] 2. Experimental steps:

[0123] Days 1-10: Organize historical data from 3 courses (500,000 records in total), train MOOC-BERT and improve the random forest model;

[0124] Days 11-15: Optimize model parameters using 5-fold cross-validation;

[0125] Days 16-20: Evaluate the three courses using both systematic and traditional methods, and calculate the deviation rate by comparing the scores with expert scores.

[0126] Experiment 3: Validation of the effectiveness of closed-loop optimization

[0127] 1. Experimental Methods: The "Python Data Analysis in Practice" course implemented a system-generated optimization strategy (experimental group), the "Advanced Excel Applications" course adopted a traditional manual optimization strategy (control group), and the "SQL Database Development" course was not optimized (blank group). After 30 days, the changes in the comprehensive market influence index and core indicators of the three courses were compared.

[0128] 2. Experimental steps:

[0129] Day 1: Obtain the initial assessment index (T0) for the three courses;

[0130] Days 2-31: The experimental group implemented a system optimization strategy (channel adjustment + package optimization), the control group implemented manual optimization (increasing the number of exercises), and the blank group maintained the status quo;

[0131] Day 32: Reassess the overall index (T1) of the three courses and calculate the improvement ((T1-T0) / T0×100%).

[0132] (iv) Experimental Results and Analysis

[0133] Experiment 1: Results of Data Acquisition Integrity

[0134]

[0135] Analysis: The system achieved full coverage of 39 planning dimensions through multi-source interfaces and hierarchical collection mechanisms, with an average daily data collection volume of 109,900 records, which is 12.7 times higher than traditional manual collection (8,000 records per day), and the data integrity meets the requirements of subsequent evaluation.

[0136] Experiment 2: Results of evaluating model accuracy

[0137]

[0138] Analysis: The average bias rate of this system is only 22.5% of that of traditional methods. The MOOC-BERT model achieved an accuracy of 86.2% in classifying public opinion, while the improved random forest model maintained an MSE below 4.3. Furthermore, the evaluation time was reduced from 8 hours to 15 minutes, validating the model's accuracy and efficiency. This result is attributed to the combined weighting mechanism of the analytic hierarchy process (AHP) and entropy weighting, as well as the BERT model optimized for educational texts, which significantly improves accuracy compared to single statistical methods.

[0139] Experiment 3: Results of the effectiveness of closed-loop optimization

[0140]

[0141] Analysis: The experimental group's overall market influence index increased by 11.6%, which was 2.97 times that of the control group (3.9%). The increases in exposure and enterprise purchasing volume were particularly significant, confirming the effectiveness of the system-generated targeted optimization strategy. This result is closely related to the scientific design of the A / B test. Validation with a 15% sample size (compliant with OpenedX platform experimental design standards) ensured the statistical significance of the strategy's effect (p<0.05), significantly improving accuracy compared to traditional manual experience-based optimization.

[0142] (V) Experimental Conclusions

[0143] 1. This system achieves 100% coverage collection of multi-source data, and the data scale and integrity far exceed those of traditional methods, laying a data foundation for accurate assessment;

[0144] 2. The average deviation rate of the evaluation model is only 4.2%, balancing accuracy and efficiency, and solving the problems of strong subjectivity and time-consuming traditional evaluation.

[0145] 3. The closed-loop optimization mechanism increased the course market influence index by 11.6%, and the core business indicators improved significantly. It achieved a virtuous cycle of "evaluation-optimization-verification", which is more scientific and practical than manual optimization.

[0146] This is a preferred embodiment of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A market influence course effectiveness evaluation and optimization system based on big data, characterized in that, It includes a hardware layer and a functional layer, which communicate bidirectionally with the functional layer via a TCP / IP protocol data bus; the hardware layer includes: Cloud server: configured with ≥256GB of memory, ≥16 cores of CPU and ≥10TB of distributed storage, used to store historical course data for the past 5 years (including course videos, exercises, student files) and industry market benchmark database (including annual market size, growth rate and data of leading institutions). Edge computing nodes: Deployed on local servers in the course platform's computer room, configured with ≥32GB of memory and ≥8 CPU cores, used to process high-frequency collected data of ≥1000 records per second in real time, with latency controlled within 50ms; Multi-source data interfaces: including RESTful API interfaces, direct database connection interfaces (supporting MySQL and MongoDB), and third-party platform SDK interfaces. The interfaces are configured with API adaptation modules (which can automatically adapt to different platform data formats) and permission verification units (based on the OAuth2.0 protocol to implement interface access authentication). The functional layer includes a data acquisition module, a data preprocessing module, a market influence assessment module, and a closed-loop optimization module that are connected in sequence. Each module is deployed in a containerized manner on cloud servers and edge computing nodes to achieve load balancing. The data acquisition module synchronously acquires course lifecycle data (including development, launch, and iteration phases) and external market data through multi-source data interfaces. The data preprocessing module cleans, fuses, and performs feature engineering on the collected data, outputting a standardized feature dataset. The market influence assessment module, based on a preset multi-dimensional indicator system and a trained machine learning model, performs quantitative assessment of a standardized feature dataset and generates a market influence assessment report. The closed-loop optimization module automatically generates course optimization strategies based on the deviation indicators in the evaluation report and pushes them to the course management terminal (PC / mobile) via HTTPS protocol.

2. The system according to claim 1, characterized in that, The data acquisition module includes an internal data acquisition unit and an external market data acquisition unit, and is configured with a data compression subunit (using the LZ77 algorithm for a compression rate of ≥60%). The internal data acquisition unit obtains the following data through the course platform interface: Learning behavior data includes: cumulative online time (accurate to the second), real-time interaction frequency (including the number of bullet comments sent, the number of questions asked, and the number of posts in the discussion area, with a statistical period of hourly), chapter completion rate (calculated as: number of completed chapters / total number of chapters × 100%), and active feedback error rate ratio (calculated as: number of errors actively marked by students / total learning time × 100%). Learning outcome data includes: homework grades (recorded on a percentage basis, rounded to one decimal place), exam pass rate (calculated as: number of students who passed the exam / number of students who took the exam × 100%), and skills certification pass rate (calculated as: number of students who obtained certification / number of students who registered for certification × 100%). Course operation data includes: number of students enrolled (differentiating between new and returning students), dropout rate (calculated as: number of dropouts / total number of enrollees × 100%), and repurchase rate (calculated as: number of students who purchased the same course series ≥ 2 times / total number of purchasers × 100%). The external market data acquisition unit obtains the following data through a multi-source data interface: Industry public opinion data includes social media comments (from Weibo, Xiaohongshu, and Zhihu, including text content, posting time, and number of likes) and education platform ratings; Competitive benchmarking data includes: pricing of similar courses (differentiating between basic / advanced / flagship classes), update frequency (number of updated class hours per month), and market share (calculated as: number of users of competitor courses / total number of users in the industry × 100%). Business conversion data includes: salary increase for students upon employment (calculated as: first month's salary after employment - pre-enrollment salary / pre-enrollment salary × 100%) and enterprise cooperation renewal rate (calculated as: number of renewing enterprises / total number of cooperating enterprises × 100%). Policy environment data: including industry regulatory documents (from the official websites of the Ministry of Education and the State Administration for Market Regulation, including the release date and effective date) and support policies (including subsidy amounts and applicable conditions).

3. The system according to claim 1, characterized in that, The data preprocessing module includes: Data cleaning unit: Uses Z-score method (threshold set to |Z|>3) to identify and remove outliers; uses KNN algorithm (K value set to 5) to fill missing values, using weighted average for continuous data and mode for discrete data; desensitizes public opinion text (replaces mobile phone numbers and email addresses with "", and blurs names to "Li" and "Zhang"); Data fusion unit: Based on the course's unique ID (consisting of 8 digits + 2 letters) and timestamp (accurate to milliseconds), an associated index is built, and internal and external data are spatiotemporally aligned on a weekly basis (e.g., the learning behavior data of week 1 is associated with the public opinion data of the same period), generating a unified data view containing 23 data fields; Feature engineering unit: For structured data (such as scores and duration), min-max normalization is applied (mapped to the [0,1] interval); for unstructured text data (such as comments), the BERT-base-chinese model is used for word segmentation (word-level granularity) and semantic embedding (generating a 768-dimensional vector). The Top 20 keyword features are extracted using the TextRank algorithm, and sentiment tendency features (positive / negative / neutral, quantized as scores in the [-1,1] interval) are extracted using the VADER sentiment analysis tool; finally, a 128-dimensional standardized feature vector is generated (containing 64-dimensional structured features + 64-dimensional unstructured features).

4. The system according to claim 1, characterized in that, The market influence assessment module includes an indicator system configuration unit, a model calculation unit, and a result output unit. The indicator system configuration unit presets three levels of evaluation indicators: Primary indicator: Market influence comprehensive index (out of 100 points); Secondary indicators: Market penetration (weight 35%), word-of-mouth marketing (weight 30%), and commercial conversion rate (weight 35%). The third-level indicators include 12 specific indicators such as course exposure (daily average UV), student referral rate (calculated as: number of active referrals / total number of students × 100%), and enterprise purchase volume (annual purchase of course hours). The weight of each indicator is determined by a combination of the analytic hierarchy process (inviting 5-7 industry experts to score) and the entropy weight method (calculating information entropy based on data from the past 3 years) (combined weight = 0.6 × expert weight + 0.4 × entropy weight). The model computation unit adopts a dual-model architecture: The first model is a public opinion sentiment classification model based on MOOC-BERT (the input is a 768-dimensional text embedding vector, the output is a word-of-mouth communication power score, and the accuracy is ≥85%). The second model is an improved random forest regression model (containing 500 decision trees, with a maximum depth of 15, the input being the remaining 112-dimensional feature vectors, and the output being market penetration and commercial conversion scores, with a mean squared error MSE ≤ 5). The final score is obtained by weighted summation (overall index = penetration × 35% + reputation × 30% + conversion rate × 35%). The result output unit generates a visual evaluation report, including: a radar chart of scores for each indicator, a bar chart of deviation from the industry average (deviation rate = (score of this course - industry average) / industry average × 100%), and a list of the top 3 strength indicators and the top 3 weaknesses indicators (weak indicators are defined as indicators with scores < 80% of the industry average).

5. The system according to claim 1, characterized in that, The closed-loop optimization module includes a strategy generation unit, a strategy verification unit, and a dynamic adjustment unit. The strategy generation unit generates targeted optimization strategies for weak indicators: If market penetration is insufficient (score < 60 points), generate a promotion channel adjustment plan (including ROI analysis of each channel, recommendations for new channels and budget allocation suggestions, such as reducing the budget of inefficient channels by 20% and transferring it to short video platforms). If the word-of-mouth effect is low (score <60 points), generate content iteration suggestions (including a case update list, requiring the addition of ≥3 new industry cases per month); Interactive design optimization solutions, such as adding an instant reward mechanism for chapter quizzes); If the business conversion rate is weak (score <60 points), generate a pricing and service package optimization plan (such as launching a "basic class + 1-on-1 consultation" package with a premium of 15%-20%). The strategy verification unit adopts an A / B testing mechanism: 10%-20% of the user group is randomly selected as the experimental group (experience optimization strategy), and the rest are used as the control group. The verification period is set according to the strategy type (7 days for promotion channels and 30 days for content iteration). The evaluation index is the difference in index scores between the experimental group and the control group (the significance standard for difference is p<0.05). The dynamic adjustment unit updates the strategy parameters based on the verification results (e.g., increasing the channel budget by 10% for channels with a conversion rate increase of ≥5%), and feeds back the optimization effect (e.g., changes in indicator scores) to the market influence assessment module, triggering the next round of assessment, thus achieving a closed-loop iteration at least once per quarter.

6. The system according to claim 4, characterized in that, The model computation unit also includes a model update unit: each quarter, based on newly added course data (≥100,000 records) and market data (≥50,000 records), the improved random forest regression model is retrained using the sliding window method (window size is 12 months); an early stop mechanism is set during training (training stops when the validation set MSE increases for 5 consecutive rounds); seamless switching between old and new models is achieved through model version management tools (such as MLflow), and the system can still provide evaluation services based on the old model during the update process, with a switching interruption time of ≤10 seconds.

7. The system according to claim 2, characterized in that, The external market data collection unit is equipped with a real-time monitoring subunit: industry public opinion data is collected at a high frequency of 5 minutes (the platform's new content is scanned once every 5 minutes), competitor benchmarking data is collected at a daily level (fully updated at 3:00 AM every day), and commercial conversion data is collected at a weekly level (the data for the week is summarized at 24:00 every Sunday). The collection frequency can be manually adjusted through the management terminal (customized periods of 1 minute to 30 days are supported), and the adjustment will take effect within 30 seconds.

8. The system according to claim 1, characterized in that, It also includes a user interaction module: providing a web-based visual interface, including: Customizable evaluation metrics interface: Allows course operators to add evaluation dimensions (such as "geographical distribution of students") and set custom weights via sliders (the total weight is forced to 100%). Optimization strategy manual intervention entry: Users can edit system-generated strategies (such as modifying budget allocation ratios), pause or terminate execution, and record the reasons for intervention; Historical data backtracking function: Supports querying evaluation reports, optimization strategies and execution results at any point in time in the past 3 years, and provides a data export interface.

Images