A Machine Learning-Based AI Collaborative Scoring Method

Abstract

This invention discloses an AI collaborative scoring method based on machine learning, comprising the following steps: Step 1: collecting raw scoring data from a generative search platform; Step 2: standardizing the raw scoring data; Step 3: generating a question structure decoupling vector using an improved SimCSE model; Step 4: converting the question structure decoupling vector into AI-generated content and extracting content cognitive feature vectors; Step 5: generating structural clustering center vectors and question weight coefficients based on the question structure decoupling vectors; Step 6: using an attention scoring model to obtain the content score of the AI-generated content; Step 7: performing usability determination and content recommendation analysis based on the content score. This invention improves the accuracy and interpretability of collaborative scoring through the improved SimCSE model and attention scoring model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and generative search optimization technology, and particularly to an AI collaborative scoring method based on machine learning. Background Technology

[0002] With the widespread application of generative artificial intelligence technology, conversational information retrieval via natural language and AI search engines has become the mainstream interaction method. In generative search platforms, key technical issues for improving search experience and commercial conversion efficiency include how to structurally model user queries, accurately assess the usability and quality of AI-generated content, and optimize keywords and recommend content. Existing AI content scoring methods generally suffer from the following shortcomings:

[0003] Scoring systems often rely on preset rules or static templates, making it difficult to adapt to the complex and ever-changing semantics and structural intent of user questions. They also lack the ability to jointly model user behavior patterns and product information. Mainstream text similarity algorithms, such as the standard SimCSE and BERT, lack product semantic bias and keyword guidance mechanisms when modeling the structure of user questions. They cannot distinguish the distribution structure of questions under different intents, such as category selection and brand comparison, resulting in fuzzy structural classification or unstable optimization word extraction. The scoring process lacks a multi-dimensional collaborative mechanism and fails to fully integrate structural features, content cognitive features, and question clustering features, making it difficult to accurately output interpretable quality scores, which in turn affects the credibility of content recommendations and the efficiency of commercial reach.

[0004] Therefore, how to provide an AI collaborative scoring method based on machine learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an AI collaborative scoring method based on machine learning. This invention fully integrates structure-supervised contrastive learning and attention-based scoring mechanisms, and details the process from user question modeling to AI content scoring in a generative search platform. It uses an improved SimCSE model to structurally model user questions, combines a large language model to construct AI-generated content drafts and extract multi-dimensional content cognitive features, and constructs a two-dimensional question structure space and a structure-guided attention scoring model to score content quality and determine usability. This invention possesses advantages such as strong structural interpretability, high scoring accuracy, and excellent recommendation decision-making performance.

[0006] According to an embodiment of the present invention, an AI collaborative scoring method based on machine learning includes the following steps:

[0007] Step 1: Collect raw rating data from the generative search platform;

[0008] Step 2: Standardize the raw rating data to generate a standard rating dataset;

[0009] Step 3: Input the standard scoring dataset into the improved SimCSE model, perform problem structure embedding decoupling modeling, and output the problem structure decoupling vector; the improved SimCSE model includes a semantic encoder module, a keyword guidance module, a product information bias injection module, a structure-supervised contrastive learning module, and a dual-channel decoupling module;

[0010] Step 4: Load the pre-trained large language model, transform the problem structure decoupling vector into AI-generated content, and extract the content cognitive feature vector;

[0011] Step 5: Based on the problem structure decoupling vector, generate the structure clustering center vector and problem weight coefficients by constructing a two-dimensional problem structure space;

[0012] Step 6: Input the content cognitive feature vector, structural cluster center vector and question weight coefficient into the attention scoring model to score the content quality of the AI-generated content and obtain the content score value;

[0013] Step 7: Based on the content rating, perform usability assessment and content recommendation analysis on the AI-generated content.

[0014] Optionally, the raw rating data includes user natural language question text, brand product identification information, product information, and user search behavior data.

[0015] Optionally, step two specifically includes:

[0016] The standardization process includes text cleaning, word segmentation, and keyword extraction.

[0017] The user search behavior data is sorted in chronological order to form the user search behavior text;

[0018] The text cleaning process specifically involves removing non-semantic symbols from the user's natural language question text, brand product identification information, product information, and user search behavior text to obtain user question text sequences, product identification text sequences, product information text sequences, and user search behavior text sequences.

[0019] The word segmentation process specifically involves: dividing the user question text sequence, product identifier text sequence, product information text sequence, and user search behavior text sequence into word units to obtain user question word unit sequences, product identifier word unit sequences, product information word unit sequences, and user search behavior word unit sequences, respectively.

[0020] The keyword extraction specifically involves: extracting target keywords that are semantically related to brand product identification information and product information based on the user's question word sequence, thereby obtaining a set of target keywords;

[0021] The standard scoring dataset is composed of user question word sequences, product identifier word sequences, product information word sequences, and user search behavior word sequences, along with the target keyword set.

[0022] Optionally, step three specifically includes:

[0023] The semantic encoder module uses a pre-trained RoBERTa model to embed and map the user question word sequence, product identifier word sequence, product information word sequence, and user search behavior word sequence with the target keyword set, and performs average pooling in the word dimension to obtain the user question semantic feature vector, product identifier feature vector, product information feature vector, user behavior feature vector, and target keyword feature vector, respectively.

[0024] The product identification feature vector and the product information feature vector are concatenated to obtain the product information feature vector.

[0025] In the keyword guidance module, the vector difference between the semantic feature vector of the user's question and the feature vector of the target keyword is calculated to obtain the keyword difference vector;

[0026] The keyword difference vector is normalized to obtain the standard keyword difference vector.

[0027] Set the perturbation intensity hyperparameter, and multiply the perturbation intensity hyperparameter with the standard keyword difference vector to obtain the keyword perturbation term;

[0028] The enhanced user question feature vector is obtained by adding the semantic feature vector of the user question to the keyword perturbation term;

[0029] The product information bias injection module modulates the enhanced user question feature vector by constructing a product bias modulation vector and performing element-wise multiplication modulation on the product bias modulation vector to obtain the bias modulation user question vector.

[0030] The structure-supervised contrastive learning module constructs a structure similarity matrix and introduces a structure space separation mechanism to perform supervised contrastive training on the bias-modulated user question vector to obtain the user question structure feature vector.

[0031] The dual-channel decoupling module includes an intent recognition channel and a user type recognition channel, which respectively perform feature mapping on the user question structure feature vector to obtain the intent embedding vector and the user type embedding vector; wherein, the intent recognition channel and the user type recognition channel are each composed of two fully connected layers;

[0032] The intent embedding vector and the user type embedding vector are concatenated to obtain the problem structure decoupling vector.

[0033] Optionally, the construction process of the product bias modulation vector specifically includes:

[0034] Set the number of semantic bias bases K, and initialize K bias base mapping weight matrices; the bias base mapping weight matrices are updated through backpropagation;

[0035] Each bias basis mapping weight matrix is ​​multiplied by the product information feature vector to obtain the product semantic bias basis vector.

[0036] The bias response weights are obtained by calculating the matching relationship between the enhanced user question feature vector and the semantic bias basis vector of each product through the Softmax function.

[0037] Multiply the bias response weights by the corresponding product semantic bias basis vectors to obtain the bias basis component vectors. Accumulate K bias basis component vectors to obtain the product semantic bias field vector.

[0038] The product bias modulation vector is obtained by calculating the vector sum of the unit vector and the product semantic bias field vector.

[0039] Optionally, the structure-supervised contrastive learning module constructs a structure similarity matrix and introduces a structure space separation mechanism to perform supervised contrastive training on the bias-modulated user question vector to obtain the user question structure feature vector, specifically including:

[0040] The current training batch is set to contain N bias-modulated user question vectors, and each bias-modulated user question vector corresponds to a structural label. The structural label is obtained by mapping the user search behavior text according to the structural classification rules.

[0041] Among them, the structural classification rules represent a two-dimensional structural tag system based on the user decision-making stage and the semantic intent of the question. The user decision-making stage includes the traffic acquisition stage, the opportunity stage, and the core conversion stage. The semantic intent of the question includes the category selection intent, the category demand intent, and the brand comparison intent.

[0042] Each structural label is transformed into a structural label semantic vector through one-hot encoding and linear mapping;

[0043] Calculate the cosine similarity between the semantic vector of the i-th structural label and the semantic vector of the j-th structural label to obtain the structural semantic similarity;

[0044] Structural semantic similarity is used as the weight factor for the contrastive loss, and the structural contrastive loss is constructed using the InfoNCE contrastive loss formula;

[0045] Construct a set of structure label center vectors, where each structure label center vector is obtained by averaging the user question vectors with the same bias of the structure label;

[0046] Calculate the Euclidean distance between each bias-modulated user question vector and the corresponding structural label center vector to obtain the structural aggregation distance value, and sum all structural aggregation distance values ​​to obtain the sample aggregation constraint term;

[0047] Calculate the Euclidean distance between any two structural label center vectors in the set of structural label center vectors to obtain the structural center spacing value, and sum all structural center spacing values ​​to obtain the structural center separation constraint term;

[0048] Set aggregation weight coefficient and separation weight coefficient, where the aggregation weight coefficient is a positive real number and the separation weight coefficient is a positive real number;

[0049] The aggregation weight coefficients are multiplied and modulated with the sample aggregation constraint term to obtain the aggregation loss term, and the separation weight coefficients are multiplied and modulated with the structure center separation constraint term to obtain the separation loss term.

[0050] The structure-aware total loss function is obtained by adding the structure contrast loss, aggregation loss term, and separation loss term.

[0051] By minimizing the total structure-aware loss function, the optimized bias-modulated user question vector is used as the user question structure feature vector.

[0052] Optionally, step four specifically includes:

[0053] The problem structure decoupling vector is concatenated with the corresponding product information feature vector and target keyword feature vector to construct the content and generate the input vector.

[0054] Load a pre-trained large language model to transform the content generation input vector into an AI-generated content manuscript, which includes a title paragraph, a body paragraph, and a link paragraph;

[0055] The AI-generated content is structurally analyzed to obtain features such as structural integrity, keyword density, conversion link layout, content quality, and product information consistency. These features are then combined to form a content cognition feature vector.

[0056] Optionally, step five specifically includes:

[0057] Construct a two-dimensional problem structure space, where the (m, n)th structural unit of the two-dimensional problem structure space corresponds to the structural label of the mth user decision stage and the nth problem semantic intent;

[0058] The problem structure decoupling vector is assigned to the corresponding structural unit according to the structure label;

[0059] All problem structure decoupling vectors in the same structural unit are used to form a problem set, and the number of problems in each structural unit is recorded.

[0060] Calculate the structural cluster center vector for each structural unit, where the structural cluster center vector represents the vector mean of all problem structural decoupling vectors within the structural unit;

[0061] Calculate the ratio of the number of problems in each structural unit to the total number of problems to obtain the problem weight coefficient for each structural unit.

[0062] Optionally, step six specifically includes:

[0063] The attention scoring model includes a structure-guided attention module and a main scoring network;

[0064] The structure-guided attention module is used to generate structure-modulated content feature vectors, specifically:

[0065] The structural cluster center vectors are mapped to structural query vectors through a trainable query mapping matrix; the content cognitive feature vectors are mapped to content key vectors and content value vectors through trainable key mapping matrices and value mapping matrices, respectively.

[0066] The structure query vector and the content key vector are multiplied by a dot product and then normalized using Softmax to generate structure-guided attention weights.

[0067] The problem weight coefficient is multiplied and modulated with the structure-guided attention weight to generate the structure-modulated attention weight;

[0068] Based on the structure modulation attention weight, the content value vector is multiplied element by element to obtain the structure modulation content feature vector;

[0069] The structure-modulated content feature vector is used to generate a scoring master vector through a scoring master network. The scoring master vector has a dimension of 5 and represents the structural integrity score, keyword density score, conversion link layout score, content quality score, and product information consistency score, respectively. The scoring master network consists of two fully connected layers.

[0070] Define a weighted scoring vector and then weight and fuse the weighted scoring vector with the main scoring vector to obtain the content score.

[0071] Optionally, step seven specifically includes:

[0072] The availability determination is specifically as follows:

[0073] Set a pass / fail threshold and a quality threshold. If the content score is less than the pass / fail threshold, the AI-generated content will be judged as unqualified and an unusable tag will be generated.

[0074] If the content score is greater than or equal to the qualified threshold and less than the excellent threshold, the AI-generated content will be judged as qualified content and usable tags will be generated.

[0075] If the content score is greater than or equal to the quality threshold, the AI-generated content will be judged as high-quality content and a high-quality tag will be generated.

[0076] For AI-generated content with unavailable tags, rewriting is required.

[0077] For AI-generated content articles with high-quality tags, content recommendation analysis is performed, specifically as follows:

[0078] Set recommendation thresholds and conversion thresholds;

[0079] If the keyword density score is greater than the recommendation threshold, the AI-generated content will be recommended and displayed on the homepage of the generative search platform.

[0080] If the user decision-making stage is the core conversion stage, and the conversion link layout score is greater than the conversion threshold, then AI-generated content will be recommended as the priority content for conversion scenarios in the generative search platform.

[0081] Conversion scenarios include displaying purchase links, product page redirects, or registration prompts to target users.

[0082] The beneficial effects of this invention are:

[0083] This invention employs an improved SimCSE model for decoupled modeling of user question structure embedding, significantly enhancing the stability and distinguishability of user question structure recognition. The improved SimCSE model's dual-channel decoupling module separates intent embedding and user type embedding, preventing semantic overlap at different user decision-making stages and improving the generalization ability of question structure vectors in generative search scenarios. In the content generation stage, this invention jointly models the question structure feature vector with product information feature vectors and target keyword feature vectors, inputting this model into a pre-trained large language model to generate AI-generated content with controllable structure. This effectively reduces the risk of generated content deviating from the user's true search intent or exhibiting product information distortion. Simultaneously, a structure parsing mechanism extracts multi-dimensional content cognitive features such as structural integrity, keyword density, conversion link layout, content quality, and consistency with product information. In the scoring and decision-making stages, this invention constructs a two-dimensional question structure space, performs structural clustering of user questions, and generates structural cluster center vectors and question weight coefficients. It also introduces structure-guided attention into the attention scoring model, subjecting the content scoring process to the joint constraints of the content's inherent quality characteristics and the question structure distribution characteristics. This avoids the problem of traditional scoring methods relying solely on text quality while ignoring differences in user search stages. In the result output stage, this invention completes usability determination and content recommendation analysis based on content scores, and executes differentiated recommendation strategies according to different user decision-making stages and conversion scenarios, ensuring that content delivery meets usability, security, and commercial conversion requirements. This invention effectively improves the accuracy, structural consistency, and recommendation credibility of AI content quality assessment in generative search platforms, reduces content unavailability and manual review costs, and achieves interpretable, scalable, and sustainably optimizable AI collaborative scoring capabilities for GEO scenarios. Attached Figure Description

[0084] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0085] Figure 1 This is a schematic diagram of an AI collaborative scoring method based on machine learning proposed in this invention;

[0086] Figure 2 This is a flowchart illustrating the structure of the structure-supervised contrastive learning module in an AI collaborative scoring method based on machine learning proposed in this invention.

[0087] Figure 3 This is a flowchart of the attention scoring model structure in an AI collaborative scoring method based on machine learning proposed in this invention. Detailed Implementation

[0088] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0089] refer to Figures 1-3 A machine learning-based AI collaborative scoring method includes the following steps:

[0090] Step 1: Collect raw rating data from the generative search platform;

[0091] Step 2: Standardize the raw rating data to generate a standard rating dataset;

[0092] Step 3: Input the standard scoring dataset into the improved SimCSE model, perform problem structure embedding decoupling modeling, and output the problem structure decoupling vector; the improved SimCSE model includes a semantic encoder module, a keyword guidance module, a product information bias injection module, a structure-supervised contrastive learning module, and a dual-channel decoupling module;

[0093] Step 4: Load the pre-trained large language model, transform the problem structure decoupling vector into AI-generated content, and extract the content cognitive feature vector;

[0094] Step 5: Based on the problem structure decoupling vector, generate the structure clustering center vector and problem weight coefficients by constructing a two-dimensional problem structure space;

[0095] Step 6: Input the content cognitive feature vector, structural cluster center vector and question weight coefficient into the attention scoring model to score the content quality of the AI-generated content and obtain the content score value;

[0096] Step 7: Based on the content rating, perform usability assessment and content recommendation analysis on the AI-generated content.

[0097] In this embodiment, the raw rating data includes user natural language question text, brand product identification information, product information, and user search behavior data.

[0098] For example, in a generative search platform, a user's query "Which Haier double-door refrigerator is the quietest?" is the user's natural language question text; "Haier" is the brand product identification information, and the corresponding product information includes structured data such as model parameters, function descriptions, energy efficiency ratings, and user reviews; if the user has previously searched for terms such as "refrigerator recommendations," "kitchen appliance comparisons," and "refrigerator size standards" and browsed related pages, then these search terms and behavioral records constitute the user's search behavior data.

[0099] In this embodiment, step two specifically includes:

[0100] Standardization processes include text cleaning, word segmentation, and keyword extraction;

[0101] The user search behavior data is sorted in chronological order to form the user search behavior text;

[0102] Text cleaning specifically involves removing non-semantic symbols from the user's natural language question text, brand and product identification information, product information, and user search behavior text to obtain user question text sequences, product identification text sequences, product information text sequences, and user search behavior text sequences.

[0103] The word segmentation process specifically involves dividing the user question text sequence, product identifier text sequence, product information text sequence, and user search behavior text sequence into word units to obtain user question word unit sequences, product identifier word unit sequences, product information word unit sequences, and user search behavior word unit sequences.

[0104] Keyword extraction specifically involves: extracting target keywords that are semantically related to brand product identification information and product information based on the user's question word sequence, thereby obtaining a set of target keywords;

[0105] The standard scoring dataset is composed of user question word sequences, product identifier word sequences, product information word sequences, and user search behavior word sequences, along with the target keyword set.

[0106] In this embodiment, step three specifically includes:

[0107] The semantic encoder module uses a pre-trained RoBERTa model to embed and map the user question word sequence, product identifier word sequence, product information word sequence, and user search behavior word sequence to the target keyword set, and performs average pooling in the word dimension to obtain the user question semantic feature vector, product identifier feature vector, product information feature vector, user behavior feature vector, and target keyword feature vector, respectively.

[0108] The product identification feature vector and the product information feature vector are concatenated to obtain the product information feature vector.

[0109] In the keyword guidance module, the vector difference between the semantic feature vector of the user's question and the feature vector of the target keyword is calculated to obtain the keyword difference vector;

[0110] The keyword difference vector is normalized to obtain the standard keyword difference vector.

[0111] Set the perturbation intensity hyperparameter, and multiply the perturbation intensity hyperparameter with the standard keyword difference vector to obtain the keyword perturbation term;

[0112] The enhanced user question feature vector is obtained by adding the semantic feature vector of the user question to the keyword perturbation term;

[0113] The product information bias injection module modulates the enhanced user question feature vector by constructing a product bias modulation vector and performing element-wise multiplication modulation on the product bias modulation vector to obtain the bias modulation user question vector.

[0114] The structure-supervised contrastive learning module constructs a structure similarity matrix and introduces a structure space separation mechanism to perform supervised contrastive training on the bias-modulated user question vector to obtain the user question structure feature vector.

[0115] The dual-channel decoupling module includes an intent recognition channel and a user type recognition channel, which respectively perform feature mapping on the user question structure feature vector to obtain the intent embedding vector and the user type embedding vector; wherein, the intent recognition channel and the user type recognition channel are each composed of two fully connected layers;

[0116] The intent embedding vector and the user type embedding vector are concatenated to obtain the problem structure decoupling vector.

[0117] In this embodiment, the process of constructing the product bias modulation vector specifically includes:

[0118] Set the number of semantic bias bases K, and initialize K bias base mapping weight matrices; wherein, the bias base mapping weight matrices are updated through backpropagation;

[0119] Each bias basis mapping weight matrix is ​​multiplied by the product information feature vector to obtain the product semantic bias basis vector.

[0120] The bias response weights are obtained by calculating the matching relationship between the enhanced user question feature vector and the semantic bias basis vector of each product through the Softmax function.

[0121] Multiply the bias response weights by the corresponding product semantic bias basis vectors to obtain the bias basis component vectors. Accumulate K bias basis component vectors to obtain the product semantic bias field vector.

[0122] The product bias modulation vector is obtained by calculating the vector sum of the unit vector and the product semantic bias field vector.

[0123] In this embodiment, the structure-supervised contrastive learning module constructs a structure similarity matrix and introduces a structure space separation mechanism to perform supervised contrastive training on the bias-modulated user question vector to obtain the user question structure feature vector, specifically including:

[0124] The current training batch is set to contain N bias-modulated user question vectors, and each bias-modulated user question vector corresponds to a structural label. The structural label is obtained by mapping the user search behavior text according to the structural classification rules.

[0125] The structural classification rules represent a two-dimensional structural tag system based on the user's decision-making stage and the semantic intent of the question. The user's decision-making stage includes the traffic acquisition stage, the opportunity stage, and the core conversion stage. The semantic intent of the question includes the category selection intent, the category demand intent, and the brand comparison intent. The two-dimensional structural tag system is as follows: Traffic acquisition stage + category selection intent: structural tag 1; Traffic acquisition stage + category demand intent: structural tag 2; Traffic acquisition stage + brand comparison intent: structural tag 3; Opportunity stage + category selection intent: structural tag 4; Opportunity stage + category demand intent: structural tag 5; Opportunity stage + brand comparison intent: structural tag 6; Core conversion stage + category selection intent: structural tag 7; Core conversion stage + category demand intent: structural tag 8; Core conversion stage + brand comparison intent: structural tag 9.

[0126] Each structural label is transformed into a structural label semantic vector through one-hot encoding and linear mapping;

[0127] Calculate the cosine similarity between the semantic vector of the i-th structural label and the semantic vector of the j-th structural label to obtain the structural semantic similarity;

[0128] Structural semantic similarity is used as the weight factor for the contrastive loss, and the structural contrastive loss is constructed using the InfoNCE contrastive loss formula;

[0129] Construct a set of structure label center vectors, where each structure label center vector is obtained by averaging the user question vectors with the same bias of the structure label;

[0130] Calculate the Euclidean distance between each bias-modulated user question vector and the corresponding structural label center vector to obtain the structural aggregation distance value, and sum all structural aggregation distance values ​​to obtain the sample aggregation constraint term;

[0131] Calculate the Euclidean distance between any two structural label center vectors in the set of structural label center vectors to obtain the structural center spacing value, and sum all structural center spacing values ​​to obtain the structural center separation constraint term;

[0132] Set aggregation weight coefficient and separation weight coefficient, where the aggregation weight coefficient is a positive real number and the separation weight coefficient is a positive real number;

[0133] The aggregation weight coefficients are multiplied and modulated with the sample aggregation constraint term to obtain the aggregation loss term, and the separation weight coefficients are multiplied and modulated with the structure center separation constraint term to obtain the separation loss term.

[0134] The structure-aware total loss function is obtained by adding the structure contrast loss, aggregation loss term, and separation loss term.

[0135] By minimizing the total structure-aware loss function, the optimized bias-modulated user question vector is used as the user question structure feature vector.

[0136] In this invention, the improved SimCSE model has the same structure as the original SimCSE model. The original SimCSE model uses a semantic encoder as its core structure, including a pre-trained language model and vector pooling operations. It uses the pre-trained model to embed and map the input text, and performs average pooling or CLS pooling operations at the word level to generate sentence-level semantic feature vectors. During the training phase, the original SimCSE model constructs positive and negative sample pairs based on a contrastive learning mechanism, and constrains the semantic representation through the InfoNCE contrastive loss function, ensuring that semantically similar texts maintain high similarity in the vector space, while semantically unrelated texts maintain distance separation. This new model's overall structure centers on the semantic encoder module and the contrastive learning loss function, focusing on solving the problems of consistency and discriminability in text semantic representation. Its output is a semantic feature vector used to measure the semantic similarity relationship between texts.

[0137] In this invention, the improved SimCSE model, while maintaining the semantic encoder module and the contrastive learning training framework unchanged, introduces several structural enhancement modules specifically for generative search scenarios. Specifically, the improved SimCSE model adds a keyword guidance module, a product information bias injection module, and a dual-channel decoupling module after the semantic encoder module, and improves the structurally supervised contrastive learning module. The module comprises several key components: a keyword guidance module, a product information bias injection module, and a dual-channel decoupling module. The former maps the user question semantic feature vector to the target keyword feature vector and constructs a keyword perturbation term, enabling the model to explicitly perceive the guiding role of the target keyword in semantic modeling. The latter constructs a product bias modulation vector and modulates the enhanced user question feature vector element-wise, constraining the semantic embedding of the user question by the product information feature vector. The latter, based on the original unsupervised contrastive learning, introduces a structural similarity matrix, structural aggregation constraint term, and structural center separation constraint term based on a two-dimensional structural label system, enabling supervised contrastive training oriented towards the user decision-making stage and the semantic intent of the question. The former maps the user question structural feature vector to intent embedding vector and user type embedding vector, respectively, and modulates and concatenates them to form the final question structural feature vector, thereby completing the semantic decoupling modeling at the structural level.

[0138] Through the aforementioned structural improvements, the improved SimCSE model upgrades from the original SimCSE model's pure semantic similarity modeling to question structure-aware modeling. This not only characterizes the semantic consistency of user questions at the linguistic level but also simultaneously expresses structural attributes such as the user's decision-making stage, the question's semantic intent, and product information constraints. The keyword guidance module effectively reduces the risk of deviation between the semantic representation of the user question and the target keywords, improving the stability of keyword mining and optimized word recommendation. The product information bias injection module enables the generated semantic vectors to possess product consistency constraints during the embedding stage, reducing the probability of product information distortion during content generation. The structural supervised contrastive learning module, through structural aggregation and structural separation mechanisms, allows questions with different structural labels to form clear distribution boundaries in the vector space, significantly improving the distinguishability of question structures. The dual-channel decoupling module reduces the coupling interference between intent features and user type features, thereby improving the generalization ability and interpretability of question structure feature vectors in downstream tasks such as generative search content generation, content rating, and content recommendation.

[0139] In this embodiment, step four specifically includes:

[0140] The problem structure decoupling vector is concatenated with the corresponding product information feature vector and target keyword feature vector to construct the content and generate the input vector.

[0141] Load a pre-trained large language model to transform the content generation input vector into AI-generated content, which includes title paragraphs, body paragraphs, and link paragraphs.

[0142] The AI-generated content is structurally analyzed to obtain features such as structural integrity, keyword density, conversion link layout, content quality, and product information consistency. These features are then combined to form a content cognition feature vector.

[0143] The structural integrity feature is used to construct a structural identifier vector; the keyword density feature is used to characterize the ratio of the number of times each target keyword in the target keyword set appears in the AI-generated content to the total number of words in the content, forming a keyword frequency density vector; the conversion link layout feature is used to record the relative position index and frequency of the link segment in the body paragraph, forming a layout feature vector; the content quality feature is used to characterize the scores for language standardization, content authority, and content originality, forming a content quality feature vector; the product information consistency feature is used to characterize the semantic similarity between the AI-generated content and product information calculated through semantic matching. The presence of the title, body, and closing paragraphs is indicated by a value of 1 if present and 0 otherwise.

[0144] In this embodiment, step five specifically includes:

[0145] Construct a two-dimensional problem structure space, where the (m, n)th structural unit of the two-dimensional problem structure space corresponds to the structural label of the mth user decision stage and the nth problem semantic intent;

[0146] The problem structure decoupling vector is assigned to the corresponding structural unit according to the structure label;

[0147] All problem structure decoupling vectors in the same structural unit are used to form a problem set, and the number of problems in each structural unit is recorded.

[0148] Calculate the structural cluster center vector for each structural unit, where the structural cluster center vector represents the vector mean of all problem structural decoupling vectors within the structural unit;

[0149] Calculate the ratio of the number of problems in each structural unit to the total number of problems to obtain the problem weight coefficient for each structural unit.

[0150] In this embodiment, step six specifically includes:

[0151] The attention scoring model includes a structure-guided attention module and a scoring main network;

[0152] The structure-guided attention module is used to generate structure-modulated content feature vectors, specifically:

[0153] The structural cluster center vectors are mapped to structural query vectors through a trainable query mapping matrix; the content cognitive feature vectors are mapped to content key vectors and content value vectors through trainable key mapping matrices and value mapping matrices, respectively.

[0154] The structure query vector and the content key vector are multiplied by a dot product and then normalized using Softmax to generate structure-guided attention weights.

[0155] The problem weight coefficient is multiplied and modulated with the structure-guided attention weight to generate the structure-modulated attention weight;

[0156] Based on the structure modulation attention weight, the content value vector is multiplied element by element to obtain the structure modulation content feature vector;

[0157] The structured content feature vector is passed through a scoring master network to generate a scoring master vector. The scoring master vector has a dimension of 5 and represents the structural integrity score, keyword density score, conversion link layout score, content quality score, and product information consistency score, respectively. The scoring master network consists of two fully connected layers. The first fully connected layer maps the structured content feature vector into an intermediate feature vector through a linear transformation and the GELU activation function. The second fully connected layer maps the intermediate feature vector into the scoring master vector through a linear transformation.

[0158] Define a weighted scoring vector and then weight and fuse the weighted scoring vector with the main scoring vector to obtain the content score.

[0159] In this embodiment, step seven specifically includes:

[0160] Availability determination is specifically as follows:

[0161] Set a pass / fail threshold and a quality threshold. If the content score is less than the pass / fail threshold, the AI-generated content will be judged as unqualified and an unusable tag will be generated.

[0162] If the content score is greater than or equal to the qualified threshold and less than the excellent threshold, the AI-generated content will be judged as qualified content and usable tags will be generated.

[0163] If the content score is greater than or equal to the quality threshold, the AI-generated content will be judged as high-quality content and a high-quality tag will be generated.

[0164] For AI-generated content with unavailable tags, rewriting is required.

[0165] For AI-generated content articles with high-quality tags, content recommendation analysis is performed, specifically as follows:

[0166] Set recommendation thresholds and conversion thresholds;

[0167] If the keyword density score is greater than the recommendation threshold, the AI-generated content will be recommended and displayed on the homepage of the generative search platform.

[0168] If the user decision-making stage is the core conversion stage, and the conversion link layout score is greater than the conversion threshold, then AI-generated content will be recommended as the priority content for conversion scenarios in the generative search platform.

[0169] Conversion scenarios include displaying purchase links, product page redirects, or registration prompts to target users.

[0170] Example 1: To verify the feasibility of this invention in practice, the method was applied to the content scoring and recommendation system of a generative search platform for a home appliance brand. This platform offers generative dialogue search capabilities to all users. After a user inputs a natural language question, the system automatically calls upon a large AI model to generate the answer and displays it in the search results area. In actual operation, the platform's content operation team found that although AI can generate answer content on a large scale, the quality of the content is difficult to assess, especially in scenarios aimed at purchase conversion. Problems exist such as incomplete structure, insufficient keyword coverage, low link reach, poor content credibility, and incorrect product descriptions, affecting content usability, user satisfaction, and brand commercial value.

[0171] In the implementation of this invention, user natural language questions, product information, and user behavior sequences are collected from a search platform and standardized to generate a standard scoring dataset. The platform deploys an improved SimCSE model to generate question structure feature vectors based on the standard scoring dataset. During the content generation stage, the question structure feature vector is concatenated with product information feature vectors and target keyword feature vectors to form a content generation input vector, which is then input into a large language model to obtain structured AI-generated content. After structural analysis, content cognitive features are extracted and input along with the structure clustering center vector and question weight coefficients into an attention scoring model to obtain a content score. Based on the scoring results, the platform can classify the AI-generated content for usability and make recommendation decisions. For example, content with high structural integrity, good keyword coverage, prominent links, credible content, and consistency with product information is marked as "high-quality" and used for core page recommendations; while content with low scores triggers a rewriting task.

[0172] To further verify the effectiveness of the method of the present invention, the method of the present invention was compared with two comparative methods. Comparison method 1 was the original SimCSE model, and comparison method 2 was the BERT model and the classifier scoring mechanism. The comparison results are shown in Table 1.

[0173] Table 1. Performance Comparison of Different Scoring Methods in Generative Search Platforms

[0174] Comparison indicators Comparison Method 1 Comparison Method 2 Method of the present invention Content usability assessment accuracy (%) 83.5 86.9 93.7 The rate of AI content rewriting triggering decreased by (%) 14.6 22.3 38.4 Click-through rate increased by (%) for high-quality content recommendations 9.8 13.5 20.1 Product information misidentification rate (%) 70.3 73.2 88.1 Correlation coefficient of user behavior consistency score 0.68 0.72 0.87 Recommended content conversion rate (%) 6.1 6.7 9.4 Content ratings and human ratings RMSE 16.3 13.9 8.4

[0175] As shown in Table 1, the method of this invention significantly outperforms the comparative methods in multiple comparative indicators. Regarding the accuracy of content usability judgment, the method of this invention reaches 93.7%, significantly higher than 83.5% for comparative method 1 and 86.9% for comparative method 2, indicating a stronger ability to identify high-quality AI-generated content. In terms of reducing the AI ​​content rewriting trigger rate, this invention achieves a 38.4% reduction, effectively reducing the proportion of low-quality content being incorrectly delivered and optimizing content production efficiency. Regarding the improvement in click-through rate for high-quality content recommendations, this invention achieves a 20.1% increase, a significant improvement compared to the comparative methods, indicating that the scoring results of this invention can more accurately identify content attractive to users. In terms of product information error recognition rate, this invention achieves 88.1%, significantly higher than 70.3% for comparative method 1 and 73.2% for comparative method 2, indicating that this invention can effectively integrate product identification and product information during the user problem structure modeling stage, and has a stronger ability to identify product attribute errors, model errors, or false descriptions in AI-generated content.

[0176] Furthermore, the method of this invention achieved a correlation coefficient of 0.87 in user behavior consistency scoring, reflecting a higher consistency between the scoring results and users' actual click behavior. In terms of commercial conversion, this invention achieved a conversion rate of 9.4% for recommended content, significantly higher than the 6.1% and 6.7% of the comparative methods, demonstrating a stronger ability to drive commercial value. Regarding the RMSE of content scoring versus human scoring, the method of this invention scored only 8.4, far lower than the 16.3 and 13.9 of the comparative methods, indicating that the method of this invention performs more stably and with smaller errors in fitting human judgment standards, possessing higher scoring credibility and practicality.

[0177] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A machine learning-based AI collaborative scoring method, characterized in that, Includes the following steps: Step 1: Collect raw rating data from the generative search platform; Step 2: Standardize the raw rating data to generate a standard rating dataset; Step 3: Input the standard scoring dataset into the improved SimCSE model, perform problem structure embedding decoupling modeling, and output the problem structure decoupling vector; the improved SimCSE model includes a semantic encoder module, a keyword guidance module, a product information bias injection module, a structure-supervised contrastive learning module, and a dual-channel decoupling module; Step 4: Load the pre-trained large language model, transform the problem structure decoupling vector into AI-generated content, and extract the content cognitive feature vector; Step 5: Based on the problem structure decoupling vector, generate the structure clustering center vector and problem weight coefficients by constructing a two-dimensional problem structure space; Step 6: Input the content cognitive feature vector, structural cluster center vector and question weight coefficient into the attention scoring model to score the content quality of the AI-generated content and obtain the content score value; Step 7: Based on the content rating, perform usability assessment and content recommendation analysis on the AI-generated content.

2. The AI ​​collaborative scoring method based on machine learning according to claim 1, characterized in that, The raw rating data includes user natural language question text, brand product identification information, product information, and user search behavior data.

3. The AI ​​collaborative scoring method based on machine learning according to claim 1, characterized in that, Step two specifically includes: The standardization process includes text cleaning, word segmentation, and keyword extraction. The user search behavior data is sorted in chronological order to form the user search behavior text; The text cleaning process specifically involves removing non-semantic symbols from the user's natural language question text, brand product identification information, product information, and user search behavior text to obtain user question text sequences, product identification text sequences, product information text sequences, and user search behavior text sequences. The word segmentation process specifically involves: dividing the user question text sequence, product identifier text sequence, product information text sequence, and user search behavior text sequence into word units to obtain user question word unit sequences, product identifier word unit sequences, product information word unit sequences, and user search behavior word unit sequences, respectively. The keyword extraction specifically involves: extracting target keywords that are semantically related to brand product identification information and product information based on the user's question word sequence, thereby obtaining a set of target keywords; The standard scoring dataset is composed of user question word sequences, product identifier word sequences, product information word sequences, and user search behavior word sequences, along with the target keyword set.

4. The AI ​​collaborative scoring method based on machine learning according to claim 1, characterized in that, Step three specifically includes: The semantic encoder module uses a pre-trained RoBERTa model to embed and map the user question word sequence, product identifier word sequence, product information word sequence, and user search behavior word sequence with the target keyword set, and performs average pooling in the word dimension to obtain the user question semantic feature vector, product identifier feature vector, product information feature vector, user behavior feature vector, and target keyword feature vector, respectively. The product identification feature vector and the product information feature vector are concatenated to obtain the product information feature vector. In the keyword guidance module, the vector difference between the semantic feature vector of the user's question and the feature vector of the target keyword is calculated to obtain the keyword difference vector; The keyword difference vector is normalized to obtain the standard keyword difference vector. Set the perturbation intensity hyperparameter, and multiply the perturbation intensity hyperparameter with the standard keyword difference vector to obtain the keyword perturbation term; The enhanced user question feature vector is obtained by adding the semantic feature vector of the user question to the keyword perturbation term; The product information bias injection module modulates the enhanced user question feature vector by constructing a product bias modulation vector and performing element-wise multiplication modulation on the product bias modulation vector to obtain the bias modulation user question vector. The structure-supervised contrastive learning module constructs a structure similarity matrix and introduces a structure space separation mechanism to perform supervised contrastive training on the bias-modulated user question vector to obtain the user question structure feature vector. The dual-channel decoupling module includes an intent recognition channel and a user type recognition channel, which respectively perform feature mapping on the user question structure feature vector to obtain the intent embedding vector and the user type embedding vector; wherein, the intent recognition channel and the user type recognition channel are each composed of two fully connected layers; The intent embedding vector and the user type embedding vector are concatenated to obtain the problem structure decoupling vector.

5. The AI ​​collaborative scoring method based on machine learning according to claim 4, characterized in that, The construction process of the product bias modulation vector specifically includes: Set the number of semantic bias bases K, and initialize K bias base mapping weight matrices; the bias base mapping weight matrices are updated through backpropagation; Each bias basis mapping weight matrix is ​​multiplied by the product information feature vector to obtain the product semantic bias basis vector. The bias response weights are obtained by calculating the matching relationship between the enhanced user question feature vector and the semantic bias basis vector of each product through the Softmax function. Multiply the bias response weights by the corresponding product semantic bias basis vectors to obtain the bias basis component vectors. Accumulate K bias basis component vectors to obtain the product semantic bias field vector. The product bias modulation vector is obtained by calculating the vector sum of the unit vector and the product semantic bias field vector.

6. The AI ​​collaborative scoring method based on machine learning according to claim 4, characterized in that, The structure-supervised contrastive learning module constructs a structure similarity matrix and introduces a structure space separation mechanism to perform supervised contrastive training on the bias-modulated user question vector, thereby obtaining the user question structure feature vector, specifically including: The current training batch is set to contain N bias-modulated user question vectors, and each bias-modulated user question vector corresponds to a structural label. The structural label is obtained by mapping the user search behavior text according to the structural classification rules. Among them, the structural classification rules represent a two-dimensional structural tag system based on the user decision-making stage and the semantic intent of the question. The user decision-making stage includes the traffic acquisition stage, the opportunity stage, and the core conversion stage. The semantic intent of the question includes the category selection intent, the category demand intent, and the brand comparison intent. Each structural label is transformed into a structural label semantic vector through one-hot encoding and linear mapping; Calculate the cosine similarity between the semantic vector of the i-th structural label and the semantic vector of the j-th structural label to obtain the structural semantic similarity; Structural semantic similarity is used as the weight factor for the contrastive loss, and the structural contrastive loss is constructed using the InfoNCE contrastive loss formula; Construct a set of structure label center vectors, where each structure label center vector is obtained by averaging the user question vectors with the same bias of the structure label; Calculate the Euclidean distance between each bias-modulated user question vector and the corresponding structural label center vector to obtain the structural aggregation distance value, and sum all structural aggregation distance values ​​to obtain the sample aggregation constraint term; Calculate the Euclidean distance between any two structural label center vectors in the set of structural label center vectors to obtain the structural center spacing value, and sum all structural center spacing values ​​to obtain the structural center separation constraint term; Set aggregation weight coefficient and separation weight coefficient, where the aggregation weight coefficient is a positive real number and the separation weight coefficient is a positive real number; The aggregation weight coefficients are multiplied and modulated with the sample aggregation constraint term to obtain the aggregation loss term, and the separation weight coefficients are multiplied and modulated with the structure center separation constraint term to obtain the separation loss term. The structure-aware total loss function is obtained by adding the structure contrast loss, aggregation loss term, and separation loss term. By minimizing the total structure-aware loss function, the optimized bias-modulated user question vector is used as the user question structure feature vector.

7. The AI ​​collaborative scoring method based on machine learning according to claim 1, characterized in that, Step four specifically includes: The problem structure decoupling vector is concatenated with the corresponding product information feature vector and target keyword feature vector to construct the content and generate the input vector. Load a pre-trained large language model to transform the content generation input vector into an AI-generated content manuscript, which includes a title paragraph, a body paragraph, and a link paragraph; The AI-generated content is structurally analyzed to obtain features such as structural integrity, keyword density, conversion link layout, content quality, and product information consistency. These features are then combined to form a content cognition feature vector.

8. The AI ​​collaborative scoring method based on machine learning according to claim 1, characterized in that, Step five specifically includes: Construct a two-dimensional problem structure space, where the (m, n)th structural unit of the two-dimensional problem structure space corresponds to the structural label of the mth user decision stage and the nth problem semantic intent; The problem structure decoupling vector is assigned to the corresponding structural unit according to the structure label; All problem structure decoupling vectors in the same structural unit are used to form a problem set, and the number of problems in each structural unit is recorded. Calculate the structural cluster center vector for each structural unit, where the structural cluster center vector represents the vector mean of all problem structural decoupling vectors within the structural unit; Calculate the ratio of the number of problems in each structural unit to the total number of problems to obtain the problem weight coefficient for each structural unit.

9. The AI ​​collaborative scoring method based on machine learning according to claim 1, characterized in that, Step six specifically includes: The attention scoring model includes a structure-guided attention module and a main scoring network; The structure-guided attention module is used to generate structure-modulated content feature vectors, specifically: The structural cluster center vectors are mapped to structural query vectors through a trainable query mapping matrix; the content cognitive feature vectors are mapped to content key vectors and content value vectors through trainable key mapping matrices and value mapping matrices, respectively. The structure query vector and the content key vector are multiplied by a dot product and then normalized using Softmax to generate structure-guided attention weights. The problem weight coefficient is multiplied and modulated with the structure-guided attention weight to generate the structure-modulated attention weight; Based on the structure modulation attention weight, the content value vector is multiplied element by element to obtain the structure modulation content feature vector; The structure-modulated content feature vector is used to generate a scoring master vector through a scoring master network. The scoring master vector has a dimension of 5 and represents the structural integrity score, keyword density score, conversion link layout score, content quality score, and product information consistency score, respectively. The scoring master network consists of two fully connected layers. Define a weighted scoring vector and then weight and fuse the weighted scoring vector with the main scoring vector to obtain the content score.

10. The AI ​​collaborative scoring method based on machine learning according to claim 1, characterized in that, Step seven specifically includes: The availability determination is specifically as follows: Set a pass / fail threshold and a quality threshold. If the content score is less than the pass / fail threshold, the AI-generated content will be judged as unqualified and an unusable tag will be generated. If the content score is greater than or equal to the qualified threshold and less than the excellent threshold, the AI-generated content will be judged as qualified content and usable tags will be generated. If the content score is greater than or equal to the quality threshold, the AI-generated content will be judged as high-quality content and a high-quality tag will be generated. For AI-generated content with unavailable tags, rewriting is required. For AI-generated content articles with high-quality tags, content recommendation analysis is performed, specifically as follows: Set recommendation thresholds and conversion thresholds; If the keyword density score is greater than the recommendation threshold, the AI-generated content will be recommended and displayed on the homepage of the generative search platform. If the user decision-making stage is the core conversion stage, and the conversion link layout score is greater than the conversion threshold, then AI-generated content will be recommended as the priority content for conversion scenarios in the generative search platform. Conversion scenarios include displaying purchase links, product page redirects, or registration prompts to target users.

Images