Large model-based customer behavior semantic autoregressive analysis method, device and medium

Abstract

The application discloses a customer behavior semantic autoregressive analysis method and device based on a large model and a medium. The method comprises the following steps: collecting multi-dimensional sales data; performing behavior corpus unified processing on the multi-dimensional sales data to obtain a unified behavior corpus tensor; performing semantic analysis on the unified behavior corpus tensor by using a large language model to obtain a multi-dimensional semantic feature tensor; performing autoregressive analysis on the multi-dimensional semantic feature tensor by using the large language model to generate personalized marketing content; collecting marketing feedback of the personalized marketing content to adaptively optimize the large language model. The application integrates deep semantic modeling, autoregressive generation and closed-loop feedback mechanism of the large language model, and can realize intelligent insight into customer behavior and generation of personalized marketing strategies, and improve the accuracy, automation and sustainable optimization capability of marketing decisions.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a method, device, and medium for semantic autoregressive analysis of customer behavior based on a large model. Background Technology

[0002] In recent years, the tobacco industry has accumulated massive amounts of customer behavior data, including sales records, customer service conversations, and social media reviews. This data contains insights into user preferences, consumption patterns, and potential needs; however, traditional analytical methods are largely limited to statistical or rule-based models of structured data, making it difficult to deeply mine the semantic information within unstructured text.

[0003] Specifically, traditional customer segmentation methods often rely solely on superficial indicators such as purchase frequency or amount, failing to identify emotional shifts or complex motivations in customer reviews. Meanwhile, existing recommendation systems, such as collaborative filtering or rule-based engines, are limited by the singularity of input features, lack semantic modeling capabilities, and struggle to autonomously learn dynamic patterns of user behavior.

[0004] Furthermore, these systems lack self-regressive content generation capabilities, requiring manual writing of recommendation scripts. This is not only inefficient but also difficult to adapt to the strict compliance restrictions and consumption characteristics of the tobacco industry, such as avoiding misleading marketing or handling sensitive information. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, device and medium for semantic autoregressive analysis of customer behavior based on a large model.

[0006] The first aspect of this invention provides a semantic autoregressive analysis method for customer behavior based on a large model, comprising the following steps:

[0007] Collect multi-dimensional sales data;

[0008] The multi-dimensional sales data is subjected to unified behavioral corpus processing to obtain a unified behavioral corpus tensor.

[0009] Semantic analysis of the unified behavior corpus tensor was performed using a large language model to obtain a multi-dimensional semantic feature tensor.

[0010] Autoregressive analysis of the multidimensional semantic feature tensor is performed using a large language model to generate personalized marketing content.

[0011] The marketing feedback from the personalized marketing content is collected, and the large language model is adaptively optimized.

[0012] Furthermore, the multi-dimensional sales data includes structured data and unstructured data; wherein the structured data includes at least sales transaction records; and the unstructured data includes at least customer service conversations, social media reviews, and customer service knowledge bases.

[0013] Furthermore, the unified processing of the multi-dimensional sales data into behavioral corpus specifically includes the following steps:

[0014] For the structured data in the multi-dimensional sales data, normalization and tag extraction operations are performed to transform the structured data into a behavioral time series containing timestamps and behavioral events.

[0015] For the unstructured data in the multi-dimensional sales data, word segmentation, noise reduction, sentiment annotation and entity extraction operations are performed to convert the unstructured data into text information with sentiment tags and entity tags.

[0016] The time series of the timestamps and behavioral events, along with the text information containing sentiment tags and entity tags, are associated by timestamp and customer to construct a unified behavioral corpus tensor in the following format:

[0017] ;

[0018] in, Profile i Indicates customer information, Behavior i Indicates a behavioral event. Text i Represents text information. Sentiment i Indicates emotional tags, Entity i This represents an entity label.

[0019] Furthermore, the steps of semantic analysis of the unified behavior corpus tensor and autoregressive analysis of the multi-dimensional semantic feature tensor are completed by a pre-trained large language model; the large language model is trained based on the tobacco industry corpus to obtain industry fine-tuning parameters; by loading the industry fine-tuning parameters into the large language model and changing the weight of each word in the preset vocabulary of the large language model, the pre-trained large language model is obtained.

[0020] Furthermore, the semantic analysis module of the large language model performs the step of semantic analysis on the unified behavior corpus tensor, specifically executing the following steps:

[0021] The unified behavior corpus tensor was integrated into a structured corpus text based on the tobacco industry corpus.

[0022] The structured text is segmented into multiple words using a word segmenter, and a position code and a paragraph code are added to each word to obtain the input representation; wherein, the position code is used to indicate the position of each token ID in the structured text, and the paragraph code is used to indicate the paragraph of each token ID in the structured text.

[0023] The input representation is parsed using a self-attention mechanism to extract explicit and implicit features, thereby obtaining the original semantic feature tensor.

[0024] The original semantic feature tensor is subjected to feature decomposition to obtain feature vectors of multiple semantic types;

[0025] The feature vectors of the multiple semantic types are recombined to obtain a multi-dimensional semantic feature tensor.

[0026] Furthermore, the content generation module of the large language model performs the step of autoregressive analysis on the multi-dimensional semantic feature tensor, specifically executing the following steps:

[0027] The multi-dimensional semantic feature tensor is compressed into a condition vector, and the condition vector is concatenated with preset marketing prompt words to obtain a conditional input sequence; the marketing prompt words are used to indicate the direction of marketing text generation.

[0028] Based on the input sequence under the conditions, marketing text is generated word by word through autoregressive analysis to obtain the original text sequence;

[0029] The marketing text is checked against marketing rules to obtain marketing text content that meets marketing requirements, which is then used as personalized marketing content. The marketing text content includes user profile descriptions, reasons for product recommendations, product promotion statements, and push notification copy.

[0030] Furthermore, the step of generating marketing text word by word through autoregressive analysis based on the conditional input sequence specifically includes the following steps:

[0031] Based on the input sequence of conditions, the initial vocabulary for the marketing text is generated;

[0032] Based on the current marketing text, the probability distribution of the next word in the marketing text is generated through forward calculation, and the word with the highest probability distribution is added to the marketing text to obtain a new marketing text;

[0033] The new marketing text is used as the current marketing text. The step of generating the probability distribution of the next word of the marketing text through forward computation is returned, and the subsequent words of the marketing text are generated until the large language model generates the end words or the length of the marketing text reaches the preset length limit.

[0034] Furthermore, the collection of marketing feedback on the personalized marketing content specifically includes the following steps:

[0035] After the generated personalized marketing content is delivered to the target users, behavioral feedback data generated by the target users is collected; the behavioral feedback data specifically includes click-through rate, conversion rate, dwell time, emotional changes, and secondary behavior paths;

[0036] Identify missing data in the behavioral feedback data, complete the missing data by revisiting the target users, and construct a behavioral tag dataset;

[0037] The behavior label dataset was added to the tobacco industry corpus, and the industry fine-tuning parameters of the large language model were updated through supervised learning.

[0038] Another aspect of the present invention discloses an electronic device, including a processor and a memory;

[0039] The memory is used to store programs;

[0040] The processor executes the program to implement the above-described customer behavior semantic autoregressive analysis method based on a large model.

[0041] In another aspect, the present invention provides a computer-readable storage medium storing a program that is executed by a processor to implement the above-described customer behavior semantic autoregressive analysis method based on a large model.

[0042] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0043] The embodiments of this invention have the following beneficial effects: This invention provides a customer behavior semantic autoregressive analysis method, device, and medium based on a large model, integrating deep semantic modeling, autoregressive generation, and closed-loop feedback mechanisms of a large language model. This enables intelligent insights into customer behavior and the generation of personalized marketing strategies, improving the accuracy, automation, and sustainable optimization capabilities of marketing decisions. By leveraging a pre-trained large model and its fine-tuning, this invention can extract valuable semantic information from massive amounts of unstructured text; utilizing the autoregressive generation capabilities of the large language model, this invention can automatically generate high-quality personalized marketing content and strategy suggestions; finally, by introducing a feedback mechanism, this invention allows the model to be continuously fine-tuned based on marketing performance data, thereby proactively adapting to changes in the market environment and user behavior.

[0044] Additional aspects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description or may be learned by practice of the invention. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating the steps of a customer behavior semantic autoregressive analysis method based on a large model according to the present invention.

[0047] Figure 2 This is a schematic diagram illustrating the implementation process of the present invention, which involves collecting multi-dimensional sales data and performing unified behavioral corpus processing on the multi-dimensional sales data.

[0048] Figure 3 This is a flowchart illustrating the semantic analysis and autoregressive analysis process of the large language model of this invention.

[0049] Figure 4 This is a schematic diagram illustrating the implementation process of the closed-loop feedback mechanism of this invention;

[0050] Figure 5 This is a schematic diagram of the structure of an electronic device according to the present invention;

[0051] Figure 6 This is a schematic diagram of a computer-readable storage medium structure according to the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] In recent years, the tobacco industry has accumulated massive amounts of customer behavior data, including sales records, customer service conversations, and social media reviews. This data contains insights into user preferences, consumption patterns, and potential needs; however, traditional analytical methods are largely limited to statistical or rule-based models of structured data, making it difficult to deeply mine the semantic information within unstructured text.

[0054] For example, traditional customer segmentation often relies on purchase frequency or amount, making it difficult to identify emotional changes or hidden intentions in customer reviews. Existing recommendation systems, such as collaborative filtering or rule-based engines, are limited by input features, lack semantic modeling capabilities, struggle to learn complex behavioral patterns autonomously, and lack autoregressive content generation capabilities. They require manual writing of recommendation scripts, resulting in low efficiency and difficulty adapting to the compliance restrictions and consumption characteristics of the tobacco industry.

[0055] The rise of large-scale pre-trained language models (LLMs) has offered a new solution to the aforementioned problems. Based on the Transformer architecture, LLMs possess excellent semantic understanding and text generation capabilities. However, general-purpose models without domain adaptation cannot accurately understand tobacco industry terminology and consumer context, and the controllability and compliance of generated content are difficult to guarantee. Therefore, there is an urgent need to construct a tobacco customer behavior analysis method that combines domain fine-tuning and closed-loop optimization mechanisms to fully leverage the semantic modeling and generation advantages of LLMs.

[0056] This invention aims to overcome the technical bottlenecks in current customer behavior analysis in the tobacco industry, such as insufficient semantic understanding, coarse recommendation strategies, and lack of adaptive optimization. It proposes a customer behavior semantic autoregressive analysis method, device, and medium based on a large-scale model. The first embodiment of this invention provides a customer behavior semantic autoregressive analysis method based on a large-scale model, such as… Figure 1 As shown, it includes the following steps:

[0057] S1. Collect multi-dimensional sales data;

[0058] S2. Perform unified behavioral corpus processing on multi-dimensional sales data to obtain a unified behavioral corpus tensor;

[0059] S3. Use a large language model to perform semantic analysis on the unified behavior corpus tensor to obtain a multi-dimensional semantic feature tensor;

[0060] S4. Use a large language model to perform autoregressive analysis on multi-dimensional semantic feature tensors to generate personalized marketing content;

[0061] S5. Collect marketing feedback on personalized marketing content and adaptively optimize the large language model.

[0062] This invention integrates deep semantic modeling of large language models, autoregressive generation, and closed-loop feedback mechanisms into semantic analysis to achieve intelligent insights into customer behavior and generate personalized marketing strategies, thereby improving the accuracy, automation, and sustainable optimization capabilities of marketing decisions.

[0063] The implementation process of each step of this invention is described in detail below:

[0064] S1. Collect multi-dimensional sales data.

[0065] like Figure 2 As shown, the multi-dimensional sales data collected in this embodiment of the invention includes structured and unstructured data. Structured data includes sales transaction records (such as tabular data, containing timestamps, behavioral events, purchase amounts, product IDs, etc.). Unstructured text data includes customer service dialogues (such as call logs or chat logs), social media reviews (such as Weibo posts, forum comments, etc., which may contain text and emoticons), and customer service knowledge bases (which are semi-structured data, such as FAQ documents).

[0066] S2. Perform unified behavioral corpus processing on multi-dimensional sales data to obtain a unified behavioral corpus tensor.

[0067] like Figure 2 As shown in this embodiment of the invention, the behavioral corpus unification processing of multi-dimensional sales data specifically includes the following steps:

[0068] S2-1. For structured data in multi-dimensional sales data, perform normalization and label extraction operations to transform the structured data into a behavioral time series containing timestamps and behavioral events.

[0069] In this embodiment of the invention, step S2 performs different corpus processing on structured and unstructured data. For structured data, this embodiment first cleans the duplicate records in the structured data, then fills in the missing data in the structured data using the mean method; finally, it normalizes the numerical range of numerical fields such as purchase amount, and extracts key tags such as behavioral event type. The processing result of step S2-1 is presented as a behavioral time series in the following form:

[0070] ,in For timestamps, For specific behavioral events.

[0071] S2-2. For unstructured data in multi-dimensional sales data, perform word segmentation, noise reduction, sentiment annotation, and entity extraction operations to convert the unstructured data into text information with sentiment and entity tags.

[0072] For unstructured data, this embodiment of the invention uses word segmentation tools (including Jieba, HanLP, etc.) to segment the unstructured data into word sequences. At the same time, the word sequences are cleaned to remove irrelevant and special characters (such as emoticons, paragraph marks, etc.) to obtain text information. Then, sentiment analysis tools (such as BERT-FT or ERNIE, etc.) are used to perform sentiment analysis on the word sequences to extract several sentiment tags. Finally, a named entity recognition (NER) model is used to extract several key entity tags from the text, such as brand names, taste preferences, etc.

[0073] S2-3. Associate the time sequence of timestamps and behavioral events with text information containing sentiment tags and entity tags by timestamp and customer.

[0074] In this embodiment of the invention, after processing structured and unstructured data, different data are associated according to customer ID and timestamp to achieve feature alignment of multi-dimensional sales data and establish a unified data format. For example, sales records with the same customer ID, customer service conversations, and social media reviews can be integrated into the same instance. i Next, we construct a unified behavior corpus tensor in the following format:

[0075] ;

[0076] in, Profile i Indicates customer information, Behavior i Indicates a behavioral event. Text i Represents text information. Sentiment i Indicates emotional tags, Entity i This represents an entity label.

[0077] Each of them C i Representing the complete semantic context of a customer or a behavior instance, forming a standardized behavioral corpus tensor.

[0078] S3. Use a large language model to perform semantic analysis on the unified behavior corpus tensor to obtain a multi-dimensional semantic feature tensor.

[0079] Steps S3 and S4 in this embodiment of the invention are completed using a pre-trained large language model. For example... Figure 3 As shown, the large language model is trained based on a tobacco industry corpus to obtain industry-specific fine-tuned parameters. Preferably, the large language model employs full-parameter fine-tuning (minimizing the task loss function). ,in θ (where X is the model parameters, X is the text input, and Y is the label) or lightweight fine-tuning methods (such as LoRA, which adjusts the attention mechanism by adding a low-rank adapter, i.e.) This allows the fine-tuned model to possess domain-specific semantic awareness, enabling it to accurately understand tobacco terminology (such as "slim cigarettes" and "rich flavor") and consumption context. When performing customer behavior semantic analysis, industry-specific fine-tuning parameters are loaded into the large language model, changing the weight of each word in the model's pre-defined vocabulary, resulting in a pre-trained large language model.

[0080] For example, a tobacco company has sales records and customer feedback data from the past few years, denoted as... First, the researchers analyzed this historical data. Organize and construct the training corpus. For example, Indicates the first i A customer's historical behavior description vector, including information such as purchase history, preferred brands, and purchase frequency, is described as a structured information segment. It also collects customer feedback from questionnaires or customer service follow-ups, such as satisfaction ratings or purchase intentions, and denoted as [vector]. Then, the method of this invention was used to fine-tune a general-purpose large language model for industry applications: the model was trained under supervised supervision using the organized customer behavior corpus as training data. During the fine-tuning process, the model learned common behavioral patterns and language expressions of tobacco consumers, such as semantic features like "likes mint flavor" and "pays more attention to price".

[0081] In step S3, the semantic analysis module of the large language model performs semantic analysis on the unified behavior corpus tensor, specifically executing the following steps:

[0082] S3-1. Based on the tobacco industry corpus, the unified behavior corpus tensor is integrated into the structured corpus text.

[0083] In this embodiment of the invention, before performing semantic analysis using a large language model, the unified behavior corpus tensor is first integrated into a structured corpus text. The structured corpus text has a structure similar to natural language, and at the same time can clearly reflect the information meaning of the original unified behavior corpus tensor.

[0084] For example, customer information Profile i Convert to the following text: Gender: Male, Age: 35, VIP Level: 3; Behavioral Event Behavior i Convert to the following text: Purchased Zhonghua cigarettes last Wednesday, inquired about new e-cigarettes this Monday; Text message Text i Translated into the following text: Customer feedback: "Chunghwa cigarettes taste good, but I'd like a milder flavor." (Emotional tag) Sentiment i and physical labels Entity i This is reflected in the text above. For example, "tastes good" is an emotional label, while "Chinese" and "light flavor" are physical labels.

[0085] S3-2. Use a word segmenter to divide the structured corpus text into multiple words, and add position encoding and paragraph encoding to each word to obtain the input representation; where position encoding is used to indicate the position of each token ID in the structured corpus text, and paragraph encoding is used to indicate the paragraph of each token ID in the structured corpus text.

[0086] S3-3. Use a self-attention mechanism to perform semantic feature parsing on the input representation, mine the explicit and implicit features, and obtain the original semantic feature tensor.

[0087] After inputting the structured corpus text into the large language model, the model performs forward computation based on the Transformer architecture, capturing semantic relationships in the input through a self-attention mechanism. Specifically, the large language model obtains explicit features, including brand preferences, taste preferences, and consumption cycles, by directly parsing the structured corpus text; and identifies implicit features such as intentions and trends as implicit features by applying the self-attention mechanism to the text data. The explicit and implicit features are then combined into the original semantic feature tensor.

[0088] S3-4. Perform feature decomposition on the original semantic feature tensor to obtain feature vectors of multiple semantic types;

[0089] The original semantic feature tensor in step S3-3 is a highly compressed, information-dense "black box" representation, making it difficult to control or evaluate information in a specific dimension, thus hindering autoregressive analysis. Therefore, this embodiment of the invention first maps the original semantic feature tensor onto multiple feature spaces through the projection layer of a large language model, with each feature space representing a feature vector of a semantic type.

[0090] S3-5. Reorganize the feature vectors of multiple semantic types to obtain a multi-dimensional semantic feature tensor.

[0091] After obtaining feature vectors of multiple semantic types, these feature vectors are concatenated into a multi-dimensional semantic feature tensor in the following format:

[0092] ;

[0093] in, e i brand Indicates brand preference characteristics (such as a customer's inclination towards a specific brand); e i flavor Indicates taste preferences (such as customer preferences for mint, tobacco, etc.); e i emotion Indicates emotional characteristics (such as the customer's current emotional state); ei intent Indicates behavioral intent characteristics (such as a customer's potential purchase intent or complaint intent). e i trend It indicates the characteristics of changing trends (such as the cyclical or trend changes in consumer behavior). n The number of semantic dimensions; d For the embedded dimension.

[0094] The generated multidimensional semantic feature tensor can capture the multifaceted semantics of customer behavior, providing rich context for subsequent autoregressive analysis steps.

[0095] In this embodiment of the invention, by leveraging a pre-trained large model and its fine-tuning, the invention can extract valuable semantic information from massive amounts of unstructured text. For example, the model can understand the implicit preferences in customer reviews and analyze a customer's potential interest in a particular brand in customer service conversations, thereby constructing a more accurate customer profile. This deep semantic analysis surpasses traditional methods based on keywords or simple tags, enabling marketing decisions to be grounded in an accurate understanding of customer needs.

[0096] S4. Use a large language model to perform autoregressive analysis on multi-dimensional semantic feature tensors to generate personalized marketing content.

[0097] In step S4, the content generation module of the large language model performs autoregressive analysis on the multi-dimensional semantic feature tensor, specifically executing the following steps:

[0098] S4-1, compress the multi-dimensional semantic feature tensor into a condition vector, and concatenate the condition vector with the preset marketing prompt words to obtain the condition input sequence;

[0099] In step S4-1, the large language model compresses the multi-dimensional semantic feature tensor into a conditional vector through pooling operations. Simultaneously, based on marketing business needs, the conditional vector is concatenated with preset marketing prompts. These marketing prompts indicate the direction of marketing text generation; for example, phrases like "Generate marketing recommendations based on the following customer characteristics" can be used as marketing prompts.

[0100] S4-2. Based on the input sequence, generate marketing text word by word through autoregressive analysis to obtain the original text sequence;

[0101] In this embodiment of the invention, the large language model uses autoregressive analysis to generate text word by word, meaning the probability of each word is calculated based on previously generated words and conditional input. During the autoregressive analysis, the large language model predicts the next word in the marketing text one by one, adding each generated word to the existing text and using this as a basis for the next round of predictions. This cycle repeats until a termination signal is generated or the text reaches its maximum length. The autoregressive analysis is specifically implemented using the following formula:

[0102] ;

[0103] in, Y = y 1, y 2, ..., y T Represents the original text sequence. yt This represents the t-th word in the original text sequence. yt:t -1 indicates that the marketing text has already been generated. θ Indicates model parameters.

[0104] Preferably, step S4-2, generating marketing text word by word based on the conditional input sequence through autoregressive analysis, specifically includes the following steps:

[0105] S4-2-1. Generate the initial vocabulary for the marketing text based on the input sequence of conditions;

[0106] S4-2-2. Based on the current marketing text, generate the probability distribution of the next word in the marketing text through forward calculation, and add the word with the highest probability distribution to the marketing text to obtain a new marketing text;

[0107] S4-2-3. Take the new marketing text as the current marketing text, return to the step of generating the probability distribution of the next word of the marketing text through forward calculation, generate the subsequent words of the marketing text, until the large language model generates the end words or the length of the marketing text reaches the preset length limit.

[0108] After learning the conditional vectors of the original text sequence, the model, starting with the marketing prompt word, calculates the generation probability of each word in the model's vocabulary for the next word in the marketing text, summarizing them into a probability distribution. For example, maximizing the generation probability can be used... Perform probability calculations. Identify each element in the marketing text individually. y t Then, when the large language model generates the ending word or the marketing text reaches the preset length limit, the large language model terminates the next iteration. y t The generation process yields marketing text.

[0109] S4-3. Perform marketing rule checks on the marketing texts to obtain marketing text content that meets marketing requirements, which will then be used as personalized marketing content.

[0110] After generating marketing text using the large language model, it undergoes checks based on tobacco industry marketing rules to verify the text's rationality and compliance. For example, it checks for sensitive words (such as "health"-related terms) and automatically replaces or adjusts them, while ensuring the text closely matches the input semantic features. The final output is directly usable natural language text as personalized marketing content. This can be used for user profile summaries (for customer service and analytics systems), product recommendation reasons (to enhance persuasiveness), promotional copy, and push notification copy (for SMS and WeChat official accounts), ensuring the text meets actual business needs in terms of coherence, logic, and controllability.

[0111] For example, for new target customers, input their recent behavioral data (such as the product they purchased most recently, frequency changes, and online feedback). The model first performs semantic analysis and maps it to semantic vectors. This is used to determine the customer's interests, preferences, and potential needs. Subsequently, the model... The model generates a marketing strategy suggestion text based on the input autoregressive algorithm. The generation probability is calculated by the model's language generator to ensure the coherence and business controllability of the output text. For example, for a customer who has recently reduced their purchase frequency and mentioned wanting to try different flavors, the model might generate a strategy suggestion like: "Recommend the new product trial activity to this customer, focusing on small-packaged products of different flavors, and offer a one-time discount to encourage purchase." The marketing team implements the corresponding strategy based on this suggestion. Actual results show that the customer accepted the new product recommendation and made a purchase. After accumulating tests with multiple customers, it was found that compared to the traditional strategy before using the method of this invention, the strategy generated by this invention improved the marketing hit rate by about 20%, proving the effectiveness of model fine-tuning combined with semantic analysis and generation in improving marketing accuracy.

[0112] In this embodiment of the invention, leveraging the autoregressive generative capabilities of a large language model, the invention can automatically generate high-quality personalized marketing content and strategy suggestions. By constructing a training corpus from historical customer data, the large language model is subjected to supervised fine-tuning, enabling it to grasp the typical behavioral patterns and semantic expressions of tobacco users. Subsequently, the model can generate personalized recommendation strategies based on recent behavioral data of target customers and apply them to real-world scenarios to verify their predictive effectiveness, significantly improving marketing accuracy. This reduces the workload of manually drafting plans, ensuring that each customer receives marketing information tailored to their behavioral characteristics and interests. For example, the large language model can generate promotional content emphasizing discounts for customers who prefer slim cigarettes and value cost-effectiveness, increasing user acceptance and engagement.

[0113] S5. Collect marketing feedback on personalized marketing content and adaptively optimize the large language model.

[0114] In some embodiments, user behavior preferences may change at any time. To maintain the model's adaptability to the latest trends, the method of this invention also introduces a reinforcement learning mechanism to perform closed-loop fine-tuning of the large language model, such as... Figure 4 As shown.

[0115] Step S5 involves collecting marketing feedback on personalized marketing content, specifically including the following steps:

[0116] S5-1. After delivering the generated personalized marketing content to the target users, collect behavioral feedback data generated by the target users.

[0117] The behavioral feedback data collected in step S5-1 specifically includes click-through rate, conversion rate, dwell time, sentiment changes, and secondary behavioral paths. Click-through rate, conversion rate, and dwell time can be collected using event tracking technology, while sentiment changes require analysis using corresponding analytical models.

[0118] S5-2. Identify missing data in the behavioral feedback data, complete the missing data by revisiting the target users, and construct a behavioral label dataset;

[0119] For missing data in the behavioral feedback data, the missing data can be supplemented by revisiting the target users, such as conducting regular manual or intelligent follow-ups with key user groups. Then, a behavioral label dataset for evaluation and relearning can be constructed based on the behavioral feedback data. , For behavior feedback vectors, such as .

[0120] S5-3. Add the behavior label dataset to the tobacco industry corpus and update the industry fine-tuning parameters of the large language model through supervised learning.

[0121] After constructing the dataset, this embodiment of the invention continuously integrates feedback data into the tobacco industry corpus and uses supervised learning to update model parameters to correct biases in previous strategies. Simultaneously, this invention introduces a reinforcement learning mechanism to transform key marketing metrics (such as conversion rate and user retention rate) into reward signals. For example, the reward signal is .

[0122] in, α , β , γ For weight parameters, CVR For conversion rate, Retention For user retention rate, UserChurn Reward signals, used to quantify the success of marketing strategies, are used to measure user churn rate.

[0123] In the reinforcement learning process, this embodiment of the invention employs a Markov Decision Process (MDP). The Markov Decision Process is represented by the following formula:

[0124] ;

[0125] in For the state space, represent the client semantic tensor. , The action space represents the list of recommended products and its generated text. The state transition probability is determined by the environment. For reward functions, such as click-through rate and conversion rate feedback, The discount factor is used. The model initially uses a large language model fine-tuned with historical data to generate recommendation results. For example, generating a list of recommended tobacco products that new users might be interested in. and accompanying recommendations Then, rewards are calculated based on the actual user interaction data for that day. If a user clicks on or purchases a recommended item, it is considered a positive reward (e.g., ...). Conversely, a positive signal is a negative signal (e.g.) In reinforcement learning, the fine-tuning parameters of the model are optimized using policy gradient algorithms (such as PPO—Proximal Policy Optimization). Specifically, it is represented in the following form:

[0126] ;

[0127] in The advantage function measures the benefit of an action relative to the baseline. This guides the optimization of the model's policy, enabling the model to have environmental awareness and self-evolution capabilities.

[0128] During reinforcement learning, the model gradually increases the probability of generating high-reward outputs and reduces the tendency to generate low-reward outputs, thereby continuously improving the model's recommendation performance and enabling it to perceive the environment and evolve itself. For example, initially, the system's click-through rate for recommending new cigarette categories was only 5%, but after reinforcement learning optimization, it increased to over 15%. Furthermore, when market changes occur (such as a brand suddenly becoming popular or user preferences shifting), the model can quickly adjust its recommendation strategy based on new feedback, demonstrating faster response speed and higher user satisfaction compared to static rule systems.

[0129] Compared with the prior art, the present invention has significant beneficial effects:

[0130] (1) Deep Semantic Understanding: With the help of a pre-trained large model and its fine-tuning, this invention can extract valuable semantic information from massive amounts of unstructured text. For example, the model can understand the implicit preferences in customer reviews and analyze the potential interest of customers in a certain brand in customer service conversations, thereby constructing a more accurate customer profile. This deep semantic analysis goes beyond traditional methods based on keywords or simple tags, enabling marketing decisions to be based on an accurate understanding of customer needs.

[0131] (2) Automated Marketing Content Generation: Utilizing the autoregressive generation capabilities of large language models, this invention can automatically generate high-quality personalized marketing content and strategy suggestions. This reduces the workload of manually writing plans and ensures that each customer receives marketing information tailored to their behavioral characteristics and interests. For example, the system can generate promotional content focusing on discounts for customers who prefer slim cigarettes and value cost-effectiveness, thereby increasing user acceptance and engagement.

[0132] (3) Closed-loop optimization and adaptation: This invention achieves closed-loop optimization of the system by introducing a feedback mechanism. The model is continuously fine-tuned based on marketing performance data, enabling it to adapt to changes in the market environment and user behavior. In contrast, traditional models remain fixed once deployed, making it difficult to respond promptly to new trends or changes in consumer preferences. The dynamic optimization capability provided by this invention ensures that marketing strategies are always based on the latest data insights, significantly improving the accuracy and continuous effectiveness of recommendations.

[0133] (4) Cross-domain knowledge generalization: Due to the use of a language model pre-trained on a massive general corpus, the method of this invention possesses a certain cross-domain knowledge transfer capability. After fine-tuning in the tobacco industry, the model still retains its understanding of general consumer behavior patterns, and can apply effective strategies from other consumer fields to tobacco marketing. This generalization capability helps to expand innovative marketing ideas, while avoiding the bias that may result from relying solely on narrow domain data.

[0134] (5) Improve marketing efficiency and optimize user experience: This invention achieves partial automation of the marketing process, significantly shortens the cycle from data insight to strategy execution, reduces the workload of manual intervention, and thus reduces the manpower and time costs of marketing campaign planning. At the same time, because the recommendations and marketing content provided are more in line with users' personal preferences, the information received by users is more relevant and valuable, avoiding the interference caused by irrelevant push notifications, and significantly improving customers' acceptance and satisfaction with marketing campaigns.

[0135] Figure 5 This is a schematic diagram of the electronic device proposed in the second embodiment of the present invention. In this embodiment, the memory stores program instructions for implementing the customer behavior semantic autoregressive analysis method based on a large model in any of the above embodiments. The processor executes the program instructions stored in the memory to perform customer behavior semantic autoregressive analysis based on a large model. The processor can also be called a CPU (Central Processing Unit). The processor may be an integrated circuit chip with signal processing capabilities. The processor can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0136] The methods described in the first embodiment of the present invention are applicable to the embodiments of the present electronic device. The specific functions implemented by the embodiments of the present electronic device are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0137] Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium according to the third embodiment of the present invention. The computer-readable storage medium of the fourth embodiment of the present invention stores program instructions capable of implementing the above-described customer behavior semantic autoregressive analysis method based on a large model. These program instructions can be stored in the storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned computer-readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0138] The methods described in the first embodiment of the present invention are applicable to the computer-readable storage medium embodiment. The specific functions implemented by the computer-readable storage medium embodiment are the same as those in the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above method.

[0139] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the customer behavior semantic autoregressive analysis method based on a large model provided in the above embodiment.

[0140] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0141] Those skilled in the art will understand that modules in the device of the embodiments of the present invention can be adaptively modified and placed in one or more devices different from those embodiments. Modules, units, or components in the embodiments of the present invention can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the corresponding claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the corresponding claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0142] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0143] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0144] Furthermore, the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. In particular, for embodiments such as apparatus and devices, since they are basically similar to the method embodiments, the relevant parts can be referred to the description of the method embodiments. The apparatus, devices, and other embodiments described above are merely illustrative, and the modules, units, etc., described as separate components may or may not be physically separate, that is, they may be located in one place or distributed in multiple places, such as nodes in a system network. Specifically, some or all of the modules and units can be selected according to actual needs to achieve the purpose of the above-described embodiment solutions. Those skilled in the art can understand and implement this without creative effort.

[0145] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0146] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0147] Furthermore, the terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of technical features indicated in this embodiment. Therefore, features defined with terms such as "first" and "second" in the embodiments of this invention can explicitly or implicitly indicate that the embodiment includes at least one of those features. In the description of this invention, the word "multiple" means at least two or more, such as two, three, four, etc., unless otherwise explicitly specified in the embodiments.

[0148] In embodiments of the present invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of the present invention may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0149] Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention. Other embodiments of the present invention will readily conceive of by considering the specification and practicing the invention. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

Claims

1. A large model-based customer behavior semantic autoregressive analysis method, characterized in that, The method comprises the following steps: Collecting multi-dimensional sales data; Performing behavior corpus uniform processing on the multi-dimensional sales data to obtain a uniform behavior corpus tensor; Using a large language model to perform semantic analysis on the uniform behavior corpus tensor to obtain a multi-dimensional semantic feature tensor; Using a large language model to perform autoregressive analysis on the multi-dimensional semantic feature tensor to generate personalized marketing content; Collecting marketing feedback of the personalized marketing content to adaptively optimize the large language model; The semantic analysis module of the large language model performs the semantic analysis on the uniform behavior corpus tensor, and specifically performs the following steps: Integrating the uniform behavior corpus tensor into a structured corpus text based on a tobacco industry corpus; Using a word segmenter to cut the structured corpus text into multiple words, and adding position encoding and paragraph encoding to each word to obtain an input representation; wherein the position encoding is used to indicate the position of each token ID in the structured corpus text, and the paragraph encoding is used to indicate the paragraph of each token ID in the structured corpus text; Using a self-attention mechanism to analyze the semantic features of the input representation, and mining the explicit features and implicit features in the input representation to obtain an original semantic feature tensor; Performing feature decomposition on the original semantic feature tensor to obtain a plurality of feature vectors of semantic types; Performing feature recombination on the plurality of feature vectors of semantic types to obtain a multi-dimensional semantic feature tensor; The content generation module of the large language model performs the autoregressive analysis on the multi-dimensional semantic feature tensor, and specifically performs the following steps: Compressing the multi-dimensional semantic feature tensor into a conditional vector, and concatenating the conditional vector with a preset marketing prompt word to obtain a conditional input sequence; the marketing prompt word is used to prompt the generation direction of the marketing text; Generating the marketing text word by word through autoregressive analysis according to the conditional input sequence to obtain an original text sequence; Performing marketing rule checking on the marketing text to obtain marketing text content meeting marketing requirements as personalized marketing content; the marketing text content includes user portrait description, product recommendation reason, product promotion sentence and message push copy.

2. The method of claim 1, wherein, The multi-dimensional sales data includes structured data and unstructured data; The structured data at least includes sales transaction records; and the unstructured data at least includes customer service dialogues, social media evaluations and customer service knowledge bases.

3. The method of claim 1, wherein the method further comprises: The behavior corpus uniform processing on the multi-dimensional sales data specifically comprises the following steps: For the structured data in the multi-dimensional sales data, performing normalization and label extraction operations to convert the structured data into a behavior time series containing a timestamp and a behavior event; For the unstructured data in the multi-dimensional sales data, performing word segmentation, denoising, sentiment labeling and entity extraction operations to convert the unstructured data into text information with sentiment labels and entity labels; Associating the behavior time series of the timestamp and the behavior event with the text information with the sentiment labels and the entity labels according to the timestamp and the customer to construct a uniform behavior corpus tensor in the following format: ； wherein, Profile i represents customer information, Behavior i represents action events, Text i represents text information, Sentiment i represents sentiment labels, Entity i represents entity labels.

4. The method of claim 1, wherein the method is based on a large model. The semantic analysis step on the unified behavior corpus tensor and the autoregressive analysis step on the multi-dimensional semantic feature tensor are completed by a pre-trained large language model; the large language model is trained based on a tobacco industry corpus to obtain industry fine-tuning parameters; by loading the industry fine-tuning parameters into the large language model, the weight of each vocabulary in the preset vocabulary table of the large language model is changed, and a pre-trained large language model is obtained.

5. The method of claim 1, wherein the method is based on a large model. The marketing text is generated word by word according to the conditional input sequence through autoregressive analysis, specifically including the following steps: According to the conditional input sequence, an initial vocabulary of the marketing text is generated; Based on the current marketing text, the probability distribution of the next vocabulary of the marketing text is generated through forward calculation, and the vocabulary with the highest probability distribution is added to the marketing text to obtain a new marketing text; The new marketing text is taken as the current marketing text, and the step of generating the probability distribution of the next vocabulary of the marketing text through forward calculation is returned to generate the subsequent vocabulary of the marketing text until the large language model generates an end vocabulary or the length of the marketing text reaches a preset upper limit.

6. The method of claim 1, wherein the method further comprises: The marketing feedback of the personalized marketing content is collected, specifically including the following steps: After the generated personalized marketing content is put into target users, behavior feedback data generated by the target users is collected; the behavior feedback data specifically includes click rate, conversion rate, stay time, emotional change and secondary behavior path; Missing data in the behavior feedback data is identified, and the missing data is completed by revisiting the target users to construct a behavior label data set; The behavior label data set is added to the tobacco industry corpus, and the industry fine-tuning parameters of the large language model are updated through supervised learning.

7. An electronic device, comprising: A processor and a memory are included; The memory is used to store a program; The processor executes the program to realize the customer behavior semantic autoregressive analysis method based on a large model according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The storage medium stores a program, and the program is executed by the processor to realize the customer behavior semantic autoregressive analysis method based on a large model according to any one of claims 1-6.

Images