Consumer demand analysis method and device based on multi-objective optimization algorithm

Abstract

The invention relates to the technical field of data analysis, and discloses a consumer demand analysis method and device based on a multi-objective optimization algorithm, and the method comprises the steps: obtaining and preprocessing a plurality of pieces of consumer comment data, and obtaining a plurality of pieces of effective comment data; analyzing the effective comment data by using a preset recognition model to obtain theme probability distribution and emotion probability distribution of the effective comment data; combining the topic probability distribution and the emotion probability distribution to obtain an overall joint probability matrix; calculating by using a multi-objective optimization algorithm to obtain an optimal weight matrix; and multiplying the overall joint probability matrix by the optimal weight matrix to obtain a demand analysis result. According to the method, by obtaining the comment data of the consumer and analyzing the theme and emotional tendency of the consumer, the coverage and real-time performance of data collection are improved, the demand and preference information of the consumer is deeply mined, the demand preference of the consumer for the target product is comprehensively and dynamically mastered, and a more accurate and timely basis is provided for the market decision of an enterprise.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and in particular to a method and device for analyzing consumer demand based on a multi-objective optimization algorithm. Background Art

[0002] As market competition intensifies, companies are paying more and more attention to improving consumer satisfaction, because consumer satisfaction directly affects the brand's market performance and long-term development. Traditional consumer demand analysis survey methods mainly rely on questionnaires and face-to-face interviews. Although these methods can provide certain consumer feedback information, they have the following problems:

[0003] (1) Limited samples: Traditional questionnaire surveys can usually only cover a limited number of consumers and cannot fully reflect the opinions and feelings of all consumers in the market. This sample limitation leads to biased survey results and affects the quality of corporate decision-making.

[0004] (2) Long data update cycle: Traditional demand analysis surveys often take a long time to collect and analyze data and are unable to reflect changes in consumer demand in real time. This lag makes it difficult for companies to adjust product strategies in a timely manner to meet market demand.

[0005] (3) High cost: Organizing large-scale questionnaire surveys and consumer interviews usually requires a lot of manpower and financial resources. Especially when the survey needs to be conducted across regions, the cost issue is more prominent.

[0006] (4) Insufficient information depth: The data obtained from questionnaires and interviews are usually structured and difficult to capture consumers’ deeper emotions and preference information, such as consumers’ potential emotional connection with a certain cigarette brand.

[0007] (5) Reliance on active consumer participation: Questionnaire surveys and interviews usually require active participation from consumers, which may lead to self-selection bias in the data, that is, consumers who participate in the survey may have systematic differences in certain characteristics from those who do not participate. Summary of the invention

[0008] In view of this, the present invention provides a consumer demand analysis method and device based on a multi-objective optimization algorithm to solve the problem that the consumer demand analysis survey results are not accurate enough.

[0009] In a first aspect, the present invention provides a method for analyzing consumer demand based on a multi-objective optimization algorithm, the method comprising:

[0010] Acquire multiple consumer review data of the target product, and pre-process the consumer review data to obtain multiple valid review data;

[0011] Analyze each valid comment data using a preset topic recognition model to obtain the topic probability distribution of the valid comment data in multiple preset topics, and analyze the valid comment data using a preset sentiment analysis model to obtain the sentiment probability distribution of the valid comment data;

[0012] Combine the topic probability distribution and sentiment probability distribution of each valid comment data to obtain the overall joint probability matrix;

[0013] Based on the overall joint probability matrix, the optimal weight matrix is ​​calculated using a multi-objective optimization algorithm;

[0014] Determine the demand intensity of the target product on each topic based on the optimal weight matrix and the overall joint probability matrix.

[0015] The consumer demand analysis method based on a multi-objective optimization algorithm provided by the present invention improves the coverage and real-time performance of data collection by acquiring consumer review data and analyzing its themes and sentiment tendencies, deeply mines consumer demand and preference information, comprehensively and dynamically grasps consumer demand preferences for target products, and provides a more accurate and timely basis for the company's market decision-making.

[0016] In an optional implementation, obtaining multiple consumer review data of a target product includes:

[0017] Determine the order of social platforms for obtaining consumer review data based on the current status and preset priority of each preset social platform;

[0018] According to the keywords of the target product, crawler technology is used to obtain multiple consumer review data from each preset social platform in the order of the social platform.

[0019] The consumer demand analysis method based on a multi-objective optimization algorithm provided by the present invention utilizes crawler technology to sequentially obtain comment data from multiple social platforms, which can quickly understand market trends and the latest feedback from consumers. The wide range of data sources can reflect diverse consumer opinions, avoiding sample limitations that lead to deviations in demand analysis results. More comprehensive market insights guide companies to timely adjust their marketing strategies and product designs, thereby better meeting consumer needs.

[0020] In an optional implementation, the consumer review data is preprocessed to obtain a plurality of valid review data, including:

[0021] Semantically encode consumer review data to obtain semantic feature vectors;

[0022] Use the pre-trained large language model to perform semantic analysis on the semantic feature vector to obtain the noise in the semantic feature vector;

[0023] The consumer review data corresponding to the noise is filtered out as invalid review data to obtain valid review data, and the pre-trained large language model is dynamically adjusted according to the quality of the valid review data.

[0024] The consumer demand analysis method based on a multi-objective optimization algorithm provided by the present invention cleans the original consumer review data, screens out invalid review data, improves the credibility and quality of valid review data, reduces the processing of invalid data, saves computing resources, manpower costs and time costs, and improves data processing efficiency.

[0025] In an optional implementation, each valid comment data is analyzed using a preset topic recognition model to obtain a topic probability distribution of the valid comment data in multiple preset topics, including:

[0026] Extract feature vectors from valid comment data, and perform topic clustering on the feature vectors to obtain candidate topics from the valid comment data;

[0027] Using the preset topic identification model, the topic probability distribution of valid comment data belonging to each candidate topic is calculated based on the feature vector.

[0028] The consumer demand analysis method based on a multi-objective optimization algorithm provided by the present invention classifies the core contents of target products that users focus on by identifying the topics of effective review data, thereby facilitating rapid analysis and accurate understanding of consumer demand.

[0029] In an optional implementation, the valid comment data is analyzed using a preset sentiment analysis model to obtain the sentiment probability distribution of the valid comment data, including:

[0030] Based on the topic probability distribution of valid comment data, the preset sentiment analysis model is used to analyze the valid comment data to obtain the sentiment probability distribution corresponding to each candidate topic.

[0031] The consumer demand analysis method based on a multi-objective optimization algorithm provided by the present invention determines consumers' positive, neutral or negative emotions towards the topic by performing sentiment analysis on effective comment data of a determined topic, and further determines whether a target product is popular under the topic, which is conducive to accurately judging consumer demand and guiding product improvement.

[0032] In an optional implementation, the topic probability distribution and the sentiment probability distribution of each valid comment data are combined to obtain an overall joint probability matrix, including:

[0033] Take the outer product of the topic probability distribution and sentiment probability distribution of the valid comment data to obtain the single joint probability corresponding to the topic and sentiment in the valid comment data;

[0034] Aggregate the individual joint probabilities of each valid comment data to obtain the overall joint probability matrix.

[0035] The consumer demand analysis method based on a multi-objective optimization algorithm provided by the present invention combines the topic probability distribution with the emotion probability distribution to obtain an overall joint probability matrix of each valid comment data, which is conducive to comprehensive analysis from both the topic and emotion aspects to obtain more accurate consumer demand analysis results.

[0036] In an optional implementation, based on the overall joint probability matrix, an optimal weight matrix is ​​calculated using a multi-objective optimization algorithm, including:

[0037] The weights of different topics and emotions in the overall joint probability matrix are assigned. In order to determine the weights of each topic and emotion, an optimization objective function is constructed. This function aims to maximize the preference reflection of consumer demand. The optimization objective function is:

[0038]

[0039] Among them, w hk =[w 11 ,w 12 ,…,w n3 ] is the weight of each element in the joint probability distribution matrix, satisfying ( And w jj ≥0), J jk represents the element in the joint probability distribution matrix, j represents the topic, and k represents the emotion. Finally, the optimal weight matrix w is obtained through a multi-objective optimization algorithm (such as genetic algorithm or particle swarm algorithm) * ,These weights are used to reflect the relative importance of each topic under different sentiment tendencies.

[0040] The consumer demand analysis method based on a multi-objective optimization algorithm provided by the present invention determines the intensity of consumer demand on different topics by assigning optimal weight distribution to each element in the overall joint probability matrix, which is conducive to accurately understanding consumers' demand preferences for target products, adjusting product strategies accordingly, and providing better services to consumers.

[0041] In a second aspect, the present invention provides a consumer demand analysis device based on a multi-objective optimization algorithm, the device comprising:

[0042] An effective review data acquisition module is used to acquire multiple consumer review data of a target product and pre-process the consumer review data to obtain multiple effective review data;

[0043] A comment data analysis module is used to analyze each valid comment data using a preset topic recognition model to obtain the topic probability distribution of the valid comment data in multiple preset topics, and to analyze the valid comment data using a preset sentiment analysis model to obtain the sentiment probability distribution of the valid comment data;

[0044] The joint probability calculation module is used to combine the topic probability distribution and sentiment probability distribution of each valid comment data to obtain the overall joint probability matrix;

[0045] A weight matrix calculation module is used to calculate the optimal weight matrix based on the overall joint probability matrix using a multi-objective optimization algorithm;

[0046] The demand analysis module is used to determine the demand intensity of the target product on each topic based on the optimal weight matrix and the overall joint probability matrix.

[0047] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0048] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to cause a computer to execute the method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 is a flow chart of a method for analyzing consumer demand based on a multi-objective optimization algorithm according to an embodiment of the present invention;

[0051] Figure 2 is a flow chart of another method for analyzing consumer demand based on a multi-objective optimization algorithm according to an embodiment of the present invention;

[0052] Figure 3 is a structural block diagram of a consumer demand analysis device based on a multi-objective optimization algorithm according to an embodiment of the present invention;

[0053] Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0055] The embodiment of the present invention provides a consumer demand analysis method based on a multi-objective optimization algorithm, which obtains real-time comment data through a crawler and uses a big data model to perform topic and sentiment analysis to achieve the effect of accurately understanding consumer needs and emotional tendencies.

[0056] In recent years, with the rapid development of the Internet and social media platforms, consumers are increasingly expressing their opinions and feedback on products through online platforms, especially in the cigarette industry, where consumers often share their usage experience, product reviews and brand preferences on social media. These comments and feedback data show the following characteristics:

[0057] (1) Large amount of data: Social media platforms generate massive amounts of data every day, which reflects consumers’ real-time opinions and feelings about products. (2) Wide range of data sources: Comments and feedback data come from a variety of sources, providing companies with multi-dimensional consumer information. (3) Strong real-time performance: Comments and feedback on social media can reflect consumers’ attitudes and emotional changes in real time, providing companies with a basis for rapid market response.

[0058] However, the data on social media platforms is usually unstructured and difficult to directly analyze using traditional methods. The complexity and diversity of such data require more advanced data processing and analysis technologies. Therefore, how to effectively use the unstructured data on social media platforms to analyze consumer demand has become one of the important challenges currently facing cigarette companies.

[0059] Based on the above situation, according to an embodiment of the present invention, a consumer demand analysis method embodiment based on a multi-objective optimization algorithm is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0060] In this embodiment, a method for analyzing consumer demand based on a multi-objective optimization algorithm is provided, which can be used in the above-mentioned computer system. Figure 1is a flow chart of a method for analyzing consumer demand based on a multi-objective optimization algorithm according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0061] Step S101, obtaining a plurality of consumer review data of a target product, and preprocessing the consumer review data to obtain a plurality of valid review data.

[0062] Specifically, the target product name is used as a keyword, and distributed web crawler technology is used to dynamically collect multiple consumer review data on multiple social platforms. The data format of the consumer review data is set in advance, and when crawling, the information of each consumer review data is obtained according to the data format. The collected consumer review data contains information such as: review content, review time, user nickname, and the location of the review IP.

[0063] The acquired consumer review data includes valid review data and invalid review data, which need to be automatically cleaned and preprocessed, including removing noise data, control data, etc., to screen out valid review data for subsequent analysis.

[0064] Step S102, using a preset topic recognition model to analyze each valid comment data to obtain the topic probability distribution of the valid comment data in multiple preset topics, and using a preset sentiment analysis model to analyze the valid comment data to obtain the sentiment probability distribution of the valid comment data.

[0065] Specifically, a preset topic identification model is used to identify the topic of each valid comment data to obtain the product topic involved in the consumer comment data. For example, for cigarette products, the topic may include but is not limited to price, taste, packaging, etc. The preset topic identification model is a pre-trained large language model. The model training process includes: selecting a pre-trained large language model (the Qwen model is taken as an example in this embodiment, but is not limited to this) as the basic model for topic identification, and using its pre-training ability on massive text data to improve the accuracy of topic identification. In order to further improve the topic identification effect of the model, it is necessary to train the large prediction model. In this embodiment, low-rank adaptation technology (Low-Rank Adaptation, LoRA) is used to train the Qwen model to reduce the number of parameter updates and achieve efficient model training. LoRA introduces a low-rank matrix and trains the original weight matrix W of the Qwen model, where A is a dimensionality reduction matrix and B is a dimensionality increase matrix. By updating the weight matrix W′ of the Qwen model, the training of the Qwen model is achieved, thereby improving the effect of topic identification. The specific formula is:

[0066] W′=W+A·B

[0067] Among them, A and B are low-rank matrices that need to be learned, W is the original pre-trained weight of the Qwen model, and W′ is the new weight matrix after adjustment.

[0068] For sentiment analysis, the training process of the preset sentiment analysis model includes: selecting a pre-trained large language model suitable for the sentiment analysis task (the Qwen model is used as an example in this embodiment, but is not limited to this), and using the pre-trained large language model to extract features from the consumer review data after the topic analysis:

[0069] H i =Model Pre-trained (X i ,Topic i )

[0070] Among them, H i is the sentiment feature vector after combining the topic, X i is the data text of the i-th comment, Topic i The topic information of the i-th comment data text.

[0071] Step S103, combining the topic probability distribution and the sentiment probability distribution of each valid comment data respectively to obtain an overall joint probability matrix.

[0072] Specifically, after topic identification, the topic probability distribution of each valid comment data is obtained:

[0073] T ij =[T i1 ,T i2 ,…,T in ]

[0074] Among them, T ij It represents the association probability between the i-th valid comment data and the topic j. After sentiment recognition, the sentiment probability distribution of each valid comment data is obtained:

[0075] E i =[E i,neg ,E i,neu ,E i,pos ]

[0076] Among them, E i,neg Comment i The corresponding negative sentiment probability, E i,neu Comment i The corresponding neutral sentiment probability, E i,pos Comment i The corresponding positive sentiment probability.

[0077] Take the outer product of the above topic probability distribution and sentiment probability distribution to obtain the individual joint probability of each comment, and then aggregate the individual joint probabilities to obtain the overall joint probability matrix.

[0078] Step S104, based on the overall joint probability matrix, an optimal weight matrix is ​​calculated using a multi-objective optimization algorithm.

[0079] Specifically, based on the overall joint probability matrix, a multi-objective optimization algorithm is applied to assign weights to different topics and sentiment dimensions to determine the relative importance of each topic in consumer demand.

[0080] Step S105, determining the demand intensity of the target product on each topic according to the optimal weight matrix and the overall joint probability matrix.

[0081] After finding the optimal weight, the next step is to determine the demand intensity on each topic based on the weight. The weighted demand intensity matrix is ​​calculated by combining the probability distribution matrix and the optimal weight, and the formula is as follows:

[0082]

[0083] Among them, P j represents the demand intensity of the jth topic, which is the weighted sum of the joint probabilities of the jth row corresponding to each sentiment tendency (positive, neutral, negative) in the overall joint probability matrix J. represents the optimal weight of the jth topic in the optimal weight matrix. By calculating the weighted demand intensity matrix P, the overall demand intensity of each topic can be obtained. These results clearly show the demand intensity of consumers on different topics, and companies can optimize products and adjust market strategies based on these demand intensities.

[0084] The consumer demand analysis method based on a multi-objective optimization algorithm provided in this embodiment improves the coverage and real-time performance of data collection by acquiring and analyzing consumer review data, deeply explores consumer demand and preference information, comprehensively and dynamically grasps consumer demand preferences for target products, and provides a more accurate and timely basis for the company's market decision-making.

[0085] In this embodiment, a method for analyzing consumer demand based on a multi-objective optimization algorithm is provided, which can be used in the above-mentioned computer system. Figure 2 is a flow chart of a method for analyzing consumer demand based on a multi-objective optimization algorithm according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0086] Step S201, obtaining a plurality of consumer review data of a target product, and preprocessing the consumer review data to obtain a plurality of valid review data.

[0087] Specifically, the above step S201 includes:

[0088] Step S2011, determining the order of social platforms for acquiring consumer review data according to the current status and preset priority of each preset social platform.

[0089] Specifically, the distributed crawler master-slave architecture used to obtain consumer review data from various social platforms includes: a master node and multiple slave nodes, where: the master node is responsible for task allocation and scheduling, and the slave node is used to perform specific data crawling tasks. The slave node and the master node are connected through a communication protocol to achieve task distribution and result aggregation.

[0090] Step S2012, based on the keywords of the target product, using crawler technology to obtain multiple consumer review data from each preset social platform in the order of the social platform.

[0091] Specifically, the master node generates tasks based on the preset keywords of the target product (e.g., cigarette brand name, for example only, but not limited to this) and the target social platform, and assigns them to the slave nodes. The name of the cigarette product to be analyzed and the target social platform are input through the interactive interface, and the cigarette name and platform information are used as input parameters of the crawler program to generate crawling tasks.

[0092] According to the assigned crawling tasks, the slave node simulates user operations or calls the platform API to submit keywords to the target social platform for retrieval, automatically finds videos and graphic links related to the user input keywords, and then obtains page data through HTTP requests, parses page content, and extracts comment information. The collected comment data includes information such as comment content, comment time, user nickname, and comment IP location. Combining multiple slave nodes can realize parallel crawling of data from multiple social platforms, and dynamically adjust the number of threads through the thread pool management unit to adapt to the request restrictions and load conditions of different social platforms. The thread pool management unit dynamically increases or decreases the number of threads according to the response speed and load conditions of the social platform to ensure the efficiency and stability of data crawling. For example, when the response speed of the target social platform is slow, the number of threads is automatically reduced.

[0093] The task scheduling module further optimizes the crawling strategy according to the load and network conditions of the social platform. For example, when the network conditions are poor, the collection tasks of each slave node are automatically reduced to avoid data collection failure. It supports task priority management to ensure that comment data on important social platforms or specific keywords are captured first. When faced with the task of collecting consumer comments on multiple products at the same time, slave nodes can be assigned for priority crawling according to user needs. For example, comment data involving Class A cigarettes are given priority. All data captured from the slave nodes are aggregated to the master node for unified processing. For example, the collected comment data is shown in Table 1. The content in the table has no practical significance and is only used as an example, but is not limited to this.

[0094] Table 1

[0095]

[0096]

[0097] Step S2013, semantic encoding is performed on the consumer review data to obtain a semantic feature vector.

[0098] Specifically, the Qwen large language model is used to generate a fixed-length semantic embedding vector for each collected comment data text, and the semantic embedding vector is used to represent the semantic information of the comment.

[0099]

[0100] Among them, x i Represents the semantic information of the i-th comment data text, is the text of the i-th comment data x i The semantic feature vector of LM is a large language model used for semantic encoding. The specific representation of the semantic feature vector is: (E 用户A =[0.5,0.1,-0.3,…])、(E 用户B =[0.2,-0.4,0.6,…]).

[0101] Step S2014, using the pre-trained large language model to perform semantic analysis on the semantic feature vector to obtain the noise in the semantic feature vector.

[0102] Specifically, based on the semantic feature vector, the Qwen large language model is used to evaluate the validity and relevance of each comment data. The large language model identifies invalid information and noise in the comments through context understanding and semantic analysis. The noise detection formula is:

[0103]

[0104] in, represents the noise score of the i-th comment data, f LM It is a feature function provided by the large language model, which is used to evaluate the noise score of the semantic feature vector. It is filtered according to the relationship between the noise score and the set threshold (τ). If the noise score is greater than the set threshold, the comment data is judged to be noise data and needs to be filtered out.

[0105] In step S2015, the consumer review data corresponding to the noise is filtered out as invalid review data to obtain valid review data, and the pre-trained large language model is dynamically adjusted according to the quality of the valid review data.

[0106] Specifically, we use the adaptive capabilities of the large language model to dynamically adjust the noise threshold τ according to the overall quality of the comments, so as to improve the effect of the model in filtering out noise data and better meet the analysis needs of users:

[0107] τ=μ+kσ

[0108] Wherein, μ is the average value of the noise scores of multiple comment data, σ is the standard deviation of the noise scores of multiple comment data, and k is an adjustment factor, which is set to 1 when the standard deviation is less than the average value, and is set to 2 when the standard deviation is greater than or equal to the average value. This is for example only, but is not limited to this.

[0109] The consumer demand analysis method based on the multi-objective optimization algorithm provided in this embodiment uses crawler technology to sequentially obtain comment data from multiple social platforms, which can quickly understand market dynamics and the latest feedback from consumers. The wide range of data sources can reflect diverse consumer opinions, avoiding sample limitations that lead to deviations in demand analysis results. More comprehensive market insights guide companies to adjust marketing strategies and product designs in a timely manner to better meet consumer needs. By cleaning the original consumer comment data, filtering out invalid comment data, improving the credibility and quality of valid comment data, reducing the processing of invalid data, saving computing resources, labor costs and time costs, and improving data processing efficiency.

[0110] Step S202, using a preset topic recognition model to analyze each valid comment data to obtain the topic probability distribution of the valid comment data in multiple preset topics, and using a preset sentiment analysis model to analyze the valid comment data to obtain the sentiment probability distribution of the valid comment data.

[0111] Specifically, the above step S202 includes:

[0112] Step S2021, extracting feature vectors from the valid comment data, and subject-clustering the feature vectors to obtain candidate subjects from the valid comment data.

[0113] Specifically, a large language model fine-tuned with LoRA is used to extract feature representations from the cleaned review data:

[0114] H i =Model LoRA (X i )

[0115] Among them, H i is the feature vector representation of the i-th valid comment data, X i is the input text of the i-th valid comment data. The expression of the comment feature vector is: (H 用户A =[0.6,-0.2,0.1,…])、(H 用户B =[0.4,0.3,-0.5,…]).

[0116] Use K-means clustering algorithm or hierarchical clustering algorithm to cluster H i The review feature vectors are clustered to identify implicit topic types in the review data (such as price, packaging, taste, etc.):

[0117] Cluster({H i})→{C 1 ,C 2 ,…,C k}

[0118] Among them, {C 1 ,C 2 ,…,C k} is the identified topic cluster, C 1 ,C 2 ,…,C k Indicates multiple alternative topics.

[0119] Step S2022, using a preset topic identification model, calculates the topic probability distribution of the valid comment data belonging to each candidate topic based on the feature vector.

[0120] Specifically, the Softmax function of the fine-tuned large language model is used to further calculate the probability distribution of each comment data belonging to different topic types:

[0121] θ ik =Softmax(W·H i +b)

[0122] Among them, θ ik is the probability that the ith comment belongs to the kth topic, and W and b are the parameters of the large language model. For example, the topic probability distribution of the comment "This cigarette tastes very good" is: taste (0.8), price (0.1), packaging (0.1).

[0123] From the topic probability distribution, the main topic of each comment data is extracted according to the size of the probability value, and each comment data is labeled, and the topic is divided into categories such as "taste", "price", and "brand influence". These labels reflect the core content of the topic and are convenient for users to analyze and understand. For example, the topic probability distribution of the comment "This cigarette tastes very good" is: taste (0.8), price (0.1), packaging (0.1), which is judged as "taste".

[0124]

[0125] Among them, represents the main topic of the i-th comment data.

[0126] Step S2023, based on the topic probability distribution of the valid comment data, the valid comment data is analyzed using a preset sentiment analysis model to obtain the sentiment probability distribution corresponding to each candidate topic.

[0127] Specifically, the fine-tuned Qwen model is used to extract features from the comment data after topic analysis so that sentiment analysis can be performed on the comment data in the next step:

[0128] H i =Model Pre-trained (X i ,Topic i )

[0129] Among them, H i is the sentiment feature vector after combining the topic, X i is the text of the i-th comment data, Topic i It is the topic information of the i-th comment data text.

[0130] The output layer of the Qwen model is used to transform the above sentiment feature vector H i The sentiment probability distribution of each comment data is analyzed to indicate the distribution of comments in different sentiment categories. For example, the sentiment probability distribution of the comment "This cigarette tastes very good" is: positive (0.9), neutral (0.05), negative (0.05).

[0131] P i =Softmax(W o ·H i +b o )

[0132] Among them, W o and b o is the output layer parameter of the preset sentiment analysis model, P i is the probability distribution of the i-th comment belonging to different sentiment categories.

[0133] From the sentiment probability distribution, extract the main sentiment tendency of the comments (such as positive, neutral, negative) according to the size of the probability value:

[0134]

[0135] Among them, P ik represents the predicted probability of the i-th comment data in sentiment category k, Sentiment i is the sentiment category of the ith comment.

[0136] For example, the sentiment probability distribution of the review data text "This cigarette tastes very good" is: positive (0.9), neutral (0.05), negative (0.05), and the main sentiment tendency is "positive".

[0137] The consumer demand analysis method based on the multi-objective optimization algorithm provided in this embodiment, by identifying the theme of the effective review data, classifies the core content of the target product that the user focuses on, and facilitates rapid analysis and accurate understanding of consumer demand. By performing sentiment analysis on the effective review data with a determined theme, it is determined whether the consumer has a positive, neutral or negative emotion on the theme, and then determines whether the target product is popular under the theme, which is conducive to accurately judging consumer demand and guiding product improvement.

[0138] Step S203, combining the topic probability distribution and the sentiment probability distribution of each valid comment data respectively to obtain an overall joint probability matrix.

[0139] Specifically, the above step S203 includes:

[0140] Step S2031, performing outer product of the topic probability distribution and the sentiment probability distribution of the valid comment data to obtain the single joint probability corresponding to the topic and the sentiment in the valid comment data.

[0141] Specifically, according to step S103 in the previous embodiment, the topic probability distribution of the i-th comment data is: T i =[T i1 ,T i2 ,…,T in ], the emotional probability distribution is: E i =[E i,neg ,E i,neu ,E i,pos ]. Combine the topic probability distribution and sentiment probability distribution to construct the joint probability distribution matrix J for each comment i :

[0142]

[0143] in, represents the outer product operation, and the result is a (n×3) matrix, which represents the single joint probability of each topic under different emotions. Assume that there is a review data on the cigarette evaluation of product A, and after the topic is identified, the topic probability distribution is:

[0144] T 1 =[0.2,0.3,0.5]

[0145] Among them, T 1 Theme 1 represents price, theme 2 represents taste, and theme 3 represents packaging.

[0146] The results of sentiment analysis are:

[0147] E 1 =[0.1,0.2,0.7]

[0148] Among them, 0.1, 0.2, and 0.7 represent the probabilities of negative, neutral, and positive sentiments, respectively. Then, the comment (d 1 ) is:

[0149]

[0150] Step S2032, aggregate the individual joint probabilities of each valid comment data to obtain an overall joint probability matrix.

[0151] Specifically, the joint probability distribution matrix J of all comments i Aggregate and get the overall joint probability distribution matrix J:

[0152]

[0153] Among them, m represents the total number of comment data, J is a (n×3) matrix, and each element J jk It represents the joint probability of the overall consumers under topic j and sentiment k, that is, the probability value of topic j and sentiment k appearing together.

[0154] The consumer demand analysis method based on a multi-objective optimization algorithm provided in this embodiment combines the topic probability distribution with the sentiment probability distribution to obtain an overall joint probability matrix of each valid comment data, which is conducive to comprehensive analysis from both the topic and sentiment aspects to obtain more accurate consumer demand analysis results.

[0155] Step S204, based on the overall joint probability matrix, the particle swarm algorithm is used as a multi-objective optimization algorithm to calculate the optimal weight matrix.

[0156] Specifically, the above step S204 includes:

[0157] The weights of different topics and emotions in the overall joint probability matrix are assigned, and the optimal weight matrix is ​​obtained using a multi-objective optimization algorithm. The formula of the multi-objective optimization algorithm is:

[0158]

[0159] Among them, w jk =[w 11 ,w 12 ,…,w n3 ] is the weight of each element in the joint probability distribution matrix, satisfying ( And w jk ≥0), J jk represents the element in the overall joint probability distribution matrix, j represents the topic, and k represents the sentiment. For each w jk Set the initial value, usually J jk The mean of the matrix is ​​used as the initial value, and the optimization is performed step by step, and the weights are continuously adjusted to maximize the objective function. When the maximum number of iterations is reached, the corresponding weight is used as the optimal weight.

[0160] The consumer demand analysis method based on a multi-objective optimization algorithm provided in this embodiment determines the intensity of consumer demand on different topics by assigning optimal weight distribution to each element in the overall joint probability matrix, which is conducive to accurately understanding consumers' demand preferences for target products, adjusting product strategies accordingly, and providing better services to consumers.

[0161] Step S205: Multiply the overall joint probability matrix and the optimal weight matrix to obtain the demand analysis result of the target product. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.

[0162] The consumer demand analysis method based on a multi-objective optimization algorithm provided in this embodiment improves the coverage and real-time performance of data collection by acquiring and analyzing consumer review data, deeply explores consumer demand and preference information, comprehensively and dynamically grasps consumer demand preferences for target products, and provides a more accurate and timely basis for the company's market decision-making.

[0163] In a specific embodiment, product A has three related review data d 1 d 2 d 3 , the corresponding single joint probability distribution matrices are J 1 , J 2 , J 3 , its specific value is:

[0164]

[0165] Then the overall joint probability matrix is:

[0166]

[0167] The objective function max is calculated by particle swarm optimization. w f(w,J) is solved, and when the maximum number of iterations is reached, the corresponding weight is used as the optimal weight matrix.

[0168]

[0169] Using the optimal weight matrix w * The weighted demand preference distribution of the overall consumer is calculated as:

[0170]

[0171] The demand intensity P of each topic is calculated by multiplying the joint probability distribution matrix J by the optimal weight matrix w* item by item j . For each row j:

[0172] Subject 1: P1 = 0.02 × 0.2 + 0.04 × 0.3 + 0.0933 × 0.5 = 0.004 + 0.012 + 0.04665 = 0.06265;

[0173] Subject 2: P2 = 0.03 × 0.1 + 0.06 x 0.2 + 0.15 × 0.7 = 0.003 + 0.012 + 0.105 = 0.12;

[0174] Subject 3: P3 = 0.03 × 0.4 + 0.0667 × 0.1 + 0.18 × 0.5 = 0.012 + 0.00667 + 0.09 = 0.10867.

[0175] According to the weighted demand preference distribution of the above-mentioned overall consumers, the demand intensity of consumers for product A on themes such as price, taste and packaging is generated. The larger the value, the more attention consumers pay to the theme. In this embodiment, according to the set theme order (packaging, taste, price), it is calculated that the demand intensity of taste is the highest (0.12), followed by price (0.109), and packaging is relatively low (0.063). Product A can adjust its product strategy accordingly, focusing on the optimization of price and taste.

[0176] In the present embodiment, a consumer demand analysis device based on a multi-objective optimization algorithm is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0177] This embodiment provides a consumer demand analysis device based on a multi-objective optimization algorithm, such as Figure 3 As shown, including:

[0178] The effective review data acquisition module 301 is used to acquire multiple consumer review data of a target product and pre-process the consumer review data to obtain multiple effective review data.

[0179] The comment data analysis module 302 is used to analyze each valid comment data using a preset topic recognition model to obtain the topic probability distribution of the valid comment data in multiple preset topics, and to analyze the valid comment data using a preset sentiment analysis model to obtain the sentiment probability distribution of the valid comment data.

[0180] The joint probability calculation module 303 is used to combine the topic probability distribution and the sentiment probability distribution of each valid comment data to obtain an overall joint probability matrix.

[0181] The weight matrix calculation module 304 is used to calculate the optimal weight matrix based on the overall joint probability matrix using a multi-objective optimization algorithm.

[0182] The demand analysis module 305 is used to determine the demand intensity of the target product on each topic based on the optimal weight matrix and the overall joint probability matrix.

[0183] In some optional implementations, the effective comment data acquisition module 301 includes:

[0184] The platform ranking unit is used to determine the order of social platforms for obtaining consumer review data based on the current status and preset priority of each preset social platform.

[0185] The de-duplication acquisition unit is used to obtain multiple consumer review data from each preset social platform in the order of the social platform based on the keywords of the target product using crawler technology.

[0186] The semantic encoding unit is used to semantically encode the consumer review data to obtain a semantic feature vector.

[0187] The semantic analysis unit is used to perform semantic analysis on the semantic feature vector using a pre-trained large language model to obtain the noise in the semantic feature vector.

[0188] The invalid filtering unit is used to filter out the consumer review data corresponding to the noise as invalid review data to obtain valid review data. The pre-trained large language model is dynamically adjusted according to the quality of the valid review data.

[0189] In some optional implementations, the review data analysis module 302 includes:

[0190] The topic clustering unit is used to extract feature vectors from the valid comment data and perform topic clustering on the feature vectors to obtain candidate topics in the valid comment data.

[0191] The topic probability distribution determination unit is used to use a preset topic recognition model to calculate the topic probability distribution of valid comment data belonging to each candidate topic based on the feature vector.

[0192] The sentiment probability distribution determination unit is used to analyze the valid comment data based on the topic probability distribution of the valid comment data using a preset sentiment analysis model to obtain the sentiment probability distribution corresponding to each candidate topic.

[0193] In some optional implementations, the joint probability calculation module 303 includes:

[0194] The probability distribution outer product unit is used to perform the outer product of the topic probability distribution and the sentiment probability distribution of the valid comment data to obtain the single joint probability corresponding to the topic and the sentiment in the valid comment data.

[0195] The joint probability aggregation unit is used to aggregate the individual joint probabilities of each valid comment data to obtain an overall joint probability matrix.

[0196] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0197] The consumer demand analysis device based on the multi-objective optimization algorithm in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0198] The embodiment of the present invention also provides a computer device having the above Figure 3 The consumer demand analysis device based on the multi-objective optimization algorithm is shown.

[0199] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0200] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0201] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0202] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0203] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0204] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0205] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0206] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A consumer demand analysis method based on a multi-objective optimization algorithm, characterized in that: The method comprises: Acquire multiple consumer review data of a target product, and pre-process the consumer review data to obtain multiple valid review data; Analyze each valid comment data using a preset topic recognition model to obtain a topic probability distribution of the valid comment data in multiple preset topics, and analyze the valid comment data using a preset sentiment analysis model to obtain a sentiment probability distribution of the valid comment data; Combine the topic probability distribution and sentiment probability distribution of each valid comment data to obtain the overall joint probability matrix; Based on the overall joint probability matrix, an optimal weight matrix is ​​calculated using a multi-objective optimization algorithm; The demand intensity of the target product on each topic is determined according to the optimal weight matrix and the overall joint probability matrix.

2. The method according to claim 1, characterized in that Get multiple consumer review data for the target product, including: Determine the order of social platforms for obtaining consumer review data based on the current status and preset priority of each preset social platform; According to the keywords of the target product, crawler technology is used to obtain multiple consumer review data of each preset social platform in the order of the social platforms.

3. The method according to claim 1, characterized in that: The consumer review data is preprocessed to obtain a plurality of valid review data, including: Performing semantic encoding on the consumer review data to obtain a semantic feature vector; Performing semantic analysis on the semantic feature vector using a pre-trained large language model to obtain noise in the semantic feature vector; The consumer review data corresponding to the noise is filtered out as invalid review data to obtain valid review data, and the pre-trained large language model is dynamically adjusted according to the quality of the valid review data.

4. The method according to claim 1, characterized in that: Analyze each valid comment data using a preset topic recognition model to obtain the topic probability distribution of the valid comment data in multiple preset topics, including: Extracting feature vectors from the valid comment data, and subject-clustering the feature vectors to obtain candidate subjects from the valid comment data; The preset topic recognition model is used to calculate the topic probability distribution of the valid comment data belonging to each candidate topic according to the feature vector.

5. The method according to claim 4, characterized in that The valid comment data is analyzed using a preset sentiment analysis model to obtain the sentiment probability distribution of the valid comment data, including: Based on the topic probability distribution of the valid comment data, the valid comment data is analyzed using a preset sentiment analysis model to obtain the sentiment probability distribution corresponding to each candidate topic.

6. The method according to claim 1, characterized in that Combine the topic probability distribution and sentiment probability distribution of each valid comment data to obtain the overall joint probability matrix, including: Perform an outer product of the topic probability distribution and the sentiment probability distribution of the valid comment data to obtain a single joint probability corresponding to the topic and the sentiment in the valid comment data; Aggregate the individual joint probabilities of each valid comment data to obtain the overall joint probability matrix.

7. The method according to claim 6, characterized in that Based on the overall joint probability matrix, the optimal weight matrix is ​​calculated using a multi-objective optimization algorithm, including: Weights are assigned to different topics and emotions in the overall joint probability matrix, and the optimal weight matrix is ​​obtained using a multi-objective optimization algorithm. The formula of the multi-objective optimization algorithm is: Among them, w jk =[w 11 ,w 12 ,…,w n3 ] is the weight of each element in the joint probability distribution matrix, satisfying ( And w jk ≥0), J jk represents the element in the joint probability distribution matrix, j represents the topic, and k represents the sentiment tendency.

8. A consumer demand analysis device based on a multi-objective optimization algorithm, characterized in that: The device comprises: An effective review data acquisition module is used to acquire multiple consumer review data of a target product and pre-process the consumer review data to obtain multiple effective review data; A comment data analysis module is used to analyze each valid comment data using a preset topic recognition model to obtain a topic probability distribution of the valid comment data in multiple preset topics, and to analyze the valid comment data using a preset sentiment analysis model to obtain a sentiment probability distribution of the valid comment data; The joint probability calculation module is used to combine the topic probability distribution and sentiment probability distribution of each valid comment data to obtain the overall joint probability matrix; A weight matrix calculation module, used to calculate an optimal weight matrix based on the overall joint probability matrix using a multi-objective optimization algorithm; The demand analysis module is used to determine the demand intensity of the target product on each topic based on the optimal weight matrix and the overall joint probability matrix.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images