Product Label Extraction Method Based on Internet Big Data and AI Large Language Model

Abstract

The present invention relates to the technical field of product label extraction, and specifically, to a product label extraction method based on Internet big data and AI large language models. It includes the following steps: S1. Use web crawler technology to capture the text data of products on the Internet; S2. Adopt the TF-IDF algorithm to determine the important words in the text data, and combine the Skip-Gram model to capture the semantic associations between words. In the process of capturing the semantic associations between words, introduce the weight reflecting the user browsing frequency and the user behavior feature vector to optimize the capturing process; S3. Based on the extracted important words and the semantic association information between words, use a large-scale pre-trained language model to generate product labels; S4. Combine the sequence annotation model BERT and the conditional random field CRF to locate and classify the product labels, and output the finally extracted product labels. The technology of the present invention can effectively locate and classify product labels by combining the BERT model and the conditional random field (CRF) layer.

Description

Technical Field

[0001] The present invention relates to the technical field of product label extraction, and specifically, to a product label extraction method based on Internet big data and AI large language models. Background Art

[0002] The keyword matching and rule-based algorithms of traditional label extraction methods rely on predefined thesauruses and language rules. Although simple and direct, they often perform inadequately when facing language diversity and complex semantics. Secondly, the feature expression ability of traditional machine learning models is limited, and it is difficult to capture the deep semantic relationships between contexts, resulting in the extraction effect being difficult to meet the actual needs. In addition, the scale of Internet data is huge, diverse in form (such as structured data and unstructured text), with significant differences in semantic expressions, and at the same time, hotspots and user needs change rapidly, making traditional methods appear inefficient and insufficient in adaptability. Therefore, a product label extraction method based on Internet big data and AI large language models is provided. Summary of the Invention

[0003] The purpose of the present invention is to provide a product label extraction method based on Internet big data and AI large language models to solve the limitations in the traditional product label extraction process as mentioned in the above background art.

[0004] To achieve the above purpose, the present invention aims to provide a product label extraction method based on Internet big data and AI large language models, including the following steps:

[0005] S1. Use web crawler technology to crawl the text data of products on the Internet;

[0006] S2. Adopt the TF-IDF algorithm to determine the important words in the text data, and combine the Skip-Gram model to capture the semantic associations between words. In the process of capturing the semantic associations between words, introduce the weight reflecting the user browsing frequency and the user behavior feature vector to optimize the capture process;

[0007] S3. Based on the extracted important words and the semantic association information between words, use a large-scale pre-trained language model to generate product labels;

[0008] S4. Combine the sequence annotation model BERT and the conditional random field CRF to locate and classify the product labels, and output the finally extracted product labels.

[0009] As a further improvement of this technical solution, in S2, adopting the TF-IDF algorithm to determine the important words in the text information includes the following steps:

[0010] S2.1. Preprocess the text data of the product and organize the captured text data into a corpus D, where D contains N documents d, D = {d1, d2, d3, …, d N};

[0011] S2.2. Calculate the term frequency TF(t, d) of each word t in document d:

[0012]

[0013] where cou(t, d) represents the number of occurrences of word t in document d, and ∑ w∈d (w, d) represents the total number of occurrences of all words in document d; w represents each word in document d;

[0014] S2.3. Calculate the inverse document frequency IDF(t, D) of word t in the entire corpus:

[0015]

[0016] where N represents the total number of documents in the corpus, and |{d ∈ D: t ∈ d}| represents the number of documents containing word t;

[0017] S2.4. Combine the term frequency and the inverse document frequency to obtain the importance weight TF-IDF(t, d, D) of word t in document d:

[0018] TF-IDF(t, d, D) = TF(t, d) · IDF(t, D).

[0019] As a further improvement of this technical solution, in S2, the Skip-Gram model is combined to capture the semantic associations between words, including the following steps:

[0020] S2.5. Build a vocabulary based on the preprocessed text data, and record all unique words that appear in the text and their frequencies;

[0021] S2.6. Set a fixed window size c. For each center word, take c words on each side of the center word as context words to form training pairs;

[0022] S2.7. Randomly initialize low-dimensional vectors for each word in the vocabulary and set hyperparameters;

[0023] S2.8. Set the objective function to train the Skip-Gram model, assign weights to the product word vectors to reflect the browsing frequency, introduce the user behavior feature vectors u i and u o into the objective function for optimization, and introduce the associated products as additional positive examples into the objective function for further optimization;

[0024] S2.9. The trained Skip-Gram model maps each word into a high-dimensional vector space to obtain the word vector v of each word t ;

[0025] S2.10. Combine the word frequency, inverse document frequency and the word vector v t , to form a new product word vector v d :

[0026]

[0027] S2.11. Calculate the cosine similarity of word vectors to measure the semantic similarity between words. For words with similar contexts, introduce a clustering method in the calculation process of the cosine similarity of word vectors, and cluster semantically similar words into the same class through clustering.

[0028] As a further improvement of this technical solution, in S2.8, the objective function is:

[0029]

[0030] where L neg represents the optimized objective function; σ(*) represents the Sigmoid function; v o represents the vector representation of the context word; v i represents the vector representation of the center word; k represents the number of negative samples; v j represents the vector representation of the negative sample word; w j represents the negative sample word; represents the expected value randomly selected according to a certain word distribution P(w) in the vocabulary; P(w) represents the word distribution; T represents the transpose of the vector; i represents the index of the center word; o represents the index of the context word; j represents the index of the negative sample word;

[0031] When a product frequently appears in the user's browsing path, assign a weight reflecting the browsing frequency to the product word vector, establish a user behavior matrix, and extract two user behavior feature vectors u i and u o , and introduce the user behavior feature vectors u i and u o into the objective function for optimization:

[0032]

[0033]

[0034] where L neg，uesr represents the objective function optimized for the user's browsing path; u iDenote the user behavior feature vector corresponding to the target word; u o Denote the user behavior feature vector corresponding to the context word; bp u，r Denote the browsing frequency of user u for the central product r; bp u，r1 Denote the browsing frequency of user u for the negative sample product r1;

[0035] If multiple products often appear in the browsing path of the same user, then the products are considered to be associated. For each central word v i , find the products that are often viewed together by the same user group according to the user's browsing history, and introduce the associated products as additional positive examples into the objective function for further optimization:

[0036]

[0037] Among them, L neg，be Denote the objective function after further optimization; B denotes a set of additional positive examples selected based on the user's browsing path; v b Denote the word vector corresponding to the additional positive example; u b Denote the user behavior feature vector corresponding to the additional positive example; α denotes the hyperparameter that controls the contribution degree of the user behavior positive example; b denotes the index of the additional positive example.

[0038] As a further improvement of this technical solution, in the S2.11, calculating the cosine similarity of word vectors to measure the semantic similarity between words is:

[0039]

[0040] Among them, Sim(v c , v e ) denotes the cosine similarity of word vectors; v c Denote the word vector representation of word c; v e Denote the word vector representation of word e;

[0041] For words with similar contexts, introduce a clustering method in the calculation process of the cosine similarity of word vectors, and cluster the semantically similar words into the same class:

[0042]

[0043] Among them, Sim1(v c , v e ) denotes the optimized cosine similarity of word vectors; CL(v c , v e ) denotes the similarity calculated based on the lexical clustering information; γ denotes the weighting coefficient of the clustering information.

[0044] As a further improvement of this technical solution, in step S3, based on the extracted important words and the semantic association information between the words, a large-scale pre-trained language model is used to generate product labels, including the following steps:

[0045] S3.1. Construct an input sequence X=(x1, x2, …, x n ) based on the extracted important words and the semantic association information between the words, and input the input sequence into the pre-trained language model;

[0046] S3.2. The pre-trained language model predicts the next word at each position in an autoregressive manner to generate the complete text:

[0047]

[0048] Among them, represents the generated product label; y n+1 represents the next generated word; P(y n+1 |X) represents the conditional probability distribution of the next word y n+1 calculated by the model in an autoregressive manner;

[0049] S3.3. Generate the probability distribution of the product label according to the input sequence and the already trained model parameters θ;

[0050] S3.4. Adopt a temperature sampling strategy to extract label candidates from the conditional probability distribution.

[0051] As a further improvement of this technical solution, in step S3.3, the probability distribution of the product label is:

[0052]

[0053] Among them, represents the conditional probability distribution of the generated product label ; n represents the length of the input sequence; t represents the position index in the process of generating the product label.

[0054] As a further improvement of this technical solution, in step S4, the sequence annotation model BERT and the conditional random field CRF are combined to locate and classify the product label, including the following steps:

[0055] S4.1. Sort out the product label candidates generated by the pre-trained language model;

[0056] S4.2. Create a corresponding feature vector T' h ={t' h，1 , t' h，2 , …, t' h，m} for each product label candidate, where T' hDenote the product label set after preliminary screening as \(t'\). h，m Denote the \(m\)-th product label candidate;

[0057] S4.3. Input the product label candidate sequence \(T'\) h into the BERT model. The BERT model encodes each product label candidate according to the context to capture the semantic features of the product label candidate:

[0058]

[0059] where \(R\) m denotes the BERT encoding result of the product label candidate \(t'\) h，m ; \(\theta\) denotes the parameters of the BERT model;

[0060] S4.4. Define the product label space \(L = \{l_1, l_2, \ldots, l_k\}\), where \(k\) represents the index of the product label category; k

[0061] S4.5. Apply the conditional random field layer to calculate the transition probability matrix \(A\) between labels. The transition probability matrix represents the probability of transitioning from one product label to another:

[0062] \(A = [a_{ij}]\); hm

[0063]

[0064] where \([a_{ij}]\) hm denotes the transition probability from the product label space \(l_i\) g to the product label space \(l_j\), k ; \(\psi\) denotes the parameters of the conditional random field layer; \(g\) represents the index of the product label category;

[0065] S4.6. Combine the BERT encoding result \(R\) m and the transition probabilities of the conditional random field layer to determine the optimal label sequence:

[0066]

[0067] where \(T''\) h denotes the finally determined product label set; \(t''_m\) h，m denotes the \(m\)-th final product label; \(m_1\) represents the total length of the product label sequence;

[0068] S4.7. Use the Viterbi algorithm to find the product label sequence with the highest probability.

[0069] ​​As a further improvement of this technical solution, in S4.7, the Viterbi algorithm is used to find the product label sequence with the highest probability, including the following steps:

[0070] S4.71. For the first product label candidate, calculate the initial probability of each possible product label:

[0071]

[0072] where l represents one of the product label categories; δ1(l) represents the initial probability of product label l under the condition of the given first product label candidate t” h，1 ;

[0073] S4.72. Initialize the backtracking pointer for each product label

[0074]

[0075] S4.73. For the e1-th product label candidate in the sequence, calculate the maximum cumulative probability of each possible product label and update the backtracking pointer;

[0076] S4.74. At the end of the sequence, select the product label with the highest cumulative probability as the last product label z m :

[0077] z m = argmax l δ m (l);

[0078] S4.75. Starting from the last product label, gradually backtrack forward according to the backtracking pointer to reconstruct the entire optimal product label sequence:

[0079] S4.76. After reconstruction, obtain the optimal product label sequence T” h , that is, the finally extracted product label:

[0080] T” h = (z1, z2,..., z m ).

[0081] As a further improvement of this technical solution, in S4.73, the calculation of the maximum cumulative probability of each possible product label is:

[0082]

[0083] where δ e1 (l) represents the probability of the best path reaching the e1-th position with product label l under the observation condition at the e1-th position; δe1-1 P(l') represents the probability of the best path that reaches the (e1 - 1)-th position with product label l' under the observation conditions at the given (e1 - 1)-th position; represents the maximum cumulative probability of product label l under the condition of the given (e1)-th product label candidate t”; h，e1 represents the pointer pointing to the previous optimal product label; a l'，l represents the transition probability from product label l' to product label l; t' h，e1 represents the (e1)-th product label candidate; t” h，e1 represents the finally determined (e1)-th product label.

[0084] Advantages of the present invention compared with the prior art:

[0085] 1. In the product label extraction method based on Internet big data and AI large language models, by combining the TF-IDF algorithm, Skip-Gram model, and large-scale pre-trained language models (such as GPT series), this method can identify and extract important words from a large amount of text data and capture the semantic associations between these words. This method not only considers the frequency of word occurrences but also attaches importance to the distribution of words in different documents and the context environment, thus more accurately reflecting the characteristics of products and the concerns of users. In addition, introducing user behavior data as additional positive examples to optimize the objective function further enhances the model's understanding of the potential relationships between products, making the generated product labels more in line with actual needs and improving the accuracy and semantic richness of the labels.

[0086] 2. In the product label extraction method based on Internet big data and AI large language models, by using the combination of BERT model and conditional random field (CRF) layer, the product labels can be effectively located and classified. The BERT model can provide a context-sensitive representation for each label candidate according to the context, while the CRF layer can calculate the transition probabilities between labels to ensure that the finally output label sequence not only conforms to the definition of individual labels but also has logical consistency and coherence in the whole sequence. The application of the Viterbi algorithm ensures the selection of the label sequence with the highest probability, which helps to improve the intelligent level of label classification and at the same time ensures the consistency and professionalism of the label system, facilitating subsequent data analysis and business decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 is the overall method flowchart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0088] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0089] Please refer to Figure 1 As shown, this embodiment provides a product label extraction method based on Internet big data and AI large language models, including the following steps:

[0090] S1. Use web crawler technology to capture the text data of products on the Internet, and the text data includes e-commerce platforms, product reviews, user feedback, product descriptions, etc.;

[0091] S2. Adopt the TF-IDF algorithm to determine the important words in the text data, and combine the Skip-Gram model to capture the semantic associations between words. In the process of capturing the semantic associations between words, introduce the weight reflecting the user browsing frequency and the user behavior feature vector to optimize the capture process;

[0092] In this embodiment, TF-IDF is a statistical method used to evaluate the importance of a word in a document or a set of documents. It is calculated by combining the term frequency (the frequency of a word appearing in a document) and the inverse document frequency (the rarity of a word appearing in the entire document collection); the main purpose of using the TF-IDF algorithm to determine the important words in the text information is to identify those words that appear frequently in a specific document but are relatively uncommon in the entire corpus. These words often have a high degree of distinctiveness and can well represent the theme or content of the document; by calculating the term frequency (TF) and inverse document frequency (IDF) of each word, a weight value can be assigned to each word. This weight reflects the importance of the word for a certain document, that is, its significance in that document; some very common words (such as stop words like "of", "is", "in", etc.) may appear frequently in most documents, but they contribute little to understanding the document theme. Through the IDF part, TF-IDF effectively reduces the weights of these general words, thereby reducing their impact on the analysis results;

[0093] Adopting the TF-IDF algorithm to determine the important words in the text information includes the following steps:

[0094] S2.1. Preprocess the text data of the product, and organize the captured text data into a corpus D, where D contains N documents d, D = {d1, d2, d3, …, d N};

[0095] S2.2. Calculate the term frequency TF(t, d) of each word t in document d:

[0096]

[0097] where cou(t, d) represents the number of occurrences of word t in document d, and Σ w∈d (w, d) represents the total number of occurrences of all words in document d; w represents each word in document d;

[0098] S2.3. Calculate the inverse document frequency IDF(t, D) of word t in the entire corpus:

[0099]

[0100] where N represents the total number of documents in the corpus, and |{d ∈ D: t ∈ d}| represents the number of documents containing word t;

[0101] S2.4. Combine the term frequency and the inverse document frequency to obtain the importance weight TF-IDF(t, d, D) of word t in document d:

[0102] TF-IDF(t, d, D) = TF(t, d) · IDF(t, D).

[0103] Among them, the Skip-Gram model is a neural network model for training word vectors. It learns the distributed representation of words by predicting the context words around a given word; by constructing and training the word vector model, Skip-Gram can map semantically similar or related words to nearby positions in a high-dimensional space, even if these words are completely different literally. This helps to identify synonyms, hyponymy relationships, and contextually similar words; compared with the method that only uses term frequency statistics, word vectors provide a richer feature representation, including the context information of the vocabulary. This is very important for tasks such as product label extraction, because it allows the system to infer the meaning of words according to the patterns in which they are used in different contexts; for some uncommon but possibly very important words, traditional statistical methods may underestimate their importance. And the Skip-Gram model can use the surrounding context information to enhance the understanding of these words, even if they appear infrequently;

[0104] Combining the Skip-Gram model to capture the semantic associations between words includes the following steps:

[0105] S2.5. Build a vocabulary based on the preprocessed text data, recording all unique words that appear in the text and their frequencies;

[0106] S2.6. Set a fixed window size c to determine the context range around the current word. For each center word, take c words on both the left and right sides of the center word as context words to form training pairs;

[0107] S2.7. Randomly initialize low-dimensional vectors for each word in the vocabulary, usually with dimensions between 50 and 300, and set hyperparameters, including the window size, vector dimension, minimum word frequency, learning rate, etc.;

[0108] S2.8. Set the objective function to train the Skip-Gram model. The objective function is used to maximize the probability of predicting the correct context word given the center word, assign weights to the product word vectors to reflect the browsing frequency, introduce the user behavior feature vectors u i and u o into the objective function for optimization, and introduce the associated products as additional positive examples into the objective function for further optimization;

[0109] Furthermore, the objective function is:

[0110]

[0111] where L neg represents the optimized objective (loss) function, and specifically outputs a scalar value, which represents the error degree between the prediction result of the model on the given training samples and the actual labels; σ(*) represents the Sigmoid function, which is used to map real values to the range (0,1); v o represents the vector representation of the context word; v i represents the vector representation of the center word; k represents the number of negative samples; v j represents the vector representation of the negative sample word; w j represents the negative sample word, which is drawn from the vocabulary distribution P(w) and is not the true context word of the center word v i ; represents the expected value randomly drawn from the vocabulary according to a certain vocabulary distribution P(w), and is actually approximately calculated by random sampling during the training process; P(w) represents the vocabulary distribution; T represents the transpose of the vector; i represents the index of the center word; o represents the index of the context word; j represents the index of the negative sample word;

[0112] When users frequently access certain products in the browsing path, it indicates that these products may have some internal connections. For example, they may be complementary products, substitutes, or belong to the same category. This co-occurrence pattern can help us discover semantic associations that exist in the text data but are not explicitly stated. Through the browsing path, we can identify which products usually appear in the same scenario (such as a single shopping session), thus providing more background information for the labels of these products and helping to capture their semantic features more accurately. Users' browsing behaviors often reveal their current needs or interests. Even if this information is not directly expressed in comments or descriptions, it can be used as supplementary clues to enrich the semantic representation of relevant words. For example, if many users are viewing the new models of a certain brand of mobile phones, then words such as "new model", "mobile phone", and the name of the brand may gain additional important weight as a result;

[0113] When a product frequently appears in the user's browsing path, assign a weight to the product word vector that reflects the browsing frequency, establish a user behavior matrix (each row of the user behavior matrix represents the behavior pattern of a user, and each column corresponds to a product), and extract two user behavior feature vectors u i and u o from the user behavior matrix, and introduce the user behavior feature vectors u i and u o into the objective function for optimization:

[0114]

[0115] where L neg，uesr represents the objective (loss) function after optimization for the user's browsing path; u i represents the user behavior feature vector corresponding to the target word; u o represents the user behavior feature vector corresponding to the context word; bp u，r represents the browsing frequency of user u for the central product r; bp u，r1 represents the browsing frequency of user u for the negative sample product r1;

[0116] Additional positive examples refer to products (or goods) that are considered "additional positive examples" related to the current product (or good) based on the user's browsing history, especially the products that the user has viewed, clicked on, or browsed within a certain period of time. The additional positive examples generated by combining the user's browsing path refer to using the user's browsing behavior data (such as clickstream, browsing history) to identify products or pages that are frequently viewed together by the same user or a similar user group, and considering these products or pages as positive examples of each other. In other words, if two or more products frequently appear in the same user's browsing path, they can be considered related or have similar attributes, even if such a relationship is not explicitly stated in the text description;

[0117] If multiple products frequently appear in the same user's browsing path, then the products are considered related or have similar attributes. For each central word v i , find the products that are frequently viewed together by the same user group according to the user's browsing history, and introduce the related products as additional positive examples into the objective function for further optimization:

[0118]

[0119] where L neg，be represents the objective (loss) function after further optimization; B represents a set of additional positive examples selected based on the user's browsing path; v b represents the word vector corresponding to the additional positive example; u b represents the user behavior feature vector corresponding to the additional positive example; α represents the hyperparameter that controls the contribution degree of the user behavior positive example, which is used to balance the weights between traditional positive examples and user behavior positive examples; b represents the index of the additional positive example.

[0120] S2.9. After training, the Skip-Gram model maps each word into a high-dimensional vector space to obtain the word vector v t of each word. Each word vector contains the latent semantic features of the word in the corpus. Through these word vectors, semantic relationships between words can be captured, such as synonyms, hyponymy relationships, etc.;

[0121] S2.10. Combine the word frequency, inverse document frequency, and the word vector v t to form a new product word vector v d :

[0122]

[0123] S2.11. Calculate the cosine similarity of word vectors to measure the semantic similarity between words. For words with similar contexts, introduce a clustering method in the calculation of the cosine similarity of word vectors, and cluster the semantically similar words into the same class;

[0124] Furthermore, the main purpose of calculating the cosine similarity of word vectors to measure the semantic similarity between words is to quantify and compare the semantic relationships between different words, so as to more accurately understand and utilize the meaning of words in various natural language processing tasks; by calculating the cosine value of the angle between two word vectors, their relative positions in the high-dimensional space can be obtained, which reflects the degree of semantic similarity between these two words. The higher the similarity score, the more likely it means that the two words often appear in similar contexts in the corpus, and thus may have similar or related meanings; in tasks such as product label extraction, using cosine similarity can help identify the words that are most closely related to the target concept, ensuring that the generated labels are both accurate and representative, while avoiding the limitations brought by relying solely on frequency statistics;

[0125] The calculation of the cosine similarity of word vectors to measure the semantic similarity between words is as follows:

[0126]

[0127] where Sim(v c , v e ) represents the cosine similarity of word vectors; v c represents the word vector representation of word c; v e represents the word vector representation of word e;

[0128] In natural language processing, traditional cosine similarity calculation judges the similarity of words based on the angle difference between word vectors. However, for many words, although they are semantically similar, their contexts may be very close, resulting in the traditional cosine similarity calculation being unable to accurately distinguish the subtle differences between them. Especially for synonyms, hyponymy relationships, or words with similar contexts, their vectors may be very close, resulting in the traditional cosine similarity being unable to effectively identify them. For example, "smartphone" and "mobile phone" are very close semantically, but may have different applications or importance in different contexts. The traditional cosine similarity calculation will consider them to have a very high similarity, while ignoring the differences in their usage scenarios; introducing a clustering method, by clustering, words that are semantically similar are grouped into the same class, so that when calculating the similarity, the clustering information between words is considered. In this way, the semantic discrimination ability of words can be enhanced using the clustering results, and the relationship between words can be more accurately reflected when calculating the similarity;

[0129] For words with similar contexts, in the process of calculating the cosine similarity of word vectors, a clustering method is introduced, and words that are semantically similar are grouped into the same class by clustering:

[0130]

[0131] Among them, Sim1(v c , v e ) represents the cosine similarity of the optimized word vectors; CL(v c , v e ) represents the similarity calculated based on the lexical clustering information; γ represents the weighted coefficient of the clustering information; G represents the set of all clusters, and each g is a cluster; represents the indicator function, indicating whether the word c and the word e belong to the same cluster g; S g (c, e) represents the similarity measure of the words c and e in the cluster.

[0132] S3. Generate product labels using a large-scale pre-trained language model based on the extracted important words and the semantic association information between words;

[0133] In this embodiment, the main purpose of generating product labels using a large-scale pre-trained language model (GPT series) based on the extracted important words and the semantic association information between words is to automatically create high-quality and representative labels that can accurately reflect the characteristics of the product and match the user's needs and market trends; by combining important words and the semantic associations between them, the pre-trained language model can generate labels that are more in line with the actual content of the product description and user reviews. This helps to ensure that the labels not only cover the key topics of the document but also express deeper product characteristics or user preferences; the pre-trained language model has been trained on a large amount of text data and has rich language rules, context relationships, and world knowledge. Therefore, it can generate diverse and informative labels, rather than simply extracting keywords, so as to provide a more detailed and meticulous description for each product;

[0134] Generating product labels using a large-scale pre-trained language model (GPT series) based on the extracted important words and the semantic association information between words includes the following steps:

[0135] S3.1. Construct an input sequence X = (x1, x2,..., x n ) based on the extracted important words and the semantic association information between words, and input the input sequence into the pre-trained language model so that the model can generate appropriate product labels. The large-scale pre-trained language model has learned rich language rules, context relationships, and world knowledge through a large amount of corpora and can generate labels highly relevant to the input text;

[0136] S3.2. The pre-trained language model predicts the next word at each position in an autoregressive manner to generate the complete text:

[0137]

[0138] Among them, Denote the generated product label; y n+1 Denote the next generated word; P(y n+1 |X) represents the conditional probability distribution of the model calculating the next word y in an autoregressive manner n+1 ;

[0139] S3.3. Generate the probability distribution of the product label according to the input sequence and the trained model parameters θ;

[0140] Furthermore, the probability distribution of the product label is:

[0141]

[0142] where Denote the conditional probability distribution of the generated product label ; n represents the length of the input sequence; t represents the position index in the product label generation process, used to represent each step of the model generating the label;

[0143] S3.4. Adopt the temperature sampling strategy to extract label candidates from the conditional probability distribution. This is a strategy to control the smoothness of the word distribution when generating text. Temperature sampling controls the randomness by adjusting the "temperature" parameter of the generated probability distribution.

[0144] S4. Combine the sequence labeling model BERT and the conditional random field CRF to locate and classify the product label, and output the finally extracted product label;

[0145] In this embodiment, the main purpose of combining the sequence annotation model BERT and the conditional random field (CRF) to locate and classify product labels is to achieve high-precision label recognition and classification, ensuring that the generated labels not only accurately reflect the characteristics of the products but also maintain the consistency and logic of the label sequences; the BERT model can capture context information, and BERT can provide a context-sensitive representation for each label candidate, which helps to more precisely understand the meaning of words in a specific context, thereby improving the accuracy of label classification; the CRF layer uses a transition probability matrix, and the CRF can consider the dependencies between labels, ensuring the coherence and rationality of the overall label sequence, avoiding processing each label in isolation, and thus improving the quality of the classification results; the CRF layer calculates the transition probabilities between labels, ensuring that the label sequence not only conforms to the definition of individual labels but also has logical consistency and coherence throughout the sequence. This is very important for constructing a structured label system, especially when multiple labels jointly describe a complex product or concept; by combining the deep learning ability of BERT and the sequence modeling ability of CRF, the long-term dependence problem in label generation can be effectively addressed, that is, the selection of a label depends not only on the current word but also on the context before and after. This method improves the overall efficiency and effectiveness of the label generation process;

[0146] Combining the sequence annotation model BERT and the conditional random field CRF to locate and classify product labels includes the following steps:

[0147] S4.1. Organize the product label candidates generated by the pre-trained language model to form a serialized input format;

[0148] S4.2. Create a corresponding feature vector T' h ={t' h，1 ,t' h，2 ,…,t' h，m} for each product label candidate, including word vectors, position information, etc., where T' h represents the set of product labels after preliminary screening, and t' h，m represents the m-th product label candidate;

[0149] S4.3. Input the product label candidate sequence T' h into the BERT model. The BERT model encodes each product label candidate according to the context, captures the semantic features of the product label candidates, and the output of the BERT model is the context-sensitive representation of each label candidate, and these representations contain the complex relationships between the label candidates and their surrounding environments:

[0150]

[0151] Among them, R mDenote the product label candidate t' h，m as the BERT encoding result, where represents the parameters of the BERT model;

[0152] S4.4. Define the product label space L = {l1, l2,..., l k}, where k represents the index of the product label category (such as "brand", "color", "size", etc.);

[0153] S4.5. Apply the conditional random field layer to calculate the transition probability matrix A between labels. The transition probability matrix represents the probability of transitioning from one product label to another product label:

[0154] A = [a hm ;

[0155]

[0156] where [a hm represents the transition probability from the product label space l g to the product label space l k . represents the parameters of the conditional random field layer; g represents the index of the product label category;

[0157]

[0158] S4.6. Combine the BERT encoding result R m with the transition probabilities of the conditional random field layer to determine the optimal label sequence:

[0159]

[0160] where T” h represents the finally determined product label set; t” h，m represents the m-th final product label; m1 represents the total length of the product label sequence;

[0161] S4.7. Use the Viterbi algorithm to find the product label sequence with the highest probability, that is, maximize the probability of the label sequence.

[0161] Furthermore, the main purpose of using the Viterbi algorithm to find the product label sequence with the highest probability is to ensure that the finally generated label sequence is not only the optimal choice locally (for each word or phrase), but also has the highest overall coherence and logical consistency throughout the sequence; the Viterbi algorithm searches for a path with the highest probability for the entire sequence through dynamic programming among all possible label sequences. This ensures that the selected label sequence is not simply based on the best label at each position, but comprehensively considers the optimality of the entire sequence; since the Viterbi algorithm takes into account the transition probabilities between labels (calculated by the CRF layer), it can ensure that there is a reasonable transition relationship between adjacent labels in the label sequence, avoiding the inconsistency problems that may occur when each label is selected in isolation.

[0162] Using the Viterbi algorithm to find the product label sequence with the highest probability includes the following steps:

[0163] S4.71. For the first product label candidate, calculate the initial probability of each possible product label:

[0164]

[0165] where l represents one of the product label categories; δ1(l) represents the initial probability of product label l under the condition of the given first product label candidate t”. h，1 of the product label l under the condition of the given first product label candidate t”.

[0166] S4.72. Initialize the backtracking pointer for each product label to record the best path:

[0167]

[0168] S4.73. For the e1-th product label candidate in the sequence (2 ≤ e1 ≤ m), calculate the maximum cumulative probability of each possible product label and update the backtracking pointer;

[0169] Furthermore, by calculating the maximum cumulative probability, global optimization of the entire label sequence can be achieved, rather than just local optimality. This means that the finally selected label sequence is the best solution after considering all possible paths, ensuring the overall quality and accuracy of the label sequence;

[0170] The calculation of the maximum cumulative probability of each possible product label is:

[0171]

[0172] where δ e1 (l) represents the observation at the e1-th position (i.e., the product label candidate t”). h，e1)The probability of the best path reaching the e1-th position with the product label being l under the condition; δ e1-1 (l') represents the probability of the best path reaching the (e1 - 1)-th position with the product label being l' under the observation condition at the (e1 - 1)-th position; represents the maximum cumulative probability of the product label l under the condition of the given e1-th product label candidate t” h，e1 ; represents the pointer pointing to the previous optimal product label; a l'，l represents the transition probability from the product label l' to the product label l, calculated by the CRF layer; t' h，e1 represents the product label candidate at the e1-th position; t” h，e1 represents the finally determined product label at the e1-th position;

[0173] S4.74. At the end of the sequence, select the product label with the highest cumulative probability as the last product label z m :

[0174] z m = argmax l δ m (l);

[0175] S4.75. Starting from the last product label, gradually backtrack forward according to the backtracking pointer to reconstruct the entire optimal product label sequence:

[0176]

[0177] S4.76. After the reconstruction is completed, obtain the optimal product label sequence T” h , that is, the finally extracted product label:

[0178] T” h =(z1, z2,..., z m ).

[0179] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification are only preferred examples of the present invention and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed.

Claims

1. A method for extracting product tags based on Internet big data and AI large language models, characterized in that, Including the following steps: S1. Use web crawler technology to scrape the text data of products on the Internet; S2. Adopt the TF-IDF algorithm to determine the important words in the text data, and combine the Skip-Gram model to capture the semantic associations between words. During the process of capturing the semantic associations between words, introduce the weight reflecting the user browsing frequency and the user behavior feature vector to optimize the capture process; Combining the Skip-Gram model to capture the semantic associations between words includes the following steps: S2.

5. Build a vocabulary based on the preprocessed text data, recording all unique words that appear in the text and their frequencies; S2.

6. Set a fixed window size , for each central word, take words on both the left and right of the central word as context words to form training pairs; S2.

7. Randomly initialize low-dimensional vectors for each word in the vocabulary and set hyperparameters; S2.

8. Set the objective function to train the Skip - Gram model, assign weights to the product word vectors to reflect the browsing frequency, introduce the user behavior feature vectors and into the objective function for optimization, and introduce the associated products as additional positive examples into the objective function for further optimization; S2.

9. The trained Skip-Gram model maps each word into a high-dimensional vector space to obtain the word vector of each word ; S2.

10. Combine word frequency, inverse document frequency, and word vectors to form a new product word vector : ； S2.

11. Calculate the cosine similarity of word vectors to measure the semantic similarity between words. For words with similar contexts, introduce a clustering method during the calculation of the cosine similarity of word vectors, and cluster the semantically similar words into the same class through clustering; S3. Based on the extracted important words and the semantic association information between words, use a large-scale pre-trained language model to generate product labels; S4. Combine the sequence annotation model BERT and the conditional random field CRF to locate and classify product labels, and output the finally extracted product labels.

2. The product label extraction method based on Internet big data and AI large language models according to claim 1, wherein: In S2, adopting the TF-IDF algorithm to determine the important words in the text information includes the following steps: S2.

1. Preprocess the text data of the product and organize the captured text data into a corpus , where contains documents , ; S2.

2. Calculate each word in the document word frequency : ； Among them, represents the number of occurrences of a word in the document, and represents the total number of occurrences of all words in the document; represents each word in the document. S2.

3. Calculate the inverse document frequency in the entire corpus : ； Among them, represents the total number of documents in the corpus, represents the number of documents containing the word ; S2.

4. Combine the word frequency and the inverse document frequency to obtain the importance weight of the word in the document : 。 3. The product label extraction method based on Internet big data and AI large language models according to claim 2, characterized in that: In S2.8, the objective function is: ； Among them, represents the optimized objective function; represents the Sigmoid function; represents the vector representation of the context word; represents the vector representation of the central word; represents the number of negative samples; represents the vector representation of the negative sample word; represents the negative sample word; represents the expected value randomly sampled according to a certain vocabulary distribution in the vocabulary list ; represents the vocabulary distribution; represents the transpose of the vector; represents the index of the central word; represents the index of the context word; represents the index of the negative sample word; When a product frequently appears in the user's browsing path, assign weights reflecting the browsing frequency to the product word vectors, establish a user behavior matrix, and extract two user behavior feature vectors from the user behavior matrix and , and introduce the user behavior feature vectors and into the objective function for optimization: ； Among them, represents the objective function after optimizing the user's browsing path; represents the user behavior feature vector corresponding to the target word; represents the user behavior feature vector corresponding to the context word; represents the user 's browsing frequency of the central product ; represents the user 's browsing frequency of the negative sample product ; If multiple products often appear in the browsing path of the same user, then the products are considered to be associated. For each central word , find the products that are often viewed together by the same user group according to the user's browsing history, and introduce the associated products as additional positive examples into the objective function for further optimization: ； Among them, represents the objective function after further optimization; represents a set of additional positive examples selected based on the user browsing path; represents the word vectors corresponding to the additional positive examples; represents the user behavior feature vectors corresponding to the additional positive examples; represents the hyperparameter that controls the contribution degree of the user behavior positive examples; represents the index of the additional positive examples.

4. The product label extraction method based on Internet big data and AI large language models according to claim 3, characterized in that: In S2.11, calculating the cosine similarity of word vectors to measure the semantic similarity between words is: ； Among them, represents the cosine similarity of word vectors; represents the word word vector representation; represents the word word vector representation; For words with similar contexts, introduce a clustering method during the calculation of the cosine similarity of word vectors, and cluster the semantically similar words into the same class through clustering: ； Among them, represents the cosine similarity of the optimized word vectors; represents the similarity calculated based on the lexical clustering information; represents the weighting coefficient of the clustering information.

5. The product label extraction method based on Internet big data and AI large language models according to claim 4, characterized in that: In S3, based on the extracted important words and the semantic association information between words, using a large-scale pre-trained language model to generate product labels includes the following steps: S3.

1. Construct an input sequence based on the extracted important words and the semantic association information between the words , and input the input sequence into the pre-trained language model; S3.

2. The pre-trained language model predicts the next word at each position in an autoregressive manner to generate the complete text; ； Among them, represents the generated product label; represents the next generated word; represents the conditional probability distribution calculated by the model for the next word in an autoregressive manner; S3.

3. Generate the probability distribution of product labels according to the input sequence and the already trained model parameters Generate the probability distribution of product labels; S3.

4. Adopt a temperature sampling strategy to extract label candidates from the conditional probability distribution.

6. The product label extraction method based on Internet big data and AI large language models according to claim 5, wherein: In S3.3, the probability distribution of product labels is: ； Among them, represents the conditional probability distribution of the generated product label ; represents the length of the input sequence; represents the position index in the product label generation process.

7. The product label extraction method based on Internet big data and AI large language models according to claim 6, characterized in that: In S4, combining the sequence annotation model BERT and the conditional random field CRF to locate and classify product labels includes the following steps: S4.

1. Organize the product label candidates generated by the pre-trained language model; S4.

2. Create a corresponding feature vector for each product label candidate , where represents the product label set after preliminary screening, represents the th product label candidate; S4.

3. Input the product label candidate sequence into the BERT model. The BERT model encodes each product label candidate according to the context to capture the semantic features of the product label candidate: ； Among them, represents the BERT encoding result of the product label candidate , represents the parameters of the BERT model; S4.

4. Define the product label space , where represents the index of the product label category; S4.

5. Calculate the transition probability matrix between labels using a conditional random field layer , where the transition probability matrix represents the probability of transitioning from one product label to another product label: ； ； Among them, represents the product label space The transition probability to the product label space of, represents the parameters of the conditional random field layer; represents the index of the product label category; S4.

6. Combine the results encoded by BERT with the transition probabilities of the conditional random field layer to determine the optimal label sequence: ； Among them, represents the finally determined product label set; represents the th final product label; represents the total length of the product label sequence; S4.

7. Use the Viterbi algorithm to find the product label sequence with the highest probability.

8. The product label extraction method based on Internet big data and AI large language models according to claim 7, characterized in that: In S4.7, using the Viterbi algorithm to find the product label sequence with the highest probability includes the following steps: S4.

71. For the first product label candidate, calculate the initial probability of each possible product label; ； Among them, represents one of the product label categories; represents that under the condition of a given first product label candidate the initial probability of product label is; S4.

72. Initialize the backtracking pointer for each product label : ； S4.

73. For the th product label candidate in the sequence, calculate the maximum cumulative probability of each possible product label and update the backtracking pointer; S4.

74. At the end of the sequence, select the product label with the highest cumulative probability as the last product label : ； S4.

75. Starting from the last product label, trace back step by step according to the backtracking pointer to reconstruct the entire optimal product label sequence: S4.

76. After reconstruction, the optimal product label sequence is obtained , that is, the finally extracted product label: 。 9. The product label extraction method based on Internet big data and AI large language model according to claim 8, wherein: In S4.73, calculating the maximum cumulative probability of each possible product label is: ； ； wherein, represents the probability of the best path reaching the th position with the product label being under the observation conditions at the th position; represents the probability of the best path reaching the th position with the product label being under the observation conditions at the th position; represents the maximum cumulative probability of the product label under the condition of the th product label candidate; ; represents the pointer to the previous optimal product label; represents the transition probability from the product label to the product label ; represents the th product label candidate at the position; represents the product label finally determined at the th position.

Images