Label construction method, electronic device, storage medium, and computer program product

Abstract

This application discloses a label construction method, an electronic device, a storage medium, and a computer program product. The method includes: obtaining first text of a first product from different information sources; one information source corresponding to one or more first texts; inputting each first text into a first model to obtain a first prediction result for each first text, wherein the first prediction result indicates a first probability of each first category label of the first product; processing the first prediction results of all first texts based on the confidence level of the information source corresponding to each first text to obtain a second probability of each first category label of the first product; and determining the final second category label of the first product among all first category labels of the first product based on the second probability of each first category label of the first product.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a tag construction method, electronic device, storage medium, and computer program product. Background Technology

[0002] With the development of digitalization, product tagging has become a key focus. The accuracy of product tags is not only directly related to market positioning but also the cornerstone for achieving refined operations and continuous innovation. Existing product tagging methods suffer from low accuracy when dealing with highly semantically complex and diverse product descriptions. Summary of the Invention

[0003] To address the related technical issues, embodiments of this application provide a tag construction method, an electronic device, a storage medium, and a computer program product.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a tag construction method, the method including:

[0006] Obtain the first text of the first product from different information sources; one information source corresponds to one or more first texts;

[0007] Each first text is input into the first model to obtain the first prediction result for each first text. The first prediction result indicates the first probability of each first category label of the first product.

[0008] Based on the confidence level of the information source corresponding to each first text, the first prediction results of all first texts are processed to obtain the second probability of each first category label of the first product;

[0009] Based on the second probability of each first category label of the first product, the final second category label of the first product is determined from all first category labels of the first product.

[0010] In the above scheme, the step of processing the first prediction results of all first texts based on the confidence level of the information source corresponding to each first text to obtain the second probability of each first category label of the first product includes:

[0011] The first value is obtained by summing all the first values ​​of the first product; wherein, the first value is determined based on the first probability of the third category label of the second text and the confidence of the source corresponding to the second text; the second text represents any first text or any first text belonging to the same source; the third category label represents any first category label of the first product;

[0012] The second probability of the third category label is determined based on the second value.

[0013] In the above scheme, before inputting each first text into the first model, the method further includes:

[0014] Construct a training sample set; wherein the training sample set includes multiple third texts and a first confidence score of each first category label for each third text; the multiple third texts include multiple fourth texts of second products and a fifth text processed from the fourth texts; one second product corresponds to one or more fourth texts;

[0015] The first model is trained based on the training sample set until the set conditions are met; during the training of the first model, the first model is used to predict the third probability of each first category label of the third text based on the third text.

[0016] In the above scheme, training the first model based on the training sample set until a set condition is met includes:

[0017] Input multiple third texts into the first model to obtain the third probability of each first category label for each third text;

[0018] The first loss value is determined based on the first confidence and third probability of each first category label for each third text;

[0019] Based on the first loss value, the parameters of the first model are updated until the set conditions are met.

[0020] In the above scheme, the first model includes an input layer, a first layer, a second layer, and an output layer; the step of inputting multiple third texts into the first model to obtain the third probability of each first category label for each third text includes:

[0021] The third text is input through the input layer;

[0022] The third text is input into the first layer for feature extraction to obtain the first feature information of the third text;

[0023] The first feature information of the third text is input into the second layer to predict the label probability, so as to obtain the third probability of each first category label of the third text.

[0024] The third probability of each first category label of the third text is output through the output layer.

[0025] In the above scheme, the first layer includes an embedding layer and an encoder layer; the step of inputting the third text into the first layer for feature extraction to obtain the first feature information of the third text includes:

[0026] The third text is input into the embedding layer for vector transformation to obtain a vector sequence of the third text;

[0027] The vector sequence of the third text is input into the encoder layer for text feature extraction to obtain the first feature information of the third text.

[0028] In the above scheme, the second layer includes a third layer, a fourth layer, and a fifth layer; the step of inputting the first feature information of the third text into the second layer for label probability prediction to obtain the third probability of each first category label of the third text includes:

[0029] The first feature information of the third text is input into the third layer for sentence-level tagging to obtain the first feature information of the third text after tagging.

[0030] The first feature information after the third text is labeled is input into the fourth layer for convolution and pooling processing to obtain the second confidence scores of each first category label of the third text;

[0031] The second confidence level of each first category label of the third text is input into the fifth layer for label probability prediction to obtain the third probability of each first category label of the third text.

[0032] In the above scheme, the fifth layer includes a fully connected layer and an activation function layer; the step of inputting the second confidence scores of each first category label of the third text into the fifth layer for label probability prediction to obtain the third probability of each first category label of the third text includes:

[0033] The second confidence scores of each first category label of the third text are input into the fully connected layer for normalization processing to obtain the normalized second confidence scores of each first category label of the third text.

[0034] The normalized second confidence scores of each first category label of the third text are input into the activation function layer to predict the label probability, thereby obtaining the third probability of each first category label of the third text.

[0035] In the above scheme, determining the first loss value based on the first confidence level and third probability of each first category label of each third text includes:

[0036] Based on the first confidence and third probability of each first category label of each third text, determine the second loss value for each third text;

[0037] The mean of the second loss values ​​of all third texts is determined as the first loss value.

[0038] In the above scheme, the fifth text is obtained by replacing the keywords in the fourth text with alternative words, and / or by generating text based on alternative words; the alternative words include one or more of the following: synonyms, near-synonyms, and related words of the keywords in the fourth text.

[0039] This application embodiment also provides a label building apparatus, the apparatus comprising:

[0040] The acquisition module is used to acquire the first text of the first product from different information sources; one information source corresponds to one or more first texts;

[0041] The prediction module is used to input each first text into the first model and obtain a first prediction result for each first text. The first prediction result indicates the first probability of each first category label of the first product.

[0042] The processing module is used to process the first prediction results of all first texts based on the confidence level of the information source corresponding to each first text, and obtain the second probability of each first category label of the first product;

[0043] The determining module is used to determine the final second category label of the first product among all first category labels of the first product based on the second probability of each first category label of the first product.

[0044] This application also provides an electronic device, including: a processor and a communication interface; wherein,

[0045] The processor is configured to acquire first text of a first product from different information sources; one information source corresponds to one or more first texts; input each first text into a first model to obtain a first prediction result for each first text, the first prediction result indicating the first probability of each first category label of the first product; process the first prediction results of all first texts based on the confidence level of the information source corresponding to each first text to obtain a second probability of each first category label of the first product; and determine the final second category label of the first product among all first category labels of the first product based on the second probability of each first category label of the first product.

[0046] This application also provides an electronic device, including a processor and a memory for storing a computer program capable of running on the processor.

[0047] When the processor runs the computer program, it executes the steps of any of the above methods.

[0048] This application also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the above methods.

[0049] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0050] The tag construction method, electronic device, storage medium, and computer program product provided in this application embodiment obtain first text of a first product from different information sources, with one information source corresponding to one or more first texts; each first text is input into a first model to obtain a first prediction result for each first text, the first prediction result indicating the first probability of each first category label of the first product; based on the confidence level of the information source corresponding to each first text, the first prediction results of all first texts are processed to obtain a second probability of each first category label of the first product; based on the second probability of each first category label of the first product, the final second category label of the first product is determined from all first category labels of the first product. In the above scheme, the first text of the first product is obtained from different information sources, which ensures the richness of the content of the first text of the first product. Obtaining the first prediction result of the first text through the first model and processing the first prediction results of all first texts based on the confidence level of the information source corresponding to the first text comprehensively considers the first prediction results of all first texts and the confidence level of the corresponding information source, resulting in higher accuracy of the second probability of each first category label of the first product. Determining the final second category label of the first product based on the second probability of each first category label of the first product improves the accuracy of the final second category label of the first product. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of a label construction method according to an embodiment of this application;

[0052] Figure 2 This is a flowchart illustrating the application of the label construction method in an embodiment of this application.

[0053] Figure 3 This is a schematic diagram of the structure of the first model in the embodiment of this application;

[0054] Figure 4 This is a schematic diagram of the training process of the first model in the embodiments of this application;

[0055] Figure 5 This is a schematic diagram of the label construction device according to an embodiment of this application;

[0056] Figure 6 This is a schematic diagram of the electronic device structure according to an embodiment of this application. Detailed Implementation

[0057] With the development of digitalization, product labeling has become a key focus. The accuracy of product labels not only directly affects market positioning but is also the cornerstone for achieving refined operations and continuous innovation. Product labeling is also crucial for creating industry chain maps, which provide companies with a comprehensive industry view, optimize product portfolios, enhance competitiveness, and promote collaborative and sustainable industrial development.

[0058] In traditional technologies, product labeling relies primarily on manual classification and management. While this approach is relatively effective for small datasets, its limitations—inefficiency, high cost, and susceptibility to errors—become increasingly apparent when faced with rapidly growing volumes of product information. For instance, manual classification often requires significant human resources and is susceptible to subjective judgment when dealing with complex or ambiguous concepts, leading to inconsistent or incorrect labels that severely impact the accuracy and usability of product labels.

[0059] With the development of digital technology, product tagging can also be automated based on rules and statistical learning. However, current methods for automatically building product tags suffer from poor accuracy and robustness when dealing with product descriptions that are highly semantically complex and diverse. For example, rule-based tagging methods are costly to maintain and update rule sets and are prone to failure when dealing with unseen or rarely seen product entities. Statistical tagging methods struggle to capture deep semantic relationships. Existing automated product tagging methods suffer from low accuracy.

[0060] Based on this, in various embodiments of this application, the first text of the first product is obtained from different information sources, with one information source corresponding to one or more first texts; each first text is input into a first model to obtain a first prediction result for each first text, the first prediction result indicating the first probability of each first category label of the first product; based on the confidence level of the information source corresponding to each first text, the first prediction results of all first texts are processed to obtain the second probability of each first category label of the first product; based on the second probability of each first category label of the first product, the final second category label of the first product is determined from all first category labels of the first product. In the above scheme, the first text of the first product is obtained from different information sources, which ensures the richness of the content of the first text of the first product. Obtaining the first prediction result of the first text through the first model and processing the first prediction results of all first texts based on the confidence level of the information source corresponding to the first text comprehensively considers the first prediction results of all first texts and the confidence level of the corresponding information source, resulting in higher accuracy of the second probability of each first category label of the first product. Determining the final second category label of the first product based on the second probability of each first category label of the first product improves the accuracy of the final second category label of the first product.

[0061] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0062] This application provides a tag construction method applied to an electronic device. The electronic device may have a display screen or display function, or may be connected to a display screen; the electronic device includes a terminal and / or a device with a display screen or display function. Figure 1 As shown, it includes:

[0063] Step 101: Obtain the first text of the first product from different information sources.

[0064] One source corresponds to one or more first texts.

[0065] Here, "source of information" refers to the information source of the first text of the first product. For example, sources may include one or more of the following: official website, research report, social media platform. "First product" refers to the product for which the tag is to be constructed. "First text" refers to the product description text and / or product business description text and / or product function description text of the first product. For example, the first text could be "Battery manufacturing; New energy power equipment manufacturing; New energy power equipment sales; New energy vehicle battery swapping facility sales".

[0066] Obtaining the first text of a first product from different sources can be achieved through web scraping technology. For example, web scraping frameworks in the Python programming language can be used to automatically collect the first text of the first product from different sources. These frameworks include, but are not limited to, one or more of the following: Scrapy, BeautifulSoup.

[0067] Step 102: Input each first text into the first model to obtain the first prediction result for each first text.

[0068] The first prediction result indicates the first probability of each first category label of the first product.

[0069] Here, the first model is used at least to output a first prediction result based on the first text. The various first category labels for the first product can be pre-set or obtained. These labels can be set or obtained based on the first product itself, or based on the product category to which the first product belongs; no specific limitation is imposed. For example, the first category labels for the first product may include energy equipment and services, lithium-ion batteries, and batteries. The first model can also be used to output predicted labels based on the first text.

[0070] Taking the first product as an example, which has C first category labels, the predicted label for the first text i is: Among them, y^ (i) A vector representing the predicted label of the first text i. The confidence level of the first category label of the first text i is represented. The confidence level of the second first-category label representing the first text i. This represents the confidence level of the C-th first-category label of the first text i. For example... A value of 1 indicates that the Cth first category label is the label of the first text i; A value of 0 indicates that the Cth label of the first category is not a label of the first text i. It is worth noting that the representation of the predicted labels is not limited to vectors. The first prediction result for the first text i includes... in, The first probability representing the first category label of the first text i. The first probability representing the second first category label of the first text i. The first probability represents the Cth first category label of the first text i.

[0071] Before inputting each piece of first text into the first model, the first text can be processed, and the processed first text is then input into the first model. Processing the first text may include one or more of the following: removing Hypertext Markup Language (HTML) tags from the first text, stemming the first text, standardizing the format of the first text, and processing the first text using natural language processing tools. For example, natural language processing tools include, but are not limited to, one or more of the following: Natural Language Toolkit (NLTK), spaCy; spaCy is a natural language text processing library for Natural Language Processing (NLP). Processing the first text can improve the quality of the first text input into the first model, thereby improving the accuracy of the first prediction result of the first text.

[0072] Step 103: Based on the confidence level of the information source corresponding to each first text, process the first prediction results of all first texts to obtain the second probability of each first category label of the first product.

[0073] Here, the confidence level of a source can also be understood as the weight of the source. The first text of the first product is obtained from different sources, for example, from N sources. w j The confidence level of the j-th source among N sources, 0 ≤ w j ≤1.

[0074] Based on the confidence level of the information source corresponding to each first text, the first prediction results of all first texts are processed to obtain the second probability of each first category label of the first product. This can be achieved by weighted summation of the first probabilities of any first category label of the first product based on the confidence level of the information source corresponding to the first text and the first prediction results of the first text, to obtain the second probability of that first category label, i.e., summing all the first values ​​of the first product to obtain the second value, which is then determined as the second probability of the third category label. Alternatively, based on the confidence level of the information source corresponding to the first text and the first prediction results of the first text, the first probabilities of any first category label of the first product are weighted summation to obtain the sum value, which is then divided by the number of information sources or the number of first texts to obtain the second probability of that first category label, i.e., summing all the first values ​​of the first product to obtain the second value, which is then used to determine the second probability of the third category label. Here, the first value is determined based on the first probability of the third category label of the second text and the confidence level of the information source corresponding to the second text; the second text represents any first text or any first text belonging to the same information source; the third category label represents any first category label of the first product.

[0075] In one embodiment, the step of processing the first prediction results of all first texts based on the confidence level of the information source corresponding to each first text to obtain the second probability of each first category label of the first product includes:

[0076] The first value is obtained by summing all the first values ​​of the first product; wherein, the first value is determined based on the first probability of the third category label of the second text and the confidence of the source corresponding to the second text; the second text represents any first text or any first text belonging to the same source; the third category label represents any first category label of the first product;

[0077] The second probability of the third category label is determined based on the second value.

[0078] Here, the second value is obtained by summing all the first values ​​of the first product, including:

[0079] When the second text represents any first text, the first value of the second text is determined by multiplying the first probability of the third category label of the second text by the confidence level of the source corresponding to the second text, or by multiplying the first probability of the third category label of the second text, the confidence level of the source corresponding to the second text, and the first deviation coefficient; the first deviation coefficient can be set based on actual needs.

[0080] Sum the first values ​​of all the second texts of the first product to obtain the second value;

[0081] In the case where the second text represents any first text belonging to the same information source,

[0082] The first value of the first source is obtained by summing the product of the first probability of the third category label of the second text belonging to the first source and the confidence level of the first source; or, the first value of the first source is obtained by summing the product of the first probability of the third category label of the second text belonging to the first source and the confidence level of the first source, and multiplying the sum by the second deviation coefficient; or, the first value of the first source is obtained by summing the first probabilities of the third category labels of all the second texts belonging to the first source and multiplying the sum by the confidence level of the first source; or, the first value of the first source is obtained by summing the first probabilities of the third category labels of all the second texts belonging to the first source, multiplying the sum, the confidence level of the first source, and the third deviation coefficient; wherein, the first source represents any source of the first text of the first product; the second deviation coefficient and the third deviation coefficient can be set based on actual needs;

[0083] The second value is obtained by summing the first values ​​of all the first sources of the first product.

[0084] The second probability of determining the third category label based on the second value can be: determining the second value as the second probability of the third category label; dividing the second value by the total number of sources of the first text of the first product to obtain the second probability of the third category label; or dividing the second value by the total number of first texts of the first product to obtain the second probability of the third category label.

[0085] In practical applications, the second probability of the third category label is calculated using the following formula:

[0086]

[0087] Among them, P k The second probability represents the k-th first category label of the first product, N represents the total number of information sources for the first text of the first product, and w j p represents the confidence level of the j-th source among the N sources representing the first text of the first product. j (k) represents the sum of the first probabilities of the k-th first category label of all first texts belonging to the j-th source of the first product.

[0088] In this embodiment, by comprehensively considering the first prediction results of all first texts and the confidence level of the corresponding information sources, the accuracy of the second probability of each first category label of the first product can be improved.

[0089] It is worth noting that the confidence levels of each information source for the first text of the first product can be adjusted based on actual needs, the deviation between the predicted labels of the first text from each information source and the final second category label of the first product, or the posterior function. The posterior function is determined based on the prior function and the likelihood function. For example, the prior function could be Beta(α,β), where α represents the number of texts with accurate predicted labels from each of the initially set information sources, and β represents the number of texts with inaccurate predicted labels from each of the initially set information sources. The likelihood function in the Bayesian formula is L(w|r,n)=w r (1-w) n-r Let w represent the confidence level of the information source, r represent the number of texts with accurate predicted labels in each actual information source, and n represent the total number of texts in each actual information source. In this case, the posterior function is Beta(α+r,β+nr). It should be understood that the Beta distribution is a continuous probability distribution, usually defined in the interval [0,1], used to describe the uncertainty of probability. The Beta distribution is the conjugate prior of the binomial distribution. When determining the posterior function value, using the Beta distribution as the prior function can keep the form of the posterior function unchanged and simplify the calculation.

[0090] Step 104: Based on the second probability of each first category label of the first product, determine the final second category label of the first product among all first category labels of the first product.

[0091] Here, based on the second probability of each first category label of the first product, the final second category label of the first product is determined from all the first category labels of the first product. This can be achieved by determining a set number of first category labels of the first product as the final second category label according to the second probability from largest to smallest, or by determining the first category labels of the first product whose second probability is greater than a first set threshold as the final second category label. The set number and the first set threshold can be set based on actual needs. For example, the set number could be 3.

[0092] In this embodiment, considering that the quality of the first text obtained from different information sources varies, the different quality of the first text may lead to low label accuracy of the first product. Therefore, the first text of the first product is obtained from different information sources, which can ensure the richness of the content of the first text of the first product. The first prediction result of the first text is obtained through the first model. The first prediction results of all the first texts are processed by the confidence of the information source corresponding to the first text. The first prediction results of all the first texts and the confidence of the corresponding information source can be comprehensively considered, so that the accuracy of the second probability of each first category label of the first product is high. The final second category label of the first product is determined based on the second probability of each first category label of the first product, which can improve the accuracy of the final second category label of the first product.

[0093] The first model processes the first text of the first product to obtain the predicted label of the first product. However, when the first model encounters situations where the first text contains synonyms and / or polysemous words and / or complex semantic relationships and / or complex contextual understanding and / or technical terms and / or complex text, it is prone to bias, leading to low accuracy in the predicted label. Therefore, in one embodiment, before inputting each piece of first text into the first model, the label construction method further includes:

[0094] Construct a training sample set; wherein the training sample set includes multiple third texts and a first confidence score of each first category label for each third text; the multiple third texts include multiple fourth texts of second products and a fifth text processed from the fourth texts; one second product corresponds to one or more fourth texts;

[0095] The first model is trained based on the training sample set until the set conditions are met; during the training of the first model, the first model is used to predict the third probability of each first category label of the third text based on the third text.

[0096] It should be understood that the first model that takes each first text as input is the first model that has reached the set conditions after training.

[0097] Here, the fourth text represents the product description text and / or product business description text and / or product function description text of the second product. The second product represents a sample product with real labels. The product category to which the second product belongs is the same as or related to the product category to which the first product belongs, that is, the first category labels of multiple second products are the same as the first category labels of the first product. The fifth text can be the text obtained by processing each fourth text, with one fourth text corresponding to one or more fifth texts; or it can be the text obtained by processing a portion of the fourth text, with each fourth text in the portion of the fourth text corresponding to one or more fifth texts.

[0098] The training sample set includes multiple third texts and the first confidence scores of each first category label for each third text, and may also include the true labels for each third text.

[0099] Taking the second product as an example, which has C first category labels, the actual label of the third text t is... Among them, y (t) The vector representing the true label of the third text t. The first confidence level representing the first category label of the third text t. The first confidence level representing the second first-class label of the third text t. The first confidence level of the C-th first category label of the third text t. For example A value of 1 indicates that the Cth first-category label is a label of the third text t; A value of 0 indicates that the Cth first-category label is not a label of the third text t. It is worth noting that the representation of the true label is not limited to vectors.

[0100] The first model is trained on the training sample set until certain conditions are met. These conditions include one or more of the following: the accuracy of the first model is greater than or equal to a second threshold, and the first loss value is less than or equal to a third threshold. The accuracy of the first model can be determined based on the number of accurately predicted third texts and the total number of third texts. The first loss value can be determined based on the first confidence level and third probability of each first category label for each third text. The second and third thresholds can be set based on actual needs. For example, the accuracy of the first model can be obtained by dividing the number of accurately predicted third texts by the total number of third texts.

[0101] During the training of the first model, the first model is used to predict the third probability of each first category label of the third text based on the third text. The first model can also be used to predict the predicted label of the third text based on the third text.

[0102] In this embodiment, the training sample set includes not only the fourth text of multiple second products, but also the fifth text after processing the fourth text. The fifth text is an expansion of the fourth text, making the third text in the training sample set richer. Training the first model based on the training sample set can improve the training effect and generalization ability of the first model.

[0103] In one embodiment, the fifth text is obtained by replacing keywords in the fourth text with alternative words, and / or by generating text based on alternative words; the alternative words include one or more of the following: synonyms, near-synonyms, and related words of keywords in the fourth text.

[0104] The substitute words include synonyms and / or near-synonyms and / or related words of the keywords in the fourth text. The fifth text is generated based on the substitute words; it can be generated using the substitute words from the fourth text as an example.

[0105] In this embodiment, keyword substitution and / or text generation through alternative words can expand the fourth text, increase training samples, and thus help improve the generalization ability and semantic understanding ability of the first model. This enables the trained first model to handle situations where the first text contains synonyms and / or polysemous words and / or complex semantic relationships and / or complex contextual understanding and / or professional terminology and / or complex text.

[0106] In one embodiment, training the first model based on the training sample set until a set condition is met includes:

[0107] Input multiple third texts into the first model to obtain the third probability of each first category label for each third text;

[0108] The first loss value is determined based on the first confidence and third probability of each first category label for each third text;

[0109] Based on the first loss value, the parameters of the first model are updated until the set conditions are met.

[0110] Here, the first loss value is determined based on the first confidence level and third probability of each first category label for each third text. Alternatively, the second loss value for each third text can be determined based on the first confidence level and third probability of each first category label, and the first loss value is determined by multiplying the sum of the second loss values ​​of all third texts by the fourth bias coefficient. Or, the second loss value for each third text can be determined based on the first confidence level and third probability of each first category label, and the first loss value is determined by averaging the second loss values ​​of all third texts. The fourth bias coefficient can be set based on actual needs.

[0111] In this embodiment, the process of training the first model based on the training sample set is clearly defined. Based on the first loss value, the parameters of the first model are updated until the set conditions are met, which can ensure that the accuracy of the first model is relatively high.

[0112] In one embodiment, the first model includes an input layer, a first layer, a second layer, and an output layer; the step of inputting multiple third texts into the first model to obtain the third probability of each first category label for each third text includes:

[0113] The third text is input through the input layer;

[0114] The third text is input into the first layer for feature extraction to obtain the first feature information of the third text;

[0115] The first feature information of the third text is input into the second layer to predict the label probability, so as to obtain the third probability of each first category label of the third text.

[0116] The third probability of each first category label of the third text is output through the output layer.

[0117] Here, the third text is input through the input layer. This can be achieved by encoding the third text and its actual label using one-hot encoding. The encoded third text and its actual label are then input through the input layer. For example, an encoded third text and its actual label might be "Third text: Battery manufacturing; New energy power equipment manufacturing; New energy power equipment sales; New energy vehicle battery swapping facility sales, 1, 1, 1, 0...". Here, the position of "1" represents the position of the three first-category labels "lithium-ion battery, energy equipment and services, battery" in the product label vocabulary. The product label vocabulary consists of all first-category labels.

[0118] The third text is input into the first layer for feature extraction to obtain the first feature information of the third text. Alternatively, the third text can be input into the first layer for vector transformation and text feature extraction to obtain the first feature information of the third text.

[0119] The first feature information of the third text is input into the second layer for label probability prediction to obtain the third probability of each first category label of the third text. This can be done by inputting the first feature information of the third text into the second layer for sentence-level labeling, convolution processing, pooling processing, and label probability prediction to obtain the third probability of each first category label of the third text; or by inputting the first feature information of the third text into the second layer for sentence-level labeling, convolution processing, pooling processing, normalization processing, and label probability prediction to obtain the third probability of each first category label of the third text.

[0120] In this embodiment, the first model is defined to include an input layer, a first layer, a second layer, and an output layer, and the operation of each layer of the first model is defined, which can improve the processing capability and processing efficiency of the first model.

[0121] In one embodiment, the first layer includes an embedding layer and an encoder layer; the step of inputting the third text into the first layer for feature extraction to obtain the first feature information of the third text includes:

[0122] The third text is input into the embedding layer for vector transformation to obtain a vector sequence of the third text;

[0123] The vector sequence of the third text is input into the encoder layer for text feature extraction to obtain the first feature information of the third text.

[0124] Here, the third text is input into the embedding layer and transformed into a vector sequence. This can be achieved by setting placeholders (PlaceHolders) and using word embeddings from a variant of the bidirectional transformer model (ALBERT, A Lite Bidirectional Encoder Representation from Transformers) to convert the third text into a vector representation, resulting in a vector sequence of the third text. ALBERT can also be understood as A Lite BERT. For example, the placeholders include input_ids (input sequence), input_masks (input masks), segment_ids (input sentence-level distinctions), and label_ids (labels corresponding to each third text).

[0125] It should be understood that PlaceHolder is a special type of tensor that allows an entry point for data when constructing the computation graph. When performing a session run, such as inputting third-party text into the embedding layer, the actual data can be passed to the placeholders through a feed_dict mechanism, allowing the first model to dynamically accept input data instead of fixing it when defining the first model. input_ids is a series of numeric identifiers (IDs) used to represent each word or sub-token in the third-party text and its ground truth labels after being processed by the tokenizer; input_masks is used to indicate which positions in the third-party text and its ground truth labels are meaningful tokens and which positions are padding tokens, for example, meaningful positions can be marked as 1 and padding positions as 0; segment_ids is used to identify how many different sentences an input consists of; label_ids indicates the encoding form of the ground truth label corresponding to each third-party text during the training of the first model. ALBERT is a pre-trained language model that, compared to the Bidirectional Encoder Representation from Transformers (BERT), reduces the number of model parameters and improves training efficiency, enabling ALBERT to be applied even with limited computing resources.

[0126] The vector sequence of the third text is input into the encoder layer for text feature extraction, yielding the first feature information of the third text. This encoder layer can use an ALBERT encoder; the ALBERT encoder employs a bidirectional deep learning model, the Transformer, which includes a hidden state layer. The hidden state layer is used to capture deeper semantic information of the text. The first feature information of the third text can be represented by a three-dimensional vector, for example, (batch_size, sequence_length, hidden_size), where batch_size represents the number of training samples captured in one training iteration, sequence_length represents the length of each input training sample, and hidden_size represents the size of the hidden state layer.

[0127] In this embodiment, the first layer is defined to include an embedding layer and an encoder layer, the structure of the first layer is refined, and the processing capability and efficiency of the first model are further improved.

[0128] In one embodiment, the second layer includes a third layer, a fourth layer, and a fifth layer; the step of inputting the first feature information of the third text into the second layer for label probability prediction to obtain the third probability of each first category label of the third text includes:

[0129] The first feature information of the third text is input into the third layer for sentence-level tagging to obtain the first feature information of the third text after tagging.

[0130] The first feature information after the third text is labeled is input into the fourth layer for convolution and pooling processing to obtain the second confidence scores of each first category label of the third text;

[0131] The second confidence level of each first category label of the third text is input into the fifth layer for label probability prediction to obtain the third probability of each first category label of the third text.

[0132] Here, the third layer can be understood as the CLS (Classification) vector layer. The CLS vector is located at the beginning of each input and serves as a "handle" or "summary" of the input, providing a fixed global sentence-level representation for subsequent layers of the first model. Inputting the first feature information of the third text into the third layer for sentence-level labeling—for example, the third layer uses the CLS vector to perform sentence-level labeling on the first feature information of the third text, resulting in the labeled first feature information of the third text—is a common practice.

[0133] The first feature information after the third text is labeled is input into the fourth layer for convolution and pooling processing to obtain the second confidence scores of each first category label of the third text. Alternatively, the fourth layer can extract features from the first feature information after the third text is labeled using convolution kernels to obtain the second feature information of the third text, and then process this second feature information using max pooling to obtain the second confidence scores of each first category label of the third text. The fourth layer can use convolution kernels of different sizes to extract features from the first feature information after the third text is labeled, in order to capture multi-character features and hidden correlations between multiple adjacent characters in the third text. The window size of the convolution kernel can be set based on actual needs. For example, since the length of product labels is usually 2-5 characters, the window size of the convolution kernel can be set to one or more of the following: 2, 3, 4, 5. Max pooling can be used to process the second feature information of the third text, discarding secondary features and reducing dimensionality. The fourth layer can use a Text Convolutional Neural Network (TextCNN) to perform convolution and pooling processing on the first feature information after the third text is labeled, in order to effectively capture local features in the third text. The fourth layer can be understood as the TextCNN layer.

[0134] The second confidence scores of each first-class label of the third text are input into the fifth layer for label probability prediction, resulting in the third probability of each first-class label of the third text. The fifth layer can use the sigmoid function to predict the label probability of each first-class label of the third text based on the second confidence scores, thus obtaining the third probability of each first-class label of the third text. The sigmoid function can also be understood as a logistic function.

[0135] In this embodiment, the second layer is defined to include the third, fourth and fifth layers, the structure of the second layer is refined, and the operation of each layer in the second layer is defined, so as to further improve the processing capability and processing efficiency of the first model.

[0136] In one embodiment, the fifth layer includes a fully connected layer and an activation function layer; the step of inputting the second confidence scores of each first category label of the third text into the fifth layer for label probability prediction to obtain the third probability of each first category label of the third text includes:

[0137] The second confidence scores of each first category label of the third text are input into the fully connected layer for normalization processing to obtain the normalized second confidence scores of each first category label of the third text.

[0138] The normalized second confidence scores of each first category label of the third text are input into the activation function layer to predict the label probability, thereby obtaining the third probability of each first category label of the third text.

[0139] Here, the fully connected layer can use the Sigmoid function to normalize the second confidence scores of each first-class label of the third text. The activation function layer can use the Sigmoid function to predict the label probability based on the normalized second confidence scores of each first-class label of the third text, so that the third probability of each first-class label of the third text is between (0, 1).

[0140] In this embodiment, the fifth layer is defined as including a fully connected layer and an activation function layer. The second confidence scores of each first category label of the third text are normalized by the fully connected layer, which can ensure data consistency and improve the accuracy of the third probability of each first category label of the third text.

[0141] In one embodiment, determining the first loss value based on the first confidence level and third probability of each first category label of each third text includes:

[0142] Based on the first confidence and third probability of each first category label of each third text, determine the second loss value for each third text;

[0143] The mean of the second loss values ​​of all third texts is determined as the first loss value.

[0144] Here, based on the first confidence level and third probability of each first category label of each third text, the second loss value of each third text is determined. This can be achieved by using the binary cross-entropy loss function to process the first confidence level and third probability of each first category label of each third text to obtain the second loss value of each third text.

[0145] The first loss value can be calculated using the following formula:

[0146]

[0147] L ss M represents the first loss value, L represents the total number of third texts, and L represents the first loss value. ss (t) Let C represent the second loss value of the t-th third text, and let C represent the total number of first category labels of the third text. The loss value representing the a-th first-class label of the t-th third text. The first confidence level represents the a-th first category label of the t-th third text. The third probability represents the a-th first category label of the t-th third text.

[0148] Understandably, the first loss value can also be calculated using the following formula:

[0149]

[0150] In this embodiment, the mean of the second loss values ​​of all third texts is determined as the first loss value, which can reduce the computational difficulty. Subsequently, the parameters of the first model are updated based on the first loss value, which can improve the accuracy of the first model.

[0151] The following section provides a more detailed description of this application with reference to application examples.

[0152] The application flowchart of the tag construction method provided in this application embodiment is as follows: Figure 2 As shown, it includes:

[0153] Step 1: Construct the training sample set.

[0154] Obtain the fourth texts for multiple second products, with one second product corresponding to one or more fourth texts. Preprocess the fourth texts using data analysis packages such as Pandas, including cleaning and standardization, to obtain processed fourth texts. Determine the true labels for each processed fourth text through manual annotation. Process the fourth texts based on substitution words to obtain fifth texts. Merge the fifth and fourth texts, perform quality checks and deduplication, to obtain multiple third texts. Based on the multiple third texts and their true labels (i.e., the first confidence scores of each first-category label), construct a training sample set. For example, there may be 300,000 third texts; the exact number of third texts is not limited. The training sample set can be divided into a training set, a validation set, and a test set in an 8:1:1 ratio.

[0155] Step 2: Train the first model based on the training sample set until the set conditions are met.

[0156] A schematic diagram of the structure of the first model is shown below. Figure 3 As shown. The first model includes an input layer, an embedding layer, an encoder layer, a third layer, a fourth layer, a fully connected layer, an activation function layer, and an output layer; the operation of each layer of the first model has been described in detail in the above embodiments, and will not be repeated here.

[0157] A schematic diagram of the training process of the first model is shown below. Figure 4As shown. The training process of the first model includes a forward propagation process and a backpropagation process. The forward propagation process inputs multiple third texts into the first model, executing the steps sequentially: input layer, embedding layer, encoder layer, third layer, fourth layer, fully connected layer, activation function layer, and output layer. This yields the third probability of each first category label for each third text. Based on the first confidence level and third probability of each first category label for each third text, a first loss value is determined. The backpropagation process updates the parameters of the first model based on the first loss value until certain conditions are met. These conditions include the first model's accuracy being greater than or equal to a second predetermined threshold (to make the accuracy of the first model approach 1), and the first loss value being less than or equal to a third predetermined threshold (to make the first loss value approach 0.0001).

[0158] Step 3: Obtain the first text of the first product from different information sources and process the first text.

[0159] For example, using a web crawler framework in the Python programming language, the first text of the first product can be automatically collected from different information sources; the HTML tags of the first text can be removed, the first text can be stemmed, the first text can be standardized in format, and the first text can be processed using natural language processing tools to obtain the processed first text.

[0160] Step 4: Determine the second probability of each first category label for the first product.

[0161] Each first text, i.e. the processed first text, is input into the first model, i.e., the first model that meets the set conditions, to obtain the first prediction result for each first text; the first prediction result indicates the first probability of each first category label of the first product; based on the confidence level of the information source corresponding to each first text, the first prediction results of all first texts are processed to obtain the second probability of each first category label of the first product.

[0162] Step 5: Determine the final second category label for the first product.

[0163] The three first-category labels of the first product are determined by ranking the probabilities of each first-category label from largest to smallest, and these three labels are then used as the final second-category labels for the first product.

[0164] The tag construction method provided in this application, through a first model, enables automated understanding and classification of highly complex and semantically diverse product descriptions. The fourth text is processed to obtain a fifth text, constructing a training sample set. This allows the first model to effectively handle synonyms, near-synonyms, and related words, covering a wider range of vocabulary and contexts. This not only increases the scale of the training samples but also enhances the first model's adaptability to complex contexts, ensuring high accuracy and broad applicability, and helping to improve the accuracy and consistency of product tag construction. The first model can self-optimize based on new training samples without manual rule adjustments, ensuring its continued effectiveness and market relevance. The first model includes an input layer, an embedding layer, an encoder layer, a third layer, a fourth layer, and a fully connected layer. The system, consisting of a transition layer, activation function layer, and output layer, enables efficient text feature extraction, further improving the accuracy of label construction. While maintaining high precision, it significantly reduces computational resource requirements, making it suitable for large-scale text classification applications. To address the issue of uneven quality among multiple information sources, the system obtains the first text of the first product from different sources and sets a confidence level for each source individually. The source confidence levels can be dynamically adjusted to gradually improve their accuracy and adaptability. By fusing the first probabilities of each first-category label of the first product from different sources with the source confidence levels, the system determines the second probabilities of each first-category label of the first product. This achieves high precision and efficiency in label generation, enhancing the stability and practicality of label construction.

[0165] To implement the tag building method provided in this application embodiment, this application embodiment also provides a tag building apparatus, such as... Figure 5 As shown, the device includes:

[0166] The acquisition module 501 is used to acquire the first text of the first product from different information sources; one information source corresponds to one or more first texts;

[0167] Prediction module 502 is used to input each first text into the first model and obtain a first prediction result for each first text. The first prediction result indicates the first probability of each first category label of the first product.

[0168] The processing module 503 is used to process the first prediction results of all first texts based on the confidence level of the information source corresponding to each first text, and obtain the second probability of each first category label of the first product.

[0169] The determining module 504 is used to determine the final second category label of the first product among all the first category labels of the first product based on the second probability of each first category label of the first product.

[0170] In one embodiment, the processing module 503 is specifically used to sum all the first values ​​of the first product to obtain a second value; wherein, the first value is determined based on the first probability of the third category label of the second text and the confidence level of the information source corresponding to the second text; the second text represents any first text or any first text belonging to the same information source; the third category label represents any first category label of the first product;

[0171] The second probability of the third category label is determined based on the second value.

[0172] In one embodiment, before inputting each first text into the first model, the apparatus further includes:

[0173] A construction module is used to construct a training sample set; wherein, the training sample set includes multiple third texts and a first confidence score of each first category label of each third text; the multiple third texts include multiple fourth texts of second products and a fifth text processed from the fourth texts; one second product corresponds to one or more fourth texts;

[0174] The training module is used to train the first model based on the training sample set until a set condition is met; during the training of the first model, the first model is used to predict the third probability of each first category label of the third text based on the third text.

[0175] In one embodiment, the training module is specifically used to input multiple third texts into a first model to obtain the third probability of each first category label of each third text;

[0176] The first loss value is determined based on the first confidence and third probability of each first category label for each third text;

[0177] Based on the first loss value, the parameters of the first model are updated until the set conditions are met.

[0178] In one embodiment, the first model includes an input layer, a first layer, a second layer, and an output layer; the training module is specifically used to input third text through the input layer.

[0179] The third text is input into the first layer for feature extraction to obtain the first feature information of the third text;

[0180] The first feature information of the third text is input into the second layer to predict the label probability, so as to obtain the third probability of each first category label of the third text.

[0181] The third probability of each first category label of the third text is output through the output layer.

[0182] In one embodiment, the first layer includes an embedding layer and an encoder layer; the training module is specifically used to input the third text into the embedding layer for vector transformation to obtain a vector sequence of the third text;

[0183] The vector sequence of the third text is input into the encoder layer for text feature extraction to obtain the first feature information of the third text.

[0184] In one embodiment, the second layer includes a third layer, a fourth layer, and a fifth layer; the training module is specifically used to input the first feature information of the third text into the third layer for sentence-level labeling to obtain the first feature information of the third text after labeling.

[0185] The first feature information after the third text is labeled is input into the fourth layer for convolution and pooling processing to obtain the second confidence scores of each first category label of the third text;

[0186] The second confidence level of each first category label of the third text is input into the fifth layer for label probability prediction to obtain the third probability of each first category label of the third text.

[0187] In one embodiment, the fifth layer includes a fully connected layer and an activation function layer; the training module is specifically used to input the second confidence scores of each first category label of the third text into the fully connected layer for normalization processing, so as to obtain the normalized second confidence scores of each first category label of the third text.

[0188] The normalized second confidence scores of each first category label of the third text are input into the activation function layer to predict the label probability, thereby obtaining the third probability of each first category label of the third text.

[0189] In one embodiment, the training module is specifically used to determine a second loss value for each third text based on a first confidence level and a third probability of each first category label of each third text;

[0190] The mean of the second loss values ​​of all third texts is determined as the first loss value.

[0191] In one embodiment, the fifth text is obtained by replacing keywords in the fourth text with alternative words, and / or by generating text based on alternative words; the alternative words include one or more of the following: synonyms, near-synonyms, and related words of keywords in the fourth text.

[0192] In practical applications, the acquisition module 501, the prediction module 502, the processing module 503, the determination module 504, the construction module, and the training module can be implemented by the processor in the label construction device.

[0193] It should be noted that the label building apparatus provided in the above embodiments is only illustrated by the division of the above-described program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the label building apparatus and the label building method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0194] Based on the hardware implementation of the above program modules, and in order to implement the tag construction method provided in this application embodiment, this application embodiment also provides an electronic device, such as... Figure 6 As shown, the electronic device 600 includes:

[0195] The communication interface 601 enables information exchange with other network nodes;

[0196] The processor 602 is connected to the communication interface 601 to enable information interaction with other network nodes and to execute the methods provided by one or more of the above-described technical solutions when running a computer program. The computer program is stored in the memory 603.

[0197] Specifically, the processor 602 is configured to acquire first text of a first product from different information sources; one information source corresponds to one or more first texts; input each first text into a first model to obtain a first prediction result for each first text, the first prediction result indicating the first probability of each first category label of the first product; process the first prediction results of all first texts based on the confidence level of the information source corresponding to each first text to obtain a second probability of each first category label of the first product; and determine the final second category label of the first product among all first category labels of the first product based on the second probability of each first category label of the first product.

[0198] In one embodiment, the processor 602 is specifically configured to sum all the first values ​​of the first product to obtain a second value; wherein, the first value is determined based on the first probability of the third category label of the second text and the confidence level of the source corresponding to the second text; the second text represents any first text or any first text belonging to the same source; the third category label represents any first category label of the first product;

[0199] The second probability of the third category label is determined based on the second value.

[0200] In one embodiment, before inputting each first text into the first model, the processor 602 is further configured to construct a training sample set; wherein the training sample set includes a plurality of third texts and a first confidence level of each first category label for each third text; the plurality of third texts include a plurality of fourth texts of second products and a fifth text processed from the fourth texts; one second product corresponds to one or more fourth texts;

[0201] The first model is trained based on the training sample set until the set conditions are met; during the training of the first model, the first model is used to predict the third probability of each first category label of the third text based on the third text.

[0202] In one embodiment, the processor 602 is specifically configured to input multiple third texts into a first model to obtain the third probability of each first category label of each third text;

[0203] The first loss value is determined based on the first confidence and third probability of each first category label for each third text;

[0204] Based on the first loss value, the parameters of the first model are updated until the set conditions are met.

[0205] In one embodiment, the first model includes an input layer, a first layer, a second layer, and an output layer; the processor 602 is specifically configured to input third text through the input layer.

[0206] The third text is input into the first layer for feature extraction to obtain the first feature information of the third text;

[0207] The first feature information of the third text is input into the second layer to predict the label probability, so as to obtain the third probability of each first category label of the third text.

[0208] The third probability of each first category label of the third text is output through the output layer.

[0209] In one embodiment, the first layer includes an embedding layer and an encoder layer; the processor 602 is specifically used to input the third text into the embedding layer for vector transformation to obtain a vector sequence of the third text;

[0210] The vector sequence of the third text is input into the encoder layer for text feature extraction to obtain the first feature information of the third text.

[0211] In one embodiment, the second layer includes a third layer, a fourth layer, and a fifth layer; the processor 602 is specifically used to input the first feature information of the third text into the third layer for sentence-level tagging to obtain the first feature information of the third text after tagging;

[0212] The first feature information after the third text is labeled is input into the fourth layer for convolution and pooling processing to obtain the second confidence scores of each first category label of the third text;

[0213] The second confidence level of each first category label of the third text is input into the fifth layer for label probability prediction to obtain the third probability of each first category label of the third text.

[0214] In one embodiment, the fifth layer includes a fully connected layer and an activation function layer; the processor 602 is specifically used to input the second confidence scores of each first category label of the third text into the fully connected layer for normalization processing to obtain the normalized second confidence scores of each first category label of the third text.

[0215] The normalized second confidence scores of each first category label of the third text are input into the activation function layer to predict the label probability, thereby obtaining the third probability of each first category label of the third text.

[0216] In one embodiment, the processor 602 is specifically configured to determine a second loss value for each third text based on a first confidence level and a third probability of each first category label of each third text;

[0217] The mean of the second loss values ​​of all third texts is determined as the first loss value.

[0218] In one embodiment, the fifth text is obtained by replacing keywords in the fourth text with alternative words, and / or by generating text based on alternative words; the alternative words include one or more of the following: synonyms, near-synonyms, and related words of keywords in the fourth text.

[0219] It should be noted that the specific processing procedure of processor 602 can be understood by referring to the above method.

[0220] Of course, in practical applications, the various components in electronic device 600 are coupled together through bus system 604. It can be understood that bus system 604 is used to realize the connection and communication between these components. In addition to a data bus, bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 6 The general designated all buses as Bus System 604.

[0221] The memory 603 in this embodiment is used to store various types of data to support the operation of the electronic device 600. Examples of such data include any computer program used to operate on the electronic device 600.

[0222] The methods disclosed in the embodiments of this application can be applied to the processor 602, or implemented by the processor 602. The processor 602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 602 or by instructions in the form of software. The processor 602 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 602 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the memory 603. The processor 602 reads the information in the memory 603 and combines its hardware to complete the steps of the aforementioned method.

[0223] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0224] It is understood that the memory 603 in this embodiment can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 603 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0225] In an exemplary embodiment, this application also provides an electronic device, including a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of any of the methods described above.

[0226] This application embodiment also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 603 that stores a computer program. The computer program can be executed by the processor 602 of the electronic device 600 to complete the steps described in the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0227] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0228] It should be noted that terms such as "first" and "second" are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. The term "and / or" in this article merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, the term "one or more" in this article refers to any combination of at least two of any one or more items from a set of A, B, and C. For example, "one or more of A, B, and C" can represent any one or at least two or more elements selected from the set of A, B, and C.

[0229] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0230] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A tag construction method, characterized in that, The method includes: Obtain the first text of the first product from different information sources; one information source corresponds to one or more first texts; Each first text is input into the first model to obtain the first prediction result for each first text. The first prediction result indicates the first probability of each first category label of the first product. Based on the confidence level of the information source corresponding to each first text, the first prediction results of all first texts are processed to obtain the second probability of each first category label of the first product; Based on the second probability of each first category label of the first product, the final second category label of the first product is determined from all first category labels of the first product.

2. The method according to claim 1, characterized in that, The first prediction results of all first texts are processed based on the confidence level of the information source corresponding to each first text to obtain the second probability of each first category label of the first product, including: The first value is obtained by summing all the first values ​​of the first product; wherein, the first value is determined based on the first probability of the third category label of the second text and the confidence of the source corresponding to the second text; the second text represents any first text or any first text belonging to the same source; the third category label represents any first category label of the first product; The second probability of the third category label is determined based on the second value.

3. The method according to claim 1 or 2, characterized in that, Before inputting each first text into the first model, the method further includes: Construct a training sample set; wherein the training sample set includes multiple third texts and a first confidence score of each first category label for each third text; the multiple third texts include multiple fourth texts of second products and a fifth text processed from the fourth texts; one second product corresponds to one or more fourth texts; The first model is trained based on the training sample set until the set conditions are met; during the training of the first model, the first model is used to predict the third probability of each first category label of the third text based on the third text.

4. The method according to claim 3, characterized in that, The step of training the first model based on the training sample set until a set condition is met includes: Input multiple third texts into the first model to obtain the third probability of each first category label for each third text; The first loss value is determined based on the first confidence and third probability of each first category label for each third text; Based on the first loss value, the parameters of the first model are updated until the set conditions are met.

5. The method according to claim 4, characterized in that, The first model includes an input layer, a first layer, a second layer, and an output layer; the step of inputting multiple third texts into the first model to obtain the third probability of each first category label for each third text includes: The third text is input through the input layer; The third text is input into the first layer for feature extraction to obtain the first feature information of the third text; The first feature information of the third text is input into the second layer to predict the label probability, so as to obtain the third probability of each first category label of the third text. The third probability of each first category label of the third text is output through the output layer.

6. The method according to claim 5, characterized in that, The first layer includes an embedding layer and an encoder layer; the step of inputting the third text into the first layer for feature extraction to obtain the first feature information of the third text includes: The third text is input into the embedding layer for vector transformation to obtain a vector sequence of the third text; The vector sequence of the third text is input into the encoder layer for text feature extraction to obtain the first feature information of the third text.

7. The method according to claim 5, characterized in that, The second layer includes a third layer, a fourth layer, and a fifth layer; the step of inputting the first feature information of the third text into the second layer for label probability prediction to obtain the third probability of each first category label of the third text includes: The first feature information of the third text is input into the third layer for sentence-level tagging to obtain the first feature information of the third text after tagging. The first feature information after the third text is labeled is input into the fourth layer for convolution and pooling processing to obtain the second confidence scores of each first category label of the third text; The second confidence level of each first category label of the third text is input into the fifth layer for label probability prediction to obtain the third probability of each first category label of the third text.

8. The method according to claim 7, characterized in that, The fifth layer includes a fully connected layer and an activation function layer; the step of inputting the second confidence scores of each first category label of the third text into the fifth layer for label probability prediction to obtain the third probability of each first category label of the third text includes: The second confidence scores of each first category label of the third text are input into the fully connected layer for normalization processing to obtain the normalized second confidence scores of each first category label of the third text. The normalized second confidence scores of each first category label of the third text are input into the activation function layer to predict the label probability, thereby obtaining the third probability of each first category label of the third text.

9. The method according to any one of claims 4 to 8, characterized in that, The determination of the first loss value based on the first confidence level and third probability of each first category label of each third text includes: Based on the first confidence and third probability of each first category label of each third text, determine the second loss value for each third text; The mean of the second loss values ​​of all third texts is determined as the first loss value.

10. The method according to claim 3, characterized in that, The fifth text is obtained by replacing the keywords in the fourth text with alternative words, and / or by generating text based on the alternative words; the alternative words include one or more of the following: synonyms, near-synonyms, and related words of the keywords in the fourth text.

11. An electronic device, characterized in that, This includes a processor and memory for storing computer programs that can run on the processor. When the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 10.

12. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images