Deep learning-based text precise labeling method and device, computer equipment and readable storage medium

Abstract

The invention discloses a text precise labeling method and device based on deep learning, computer equipment and a readable storage medium, and the method comprises the steps: firstly obtaining advertisement text data from a public advertisement platform and an enterprise internal system, carrying out the preprocessing of the advertisement text data, and carrying out the labeling of enterprise historical business data, and obtaining a sample set; vectorizing the preprocessed data by means of a pre-training large model based on a Transform self-attention mechanism to obtain a target feature vector containing long-distance dependency relationship information, and performing fine tuning on the model based on a sample set; and finally, inputting the target feature vector into a multi-label classification network to obtain a label type corresponding to the advertisement text. According to the method, deep learning and enterprise business data are effectively fused, the accuracy and efficiency of advertisement text tagging are improved, and support is provided for refined operation of advertisement businesses.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method, apparatus, computer equipment and readable storage medium for accurate text labeling based on deep learning. Background Art

[0002] In the advertising business, accurate ad text labeling is crucial for ad placement, management, and analysis. Traditional ad text labeling methods rely heavily on manual annotation or simple rule matching, resulting in low efficiency and poor accuracy. Existing deep learning-based labeling solutions often struggle to fully utilize enterprise business data and are inadequate for handling the semantic complexity and dynamic changes of ad text, failing to meet enterprises' growing demand for precision. Summary of the Invention

[0003] The purpose of the present invention is to provide a method, device, computer equipment and readable storage medium for accurate text labeling based on deep learning.

[0004] In a first aspect, the present invention provides a method for accurate text labeling based on deep learning, comprising:

[0005] Acquire advertisement text data from public advertisement platforms and internal enterprise systems, and pre-process the advertisement text data;

[0006] Use the company's historical business data for annotation to obtain a sample set of labeled advertising text data;

[0007] Inputting the pre-processed advertising text data into a pre-trained large model based on the Transformer self-attention mechanism for vector representation, obtaining a target feature vector corresponding to the pre-processed advertising text data, wherein the target feature vector contains information about long-range text dependencies, and the pre-trained large model is fine-tuned based on the advertising text data sample set;

[0008] The target feature vector is input into a multi-label classification network to obtain a label type corresponding to the advertisement text data.

[0009] In a possible implementation, inputting the target feature vector into a multi-label classification network to obtain a label type corresponding to the advertisement text data includes:

[0010] Inputting the target feature vector into a multi-label classification network including a fully connected layer and a nonlinear activation function, and outputting the probabilities of multiple undetermined labels corresponding to the advertisement text data;

[0011] According to the label distribution based on business data statistics, obtain the independent probability threshold corresponding to each pending label;

[0012] Based on the probability of each pending tag and the independent probability threshold corresponding to each pending tag, low-probability pending tags are filtered out and high-probability tags are integrated to obtain the tag type corresponding to the advertisement text data.

[0013] In one possible implementation, the loss function of the multi-label classification network is:

[0014] Among them, y i is the actual label, p i is the model prediction label, and N is the number of samples.

[0015] In one possible implementation, the method further includes:

[0016] Obtain user feedback information and enterprise real-time business data for the tag type, and adjust the pre-trained large model based on the feedback information and the enterprise real-time business data.

[0017] In a possible implementation, adjusting the pre-trained large model based on the feedback information and the enterprise's real-time business data includes:

[0018] Integrate the feedback information and the enterprise's real-time business data to obtain an updated data set;

[0019] The pre-trained large model is fine-tuned for a second time using the updated data set to adjust the parameters of the pre-trained large model to improve the accuracy of the model in labeling the advertising text data.

[0020] In a possible implementation, the tagging using the enterprise's historical business data includes:

[0021] Building a labeling rule library, which contains labeling rules based on enterprise business knowledge and advertising industry standards;

[0022] The historical business data of the enterprise is labeled according to the labeling rule library to obtain a sample set of advertising text data with labels.

[0023] In a possible implementation, obtaining the independent probability threshold corresponding to each pending tag according to the tag distribution based on business data statistics includes:

[0024] Analyze the frequency and co-occurrence relationship of each tag in the company's historical business data;

[0025] Based on the occurrence frequency and the co-occurrence relationship, an independent probability threshold is set for each pending tag in combination with preset business requirements.

[0026] In a second aspect, an embodiment of the present invention provides a method and apparatus for accurate text labeling based on deep learning, comprising:

[0027] An acquisition module is used to obtain advertising text data from public advertising platforms and internal enterprise systems, and pre-process the advertising text data; annotate the data using the enterprise's historical business data to obtain a sample set of labeled advertising text data;

[0028] The execution module is used to input the preprocessed advertising text data into a pre-trained large model based on the Transformer self-attention mechanism for vector representation, thereby obtaining a target feature vector corresponding to the pre-processed advertising text data, wherein the target feature vector includes long-distance text dependency information, and the pre-trained large model is fine-tuned based on the advertising text data sample set; and input the target feature vector into a multi-label classification network to obtain a label type corresponding to the advertising text data.

[0029] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor and a non-volatile memory storing computer instructions, wherein when the computer instructions are executed by the processor, the computer device executes the method described in the first aspect.

[0030] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, wherein the readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method described in the first aspect.

[0031] Compared with the existing technology, the beneficial effects provided by the present invention include: adopting a method, device, computer equipment and readable storage medium for accurate text labeling based on deep learning disclosed by the present invention, obtaining and preprocessing advertising text data from public advertising platforms and internal enterprise systems, and then annotating the sample set using the company's historical business data. The pre-processed data is vectorized with the help of a pre-trained large model based on the Transformer self-attention mechanism to obtain the target feature vector containing long-distance dependency information, and the model is fine-tuned based on the sample set. Finally, the target feature vector is input into a multi-label classification network to obtain the label type corresponding to the advertising text. This method effectively integrates deep learning with enterprise business data, improves the accuracy and efficiency of advertising text labeling, and provides support for the refined operation of the advertising business. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly describes the drawings required for use in the embodiments. It should be understood that the following drawings illustrate only certain embodiments of the present invention and should not be construed as limiting the scope of the present invention. Those skilled in the art can, without inventive effort, derive other relevant drawings from these drawings.

[0033] Figure 1 A schematic diagram of the steps of the method for accurate text labeling based on deep learning provided by an embodiment of the present invention;

[0034] Figure 2 A schematic block diagram of the structure of a method and apparatus for accurate text labeling based on deep learning provided by an embodiment of the present invention;

[0035] Figure 3 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present invention. It should be understood that the described embodiments are only a portion of the embodiments of the present invention, not all of them. Generally, the components of the embodiments of the present invention described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

[0037] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0038] In order to solve the technical problems in the above background technology, Figure 1 This is a flow chart of a method for accurate text labeling based on deep learning provided in an embodiment of the present disclosure. The method for accurate text labeling based on deep learning is introduced in detail below.

[0039] Step S201: Acquire advertisement text data from a public advertisement platform and an internal enterprise system, and pre-process the advertisement text data;

[0040] Step S202: using the enterprise's historical business data for labeling to obtain a sample set of advertising text data with labels;

[0041] Step S203: Input the pre-processed advertising text data into a pre-trained large model based on the Transformer self-attention mechanism for vectorization representation, thereby obtaining a target feature vector corresponding to the pre-processed advertising text data, wherein the target feature vector includes information about long-range text dependencies. The pre-trained large model is fine-tuned based on the advertising text data sample set.

[0042] Step S204: input the target feature vector into a multi-label classification network to obtain a label type corresponding to the advertisement text data.

[0043] In the embodiments of the present invention, public advertising platforms, such as well-known search engine advertising platforms and social media advertising platforms, are exemplary sources of massive amounts of advertising text. For example, on search engine advertising platforms, numerous businesses place a wide variety of keyword ads daily, with these ads containing a wide variety of content, including product introductions and promotional activities. The server, through data interfaces established with these public advertising platforms, acquires advertising text data periodically or in real time, according to established rules and frequencies.

[0044] Internal enterprise systems are also an indispensable data source. Enterprise websites and e-commerce platforms contain a large amount of advertising content. For example, a large clothing company's official website features advertisements for new product promotions and exclusive member discounts. Its internal e-commerce platform also features advertising for various promotional events. The server collects this advertising data by accessing the company's internal databases and content management systems.

[0045] The acquired advertising text data is often raw and messy, and the server needs to perform a series of preprocessing operations on it.

[0046] Cleaning: Advertisement text may contain various noise data, such as garbled characters, special symbols, HTML tags and other useless information. For example, the advertisement text obtained from a public advertising platform contains some HTML tags used for web page layout. ”" etc., the server will use a specific text cleaning algorithm to remove these tags and other irrelevant garbled characters and special symbols, and only retain the pure text content.

[0047] Word Segmentation: The text after cleaning is usually continuous sentences. For the convenience of subsequent processing, the server will perform word segmentation on it. Taking a clothing advertisement text "New autumn fashion dress, made of high-quality fabric, comfortable and beautiful" as an example, the server will use professional word segmentation tools, such as the commonly used Jieba word segmenter in Chinese, to split it into single words or phrases such as "new style", "autumn", "fashion", "dress", "made of", "high-quality", "fabric", "comfortable", "and", "beautiful".

[0048] Standardization: The advertisement texts from different sources may have format differences. The server will perform standardization processing on the word-segmented text. For example, unify the case form of the text, convert all English words to lowercase; unify the expression of numbers, such as converting "1st" to "1", etc. At the same time, it will also perform normalization processing on some synonyms, such as unifying "clothes" and "dress" to "clothing", making the data format more unified and standardized.

[0049] The enterprise's historical business data contains rich business knowledge and label information. The server uses this data to annotate the preprocessed advertisement text.

[0050] Taking an electronic product sales enterprise as an example, its historical business data records the effects of past advertisement placements, customer purchase behaviors, and relevant product information, etc. The server first constructs an annotation rule library, which is formulated based on the business knowledge accumulated by the enterprise for a long time and the norms of the electronic product advertisement industry. For example, for the advertisement text that mentions "with a high-definition display screen", according to the rule library, it can be annotated with the label "display feature - high definition"; if the advertisement mentions "suitable for gamers to use", it is annotated with the label "target user - gamers".

[0051] The annotation process can be carried out in the way of manual annotation or semi-automatic annotation. In manual annotation, the enterprise's business personnel or professional annotators will carefully read each advertisement text according to the annotation rule library and add corresponding labels. For example, for a mobile phone advertisement text "Brand new smartphone, equipped with a high-performance processor, with a large memory, and excellent photo-taking effect", the annotator will annotate labels such as "product type - mobile phone", "performance feature - high-performance processor", "performance feature - large memory", "function feature - excellent photo-taking" according to the rules.

[0052] Semi-automatic labeling involves the server applying a specific algorithm to make preliminary label predictions based on historical labeling data and a rule base. Labelers then review and revise the predictions. For example, the server analyzes a large amount of previously labeled data from similar mobile phone ads to build a simple machine learning model. When a new mobile phone ad is encountered, the model automatically predicts possible labels. Labelers then simply check the predictions and make any necessary adjustments.

[0053] Through this labeling process, the server obtains a sample set of labeled advertising text data, which will become important data support for subsequent model training.

[0054] The server selects large pre-trained models based on the Transformer self-attention mechanism, such as BERT (Bidirectional Encoder Representations from Transformers). The BERT model has powerful semantic understanding capabilities in the field of natural language processing. Its core Transformer self-attention mechanism can effectively capture long-range dependencies in text.

[0055] Before feeding the pre-processed ad text data into the model, the server fine-tunes the pre-trained model using the previously obtained sample set of labeled ad text data. For example, using the BERT model, the server feeds the ad text and its corresponding labels from the sample set into the model. Using the backpropagation algorithm, the model's parameters are adjusted to better understand the relationship between ad text and labels.

[0056] For example, during training, for an advertisement with the text "Limited-time offer, buy a laptop and enjoy a discount" and its annotated labels "Promotion - Discount" and "Product Type - Laptop," the BERT model first converts each word in the text into a corresponding word vector. Then, through a multi-layer Transformer encoder, the model uses a self-attention mechanism to calculate the attention weights between each word and other words, thereby capturing the semantic dependencies between "laptop" and words like "buy," "discount," and so on. Based on the discrepancies between predicted and actual labels, the model continuously adjusts its parameters to improve its understanding of the advertisement text and its label prediction capabilities.

[0057] After fine-tuning training, the server feeds the newly pre-processed ad text data into the pre-trained large model. For example, consider a new ad text: "Fashionable smartwatch with heart rate monitoring, perfect for sports enthusiasts." The model converts each word in the text into a word vector based on the patterns learned during training. These vectors are then further processed using the Transformer self-attention mechanism.

[0058] During this process, the self-attention mechanism calculates the attention score between each word and other words to determine each word's importance and semantic relevance within the text. For example, there's a clear semantic connection between "heart rate monitoring function" and "sports enthusiast," and the self-attention mechanism assigns them a higher attention score, better capturing this long-range dependency. Ultimately, the model outputs a target feature vector that incorporates both text semantics and long-range dependency information. This vector serves as input to the subsequent multi-label classification network.

[0059] The multi-label classification network built by the server includes fully connected layers and nonlinear activation functions, such as the Sigmoid function. The fully connected layer further transforms the input target feature vector to extract a feature representation more suitable for label classification.

[0060] When the target feature vector is input into the multi-label classification network, the fully connected layer first performs a linear transformation on it. For example, if the input target feature vector has a dimension of 100, the fully connected layer uses a set of learnable weight matrices and bias terms to map it to a dimensional space related to the number of labels. Assuming that there are 20 possible labels in the advertisement text classification task, after the transformation by the fully connected layer, the output is a vector of dimension 20.

[0061] The nonlinear activation function, Sigmoid, then processes the output vector of the fully connected layer. The Sigmoid function compresses each value in the output vector to a value between 0 and 1, allowing it to be interpreted as a probability of the corresponding label. For the 20-dimensional output vector, after Sigmoid processing, each element becomes a numeric value between 0 and 1, representing the probability of the ad text belonging to each pending label. For example, for the ad text "Fashionable smartwatch with heart rate monitoring, suitable for sports enthusiasts," after processing it through the multi-label classification network, the probability of the label "Product Type - Smartwatch" is 0.9, the probability of the label "Features - Heart Rate Monitoring" is 0.85, and the probability of the label "Target User - Sports Enthusiast" is 0.8, etc.

[0062] The server uses the tag distribution based on business data statistics to determine the independent probability threshold for each pending tag. For example, by analyzing the frequency and co-occurrence of tags in the company's historical business data, combined with business needs and experience, it sets a threshold of 0.7 for tags related to "product type" and 0.6 for tags related to "features."

[0063] The server then filters out low-probability tags and merges high-probability tags based on the probability of each pending tag and the corresponding independent probability threshold, ultimately determining the tag type for the ad text data. For the aforementioned ad text, the probability of "Product Type - Smartwatch" is 0.9, which is greater than the threshold of 0.7; the probability of "Features - Heart Rate Monitoring" is 0.85, which is greater than the threshold of 0.6; and the probability of "Target User - Sports Enthusiast" is 0.8, which is greater than the threshold of 0.6. Therefore, these tags are retained, ultimately determining the tag type for the ad text as "Product Type - Smartwatch," "Features - Heart Rate Monitoring," and "Target User - Sports Enthusiast."

[0064] Through the detailed steps and scenario examples above, the server can achieve accurate labeling of advertising text based on deep learning, providing companies with strong data support and decision-making basis in advertising management, delivery strategy formulation, and user analysis.

[0065] In the embodiment of the present invention, inputting the target feature vector into a multi-label classification network to obtain the label type corresponding to the advertisement text data can be implemented through the following example.

[0066] Inputting the target feature vector into a multi-label classification network including a fully connected layer and a nonlinear activation function, and outputting the probabilities of multiple undetermined labels corresponding to the advertisement text data;

[0067] According to the label distribution based on business data statistics, obtain the independent probability threshold corresponding to each pending label;

[0068] Based on the probability of each pending tag and the independent probability threshold corresponding to each pending tag, low-probability pending tags are filtered out and high-probability tags are integrated to obtain the tag type corresponding to the advertisement text data.

[0069] In this embodiment of the present invention, the server obtains a target feature vector, obtained by processing a pre-trained large model based on the Transformer self-attention mechanism. For example, let's take a beauty company's advertising text as an example. For example, suppose the text is "Newly launched moisturizing liquid foundation, lightweight texture, excellent coverage, suitable for those with dry skin." After vectorization using the pre-trained large model, a target feature vector is obtained that contains the text's semantics and long-range dependency information.

[0070] The multi-label classification network constructed by the server consists of fully connected layers and nonlinear activation functions (such as the Sigmoid function). When the server inputs the target feature vector into the multi-label classification network, the fully connected layer begins to operate. The neurons in the fully connected layer are connected to all elements of the input target feature vector. It performs a linear transformation on the target feature vector using a set of learnable weight matrices and bias terms.

[0071] Assume the target feature vector has a dimension of 200, while the beauty ad text classification task has 30 possible labels. The fully connected layer uses weight matrix operations to map the 200-dimensional target feature vector to a 30-dimensional space, generating a new vector. This process is like a feature recombination and transformation of the target feature vector, ensuring that each element in the new vector contains information related to each pending label.

[0072] Next, the nonlinear activation function, the Sigmoid function, processes the 30-dimensional vector output by the fully connected layer. The Sigmoid function maps any real-valued input value, $z$, to the interval $(0,1)$. The Sigmoid function is calculated for each element in the fully connected layer's output vector. For example, the value obtained after the Sigmoid function calculates the probability that the ad text belongs to the corresponding undetermined label (assuming the label is "texture characteristics - lightness").

[0073] Through this process, the multi-label classification network outputs the probabilities of 30 potential labels corresponding to the beauty ad text. For example, the probability of the label "Product Type - Liquid Foundation" is 0.92, the probability of the label "Functional Characteristics - Moisturizing" is 0.88, the probability of the label "Functional Characteristics - Excellent Concealing" is 0.85, and the probability of the label "Target User - People with Dry Skin" is 0.8.

[0074] The server holds a large amount of historical business data from the enterprise, which includes rich advertising text tag information. The server conducts in-depth analysis of this historical business data and counts the frequency and co-occurrence of each tag.

[0075] For example, a server analyzing the past year's advertising and business records for this cosmetics company found that the "product type" tag appeared relatively frequently in all ad text and was crucial for business decision-making. After detailed data mining and analysis, the server determined the probability threshold for the "product type" tag to be 0.7.

[0076] Although the frequency of appearance of tags related to "functionality" is also high, since the functional characteristics of beauty products are relatively diverse and have different emphases in different advertisements, the server sets the probability threshold of tags related to "functionality" to 0.6 based on actual business needs and data distribution.

[0077] As for the "target user" related tags, considering the importance of accurately positioning target users in beauty advertising and data statistical results, the server sets its probability threshold to 0.65.

[0078] In this way based on business data statistics, the server obtains the corresponding independent probability threshold for each pending tag, and these thresholds will be used for subsequent screening of pending tags.

[0079] The server compares the probability of each pending label output by the multi-label classification network with the corresponding independent probability threshold. For the aforementioned beauty ad, the probability of the label "Product Type - Liquid Foundation" is 0.92, which is greater than the threshold of 0.7 set for the "Product Type" label. Therefore, this label is considered a high-probability label and is retained.

[0080] The probability of the "Functional Characteristics - Moisturizing" label is 0.88, which is greater than the threshold of 0.6 for "Functional Characteristics" related labels and is also retained. The probability of the "Functional Characteristics - Excellent Concealing" label is 0.85, which is also greater than the threshold of 0.6 and is retained.

[0081] The probability of the label "target user - dry skin people" is 0.8, which is greater than the threshold of 0.65 for the "target user" related label and is also retained.

[0082] As for some other pending tags, assuming that the probability of the "fragrance characteristics - elegant fragrance" tag is 0.4, which is less than the threshold of 0.6 for the "functional characteristics" related tags, the server will filter out this low-probability pending tag.

[0083] After this screening process, the server integrated the high-probability tags that were retained and ultimately determined that the tag types for the beauty ad text were "Product Type - Liquid Foundation", "Functional Characteristics - Moisturizing", "Functional Characteristics - Excellent Concealing", and "Target Users - People with Dry Skin".

[0084] Through the detailed steps and scenario examples above, the server uses a multi-label classification network to accurately determine the label type corresponding to the advertising text based on the target feature vector and the rules of business data statistics. This provides important basis and support for beauty companies in areas such as precise advertising delivery, product positioning, and user market segmentation.

[0085] In this embodiment of the present invention, the loss function of the multi-label classification network is:

[0086] Among them, y i is the actual label, p i is the model prediction label, and N is the number of samples.

[0087] In this embodiment of the present invention, for example, when a server runs a deep learning-based method for accurate text labeling, the loss function of the multi-label classification network plays a key role. It is used to measure the difference between the model's predictions and the actual labels, thereby optimizing the model. The following describes a detailed scenario using advertising text processing for a toy company.

[0088] A toy company's server retrieves a large amount of advertising text data from public advertising platforms and internal systems, such as "New educational building block toys, develop children's thinking skills, suitable for children aged 3-6." After preprocessing this advertising text data and annotating it with the company's historical business data, a labeled advertising text data sample set was obtained.

[0089] The server feeds the pre-processed ad text data into a pre-trained large model based on the Transformer self-attention mechanism to obtain the target feature vector, which is then fed into a multi-label classification network. The multi-label classification network outputs the probabilities of multiple pending labels corresponding to each ad text.

[0090] Take a specific advertisement text as an example: "Cool remote control racing car toys, super fast, boys love it." In the company's historical business data annotation, this advertisement text was given actual labels such as "Product Type - Remote Control Racing Car," "Functional Features - Fast Speed," and "Target User - Boys." These labels are the y in the loss function. i .

[0091] After processing the target feature vector of the advertisement text, the multi-label classification network predicts that the probability of "product type - remote control racing car" is 0.8, the probability of "functional characteristics - fast speed" is 0.75, and the probability of "target user - boy" is 0.7. These predicted probabilities are the p in the loss function. i .

[0092] The server uses the loss function of the multi-label classification network to evaluate the accuracy of the model's predictions. In this toy advertisement text scenario, the number of samples N is 1 (just one advertisement text). When calculating the loss value, the server calculates it for each actual label and the corresponding predicted label.

[0093] For the label "Product Type - Remote Control Racing Car", the actual label y i is 1 (indicating that the ad text does belong to this label category), and the predicted probability p i 0.8, which is substituted into the corresponding part of the loss function. Similarly, similar calculations are performed for labels such as "Features - Fast Speed" and "Target User - Boys." These results are then aggregated to obtain the loss value for this ad text.

[0094] When the server processes multiple ad texts (the number of samples N is greater than 1), it calculates the loss value for each ad text as described above and then averages the results to obtain the overall loss value L. This loss value reflects the degree of deviation between the multi-label classification network's predictions for this batch of ad text samples and the actual labels.

[0095] The server adjusts the parameters of the multi-label classification network based on this loss value. If the loss value is large, it indicates that the model's prediction differs significantly from the actual label. The server then uses algorithms such as backpropagation to adjust parameters such as the weights of the fully connected layers in the network, bringing the model's subsequent predictions closer to the actual label. This continuously optimizes the performance of the multi-label classification network and improves the accuracy of ad text label predictions.

[0096] In the embodiments of the present invention, the following implementation modes are also provided.

[0097] Obtain user feedback information and enterprise real-time business data for the tag type, and adjust the pre-trained large model based on the feedback information and the enterprise real-time business data.

[0098] In this embodiment of the present invention, for example, a sports equipment company places various advertisements on its online platform, such as "Professional-grade treadmills with multiple exercise modes for efficient fitness." The server processes these advertisement texts according to the previous steps, obtaining corresponding tag types, such as "Product Type - Treadmill," "Features - Multiple Exercise Modes," and "Usage Effects - Efficient Fitness." These tags are then displayed to users, for example, on product details pages and ad placement pages.

[0099] When users browse advertisements and related product information, they judge label types based on their own knowledge and actual experience. Some users may believe that the labels in a particular ad are inaccurate or incomplete. For example, a user who purchased a treadmill discovered that while the treadmill offered multiple exercise modes, the "incline adjustment function" significantly impacted their fitness plan. However, this feature was not reflected in the ad's label. This user then submitted feedback to the server through the platform's feedback channels, such as feedback forms and customer service messages, stating that the "Features" label should include "incline adjustment function."

[0100] Furthermore, some users may have different opinions on how to categorize tags. For example, some users believe that the tag "helps with efficient fitness" is too general and should be categorized into more specific tags such as "improve cardiopulmonary function" or "enhance muscle strength." The server collects this feedback from different users in real time and stores it in a dedicated database for further processing.

[0101] Sports equipment companies' businesses are constantly evolving, and servers need to access their business data in real time. In daily operations, the company's sales system records every sale of sports equipment, including product name, sales quantity, customer characteristics (such as age, gender, and region), and purchase date. For example, the server retrieved recent sales data for a particular spinning bike from the sales system and found that the majority of customers were men aged 30-40, primarily concentrated in first-tier cities.

[0102] At the same time, the company's marketing department constantly launches new marketing campaigns, such as collaborating with fitness bloggers to promote products and hosting online fitness challenges. The server captures data related to these campaigns, including the number of participants, user interactions (such as likes, comments, and shares), and the number of new customers generated through the campaigns. For example, a campaign promoting new dumbbells in collaboration with a fitness blogger attracted significant user attention and engagement, and the server recorded this real-time business data in detail.

[0103] Furthermore, a company's R&D department also generates relevant business data when improving sports equipment and launching new products. For example, if the R&D department upgrades the material of a yoga mat to improve its anti-slip properties and comfort, the server will obtain detailed information about the product improvement, including the specific details of the improvement and test data.

[0104] The server integrates and processes collected user feedback with real-time business data. The server then categorizes and analyzes user feedback to determine which feedback is common and important. For example, if multiple users mention the "slope adjustment feature" without a tag, the server will consider it important.

[0105] The server mines real-time business data for information related to ad text labels and model performance. For example, after discovering the purchasing preferences of a specific customer group in exercise bike sales data, the server considers how to adjust the pre-trained large model so that it can more accurately label these customer groups when processing related ad text, such as "Target user - males aged 30-40."

[0106] The server then uses this integrated data to adjust the pre-trained large model. Specifically, the server uses this new data as training samples and re-enters the pre-trained large model for fine-tuning. During training, the model adjusts its parameters based on the label information and text content in the new data. For example, to better identify and label the functional characteristics of sports equipment, the model adjusts its internal parameters related to semantic understanding and feature extraction. This allows it to more accurately capture important information such as "slope adjustment function" when processing similar advertising text and annotate it with the corresponding label.

[0107] By continuously obtaining user feedback and real-time business data from enterprises and adjusting the pre-trained large model based on this, the server can adapt the model to changing business needs and user expectations, continuously improve the ability of text precision labeling methods, and provide more reliable support for sports equipment companies' advertising, product promotion, and market analysis.

[0108] In an embodiment of the present invention, adjusting the pre-trained large model based on the feedback information and the enterprise real-time business data can be implemented through the following examples.

[0109] Integrate the feedback information and the enterprise's real-time business data to obtain an updated data set;

[0110] The pre-trained large model is fine-tuned for a second time using the updated data set to adjust the parameters of the pre-trained large model to improve the accuracy of the model in labeling the advertising text data.

[0111] In this embodiment of the present invention, for example, a home furnishings company places an advertisement on major e-commerce platforms, social media, and other public advertising platforms, such as "Simple style solid wood dining table, stable, durable, and easy to clean." The server processes the ad text through a predetermined process and displays a label, and users provide feedback based on their own understanding.

[0112] After purchasing this solid wood dining table, some users commented in the platform's review section, noting that its "rounded corners" effectively prevented bumps and scratches, a crucial feature not reflected in the label. Other users suggested that the "easy to clean" label should be further refined to include more descriptive terms like "stains are easily removed." The server uses a dedicated data scraping program to collect these user feedbacks, scattered across different platforms, in real time and store them in a unified feedback database.

[0113] Sales data: A company's sales management system records every household item sold in real time. The server periodically retrieves data from the system. For example, recent sales data for a sofa shows that most purchases are by newlyweds, concentrated in second-tier cities. This information includes details such as the date of purchase, quantity, and payment method.

[0114] Marketing data: The company launched a promotional campaign with a well-known blogger in the home furnishings sector, and the server captured relevant data during the campaign. For example, after the blogger published a video promoting a particular bedding product, the server accurately recorded data such as the number of views, likes, comments, and traffic to the product page via the video link. Furthermore, the company held an offline home furnishings exhibition, and the server also collected information such as the number of participants and the number of on-site orders.

[0115] Product R&D data: The company's R&D department upgraded a curtain product, using a new light-blocking material. The server captured test data from the R&D process, including light-blocking rates under different lighting conditions, durability test results of the new material, and detailed documentation for the product upgrade.

[0116] The server launches a data integration program to process the collected feedback and the company's real-time business data. Natural language processing technology is used to perform text classification and keyword extraction on the feedback. For example, feedback regarding additional dining table features is categorized as "product feature feedback," and keywords such as "rounded corner design" are extracted.

[0117] The server cleans and formats real-time business data. For example, it standardizes the names of different cities in sales data and organizes various indicators in marketing data according to a unified statistical caliber.

[0118] The server then integrates the processed feedback with business data. For example, for a dining table, feedback about its "rounded corners" design is linked to its sales and marketing data, creating a complete record containing fields such as product name, original label, user feedback, sales-related information, and marketing status. This process generates an updated dataset for the server, providing a rich and targeted data foundation for subsequent adjustments to the pre-trained model.

[0119] The server builds a framework suitable for fine-tuning large pre-trained models within its own computing resources. For large pre-trained models based on the Transformer self-attention mechanism, the server is configured with the corresponding deep learning library, such as TensorFlow or PyTorch, and is allocated sufficient computing memory and CPU and GPU resources to ensure efficient fine-tuning.

[0120] The server feeds the updated dataset into the pre-trained large model in batches of a specified size. For example, each batch contains 100 records. During this input process, the model vectorizes the ad text in the updated dataset, using the Transformer self-attention mechanism to capture semantic information and long-range dependencies in the text, just as it did with the original ad text data.

[0121] After updating the dataset input, the model uses backpropagation to adjust parameters based on the discrepancies between the actual labels in the data (e.g., accurate labels supplemented with user feedback) and its own predicted labels. For example, if the model previously failed to accurately predict the label for the newly added important feature "rounded corners" when processing related ad text, during fine-tuning, the model will adjust internal parameters related to semantic understanding and feature extraction, enabling it to more accurately recognize and predict the label the next time it encounters similar text.

[0122] Specifically, the model's weights are updated based on the results of the loss function. The loss function measures the difference between the model's predicted labels and the actual labels. The model optimizes its parameters by continuously reducing the loss value. During this process, the model may adjust the weights of the fully connected layers to better transform the feature vectors, or it may adjust the parameters of the Transformer layers to more accurately capture the text's semantics.

[0123] After a certain number of rounds of fine-tuning training, the server uses a reserved portion of the updated dataset as a validation set to evaluate the fine-tuned pre-trained model. By calculating metrics such as precision and recall, the server determines whether the model's accuracy in labeling ad text data has improved.

[0124] If the evaluation results show that the model performance still does not meet expectations, the server will continue to adjust the fine-tuning parameters, such as increasing the number of training rounds and adjusting the learning rate, and conduct fine-tuning training and evaluation again until the model's labeling accuracy of advertising text data is significantly improved, which can better meet the business needs of home furnishing companies and provide more accurate support for advertising, product recommendations, etc.

[0125] In the embodiment of the present invention, the tagging using the enterprise's historical business data can be implemented through the following examples.

[0126] Building a labeling rule library, which contains labeling rules based on enterprise business knowledge and advertising industry standards;

[0127] The historical business data of the enterprise is labeled according to the labeling rule library to obtain a sample set of advertising text data with labels.

[0128] In the embodiment of the present invention, for example, an electronic product manufacturing enterprise has accumulated a large amount of business knowledge in the long-term operation process and also complies with relevant regulations of the advertising industry. The server undertakes the important task of building a labeling rule library.

[0129] Companies offer a wide range of products, including mobile phones, tablets, smartwatches, and other electronic gadgets. For example, based on product functionality, the server develops labeling rules based on the company's knowledge of mobile phone R&D, production, and market positioning. If a phone has a high-pixel camera, the labeling rules might include a "Photo Function - High-Pixel Camera" tag. For phones with fast charging capabilities, the labeling rules might include "Charging Function - Fast Charging."

[0130] From a marketing perspective, if a phone is primarily targeted at game enthusiasts, the server will create a labeling rule for "Target User - Game Enthusiasts." Furthermore, based on the phone's design, such as a full-screen design, a labeling rule for "Appearance Design - Full Screen" will be established.

[0131] The advertising industry has standards for product promotion and description. The server complies with these standards to improve its annotation rule library. For example, exaggerated terms used in advertisements require accurate semantic conversion and annotation. If a phrase like "superb performance" appears in an ad, the server will formulate rules to annotate it as "Performance Features - High Performance" based on industry standards and the company's actual definition of product performance. This avoids overly exaggerated terms and ensures accurate and standardized annotation.

[0132] The server also has rules regarding intellectual property and copyright related content in advertisements. If a specific technology patent is cited in the ad text, it will be marked with "Technical Features - (Specific Patent Name)" to comply with industry intellectual property regulations.

[0133] The server integrates various tagging rules developed based on enterprise business knowledge and advertising industry standards. During the integration process, the rules are streamlined and optimized to avoid conflicts and duplication. For example, for mobile phone performance descriptions, the relevant tags are standardized to ensure consistency across different ad text tags.

[0134] At the same time, the server will continuously update and improve the labeling rule library based on the company's business development and changes in the advertising industry. When a company launches a new electronic product line or adopts a new advertising strategy, the corresponding labeling rules will be added or adjusted in a timely manner to ensure the timeliness and practicality of the rule library.

[0135] The server retrieves historical business data from the company's multiple business systems. This includes technical documentation from the product development department, which details the functional characteristics and technical specifications of various electronic products. For example, the technical documentation for a tablet computer details its processor model, screen resolution, battery capacity, and other parameters.

[0136] The sales department's business records are also an important data source, containing information on sales regions, customer group characteristics, sales volumes, and other information for different electronic products. For example, sales data for a particular smartwatch in a specific region shows that the majority of buyers are young people, many of whom are sports enthusiasts.

[0137] In addition, the advertising placement records of the marketing department are also collected by the server, including information such as advertising copy content, delivery platform, delivery time and effect evaluation.

[0138] The server uses a pre-built annotation rule library to annotate advertising text within the company's historical business data. For example, consider a mobile phone ad with the text "Brand new high-performance phone, equipped with an ultra-clear camera, enjoy all kinds of games."

[0139] According to the labeling rule library, the "high performance" in "new high-performance mobile phone" corresponds to the "performance features-high performance" label; "equipped with ultra-clear camera" can be labeled as "photographing function-ultra-clear camera"; and "enjoy playing all kinds of games" complies with the labeling rule of "target users-game enthusiasts".

[0140] The server performs annotation through a combination of automated annotation programs and manual review. The automated program initially annotates the ad text based on a library of annotation rules, followed by review and revision by professional annotators. For semantically ambiguous or controversial expressions, annotators will reference additional business data and industry standards for accurate annotation.

[0141] After labeling, the server organizes and stores each ad text and its corresponding label, forming a labeled ad text data sample set. This sample set contains rich information, such as ad text content, label type, product category, sales region, and other fields.

[0142] These labeled advertising text data sample sets will become an important data foundation for subsequent fine-tuning training of pre-trained large models, helping the model learn the correspondence between electronic product advertising text and labels, thereby improving the ability to accurately label advertising text and providing strong data support for enterprises in advertising placement, product promotion and market analysis.

[0143] In the embodiment of the present invention, obtaining the independent probability threshold corresponding to each pending tag according to the tag distribution based on business data statistics can be implemented through the following example.

[0144] Analyze the frequency and co-occurrence relationship of each tag in the company's historical business data;

[0145] Based on the occurrence frequency and the co-occurrence relationship, an independent probability threshold is set for each pending tag in combination with preset business requirements.

[0146] In this embodiment of the present invention, for example, a server at a car sales company collects historical business data from multiple data sources. The company's customer management system records customer car purchase information, including age, gender, occupation, model, and purchase date. For example, the data shows that over the past year, 3,000 customers purchased various car models, including 800 young customers aged 25-35 and 1,200 who purchased SUVs.

[0147] The sales system also stores detailed information about each transaction, such as vehicle configuration, price, promotions, etc. For example, during a promotion, customers who purchased a specific model received a 10% discount, and all this information is fully recorded.

[0148] At the same time, the advertising delivery system stores the text and related data of automobile advertisements placed by companies on various media platforms, including the number of ad impressions, clicks, conversion rates, etc. For example, an advertisement for a luxury car was displayed 100,000 times on social media, received 5,000 clicks, and ultimately led to 200 purchases.

[0149] The server sorts through the collected business data and extracts the relevant tags. In automotive advertising and sales scenarios, common tags include "Model - SUV," "Target Customer - Young People," "Promotional Activities - Discounts," and "Vehicle Configuration - Panoramic Sunroof."

[0150] The server then counts the frequency of each tag in historical business data. For example, analyzing sales records for "Car Type - SUV" reveals that this tag appears 1,200 times, for a frequency of 40% (1,200 ÷ 3,000). The tag "Target Customer - Young Group" appears 800 times, for a frequency of 26.7%.

[0151] In addition to frequency of occurrence, the server also analyzes the co-occurrence relationship between tags. For example, during the statistical process, it was found that when the tag "Model - SUV" appeared, the tag "Target Customer - Young Group" also appeared 400 times, indicating that the co-occurrence frequency of these two tags was 33.3% (400 ÷ 1200). This indicates that young people account for a certain proportion of customers who purchase SUV models, and there is a certain correlation between the two.

[0152] For example, the "Promotions - Discounts" tag and the "Vehicle Configuration - Panoramic Sunroof" tag also co-occur. During promotions with discounts, some vehicles were equipped with panoramic sunroofs. The server analyzed data to determine the specific frequency and degree of correlation between these two tags.

[0153] Based on their market positioning and sales strategies, automobile retailers have predefined business requirements. They aim to prioritize SUV models to attract younger customers and increase the sales proportion of higher-end models. Based on these requirements, they have varying expectations regarding the importance of different tags in ad text classification and labeling.

[0154] For the "Car Type - SUV" label, the company wants to heavily promote this model, it appears frequently in historical business data (40%), and it also has a certain co-occurrence relationship with the target customer group label. Taking into account business needs and data conditions, the server sets a relatively high probability threshold for this label, such as 0.6. This means that only when the multi-label classification network predicts that a particular ad text belongs to the "Car Type - SUV" label with a probability of 0.6 or above is it determined as the final label.

[0155] The "target customer - young group" tag, while appearing relatively infrequently (26.7%), is relevant to the company's key marketing priorities. Based on its co-occurrence with other tags and business requirements, the server sets a probability threshold of 0.5.

[0156] For the "Vehicle Configuration - Panoramic Sunroof" label, since the company hopes to highlight the vehicle's high-end configuration to attract customers, although its frequency of appearance may not be as high as the vehicle model and customer group labels, the probability threshold is set at 0.45 based on the business's emphasis on configuration promotion and its association with other labels.

[0157] For the "Promotion-Discount" label, the probability threshold is set to 0.55, considering the importance of promotions in sales and their occurrence and co-occurrence in historical data.

[0158] Once the server sets probability thresholds, they aren't set in stone. As the company's business grows and market conditions change, the server continuously monitors and evaluates the rationality of these thresholds. If a label's prediction performance is suboptimal or business requirements change, the server reanalyzes historical business data and adjusts and optimizes the probability thresholds based on the new frequency and co-occurrence relationships. This ensures that the multi-label classification network accurately determines the appropriate label type for automotive ad text, better serving the company's advertising and sales strategies.

[0159] In order to more clearly describe the solution provided by the embodiment of the present invention, a relatively complete implementation method is provided below.

[0160] This invention provides a text precision tagging system based on deep learning and business data. Its core lies in using a pre-trained large model for deep semantic encoding and combining it with the company's accumulated advertising business data for customized secondary training, thereby achieving precise tagging. The specific technical solution is as follows:

[0161] 1. Overall system structure

[0162] The system mainly includes the following modules:

[0163] Data acquisition and preprocessing module:

[0164] Collect advertising text data from Internet advertising platforms and internal enterprise systems;

[0165] Clean, denoise, segment and standardize the collected text to ensure data quality;

[0166] In particular, the company's historical business data is used to label data, providing high-quality training sets for subsequent model fine-tuning.

[0167] Deep Semantic Coding Module:

[0168] Use pre-trained large models (such as BERT, GPT, RoBERTa, etc.) to vectorize the pre-processed text;

[0169] By combining proprietary business data and fine-tuning the model twice, we made it more aligned with the language characteristics and semantic expressions of the advertising industry.

[0170] The Transformer self-attention mechanism is used to capture long-distance dependencies in text. Its core calculation formula is:

[0171]

[0172] Among them, Q, K, and V are vector representations of query, key, and value respectively, and d k is the vector dimension.

[0173] Multi-label classification module:

[0174] Based on deep semantic features, a fully connected layer and nonlinear activation (such as Sigmoid function) are designed to achieve multi-label output;

[0175] Based on the label distribution of business data statistics, set a personalized probability threshold and judge the independent probability of each label output;

[0176] The loss function uses binary cross entropy loss, and the formula is as follows:

[0177]

[0178] Among them, y i is the actual label, p i is the model prediction label, and N is the number of samples.

[0179] Post-processing and feedback module:

[0180] According to the classification results, low-probability outputs are filtered out and high-probability labels are optimized and fused;

[0181] The system supports online learning and uses real-time business data and user feedback to continuously adjust model parameters to achieve dynamic adaptive optimization of the model.

[0182] 2. Workflow and steps

[0183] The system workflow is divided into the following steps:

[0184] Text data input: Users or the backend upload advertising text data, and at the same time import the company's historical business data as training and fine-tuning samples.

[0185] Data preprocessing: cleaning, word segmentation, stop word filtering, etc. of input text to ensure data consistency and accuracy;

[0186] Feature extraction and fine-tuning training:

[0187] The pre-processed text is input into the pre-trained large model for vectorized feature extraction;

[0188] Use the company's own business data to fine-tune the pre-trained model to make it more suitable for the semantic characteristics of advertising text;

[0189] Label prediction: The feature vector is processed through a multi-label classification network and the probability of each possible label is output;

[0190] Label determination and output: Based on the preset personalized threshold, the final label set is determined and returned to the user or stored in the database;

[0191] Online model update: By continuously introducing new business data and user feedback, the model can be continuously fine-tuned and optimized to improve the adaptability and accuracy of the system.

[0192] 3. Algorithms and Mathematical Models

[0193] The core of this system lies in the effective combination of Transformer structure and multi-label classification algorithm, while making full use of enterprise business data for customized training.

[0194] Self-attention mechanism: used to fully capture the correlation between each word in the advertising text;

[0195] Business data fine-tuning strategy: Based on the pre-trained model, fine-tune it using a large amount of annotated data within the enterprise, and further improve the model's performance in specific areas through optimization algorithms (such as the Adam optimizer);

[0196] Multi-label probability output: uses the Sigmoid activation function, which is suitable for scenarios where labels are not mutually exclusive;

[0197] Joint optimization strategy: Use the joint loss function to comprehensively optimize model parameters to ensure the coordinated training of feature extraction and label classification modules.

[0198] Through the above technical means, this system fully integrates the advantages of deep learning and corporate business data, and has significant advantages in processing the diversity, dynamics and semantic complexity of advertising texts.

[0199] In summary, the key innovation of the present invention lies in the deep integration of deep learning and business data to achieve a method for accurate text labeling. First, by combining the pre-trained large model with the company's own business data for secondary fine-tuning, the deep semantic modeling is strengthened, and the ability to accurately capture the semantics of advertising text is greatly improved. Second, a multi-label classification network based on Sigmoid activation is designed. According to the non-mutually exclusive characteristics of advertising text labels, the threshold is customized according to business data, which significantly improves the accuracy of label allocation. Third, with the help of online learning mechanism, the model parameters are optimized according to real-time business data and user feedback to enhance the robustness and adaptability of the system. Fourth, the joint loss function and self-attention mechanism are used to promote the collaborative training of feature extraction and label classification modules.

[0200] The embodiments of the present invention cover the overall system integration architecture, innovatively combining modules such as data preprocessing, deep semantic coding, business data fine-tuning, multi-label classification and post-processing feedback; propose a deep semantic coding solution based on the Transformer structure and an implementation method combined with customized fine-tuning of enterprise business data; clarify the specific multi-label classification algorithm design and joint optimization strategy, including relevant mathematical formulas, model training and threshold setting methods; explain the online learning mechanism and model adaptive update method, and realize the technical path of using real-time business data to improve the accuracy of advertising text labeling.

[0201] Please refer to Figure 2 , Figure 2 A method and apparatus 110 for accurate text labeling based on deep learning provided in an embodiment of the present invention includes:

[0202] The acquisition module 1101 is used to obtain advertising text data from public advertising platforms and internal enterprise systems, and pre-process the advertising text data; annotate the data using the enterprise's historical business data to obtain a sample set of labeled advertising text data;

[0203] Execution module 1102 is used to input the pre-processed advertising text data into a pre-trained large model based on the Transformer self-attention mechanism for vector representation, thereby obtaining a target feature vector corresponding to the pre-processed advertising text data, wherein the target feature vector includes long-distance text dependency information, and the pre-trained large model is fine-tuned based on the advertising text data sample set; and input the target feature vector into a multi-label classification network to obtain a label type corresponding to the advertising text data.

[0204] It should be noted that the implementation principles of the aforementioned deep learning-based text accurate labeling method device 110 can refer to the implementation principles of the aforementioned deep learning-based text accurate labeling method method, and will not be repeated here. It should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a physical entity, or physically separated. Moreover, these modules can be implemented entirely in the form of software called by processing elements; or entirely in the form of hardware; or some modules can be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the deep learning-based text accurate labeling method device 110 can be a separate processing element, or it can be integrated into a chip of the above device. In addition, it can be stored in the form of program code in the memory of the above device, and called by a processing element of the above device to execute the functions of the above deep learning-based text accurate labeling method device 110. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each module above may be completed by an integrated logic circuit of hardware in a processor element or by instructions in the form of software.

[0205] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code on a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0206] The embodiment of the present invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned method and apparatus 110 for accurate text labeling based on deep learning. Figure 3 As shown, Figure 3 This is a block diagram of the structure of a computer device 100 provided in an embodiment of the present invention. The computer device 100 includes a method and apparatus for accurately labeling text based on deep learning 110 , a memory 111 , a processor 112 , and a communication unit 113 .

[0207] In order to realize the transmission or interaction of data, the memory 111, the processor 112 and the communication unit 113 are electrically connected to each other directly or indirectly. For example, the electrical connection between these elements can be achieved through one or more communication buses or signal lines. The text precision labeling method device 110 based on deep learning includes at least one software function module that can be stored in the memory 111 in the form of software or firmware or solidified in the operating system (OS) of the computer device 100. The processor 112 is used to execute the text precision labeling method device 110 based on deep learning stored in the memory 111, such as the software function modules and computer programs included in the text precision labeling method device 110 based on deep learning.

[0208] An embodiment of the present invention provides a readable storage medium, which includes a computer program. When the computer program is running, it controls the computer device where the readable storage medium is located to execute the aforementioned deep learning-based text precise labeling method device 110.

[0209] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in light of the above teachings. These embodiments have been selected and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the present disclosure and to utilize various embodiments with various modifications as appropriate for the specific application contemplated.

Claims

1. A method for accurate text labeling based on deep learning, characterized in that: include: Acquire advertisement text data from public advertisement platforms and internal enterprise systems, and pre-process the advertisement text data; Use the company's historical business data for annotation to obtain a sample set of labeled advertising text data; Inputting the pre-processed advertising text data into a pre-trained large model based on the Transformer self-attention mechanism for vector representation, obtaining a target feature vector corresponding to the pre-processed advertising text data, wherein the target feature vector contains information about long-range text dependencies, and the pre-trained large model is fine-tuned based on the advertising text data sample set; The target feature vector is input into a multi-label classification network to obtain a label type corresponding to the advertisement text data.

2. The method according to claim 1, characterized in that Inputting the target feature vector into a multi-label classification network to obtain a label type corresponding to the advertisement text data includes: Inputting the target feature vector into a multi-label classification network including a fully connected layer and a nonlinear activation function, and outputting the probabilities of multiple undetermined labels corresponding to the advertisement text data; According to the label distribution based on business data statistics, obtain the independent probability threshold corresponding to each pending label; Based on the probability of each pending tag and the independent probability threshold corresponding to each pending tag, low-probability pending tags are filtered out and high-probability tags are integrated to obtain the tag type corresponding to the advertisement text data.

3. The method according to claim 2, characterized in that The loss function of the multi-label classification network is: Among them, y i is the actual label, p i is the model prediction label, and N is the number of samples.

4. The method according to claim 1, wherein The method further comprises: Obtain user feedback information and enterprise real-time business data for the tag type, and adjust the pre-trained large model based on the feedback information and the enterprise real-time business data.

5. The method according to claim 4, characterized in that The adjusting the pre-trained large model based on the feedback information and the real-time business data of the enterprise includes: Integrate the feedback information and the enterprise's real-time business data to obtain an updated data set; The pre-trained large model is fine-tuned for a second time using the updated data set to adjust the parameters of the pre-trained large model to improve the accuracy of the model in labeling the advertising text data.

6. The method according to claim 1, characterized in that The use of the enterprise's historical business data for labeling includes: Building a labeling rule library, which contains labeling rules based on enterprise business knowledge and advertising industry standards; The historical business data of the enterprise is labeled according to the labeling rule library to obtain a sample set of advertising text data with labels.

7. The method according to claim 2, characterized in that Obtaining the independent probability threshold corresponding to each pending tag according to the tag distribution based on business data statistics includes: Analyze the frequency and co-occurrence relationship of each tag in the company's historical business data; Based on the occurrence frequency and the co-occurrence relationship, an independent probability threshold is set for each pending tag in combination with preset business requirements.

8. A method and device for accurate text labeling based on deep learning, characterized in that: include: An acquisition module, configured to acquire advertisement text data from public advertisement platforms and internal enterprise systems, and to pre-process the advertisement text data; Use the company's historical business data for annotation to obtain a sample set of labeled advertising text data; The execution module is used to input the preprocessed advertising text data into a pre-trained large model based on the Transformer self-attention mechanism for vector representation, thereby obtaining a target feature vector corresponding to the pre-processed advertising text data, wherein the target feature vector includes long-distance text dependency information, and the pre-trained large model is fine-tuned based on the advertising text data sample set; and input the target feature vector into a multi-label classification network to obtain a label type corresponding to the advertising text data.

9. A computer device, characterized in that: The computer device includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device executes the method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images