Content generation method and device, storage medium and program product

Abstract

The embodiment of the invention provides a content generation method and device, a storage medium and a program product. In the embodiment of the invention, the method comprises the following steps: firstly, performing category prediction on initial text content through a category prediction model, obtaining at least one keyword from the text content based on the predicted category, and generating content corresponding to each theme according to at least one keyword under each theme classification and predicted category information to obtain target content. On one hand, the keyword of the initial text content can be extracted according to the category predicted according to the main semantics of the initial text content, and it can be guaranteed that the keyword is matched with the main category corresponding to the initial text content. And on the other hand, subject classification is performed on the keywords, and the detail description corresponding to each subject is generated according to the keywords under each subject classification, so that the generated target content can be structured, the user can conveniently read and quickly find the bright spots of the post content, and the user traffic can be increased.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a content generation method, device, storage medium and program product. Background Art

[0002] In the field of post content generation, users are usually required to manually input a large amount of initial information, and generation tools based on certain rules and templates generate post content. This method is not only time-consuming and labor-intensive, but also limits the diversity and creativity of generated content. Alternatively, post content can be generated based on natural language processing methods based on machine learning, but such methods usually require a large amount of training data so that the model can learn effective language generation patterns, and the training process is complex and time-consuming.

[0003] Currently, there is a lack of effective solutions to the above problems. Summary of the invention

[0004] Multiple aspects of the present application provide a content generation method, device, storage medium and program product to improve the efficiency of model training, increase the diversity and creativity of text content generated by the model, structure the text content, improve the readability of the text content, and attract user traffic.

[0005] An embodiment of the present application provides a method for generating text content, including: obtaining initial text content of target content to be generated, the initial text content corresponding to a target category, and the target category is obtained by predicting the initial text content using a target category prediction model; inputting a prompt word, the initial text content and its corresponding target category into a target AI language model based on artificial intelligence, extracting at least one keyword matching the target category in the initial text content and subject-classifying the at least one keyword, and generating detailed description information corresponding to the topic based on at least one keyword under each topic to obtain the target text content; wherein the AI ​​language model based on artificial intelligence is obtained by training the initial AI language model based on artificial intelligence based on multiple text contents containing at least one detailed description information marked with a topic tag as training samples.

[0006] An embodiment of the present application also provides an electronic device, including: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the above method.

[0007] An embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements the steps in the above method.

[0008] An embodiment of the present application also provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the processor is enabled to implement the steps in the above method.

[0009] In an embodiment of the present application, the category prediction model is first used to predict the category of the initial text content, at least one keyword is obtained from the text content based on the predicted category, and the keywords are subject-classified, and the content corresponding to each theme is generated according to at least one keyword under each theme classification and the predicted category information to obtain the target content. On the one hand, the use of a large artificial intelligence model can reduce the training cost and improve the training efficiency of the model; on the other hand, the keywords of the initial text content can be extracted according to the categories predicted by the main semantics of the initial text content, which can ensure that the keywords match the categories corresponding to the main content of the initial text content. On the other hand, the keywords are subject-classified, and the detailed description corresponding to each theme is generated according to the keywords under each theme classification, so that the generated target content can be structured, comprehensive, and fluent in language, which is convenient for users to read and quickly find the highlights of the post content, improve the readability of the text content, and attract user traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0011] Figure 1a A flowchart of a content generation method provided by an exemplary embodiment of the present application;

[0012] Figure 2a A flow chart of a category prediction model training method provided by an exemplary embodiment of the present application;

[0013] Figure 2b A schematic diagram of the structure of an initial category prediction model provided by an exemplary embodiment of the present application;

[0014] Figure 2c A schematic diagram of the structural flow of a target category prediction model provided for an exemplary embodiment of the present application;

[0015] Figure 3a A schematic diagram of a flow chart of a category prediction method provided by an exemplary embodiment of the present application;

[0016] Figure 3b A schematic diagram of a front-end display page provided for an exemplary embodiment of the present application;

[0017] Figure 3c A schematic diagram of a backend display page provided for an exemplary embodiment of the present application;

[0018] Figure 4a A flowchart of a cross-category recognition model training method provided for an exemplary embodiment of the present application;

[0019] Figure 4b A schematic diagram of a prompt structure provided for an exemplary embodiment of the present application;

[0020] Figure 4c Another schematic diagram of a prompt word structure provided for another exemplary embodiment of the present application;

[0021] Figure 5 A schematic diagram of a cross-category identification method provided for an exemplary embodiment of the present application;

[0022] Figure 6 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0024] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0025] In view of the above technical problems, in an embodiment of the present application, the category prediction model is first used to predict the category of the initial text content, and at least one keyword is obtained from the text content based on the predicted category, and the keywords are subject-classified, and the content corresponding to each theme is generated according to at least one keyword under each theme classification and the predicted category information to obtain the target content. On the one hand, the use of a large artificial intelligence model can reduce the training cost and improve the training efficiency of the model; on the other hand, the keywords of the initial text content can be extracted according to the categories predicted by the main semantics of the initial text content, which can ensure that the keywords match the categories corresponding to the main content of the initial text content. On the other hand, the keywords are subject-classified, and the detailed description corresponding to each theme is generated according to the keywords under each theme classification, so that the generated target content can be structured, comprehensive and fluent in language, which is convenient for users to read and quickly find the highlights of the post content, improve the readability of the text content, and attract user traffic. The following is a detailed description of a solution provided by an embodiment of the present application in conjunction with the accompanying drawings.

[0026] Figure 1a The following is a flow chart of a content generation method provided by an exemplary embodiment of the present application. Figure 1a As shown, the method includes:

[0027] 101. Obtaining initial text content of the target content to be generated, the initial text content corresponds to a target category, and the target category is predicted based on main semantic information of the initial text content using a target category prediction model;

[0028] 102. Input the prompt word, the initial text content and the corresponding target category into the target AI language model based on artificial intelligence, extract at least one keyword matching the target category in the initial text content and classify the at least one keyword into topics, and generate detailed description information corresponding to the topic based on at least one keyword under each topic to obtain the target text content;

[0029] Among them, the AI ​​language model based on artificial intelligence is obtained by training the initial AI language model based on artificial intelligence based on multiple text contents containing detailed description information marked with at least one topic tag as training samples.

[0030] In the embodiment of the present application, content generation includes two application scenarios. The first application scenario is to generate target text content based on the text content entered by the user on the post publishing page, and the second application scenario is to re-write the content based on any post published by the user.

[0031] In the embodiment of the present application, for the first application scenario described above, obtaining the initial text content to be generated as the target content includes: in response to the user's input operation in the content input box on the post publishing page, obtaining the initial text content so as to generate the target text content based on the initial text content, the post publishing page such as Figure 3b shown.

[0032] In the embodiment of the present application, for the above-mentioned second application scenario, obtaining the initial text content of the target content to be generated includes: in response to the user's selection operation on any published post on the post rewriting page, obtaining the published post content as the initial text content, so as to generate the target text content based on the initial text content. In this application scenario, the user's post copying demand information can be input by voice or text through the user and intelligent customer service conversation page provided by the platform, such as "I want to re-copy a new post based on the first post published under my account", then based on the obtained user's demand information and the user's registration information on the platform, the first post is found from the user account and a new post is re-copy based on the post content of the first post and published.

[0033] It should be noted that the implementation methods of obtaining the initial text content of the target content to be generated in the two application scenarios can be interchangeable, and the two can also use the same implementation method.

[0034] In an embodiment of the present application, the target AI language model based on artificial intelligence includes: a semantic information extraction layer, a semantic information classification layer and a content generation layer; the prompt word, the initial text content and its corresponding target category are input into the AI ​​language model based on artificial intelligence, at least one keyword matching the target category in the initial text content is extracted and the at least one keyword is subject-classified, and based on at least one keyword under each subject, detailed description information corresponding to the subject is generated to obtain the target text content, including: the initial text content and its corresponding target category are input into the semantic information extraction layer, and the semantic information of the target category and the semantic information of the initial text are extracted; based on the semantic information of the target category, at least one keyword semantic information adapted to the semantic information of the target category is extracted from the semantic information of the initial text; at least one keyword semantic information is input into the semantic information classification layer, and at least one keyword semantic information is classified to obtain at least one keyword semantic information set, and each keyword semantic information set corresponds to a subject tag; based on each keyword semantic information set, the content generation rules and prompt words of its corresponding subject tag, the detailed description information corresponding to the subject is generated; based on the detailed description information corresponding to each subject, the target text content is obtained.

[0035] Further optionally, after obtaining the initial text content to be generated, and before inputting the prompt words, the initial text content and its corresponding target category into the AI ​​language model based on artificial intelligence, the initial text content can also be input into the target category prediction model to predict at least one first-level category of the initial text content to obtain at least one first-level initial category corresponding to the initial text content.

[0036] Optionally, the initial text content is input into a target category prediction model, and at least one first-level category of the initial text content is predicted to obtain at least one first-level category corresponding to the initial text content, including: when the length of the initial text content is greater than a text length threshold supported by the target category prediction model, the initial text content is divided into at least two text segments, and the length of each text segment is not greater than the text length threshold; the target category prediction model includes at least multiple semantic information extraction layers and at least one first-level category prediction layer; the at least two text segments are input into at least two semantic information extraction layers in the target category prediction model to extract semantic features to obtain at least two semantic information; the at least two semantic information extraction layers are semantic information extraction layers selected from multiple semantic information extraction layers and the same number as the at least two text segments; based on the at least two semantic information, target semantic information corresponding to the initial text content is generated; the target semantic information is input into at least one first-level category prediction layer to predict at least one first-level category to obtain multiple category prediction results corresponding to the initial text content including at least one first-level initial category. A more detailed implementation of predicting at least one level category of the initial text content to obtain at least one level initial category corresponding to the initial text content can be found in the relevant description of the following embodiment, which will not be described here in detail.

[0037] Further optionally, in response to the user's selection operation on any category prediction result, at least one first-level category corresponding to the category prediction result can be obtained; the at least one first-level category corresponding to the category prediction result can be input into the cross-category prediction model, and the initial text content and the at least one first-level category corresponding to the category prediction result can be identified according to the main semantic information of the initial target content to see whether they match, and the recognition result can be output; when the recognition result is that the initial text content is non-cross-category, the initial text content and at least one first-level category selected by the user can be input into the AI ​​language model based on artificial intelligence to obtain the target text content.

[0038] Further optionally, before the prompt words, the initial text content and their corresponding target categories are input into the artificial intelligence-based target AI language model, the artificial intelligence-based target AI language model may also be trained.

[0039] Optionally, training the target AI language model based on artificial intelligence includes: obtaining multiple training samples; using the multiple training samples to train the initial AI language model based on artificial intelligence to obtain the target text content corresponding to at least one topic tag contained in each training sample; calculating the semantic loss function based on the text content of at least one detailed description information marked with a topic tag contained in each training sample and the target text content corresponding to at least one topic tag contained in the training sample; according to the semantic loss function, adjusting the network parameters of each network layer in the initial AI language model based on artificial intelligence until the semantic loss function meets the model training conditions to obtain the target AI language model based on artificial intelligence. For a more detailed implementation method of training the target AI language model based on artificial intelligence, please refer to the relevant description of other model training in the following embodiments, which will not be repeated here (the training methods of each model in the following embodiments are also applicable to the training of this model).

[0040] In the technical solutions provided in the above-mentioned embodiments of the present application, on the one hand, the use of a large artificial intelligence model can reduce the training cost and improve the training efficiency of the model; on the other hand, the keywords of the initial text content can be extracted according to the categories predicted based on the main semantics of the initial text content, which can ensure that the keywords match the categories corresponding to the main content of the initial text content. On the other hand, by subject-classifying the keywords and generating detailed descriptions corresponding to each topic based on the keywords under each topic classification, the generated target content can be structured, comprehensive, and fluent in language, which is convenient for users to read and quickly find the highlights of the post content, thereby improving the readability of the text content and attracting user traffic. The following is a detailed description of a solution provided in an embodiment of the present application in conjunction with the accompanying drawings.

[0041] Figure 2a The following is a flow chart of a method for training a category prediction model provided by an exemplary embodiment of the present application. Figure 2a As shown, the method includes:

[0042] 201. Obtain multiple training samples, each training sample corresponding to a labeling result of at least two levels of categories;

[0043] 202. According to the text lengths of the plurality of training samples, the training samples whose text lengths are within the same length range are divided into the same training subset to obtain a plurality of training subsets, and different training subsets correspond to different length ranges;

[0044] 203. Randomly sort multiple training subsets, and perform model training on an initial category prediction model according to the target training samples in each training subset after random sorting, so as to obtain prediction results of at least two levels of categories corresponding to each target training sample; the initial category prediction model at least includes a semantic information extraction layer and at least two levels of category prediction layers, and the prediction results of at least two levels of categories are predicted by the at least two levels of category prediction layers according to the semantic information extracted from the target training samples by the semantic information extraction layer;

[0045] 204. Calculate the loss of the same-level category according to the labeling result and prediction result of the same-level category corresponding to each target training sample to obtain the loss function of at least two levels of categories; calculate the comprehensive loss function according to the loss function of at least two levels of categories;

[0046] 105. According to the comprehensive loss function, adjust the network parameters of the semantic information extraction layer and at least two-level category prediction layers in the initial category prediction model until the comprehensive loss function satisfies the model training termination condition to obtain an intermediate category prediction model;

[0047] 106. Copy and splice the semantic information extraction layers in the intermediate category prediction model to obtain a target category prediction model that includes at least a plurality of semantic information extraction layers connected in sequence and at least two levels of category prediction layers.

[0048] In an embodiment of the present application, in order to obtain a category prediction model with high category prediction accuracy, the initial category prediction model can be trained to obtain a target category prediction model. The initial category prediction model includes at least a semantic information extraction layer and at least two levels of category prediction layers. The semantic information extraction layer is used to extract semantic information from text content. The "at least two levels" in the at least two levels of category prediction layers is used to limit the number and hierarchical relationship of category prediction layers. Each level of category prediction layer is used to predict the category of the training sample at that level based on the semantic information extracted from the text content by the semantic information extraction layer. Figure 2bThe figure is an exemplary structural diagram of the initial category prediction model, wherein the semantic information extraction layer can be implemented as a BERT (Bidirectional Encoder Representations from Transformers) model or a RoBERTa (A Robustly Optimized BERT Pretraining Approach) model. At least two levels of category prediction layers can be implemented as a Word2Vec model, a TF-IDF vector, and a logistic regression model, etc. At least two levels of category prediction layers can be implemented as the same model to simultaneously predict at least two levels of categories, or each level of category prediction layer can be implemented as a model to simultaneously predict the corresponding level categories. When each level of category prediction layer can be implemented as a model, at least two levels of category prediction layers can be limited to different models.

[0049] In an embodiment of the present application, multiple training samples can be obtained, and the initial prediction model can be trained using the multiple training samples. Among them, the multiple training samples can be posts, articles, etc. that have been published on the platform, and each training sample contains text content, and the text content includes but is not limited to: Chinese and its punctuation marks, foreign languages ​​and their punctuation marks, and each training sample corresponds to at least two levels of category annotation results. The at least two levels of category annotation results refer to each level of category annotated for each training sample, that is, each level of category selected by each training sample when it is published.

[0050] This embodiment does not limit the specific implementation method of obtaining multiple training samples. For example, the text content of multiple published posts or articles with annotation results is selected from an associated database as multiple training samples; or, the text content in multiple posts or articles is annotated with categories manually or through an annotation model as multiple training samples; and so on. In addition, in order to balance the categories, training samples can be evenly obtained according to at least two levels of categories. For example, samples are evenly obtained according to two levels of categories, and 1,000 training samples are obtained for each level of category; or, training samples can be evenly obtained according to three or more levels of categories, for example, 1,000 training samples are obtained for each category. The selection of training samples for several levels of categories depends on the specific application situation, and this embodiment does not limit this.

[0051] In an embodiment of the present application, each training sample corresponds to a text length. In order to reduce the difference between the results of the model's category prediction for posts of different lengths, and at the same time to improve the accuracy of the trained model's category prediction results for posts of different lengths, and to improve the generalization ability of the trained model, the training samples whose text lengths are within the same length range can be divided into the same training subset according to the text lengths of multiple training samples, so as to obtain multiple training subsets, and different training subsets correspond to different length ranges, so as to select training samples within different length ranges as target training samples to train the initial category prediction model.

[0052] Optionally, according to the text lengths of multiple training samples, the training samples whose text lengths are within the same length range are divided into the same training subset to obtain multiple training subsets, including: sorting the multiple training samples according to the text lengths of the multiple training samples; and evenly dividing the sorted multiple training samples according to the preset number of subsets to obtain multiple training subsets, each training subset including the same number of training samples. Thus, it can be ensured that the multiple training samples in each training subset are within the same length range, and when selecting at least one as the target training sample, the generalization ability of the model will not be affected by the uncertainty of the length. Alternatively, the training subset corresponds to a preset number of training sample carrying capacity, and the sample carrying capacity of each training subset can be the same. If there is a situation where the training samples cannot be evenly divided, the multiple training samples can be divided into each training subset in turn according to the maximum carrying capacity of the samples, and the remaining training samples are divided into the last training subset, and the number of remaining training samples should be less than or equal to the training sample carrying capacity corresponding to the training subset. In addition, multiple training samples can also be unevenly divided.

[0053] For example, if there are 5 training subsets, including: a first training subset, a second training subset, a third training subset, a fourth training subset and a fifth training subset, and the length ranges of the 5 training subsets can be 50-60 characters, 60-70 characters, 70-80 characters, 80-90 characters, and 90-100 characters, respectively, then the training samples with a length range of 50-60 characters are divided into the first training subset, the training samples with a length range of 60-70 characters are divided into the first training subset, the training samples with a length range of 70-80 characters are divided into the first training subset, the training samples with a length range of 80-90 characters are divided into the first training subset, and the training samples with a length range of 90-100 characters are divided into the first training subset.

[0054] In an embodiment of the present application, after dividing a plurality of training samples into a plurality of training subsets, the plurality of training subsets can also be randomly sorted, and the initial category prediction model can be trained according to the target training samples in each training subset after random sorting to obtain the prediction results of at least two levels of categories corresponding to each target training sample, and the training subsets of different length ranges can be shuffled, and the target training samples are selected from each shuffled training subset to train the initial category prediction model, which can further reduce the difference between the results of the model's category prediction for posts of different lengths, and improve the accuracy of the trained model's category prediction results for posts of different lengths and the generalization ability of the model. Among them, the prediction results of at least two levels of categories are predicted by at least two levels of category prediction layers based on the semantic information extracted from the target training samples by the semantic information extraction layer.

[0055] In some embodiments, the initial category prediction model is trained according to the target training samples in each training subset after random sorting to obtain prediction results of at least two levels of categories corresponding to each target training sample, including: for each training subset after random sorting, selecting a target training sample from the training subset; inputting the target training sample into a semantic information extraction layer to extract semantic information to obtain the semantic information of the target training sample; using the pooling layer in the initial category prediction model to pool the semantic information of the target training sample to obtain the pooled semantic information of the target training sample; inputting the pooled semantic information of the target training sample into at least two levels of category prediction layers respectively to perform prediction of at least two levels of categories to obtain prediction results of at least two levels of categories corresponding to the target training sample.

[0056] Optionally, selecting a target training sample from each training subset includes: each training sample in each training subset is used as a target training sample; or, randomly selecting a training sample or part of the training samples from each training subset as the target training sample; or, selecting a training sample with the longest text length, a text with a medium text length, and a training sample with the shortest text length or a combination thereof from each training subset as the target training sample, which is not limited in this embodiment. In addition, the number of target training samples selected from each training subset may be the same or different, and the method of selecting the target training samples from each training subset may be the same or different. For example, for some training subsets, the longest text length may be selected as the target sample, and for other training subsets, the shortest text length may be selected as the training sample; or, for all training subsets, the longest text length may be selected as the target sample.

[0057] Further optionally, since the number of target training samples selected in each subset may be multiple and the lengths of the multiple training samples are within the same range, in order to further improve the generalization ability of the model, the multiple training samples contained in each target training sample may be randomly sorted, and the randomly sorted training samples may be used for model training.

[0058] It should be noted that in the following embodiments, the number of training target samples selected in each training sample is 1 as an example for detailed description.

[0059] Optionally, the target training sample contains multiple segmentations, which refer to independent and meaningful segmentations. A segmentation can be a word, phrase, symbol, number or any other segmentation that can be processed separately. For example, the text content "We are committed to providing high-quality cleaning and washing services!" includes the following segmentations: "we", "committed to", "provide", "high-quality", "cleaning and washing services" and "!". Based on this, the target training sample is input into the semantic information extraction layer in the initial category prediction model to extract semantic information to obtain the semantic information of the target training sample, including: inputting the target training sample into the semantic information extraction layer in the initial category prediction model, and based on the self-attention learning mechanism, learning the semantic information of each segmentation and the semantic association relationship between each segmentation to obtain the semantic information of the target training sample.

[0060] Among them, the self-attention learning mechanism, also known as the intra-attention mechanism, is a mechanism that allows the model to focus on the relationship between different segmentations within the sequence when processing the segmentation sequence (an independent text content) corresponding to the text content of the target training sample. The core idea of ​​the self-attention mechanism is to allow each segmentation in the segmentation sequence to interact directly with other segmentations in the sequence, thereby capturing the short-distance and long-distance dependencies between the segmentations within the segmentation sequence, so that the model can dynamically pay attention to different segmentations in the segmentation sequence and adjust the importance of each segmentation according to the context. Among them, short-distance dependencies refer to the mutual dependencies between segments that are close to each other in the segmentation sequence data, and long-distance dependencies refer to the mutual dependencies between segments that are far apart in the segmentation sequence data. This dependency can be semantic, grammatical, or context-based. The main features of the self-attention learning mechanism include: (1) Query vector, key vector, and value vector: The self-attention mechanism maps each word in the input word sequence to three vectors: query (Q), key (K), and value (V). These vectors are obtained by multiplying the input word sequence with the learnable parameters (weight matrix) in the model. The query vector represents the word or position currently being processed, which needs to obtain information from other words or positions in the word sequence. In the self-attention mechanism, each word generates a query vector for comparison with the key vector to determine which words or positions in the sequence should be used to obtain information. The key vector determines which information or words at which positions in the word sequence are "accessible". The degree of match between the query vector and the key vector determines the degree to which the value vector is weighted and summed. In the self-attention mechanism, each word or position has a corresponding key vector, which together with the query vector determines the degree of influence of other words in the word sequence on the current word. The value vector contains the actual content or features of each position or word in the sequence, which will be weighted and summed according to the degree of match between the query and the key. The value vector represents the specific semantic information of each word in the word sequence, which will be integrated into the output according to the attention weight. (2) Attention score calculation: Calculate the similarity or match between the query vector and all key vectors to generate an attention score. The attention score represents the importance of each word in the word sequence to the current word. (3) Scaling and tiling: To avoid numerical problems, such as vanishing or exploding gradients caused by excessive dot products, the model scales the attention scores (usually by dividing by the square root of the key vector dimension) and then normalizes the scores using the Softmax function. (4) Attention-weighted features: The normalized values ​​obtained by the Softmax function are used as weights and multiplied by the value vector to obtain the final attention-weighted features. The attention-weighted features integrate the information of all words in the word sequence, and the contribution of each word is determined by its corresponding attention weight.(5) Multi-head attention: In order to capture the relationship between different subspaces in the sequence, the self-attention mechanism can be expanded to a multi-head attention mechanism. In the multi-head attention, the model executes the above process multiple times in parallel, and each "head" may focus on different relationships or features in the word segmentation sequence. (6) Residual connection and layer normalization: The output of the self-attention will be added to the original input (residual connection), and then normalized (LayerNormalization) to stabilize the training process and improve the gradient flow. (7) Calculation of the feedforward neural network layer (also known as fully connected layer calculation): After the self-attention layer, there is usually a fully connected layer, which performs further nonlinear transformation on the output of the self-attention layer to strengthen the position representation or extract more complex features. For the detailed implementation of the self-attention learning mechanism applied to this application, please refer to the relevant description of the following embodiments, which will not be repeated here.

[0061] In an embodiment of the present application, through a learning process based on a self-attention learning mechanism, the semantic information of the target training sample corresponding to each segmentation in each target training sample can be obtained, and the semantic information of the target training sample corresponding to each segmentation is the same, that is, the semantic information of multiple identical target training samples is obtained. In practical applications, one of the semantic information of multiple target training samples can be selected as the output of the semantic information extraction layer. For this, special characters (without specific semantic meanings) can be added to the target training sample, and the semantic information of the target training sample corresponding to the special characters can also be obtained based on the self-attention learning mechanism, and the semantic information of the target training sample corresponding to the special characters is used as the output of the semantic information extraction layer. Based on this, the target training sample is input into the semantic information extraction layer in the initial category prediction model to extract the semantic information to obtain the semantic information of the target training sample, including: performing a special segmentation addition operation on the target training sample to obtain a target training sample containing special characters; inputting the target training sample containing special characters into the semantic information extraction layer in the initial category prediction model, based on the self-attention learning mechanism, learning the semantic information of each segmentation and the semantic association between each segmentation, so as to obtain the target semantic information corresponding to the special character as the semantic information of the target training sample.

[0062] In some embodiments, a special word segmentation adding operation is performed on a target training sample to obtain a target training sample containing special characters, including: selecting special characters. The selection of special characters depends on the specific task and model architecture. Common special characters include <cls>(Classification) and <sep>(separated), for example, in the BERT model, <cls>The token is added to the beginning of the sentence, <sep>The tag is added to the end of the sentence; word segmentation: the text content of the target training sample is divided into multiple tokens, which can be words, subwords or characters; the tokenizer converts the text content into a token sequence according to predefined rules, and the token sequence contains multiple tokens in the text content of the target training sample; adding special characters: adding special characters to the token sequence, for example, if processing a single sentence, <cls>will be added to the beginning of the sequence, <sep>It will be added to the end of the sequence; encoding (Encoding), converting multiple participles (including special characters) into numerical representations (digital form) that the model can understand. Specifically, the numerical representation of each participle in the vocabulary (including each participle and its corresponding numerical mapping relationship) can be found to input the numerical representation corresponding to each participle into the model. The numerical representation corresponding to each participle can represent the initial semantic information of the participle. The numerical representation corresponding to each participle can be implemented in the form of a matrix, which contains the numerical representation corresponding to each participle. In addition, in order to enable the model to understand the position of multiple participles in the participle sequence, position encoding can also be added to each participle in the participle sequence, which is conducive to the model capturing the order information in the sequence data.

[0063] In some embodiments, a target training sample containing special characters is input into a semantic information extraction layer in an initial category prediction model, and based on a self-attention learning mechanism, the semantic association relationship between each word segment is learned to obtain the target semantic information corresponding to the special character as the semantic information of the target training sample, including: inputting the numerical representation corresponding to the word segmentation sequence containing special characters into the semantic information extraction layer in the initial category prediction model, performing vector mapping on the numerical representation corresponding to each word segmentation in the target training sample to obtain the query vector, key vector and value vector corresponding to each word segmentation; for each word segmentation, performing matching calculations using the query vector of the word segmentation and the key vector of each word segmentation to obtain the attention weight of each word segmentation; performing a full connection calculation operation on the value vector of each word segmentation according to the attention weight of each word segmentation to obtain the target semantic information corresponding to the special character as the semantic information of the target training sample.

[0064] Optionally, the numerical representation corresponding to the word segmentation sequence containing special characters is input into the semantic information extraction layer in the initial category prediction model, and vector mapping is performed on the numerical representation corresponding to each word segmentation in the target training sample to obtain the query vector, key vector and value vector corresponding to each word segmentation, including: mapping the numerical representation of each word segmentation in the word segmentation sequence to the query vector, key vector and value vector respectively through different weight matrices, such as mapping them to the query vector, key vector and value vector respectively through three different weight matrices W^Q, W^K, and W^V.

[0065] Optionally, for each word segment, the query vector of the word segment is used to calculate the matching degree with the key vector of each word segment to obtain the attention weight of each word segment, including, for the current word segment (each word segment), the query vector of the word segment is calculated with the key vector of each word segment (including the current word segment) to obtain multiple score values, each score value represents the attention weight of the word segment providing the key vector to the current word segment. The attention weight reflects the relative importance of one word segment to another word segment in the sequence. The higher the weight, the more important the corresponding word segment is in the current context.

[0066] Further optionally, the multiple score values ​​may include positive scores and negative scores. In order to facilitate the subsequent full connection calculation, each score value may be normalized to obtain multiple normalized values, and the multiple normalized values ​​are all positive numbers. The normalization process here may be an exponential calculation operation (the score value is used as a parameter of the exponential calculation), but is not limited thereto.

[0067] Further optionally, after obtaining the attention weight of each segmentation, before performing the pooling process, a weighted sum calculation can be performed on the value vector of each segmentation according to the attention weight of each segmentation to obtain a weighted sum vector (updated representation). The weighted sum vector incorporates the segmentations at all positions in the input sequence, but the contribution of the segmentations at different positions is determined by their attention relative to the query. This means that for each segmentation in the segmentation sequence, its weighted sum vector is a weighted combination of other segmentation information associated with it in the entire segmentation sequence, and the weight is determined by the attention score (normalized). Through this process, the weighted sum vector of each segmentation not only represents the semantic information of the segmentation itself, but also represents the synthesis of the contextual semantic information of the entire segmentation sequence, which enables the model to capture long-distance dependencies within the sequence and take into account the semantic information of the entire sequence when processing the current segmentation.

[0068] Further optionally, after obtaining the weighted sum vector of each word segment, before performing the pooling process, a fully connected calculation operation can be performed on the value vector of each word segment according to the attention weight of each word segment to obtain the target semantic information corresponding to the special character as the semantic information of the target training sample, including: inputting the value vector obtained by weighted summation into the fully connected layer, the fully connected layer includes two linear transformation functions and a nonlinear activation function such as ReLU); the first linear transformation function maps the value vector to an intermediate dimension, and then applies the nonlinear activation function to perform nonlinear calculation, and finally the second linear transformation function maps the nonlinear calculation result back to the original dimension or the target dimension to output the semantic information of the target training sample corresponding to the special character, the semantic information of the target training sample integrates the context information of the entire input word segmentation sequence, and can be used as the semantic information of the target training sample.

[0069] Further optionally, the semantic information of the target training sample is a multidimensional parameter vector. Based on this, after obtaining the semantic information of the target training sample, the semantic information of the target training sample is pooled using the pooling layer in the initial category prediction model to obtain the pooled semantic information of the target training sample, including: determining the window size of the pooling window and the sliding step of the pooling window corresponding to the pooling process, and the values ​​of the window size and the sliding step are equal; based on the window size and the sliding step, for the multidimensional parameter vector, in the area covered by each pooling window, the maximum value of the parameter is selected as the output to obtain the pooled semantic information of the target training sample; or, based on the window size and the sliding step, for the multidimensional parameter vector, in the area covered by each pooling window, the average value of all parameter values ​​is calculated as the output to obtain the pooled semantic feature vector of the target training sample; or, based on the window size and the sliding step, for the multidimensional parameter vector, in the area covered by each pooling window, the sum of all parameter values ​​is calculated as the output to obtain the pooled semantic feature vector of the target training sample.

[0070] In some embodiments, the pooled semantic information of the target training sample is respectively input into at least two levels of category prediction layers to perform prediction of at least two levels of categories to obtain prediction results of at least two levels of categories corresponding to the target training sample, including: inputting the pooled semantic information of the target training sample into at least two levels of category prediction layers, analyzing the similarity between the pooled semantic information and the semantic information of multiple known categories at the corresponding category level; based on the similarity, determining the probability value of the target training sample belonging to the multiple known categories contained in each category level as the prediction result of at least two levels of categories corresponding to the target training sample.

[0071] Optionally, analyzing the similarity between the pooled semantic information and the semantic information of multiple known categories at the corresponding category level includes: calculating the similarity between the pooled semantic information and the semantic information of each known category, and the calculation method includes cosine similarity, Euclidean distance, etc. Cosine similarity evaluates their similarity by measuring the angle between two vectors, and Euclidean distance measures the straight-line distance between them. The calculation formula of cosine similarity is: Among them, A and B are the semantic information vector after pooling and the semantic information vector of the category level, "·" represents the dot product, and ||A|| and ||B|| are the moduli of the vectors, respectively; for at least two levels of category prediction layers, it is necessary to calculate the similarity between the semantic information after pooling and all known categories at each level, which can be done by iteratively comparing the semantic vectors of each category, such as constructing a similarity matrix, in which each row represents a category level, each column represents a known category, and the values ​​in the matrix represent the similarity between the semantic information after pooling and the corresponding category. According to the similarity matrix, category prediction can be performed. Usually, the category with the highest similarity will be selected as the most likely prediction result.

[0072] In the above-mentioned embodiments of the present application, in the process of self-attention learning, only the encoding process is used to realize the extraction and comparison of semantic information, and the decoding process is not involved, thereby improving the efficiency of the self-attention learning process.

[0073] In an embodiment of the present application, in order to improve the prediction accuracy of the model, the parameters of the model can also be continuously adjusted by calculating the loss function between the network layers. Therefore, the loss calculation on the same-level category can be performed according to the labeling results and prediction results of the same-level category corresponding to each target training sample to obtain the loss function on at least two levels of categories, and the parameters of each network layer are adjusted according to the loss function on at least two levels of categories. Optionally, the loss calculation on the same-level category is performed according to the labeling results and prediction results of the same-level category corresponding to each target training sample to obtain the loss function on at least two levels of categories, including: according to the labeling results and prediction results of the same-level category corresponding to each target training sample, respectively obtain the loss function on at least two levels of categories, wherein the prediction result of the category is the predicted category of the model; wherein different category levels use different loss functions for the same loss function, or different category levels use different loss functions, and the higher the category level, the higher the complexity of the loss calculation method.

[0074] In the embodiment of the present application, in order to further improve the accuracy of the model, a comprehensive loss function can also be calculated based on the loss function on at least two levels of categories, and the network parameters of the semantic information extraction layer and at least two levels of category prediction layer in the initial category prediction model can be adjusted according to the comprehensive loss function until the comprehensive loss function meets the model training termination condition to obtain an intermediate category prediction model. Among them, the model training termination condition can be that the output result of the comprehensive loss function is less than a set threshold.

[0075] In the embodiment of the present application, the semantic information extraction layers in the intermediate category prediction model can also be copied and spliced ​​to obtain a target category prediction model that includes at least a plurality of semantic information extraction layers connected in sequence and at least two-level category prediction layers, so that when performing category prediction, multiple semantic information extraction layers can be used to extract semantic information from multiple text segments of the target training sample respectively, thereby improving the efficiency of semantic information extraction and thus improving the prediction efficiency of the category prediction model. An exemplary schematic diagram of the target category prediction model is shown in FIG. Figure 2c As shown, it contains multiple semantic information extraction layers.

[0076] Further optionally, a selection mechanism for the semantic information extraction layer can be preset. Then, when performing category prediction according to the target category prediction model, a number of semantic information extraction layers that are adapted to the text length of the text content to be predicted can be adaptively selected from multiple semantic information extraction layers according to the text length of the text content to be predicted and the selection mechanism of the semantic information extraction layer to extract semantic information from the text content to be predicted, without starting the remaining semantic information extraction layers. Thus, video memory is allocated to an adapted number of semantic information extraction layers according to the actual text length of the text content to be predicted, which can save the size of video memory occupied during prediction, improve throughput, and increase prediction speed.

[0077] For example, the target category prediction model includes 5 semantic information extraction layers, and the upper limit of the text length processed by each semantic information extraction layer at a time is 500 characters. If the text length of the current text content to be predicted is 1000 characters, then according to the preset semantic information extraction layer selection mechanism and text length, two semantic information extraction layers can be selected from the 5 semantic information extraction layers and video memory can be allocated to them to support their operation, and two text fragments (each text fragment is 500 characters) are respectively input into the two semantic information extraction layer pairs to extract the semantic information of the two text fragments.

[0078] In the above embodiments of the present application, training samples in the same length range are divided into the same training subset, and training subsets of different length ranges are shuffled, and target training samples are selected from each shuffled training subset to train the initial category prediction model, rather than training the initial category prediction model according to the rule of length range from small to large or from large to small. This can not only avoid the problem of large differences in the results of category prediction for posts of different lengths made by the trained model, but also improve the accuracy of the category prediction results of the trained model for posts of different lengths, and improve the generalization ability of the trained model. In the process of model training, the loss calculation on the same-level category is performed according to the labeling results and prediction results of the same-level category corresponding to each target training sample, and the network parameters of the semantic information extraction layer and each-level category prediction layer are adjusted to improve the calculation accuracy of each network layer and the prediction accuracy of the model; further, the comprehensive loss function can be calculated according to the loss function on at least two levels of categories, and the network parameters of the semantic information extraction layer and each-level category prediction layer in the initial category prediction model are further adjusted according to the comprehensive loss function until the comprehensive loss function meets the model training termination condition to obtain the intermediate category prediction model, thereby greatly improving the accuracy of the prediction model. Further, according to the network parameters of the trained semantic information extraction layer, the semantic information extraction layer in the intermediate category prediction model can be copied and spliced ​​to obtain a target category prediction model including multiple semantic information extraction layers connected in sequence, so that when performing category prediction, multiple semantic information extraction layers can be used to extract semantic information from multiple text fragments of the target training sample respectively, thereby improving the efficiency of semantic information extraction and thus improving the prediction efficiency of the category prediction model.

[0079] The above and Figure 2a , Figure 2b and Figure 2c The relevant embodiments specifically describe the training process of the target category prediction model. Figure 3a , Figure 3b and Figure 3c A related embodiment is a specific process of performing category prediction on text content of a to-be-predicted category based on a target category prediction model.

[0080] Figure 3a The following is a flow chart of a category prediction method provided by an exemplary embodiment of the present application. Figure 3a As shown, the method includes:

[0081] 301. Obtain text content of the category to be predicted;

[0082] 302. When the length of the text content is greater than a text length threshold supported by the target category prediction model, the text content is divided into at least two text segments, and the length of each text segment is not greater than the text length threshold; the target category prediction model includes at least a plurality of semantic information extraction layers and at least two levels of category prediction layers;

[0083] 303. Inputting at least two text segments into at least two semantic information extraction layers in the target category prediction model to extract semantic features to obtain at least two semantic information; the at least two semantic information extraction layers are semantic information extraction layers selected from a plurality of semantic information extraction layers and are the same in number as the at least two text segments;

[0084] 304. Generate target semantic information corresponding to the text content according to the at least two pieces of semantic information;

[0085] 305. Input the target semantic information into at least two-level category prediction layer to predict at least two-level categories, so as to obtain at least two-level categories corresponding to the text content.

[0086] This embodiment does not limit the specific implementation method of obtaining the text content of the category to be predicted. Figure 3b The post or article to be published page shown obtains the text content filled in by the user as the text content to be predicted. Alternatively, the text content of the post or article of the category to be predicted is selected from the associated database as the text content to be predicted.

[0087] In an embodiment of the present application, the text length threshold supported by the target category prediction model actually refers to the text length threshold that a single semantic information extraction layer in the target category prediction model can process at a time. In the case where the length of the text content is greater than the text length threshold supported by the target category prediction model, that is, in the case where the length of the text content is greater than the text length threshold that a single semantic information extraction layer can process at a time, the text content of the category to be predicted can be divided into at least two text segments according to the text length threshold that each semantic information extraction layer can process at a time, the length of each text segment is not greater than the text length threshold, and the text length of each text segment can be the same or different.

[0088] Further, after obtaining at least two text segments, video memory is allocated to the same number of semantic information extraction layers according to the number of text segments to support their operation. It should be noted that, when the text length of each text segment is different, it is necessary to adaptively select a semantic information extraction layer that supports the text length of each text segment and allocate video memory to it to support its operation.

[0089] In an embodiment of the present application, each text fragment contains multiple participles; at least two text fragments are input into at least two semantic information extraction layers in the target category prediction model to extract semantic features to obtain at least two semantic information, including: performing a special participle adding operation for each text fragment to obtain at least two text fragments containing special participles according to the number of text fragments; each text fragment is input into a semantic information extraction layer with allocated video memory, and based on a self-attention learning mechanism, the semantic information of each participle in the text fragment and the semantic association relationship between each participle are learned to obtain the target semantic information corresponding to the special character as the semantic information of the text fragment to obtain at least two semantic information.

[0090] In some embodiments, each text segment containing special characters is input into a corresponding semantic information extraction layer in the target category prediction model, and based on the self-attention learning mechanism, the semantic information of each word segment and the semantic association relationship between the word segment are learned to obtain the semantic information of the text segment corresponding to each special character, including: performing vector mapping for each word segment in each text segment to obtain the query vector, key vector and value vector corresponding to each word segment in the text segment; for each word segment in the text segment, using the query vector of the word segment and the key vector of each word segment to perform matching calculation respectively to obtain the attention weight of each word segment in the text segment; performing a full connection calculation operation on the value vector of each word segment in the text segment according to the attention weight of each word segment in the text segment to obtain at least two semantic information.

[0091] In an embodiment of the present application, after obtaining at least two pieces of semantic information, target semantic information corresponding to the text content is generated based on the at least two pieces of semantic information, including: using the pooling layer in the target category prediction model to perform pooling processing on the at least two pieces of semantic information to obtain the target semantic information corresponding to the text content.

[0092] In some embodiments, the pooling layer has a pooling window, and the window size of the pooling window and the sliding step size of the pooling window are equal in value; based on at least two semantic information, target semantic information corresponding to the text content is generated, including: inputting at least two semantic information into the pooling layer in the target category prediction model, semantically splicing the at least two semantic information to obtain the spliced ​​semantic information, and the spliced ​​semantic information is a multidimensional parameter vector; based on the window size and the sliding step size, for the multidimensional parameter vector, in the area covered by each pooling window, the maximum value of the parameter is selected as output to obtain the target semantic information corresponding to the text content; or, based on the window size and the sliding step size, for the multidimensional parameter vector, in the area covered by each pooling window, the average value of all parameter values ​​is calculated as output to obtain the target semantic information corresponding to the text content; or, based on the window size and the sliding step size, for the multidimensional parameter vector, in the area covered by each pooling window, the sum of all parameter values ​​is calculated as output to obtain the target semantic information corresponding to the text content.

[0093] In an embodiment of the present application, target semantic information is input into at least two levels of category prediction layers to perform prediction of at least two levels of categories to obtain at least two levels of categories corresponding to the text content, including: inputting the target semantic information into at least two levels of category prediction layers respectively, analyzing the similarity between the target semantic information and the semantic information of multiple known categories at the corresponding category level; determining the probability value of the target training sample belonging to the multiple known categories contained in each category level based on the similarity, the probability value of each category indicating the possibility that the text content belongs to the category, and the larger the probability value, the greater the possibility; based on the probability values ​​of the multiple known categories contained in each category level, selecting the known categories whose probability values ​​are ranked before the preset ranking as the at least two levels of categories corresponding to the text content, for example, selecting the top 10 consistent categories in the probability value ranking in each level of categories as the at least two levels of categories corresponding to the text content.

[0094] It should be noted that in the training stage of the target prediction model, at least two levels of category prediction layers have learned the unique identifier (such as ID) corresponding to each category in the platform. Then, when predicting categories, the output results of at least two levels of category prediction layers can be the identifier corresponding to each known category and its probability value, and then based on the identifier, find out which category each identifier corresponds to from the known category-identifier mapping table. Alternatively, the output results of at least two levels of category prediction layers can also be each known category and its probability value. Alternatively, the output results of at least two levels of category prediction layers can also be each known category, the identifier corresponding to each known category and its probability value. Taking "home improvement service" as an example, the corresponding prediction results can be "cate": 1043, "cateName": "home improvement service", "p": 0.5791.

[0095] It should be noted that Figure 3a , Figure 3b and Figure 3c The specific implementation methods and technical effects of each step in the relevant embodiments can be found in 1a, Figure 2b and Figure 2c The relevant descriptions of the relevant embodiments will not be repeated here.

[0096] Further optionally, when the text content is content to be posted sent by the client, at least two levels of categories corresponding to the text content are sent to the client, so that the client can display it on the posting page for the user to configure the at least two levels of categories to which the text content belongs on the posting page.

[0097] Optionally, at least two levels of categories corresponding to the text content are sent to the client, including: there can be multiple predicted categories at each level, and there is an arrangement order between multiple prediction results of the same level category, and the arrangement order is determined according to the similarity between the text content and each predicted category; the top N prediction results corresponding to each level of category are selected and pushed to the client for display.

[0098] Further optionally, taking posting as an example, the user can select any category from at least two levels of predicted categories displayed on the post-to-be-posted page of the client as the category to which the text content of the post-to-be-posted belongs.

[0099] Further optionally, in response to the user's post publishing operation, the post to be published is published to the corresponding category of the corresponding platform so that other users can view the post content.

[0100] Figure 3a It is an exemplary schematic diagram of the backend corresponding to the prediction results under each level of category predicted based on the text content in the figure. The prediction results include the ID corresponding to each predicted category, the category name and the probability value of the text content belonging to the category. 2b is an exemplary schematic diagram of at least two levels of predicted categories displayed on the client's posting page. This embodiment does not limit the ranking method of at least two levels of predicted categories displayed on the posting page. For example, for categories of the same level, they can be arranged in order of the probability value belonging to each category, or they can be arranged randomly. For categories of different levels, they can be arranged in partitions.

[0101] Figure 4a A flow chart of a cross-category recognition model training method provided by an exemplary embodiment of the present application. Figure 4a As shown, the method includes:

[0102] 401. Acquire multiple training samples, each training sample including published text content and at least one primary category selected for the text content when the text content is published;

[0103] 402. According to the prompt word structure corresponding to the target language model, a plurality of training samples are respectively converted into corresponding structured prompt words, wherein the structured prompt word corresponding to each training sample at least includes the training sample and a guide instruction, wherein the guide instruction is used to instruct the target language model to identify whether the text content in the training sample matches at least one primary category according to the main semantic information of the text content in the training sample;

[0104] 403. Input the structured prompt words corresponding to each training sample into the target language model, and under the guidance of the guide instruction, identify whether the text content in the training sample matches at least one primary category according to the main semantic information of the text content in the training sample and output the recognition result;

[0105] 404. According to the recognition result output by the target language model, each training sample is labeled as a positive training sample or a negative training sample, and the initial cross-category recognition model is trained according to the positive training samples and the negative training samples to obtain the target cross-category recognition model.

[0106] In an embodiment of the present application, in order to obtain a cross-category recognition model with high cross-category recognition accuracy, the initial cross-category recognition model can be trained to obtain a target cross-category recognition model. Specifically, multiple training samples can be obtained, and the initial cross-category recognition model can be trained using the multiple training samples. Among them, the multiple training samples can be posts, articles, etc. published by the platform, each training sample includes the published text content and at least one primary category selected for the text content when the text content is published, and the text content includes but is not limited to: Chinese and its punctuation marks, foreign languages ​​and their punctuation marks.

[0107] This embodiment does not limit the specific implementation method of obtaining multiple training samples. For example, multiple samples containing published text content and at least one first-level category selected for the text content when the text content is published are selected from an associated database as multiple training samples; or multiple samples containing text content and at least one first-level category corresponding to the text content are generated manually or by a model as multiple training samples.

[0108] In order to further improve the generalization ability and accuracy of the cross-category recognition model, multiple training samples can be labeled, and the multiple training samples can be labeled as positive training samples and negative training samples. The initial cross-category recognition model can be trained using the positive training samples and negative training samples to obtain a target cross-category recognition model with strong generalization ability and high cross-category recognition accuracy. Among them, positive training samples refer to training samples labeled as non-cross-category, and non-cross-category means that the text content matches at least one of its corresponding categories. Negative training samples refer to training samples labeled as cross-category, and non-cross-category refers to training samples whose text content does not match at least one of its corresponding categories.

[0109] In an embodiment of the present application, a target language model can be used to annotate multiple training samples. In order to further improve the training ability of multiple training samples for the initial cross-category recognition model, a prompt word structure corresponding to the target language training model is introduced, and the multiple training samples are respectively converted into corresponding structured prompt words according to the prompt word structure corresponding to the target language model. The structured prompt words corresponding to each training sample at least include the training sample and a guide instruction, and the guide instruction is used to instruct the target language model to identify whether the text content in the training sample matches at least one primary category according to the main semantic information of the text content in the training sample.

[0110] In some embodiments, the multiple training samples are converted into corresponding structured prompt words according to the prompt word structure corresponding to the target language model, including: obtaining the prompt word structure and adapted guide instructions corresponding to the target language model, adding the text content of each training sample and its adapted guide instructions to the prompt word structure to obtain structured prompt words corresponding to the multiple training samples.

[0111] Figure 4b is an exemplary prompt word structure corresponding to a training sample, wherein "{item, content}" is the text content of the training sample, "{item, cateName}" is at least one first-level category corresponding to the training sample, and "Does the text content of this post match the post category?" is the guiding instruction corresponding to the training sample.

[0112] In an embodiment of the present application, after obtaining the structured prompt words corresponding to the training samples, the structured prompt words corresponding to each training sample can be input into the target language model. Under the guidance of the guide instructions, the text content in the training sample is identified according to the main semantic information of the text content in the training sample to see whether it matches at least one first-level category and the recognition result is output.

[0113] Further optionally, in order to further improve the generalization ability and accuracy of the cross-category recognition model, the structured prompt words corresponding to each training sample may also include: background information, which is used to describe the affiliation and / or non-affiliation relationship between a specific secondary category and a specific primary category, such as Figure 4c The background information shown in the example is: "Installation and repair are not wholesale purchases", "Cutting does not belong to wholesale", etc., where installation, repair, and cutting belong to specific secondary categories, and wholesale purchases and wholesale belong to specific primary categories. In addition, when publishing text content, at least two categories can be selected for the text content.

[0114] Based on this, in some embodiments, the structured prompt words corresponding to each training sample are input into the target language model, and under the guidance of the guiding instructions, the main semantic information of the text content in the training sample is used to identify whether the text content in the training sample matches at least one first-level category and output the recognition result, including: the structured prompt words corresponding to each training sample are input into the target language model, and under the guidance of the guiding instructions, according to the main semantic information and background information of the text content in the training sample, whether the text content in the training sample matches at least two-level categories, and output the recognition result.

[0115] In some embodiments, the structured prompt words corresponding to each training sample are input into the target language model, and under the guidance of the guide instruction, whether the text content in the training sample matches at least two levels of categories is identified according to the main semantic information and background information of the text content in the training sample, and the recognition result is output, including: inputting the structured prompt words corresponding to each training sample into the target language model, performing semantic learning on the guide instruction and the background information in the structured prompt words to obtain the semantic information of the guide instruction and the semantic information of the background information; performing semantic relevance learning on the text content in the training sample and at least two levels of categories according to the semantic information of the guide instruction and the semantic information of the background information to obtain the semantic matching degree between the text content and at least two levels of categories; wherein the target semantic matching degree includes the overall semantic matching degree between the text content and at least two levels of categories, or includes the local semantic matching degree between the text content and each level of categories; according to the semantic matching degree between the text content and at least two levels of categories, whether the text content matches the at least two levels of categories is identified, and the recognition result is output, and the recognition result can be a match or a mismatch.

[0116] Optionally, the structured prompt words corresponding to each training sample are input into the target language model, and semantic learning is performed on the guiding instructions and background information in the structured prompt words to obtain the semantic information of the guiding instructions and the semantic information of the background information, including: the guiding instructions and the background information in the structured prompt words are input into the target language model, and based on the self-attention mechanism, semantic learning is performed on the guiding instructions and the background information respectively to obtain the semantic information of the guiding instructions and the semantic information of the background information.

[0117] Further optionally, the guiding instruction and background information in the structured prompt word are input into the target language model, and based on the self-attention mechanism, semantic learning is performed on the guiding instruction and the background information respectively to obtain the semantic information of the guiding instruction and the semantic information of the background information, including: inputting the guiding instruction and the background information in the structured prompt word into the target language model, performing vector mapping for each word segment in the guiding instruction and the background information to obtain the query vector, key vector and value vector corresponding to each word segment; for each word segment in the guiding instruction, using the query vector of the word segment and the key vector of each word segment in the guiding instruction to perform matching calculation respectively to obtain the attention weight of each word segment in the guiding instruction; performing a full-link calculation operation on the value vector of each word segment according to the attention weight of each word segment in the guiding instruction to obtain the semantic information of the guiding instruction; for each word segment in the background information, using the query vector of the word segment and the key vector of each word segment in the background information to perform matching calculation respectively to obtain the attention weight of each word segment in the background information; performing a full-link calculation operation on the value vector of each word segment according to the attention weight of each word segment in the background information to obtain the semantic information of the background information.

[0118] Optionally, based on the semantic information of the guidance instruction and the semantic information of the background information, the semantic relevance of the text content and at least two levels of categories in the training sample is learned to obtain the semantic matching degree between the text content and at least two levels of categories, including: inputting the text content and at least two levels of categories in the training sample into the target language model, and based on the self-attention mechanism, respectively performing semantic learning on the text content and the at least two levels of categories to obtain the semantic information corresponding to the text content and the at least two levels of categories; based on the cross-attention mechanism, learning the semantic association relationship between the semantic information corresponding to the text content, the at least two levels of categories, the guidance instruction and the background knowledge to obtain the associated semantic information between the semantic information corresponding to the text content and the at least two levels of categories; based on the associated semantic information, obtaining the target semantic matching degree between the text content and at least two levels of categories.

[0119] Further optionally, the text content and at least two levels of categories in the training sample are input into the target language model, and based on the self-attention mechanism, semantic learning is performed on the text content and the at least two levels of categories respectively to obtain semantic information corresponding to the text content and the at least two levels of categories, including: the text content and the at least two levels of categories are input into the target language model, and vector mapping is performed for each word segment in the text content and the at least two levels of categories to obtain a query vector, a key vector and a value vector corresponding to each word segment; for each word segment in the text content, the query vector of the word segment is used to map the query vector of the word segment with the guide instruction. The key vectors of each word in the text content are matched respectively to obtain the attention weight of each word in the text content; the value vectors of each word are fully linked according to the attention weight of each word in the text content to obtain the semantic information of the text content; for each word in each level of at least two levels of categories, the query vector of the word is used to perform matching calculations with the key vectors of each word in the category of the same level to obtain the attention weight of each word in each level of categories; the value vectors of each word are fully linked according to the attention weight of each word in each level of categories to obtain the semantic information of each level of categories.

[0120] Further optionally, based on the cross-attention mechanism, the semantic association relationship between the semantic information corresponding to the text content, at least two levels of categories, guiding instructions and background knowledge is learned to obtain the associated semantic information between the text content and the semantic information corresponding to the at least two levels of categories, including: respectively constructing query vectors corresponding to the text content, at least two levels of categories, guiding instructions and background knowledge; for any one of the text content, at least two levels of categories, guiding instructions and background knowledge, using the query vector of the content and the key vectors of other contents to perform matching calculations respectively to obtain the attention weight of each content; performing a full-link calculation operation on the value vectors corresponding to other contents according to the attention weight corresponding to each content, to obtain the associated semantic information between the text content and the semantic information corresponding to the at least two levels of categories.

[0121] In an embodiment of the present application, after obtaining the recognition result based on the target language model, the initial cross-category recognition model may also be trained based on positive training samples and negative training samples to obtain a target cross-category recognition model.

[0122] In some embodiments, after obtaining the recognition result based on the target language model, the initial cross-category recognition model can also be trained based on the positive training samples and the negative training samples to obtain the target cross-category recognition model, including: taking the positive training samples and the negative training samples as the samples to be recognized whose annotation results are not cross-category and the samples to be recognized whose annotation results are cross-category, respectively, to obtain a set of samples to be recognized; inputting the samples to be recognized in the set of samples to be recognized into the initial cross-category recognition model respectively, and learning the semantic relevance between the text content and at least two levels of categories in the samples to be recognized to obtain the semantic matching degree between the text content and at least two levels of categories; performing cross-category prediction on the samples to be recognized based on the semantic matching degree between the text content and at least two levels of categories to obtain a prediction result of whether the samples to be recognized are cross-category; calculating the loss function based on the annotation result and the prediction result of whether the samples to be recognized are cross-category, and when the loss function satisfies the termination condition of the model training, adjusting the network parameters of the initial cross-category recognition model until the loss function satisfies the termination condition of the model training, to obtain the target cross-category recognition model.

[0123] In the above embodiment of the present application, the structured prompt words of each training sample are generated using the prompt word structure corresponding to the target language model. Based on the structured prompt words of the training samples, the target semantic model is used to identify and annotate whether the main semantic information of the training samples matches at least one of the corresponding primary categories under the guidance of the prompt words, and the training samples are annotated as positive samples and negative samples according to the recognition results, so as to train the initial cross-category recognition model to be trained, so as to obtain the target cross-category recognition model, so as to identify whether the published text content matches at least one of the corresponding primary categories based on the target cross-category recognition model, thereby improving the accuracy of cross-category recognition.

[0124] The following is a detailed description of the cross-category identification process. Figure 5 The specific implementation methods of each step in the relevant embodiments can be found in Figure 2a , Figure 2b , Figure 2c , Figure 3a , Figure 3b , Figure 3c , Figure 4a , Figure 4b , Figure 4c The relevant descriptions of the related embodiments are not repeated in this embodiment.

[0125] Figure 5 The following is a flow chart of a cross-category identification method provided by an exemplary embodiment of the present application. Figure 5 As shown, including:

[0126] 501. Acquire the published target content and at least one primary category selected for the target content when publishing the target content;

[0127] 502. Input the target content and at least one first-level category into the target cross-category recognition model, identify whether the target content matches at least one first-level category according to the main semantic information of the target content, and output the recognition result;

[0128] 503. Mark the target content as cross-category or non-cross-category based on the recognition result;

[0129] The target cross-category recognition model is obtained by training the initial cross-category recognition model based on negative training samples whose labeled results are cross-category and positive training samples whose labeled results are not cross-category.

[0130] Optionally, when publishing text content, at least two levels of categories are selected for the text content; the target content and at least one level of categories are input into a target cross-category recognition model, and whether the target content matches the at least one level of categories is identified according to the main semantic information of the target content, and the recognition result is output, including: based on the self-attention mechanism, semantic learning is performed on the guide instruction and the background information respectively to obtain the semantic information corresponding to the semantic information of the guide instruction and the semantic information of the background information; based on the cross-attention mechanism, the semantic association relationship between the text content and the semantic information corresponding to each of the at least two levels of categories is learned to obtain the associated semantic information between the text content and the semantic information corresponding to each of the at least two levels of categories, and the associated semantic information is used to characterize the target semantic matching degree between the text content and the at least two levels of categories; wherein the associated semantic information is used to characterize the target semantic matching degree between the text content and the at least two levels of categories, and the target semantic matching degree includes the overall semantic matching degree between the text content and the at least two levels of categories, or includes the local semantic matching degree between the text content and each level of categories; according to the semantic matching degree between the text content and the at least two levels of categories, whether the text content matches the at least two levels of categories is identified, and the recognition result is output.

[0131] Figure 6 The schematic diagram of the structure of the electronic device provided by the exemplary embodiment of the present application. Figure 6 As shown, it includes: a memory 30a and a processor 30b; the memory 30a is used to store a computer program; the processor 30b is coupled to the memory 30a and is used to execute the computer program to implement the following steps:

[0132] The initial text content of the target content to be generated is obtained, the initial text content corresponds to a target category, and the target category is predicted by using a target category prediction model based on the main semantic information of the initial text content; the prompt word, the initial text content and its corresponding target category are input into a target AI language model based on artificial intelligence, at least one keyword matching the target category in the initial text content is extracted and the at least one keyword is subject-classified, and based on at least one keyword under each topic, detailed description information corresponding to the topic is generated to obtain the target text content; wherein the AI ​​language model based on artificial intelligence is obtained by training the initial AI language model based on artificial intelligence based on multiple text contents containing at least one detailed description information marked with a topic tag as training samples.

[0133] In an embodiment of the present application, when the processor obtains the initial text content of the target content to be generated, it is specifically used to: in response to the user's input operation in the content input box of the post publishing page, obtain the initial text content, so as to generate the target text content based on the initial text content; or, in response to the user's selection operation on any published post on the post rewrite page, obtain the published post content as the initial text content, so as to generate the target text content based on the initial text content.

[0134] In an embodiment of the present application, the target AI language model based on artificial intelligence includes: a semantic information extraction layer, a semantic information classification layer and a content generation layer; when the processor inputs the prompt word, the initial text content and its corresponding target category into the AI ​​language model based on artificial intelligence, extracts at least one keyword matching the target category in the initial text content and classifies the at least one keyword by subject, and generates detailed description information corresponding to the subject based on at least one keyword under each subject to obtain the target text content, it is specifically used to: input the initial text content and its corresponding target category into the semantic information extraction layer, extract the semantic information of the target category and the semantic information of the initial text; based on the semantic information of the target category, extract at least one keyword semantic information adapted to the semantic information of the target category from the semantic information of the initial text; input at least one keyword semantic information into the semantic information classification layer, classify at least one keyword semantic information, and obtain at least one keyword semantic information set, each keyword semantic information set corresponding to a subject tag; based on each keyword semantic information set, the content generation rules and prompt words of its corresponding subject tag, generate detailed description information corresponding to the subject; based on the detailed description information corresponding to each subject, obtain the target text content.

[0135] Further optionally, after acquiring the initial text content to be generated and before inputting the prompt word, the initial text content and its corresponding target category into the AI ​​language model based on artificial intelligence, the processor can also be used to input the initial text content into the target category prediction model to predict at least one first-level category of the initial text content to obtain at least one first-level initial category corresponding to the initial text content.

[0136] Optionally, when the processor inputs the initial text content into the target category prediction model and predicts at least one first-level category of the initial text content to obtain at least one first-level category corresponding to the initial text content, it is specifically used to: when the length of the initial text content is greater than the text length threshold supported by the target category prediction model, divide the initial text content into at least two text segments, and the length of each text segment is not greater than the text length threshold; the target category prediction model includes at least multiple semantic information extraction layers and at least one first-level category prediction layer; the at least two text segments are input into the at least two semantic information extraction layers in the target category prediction model to extract semantic features to obtain at least two semantic information; the at least two semantic information extraction layers are semantic information extraction layers selected from multiple semantic information extraction layers and the same number as the at least two text segments; based on the at least two semantic information, generate target semantic information corresponding to the initial text content; input the target semantic information into at least one first-level category prediction layer to predict at least one first-level category to obtain multiple category prediction results corresponding to the initial text content including at least one first-level initial category.

[0137] Further optionally, the processor can also obtain at least one first-level category corresponding to the category prediction result in response to the user's selection operation on any category prediction result; input the at least one first-level category corresponding to the category prediction result into the cross-category prediction model, identify whether the initial text content matches the at least one first-level category corresponding to the category prediction result according to the main semantic information of the initial target content, and output the recognition result; when the recognition result is that the initial text content is non-cross-category, the initial text content and at least one first-level category selected by the user are input into the AI ​​language model based on artificial intelligence to obtain the target text content.

[0138] Further optionally, the processor may also train the artificial intelligence-based target AI language model before inputting the prompt words, initial text content and their corresponding target categories into the artificial intelligence-based target AI language model.

[0139] Optionally, when training the target AI language model based on artificial intelligence, the processor is also used to: obtain multiple training samples; use the multiple training samples to train the initial AI language model based on artificial intelligence to obtain the target text content corresponding to at least one topic tag contained in each training sample; calculate the semantic loss function based on the text content of at least one detailed description information marked with a topic tag contained in each training sample and the target text content corresponding to at least one topic tag contained in the training sample; according to the semantic loss function, adjust the network parameters of each network layer in the initial AI language model based on artificial intelligence until the semantic loss function meets the model training conditions to obtain the target AI language model based on artificial intelligence.

[0140] Furthermore, if Figure 6 As shown, the electronic device also includes: a communication component 30c, a display 30d, a power component 30e, an audio component 30f and other components. Figure 6 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 6 Components shown.

[0141] The detailed implementation methods and beneficial effects provided by the embodiments of the present application have been described in detail in the aforementioned embodiments and will not be elaborated in detail here.

[0142] The exemplary embodiments of the present application further provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is caused to implement the steps in the above-mentioned method embodiments.

[0143] An exemplary embodiment of the present application further provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the processor is enabled to implement the steps in the above-mentioned method embodiments.

[0144] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0145] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0146] The above-mentioned display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0147] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0148] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0149] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage, etc.) containing computer-usable program code.

[0150] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0151] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0152] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0153] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0154] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0155] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0156] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, commodity or device including the element.

[0157] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.< / sep> < / cls> < / sep> < / cls> < / sep> < / cls>

Claims

1. A content generation method, characterized in that: include: Acquire initial text content of target content to be generated, the initial text content corresponds to a target category, and the target category is predicted based on main semantic information of the initial text content using a target category prediction model; Input the prompt word, the initial text content and the corresponding target category into the target AI language model based on artificial intelligence, extract at least one keyword matching the target category in the initial text content and classify the at least one keyword by subject, and generate detailed description information corresponding to the subject based on at least one keyword under each subject to obtain the target text content; The AI ​​language model based on artificial intelligence is obtained by training an initial AI language model based on artificial intelligence based on multiple text contents containing at least one detailed description information marked with a topic tag as training samples.

2. The method according to claim 1, characterized in that Get the initial text content of the target content to be generated, including: In response to an input operation of a user in a content input box of a post publishing page, the initial text content is acquired so as to generate a target text content based on the initial text content; or, In response to the user's selection operation on any published post on the post rewriting page, the published post content is obtained as the initial text content, so as to generate the target text content based on the initial text content.

3. The method according to claim 1, characterized in that: The target AI language model based on artificial intelligence includes: a semantic information extraction layer, a semantic information classification layer and a content generation layer; the prompt word, the initial text content and the corresponding target category are input into the AI ​​language model based on artificial intelligence, at least one keyword matching the target category in the initial text content is extracted and the at least one keyword is subject-classified, and detailed description information corresponding to the topic is generated based on at least one keyword under each topic to obtain the target text content, including: Inputting the initial text content and its corresponding target category into the semantic information extraction layer, extracting the semantic information of the target category and the semantic information of the initial text; based on the semantic information of the target category, extracting at least one keyword semantic information adapted to the semantic information of the target category from the semantic information of the initial text; Inputting the at least one keyword semantic information into the semantic information classification layer, classifying the at least one keyword semantic information to obtain at least one keyword semantic information set, each keyword semantic information set corresponding to a topic label; Generate detailed description information corresponding to the topic based on each keyword semantic information set, its corresponding topic tag content generation rules and prompt words; Based on the detailed description information corresponding to each topic, the target text content is obtained.

4. The method according to claim 1, characterized in that After obtaining the initial text content to be generated and before inputting the prompt word, the initial text content and the corresponding target category into the AI ​​language model based on artificial intelligence, the method further includes: The initial text content is input into a target category prediction model, and at least one first-level category of the initial text content is predicted to obtain at least one first-level initial category corresponding to the initial text content.

5. The method according to claim 4, characterized in that Inputting the initial text content into a target category prediction model, predicting at least one primary category of the initial text content to obtain at least one primary category corresponding to the initial text content, including: When the length of the initial text content is greater than a text length threshold supported by the target category prediction model, the initial text content is divided into at least two text segments, and the length of each text segment is not greater than the text length threshold; the target category prediction model includes at least a plurality of semantic information extraction layers and at least one primary category prediction layer; Inputting the at least two text segments into at least two semantic information extraction layers in the target category prediction model to extract semantic features to obtain at least two semantic information; the at least two semantic information extraction layers are semantic information extraction layers selected from the plurality of semantic information extraction layers and are the same in number as the at least two text segments; Generating target semantic information corresponding to the initial text content according to the at least two semantic information; The target semantic information is input into the at least one first-level category prediction layer to perform at least one first-level category prediction, so as to obtain a plurality of category prediction results corresponding to the initial text content and including at least one first-level initial category.

6. The method according to claim 5, characterized in that Also includes: In response to a user's selection operation on any category prediction result, obtaining at least one primary category corresponding to the category prediction result; Inputting at least one first-level category corresponding to the category prediction result into a cross-category prediction model, identifying whether the initial text content matches at least one first-level category corresponding to the category prediction result according to the main semantic information of the initial target content, and outputting the identification result; When the recognition result is that the initial text content is non-cross-category, the initial text content and at least one primary category selected by the user are input into an AI language model based on artificial intelligence to obtain the target text content.

7. The method according to any one of claims 1 to 6, characterized in that Before inputting the prompt word, the initial text content and the corresponding target category into the target AI language model based on artificial intelligence, the method further includes: Obtain multiple training samples; Using the multiple training samples to train the initial AI language model based on artificial intelligence to obtain target text content corresponding to at least one topic tag included in each training sample; Calculating a semantic loss function based on the text content of at least one detailed description information marked with a topic tag contained in each training sample and the target text content corresponding to at least one topic tag contained in the training sample; According to the semantic loss function, the network parameters of each network layer in the initial AI language model based on artificial intelligence are adjusted until the semantic loss function meets the model training conditions to obtain the target AI language model based on artificial intelligence.

8. An electronic device, characterized in that: include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to implement the steps in the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps of any one of the methods of claims 1 to 7.

Images