Cross-category model training and cross-category identification method and device, storage medium and program product

Abstract

The embodiment of the invention provides a cross-category model training and cross-category identification method and device, a storage medium and a program product. In the embodiment of the invention, the structured cue word of each training sample is generated by utilizing the cue word structure corresponding to the target language model, and the target semantic model is utilized to generate the structured cue word of each training sample based on the structured cue word of the training sample. Identifying and marking whether the main semantic information of the training sample is matched with at least one level of category corresponding to the training sample or not under the guidance of a guidance instruction of the cue word, and marking the training sample as a positive sample and a negative sample according to an identification result, so as to train a to-be-trained initial cross-category identification model to obtain a target cross-category identification model; therefore, whether the published text content is matched with the corresponding at least one level of category can be identified based on the target cross-category identification model, so that the accuracy of the target cross-category identification model is improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a cross-category model training and cross-category recognition method, device, storage medium and program product. Background Art

[0002] In the post publishing field, users need to manually select which categories the posts belong to when posting, which is a cumbersome process. Especially in the field of housekeeping services, there are a large number of categories. Posting users (such as merchants) sometimes do not know which categories their posts belong to, and thus give up posting because the posting process is too long.

[0003] In this regard, with the continuous development of artificial intelligence technology, category prediction models can be used to predict the multiple categories that a post may belong to based on the content of the post to be published, and recommend them to users, so that the posting user can choose a category that matches the content of the post. However, the charging standards for posting services provided by the platform for different categories (such as popular categories or unpopular categories) are different. When the category prediction model shows that the content of the post to be published may belong to multiple categories, there may be both high-fee categories and low-fee categories in these categories. In order to save money, the posting user may choose a category that does not match the content of the post but has a lower fee, and publish the post under this category. This has a serious impact on platform management and users' selection of posts by category.

[0004] Therefore, the platform needs to identify posts that do not belong to each category to prevent users from posting across categories. However, there is currently no effective solution for accurately identifying cross-category posts from published posts. Summary of the invention

[0005] Various aspects of the present application provide a cross-category recognition model training and cross-category recognition method, device, storage medium and program product for accurately identifying the content of posts in each category that do not match the category and based on the recognition results, improving the accuracy of target cross-category recognition.

[0006] The embodiment of the present application provides a cross-category recognition model training method, comprising: obtaining multiple training samples, each training sample comprising a published text content and at least one primary category selected for the text content when the text content is published; converting the multiple training samples into corresponding structured prompt words according to the prompt word structure corresponding to the target language model, the structured prompt word corresponding to each training sample at least comprising the training sample and a guide instruction, the guide instruction being used to instruct the target language model to identify whether the text content in the training sample matches at least one primary category according to the main semantic information of the text content in the training sample; inputting the structured prompt word corresponding to each training sample into the target language model, and under the guidance of the guide instruction, identifying whether the text content in the training sample matches at least one primary category according to the main semantic information of the text content in the training sample and outputting a recognition result; marking each training sample as a positive training sample or a negative training sample according to the recognition result output by the target language model, and training an initial cross-category recognition model according to the positive training sample and the negative training sample to obtain a target cross-category recognition model.

[0007] The embodiment of the present application also provides a cross-category identification method, which obtains the published target content and at least one first-level category selected for the target content when the target content is published; inputs the target content and the at least one first-level category into a target cross-category identification model, identifies whether the target content matches the at least one first-level category according to the main semantic information of the target content and outputs the identification result; and marks the target content as cross-category or non-cross-category based on the identification result. The target cross-category identification model is obtained by training an initial cross-category identification model based on negative training samples with cross-category annotation results and positive training samples with non-cross-category annotation results.

[0008] The application embodiment also provides an electronic device, including: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the above method.

[0009] An embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements the steps in the above method.

[0010] An embodiment of the present application also provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the processor is enabled to implement the steps in the above method.

[0011] In an embodiment of the present application, a structured prompt word for each training sample is generated using a prompt word structure corresponding to a target language model, and the structured prompt word includes a training sample and its corresponding guide instruction. Based on the structured prompt words of the training sample, a target semantic model is used to identify and annotate whether the main semantic information of the text content of the training sample matches at least one of its corresponding primary categories under the guidance of the prompt word guide instruction, and the training sample is annotated as a positive sample and a negative sample according to the recognition result, so as to train the initial cross-category recognition model to be trained based on the annotated positive and negative samples to obtain a target cross-category recognition model, so as to identify whether the published text content matches at least one of its corresponding primary categories based on the target cross-category recognition model, thereby improving the accuracy of the target cross-category recognition.

[0012] In an embodiment of the present application, a target cross-category recognition model obtained through training is used to identify whether the target content matches at least one primary category according to the main semantic information of the published target content and output the recognition result. Based on the recognition result, the target content is marked as cross-category or non-cross-category, thereby improving the accuracy of cross-category recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0014] Figure 1a A flow chart of a category prediction model training method provided by an exemplary embodiment of the present application;

[0015] Figure 1b A schematic diagram of the structure of an initial category prediction model provided by an exemplary embodiment of the present application;

[0016] Figure 1c A schematic diagram of the structural flow of a target category prediction model provided for an exemplary embodiment of the present application;

[0017] Figure 2a A schematic diagram of a flow chart of a category prediction method provided by an exemplary embodiment of the present application;

[0018] Figure 2b A schematic diagram of a front-end display page provided for an exemplary embodiment of the present application;

[0019] Figure 2c A schematic diagram of a backend display page provided for an exemplary embodiment of the present application;

[0020] Figure 3a A flowchart of a cross-category recognition model training method provided for an exemplary embodiment of the present application;

[0021] Figure 3b A schematic diagram of a prompt word structure provided for an exemplary embodiment of the present application;

[0022] Figure 3c Another schematic diagram of a prompt word structure provided for another exemplary embodiment of the present application;

[0023] Figure 4 A schematic diagram of a cross-category identification method provided for an exemplary embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0026] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0027] In an embodiment of the present application, when a user publishes text content on some platforms, he or she will select at least one primary category corresponding to the text content, so that when the text content is published, the text content can be directly published to the corresponding category of the corresponding platform based on the selected at least one primary category. For some platforms, the charging standards for publishing text content of different categories are different. The charging standards for some popular categories will be relatively high, and the charging standards for some unpopular categories will be relatively low. In order to reduce the publishing cost, users usually add a small amount of cross-category text content to the text content to be published. For example, if the original text content corresponds to category A, and category A charges a higher fee, the user can add a small amount of text content belonging to category B to the original text content. Category B charges a lower fee, and the user can publish it under category B, which can save publishing costs and avoid the risk of cross-category identification. Therefore, a cross-category recognition model with a higher accuracy is required to perform cross-category identification on these cross-category text contents.

[0028] In an embodiment of the present application, the entire process of a user publishing text content to corresponding categories of some platforms and performing cross-category identification on the published text content may specifically include the following steps: (1) Performing category prediction based on the text content to be published, predicting multiple at least first-level categories corresponding to the text content to be published, so that the user can select at least one first-level target category from the multiple at least first-level categories. For example, based on the prediction of multiple at least first-level categories for the text content to be published, the top-ranked categories in the multiple at least first-level categories can be selected and pushed to the user so that the user can select a target category at each category level. The higher the match between the category and the text content, the higher the ranking. (2) Publishing the text content to be published under the target category. (3) Performing cross-category identification on the published text content.

[0029] First, before elaborating on the training process of the cross-category recognition model and the cross-category recognition process, the training of the category prediction model and the category prediction process of the text content to be published are described in detail as follows.

[0030] Figure 1a The following is a flow chart of a method for training a category prediction model provided by an exemplary embodiment of the present application. Figure 1a As shown, the method includes:

[0031] 101. Obtain multiple training samples, each training sample corresponding to at least two levels of category labeling results;

[0032] 102. According to the text lengths of the plurality of training samples, the training samples whose text lengths are within the same length range are divided into the same training subset to obtain a plurality of training subsets, and different training subsets correspond to different length ranges;

[0033] 103. Randomly sort multiple training subsets, and perform model training on an initial category prediction model according to the target training samples in each training subset after random sorting, so as to obtain prediction results of at least two levels of categories corresponding to each target training sample; the initial category prediction model at least includes a semantic information extraction layer and at least two levels of category prediction layers, and the prediction results of at least two levels of categories are predicted by the at least two levels of category prediction layers according to the semantic information extracted from the target training samples by the semantic information extraction layer;

[0034] 104. Calculate the loss of the same-level category according to the labeling result and prediction result of the same-level category corresponding to each target training sample to obtain the loss function of at least two-level categories; calculate the comprehensive loss function according to the loss function of at least two-level categories;

[0035] 105. According to the comprehensive loss function, adjust the network parameters of the semantic information extraction layer and at least two-level category prediction layers in the initial category prediction model until the comprehensive loss function satisfies the model training termination condition to obtain an intermediate category prediction model;

[0036] 106. Copy and splice the semantic information extraction layers in the intermediate category prediction model to obtain a target category prediction model that includes at least a plurality of semantic information extraction layers connected in sequence and at least two levels of category prediction layers.

[0037] In an embodiment of the present application, in order to obtain a category prediction model with high category prediction accuracy, the initial category prediction model can be trained to obtain a target category prediction model. The initial category prediction model includes at least a semantic information extraction layer and at least two levels of category prediction layers. The semantic information extraction layer is used to extract semantic information from text content. The "at least two levels" in the at least two levels of category prediction layers is used to limit the number and hierarchical relationship of category prediction layers. Each level of category prediction layer is used to predict the category of the training sample at that level based on the semantic information extracted from the text content by the semantic information extraction layer. Figure 1b The figure is an exemplary structural diagram of the initial category prediction model, wherein the semantic information extraction layer can be implemented as a BERT (Bidirectional Encoder Representations from Transformers) model or a RoBERTa (A Robustly Optimized BERT Pretraining Approach) model. At least two levels of category prediction layers can be implemented as a Word2Vec model, a TF-IDF vector, and a logistic regression model, etc. At least two levels of category prediction layers can be implemented as the same model to simultaneously predict at least two levels of categories, or each level of category prediction layer can be implemented as a model to simultaneously predict the corresponding level categories. When each level of category prediction layer can be implemented as a model, at least two levels of category prediction layers can be different models.

[0038] In an embodiment of the present application, multiple training samples can be obtained, and the initial prediction model can be trained using the multiple training samples. Among them, the multiple training samples can be posts, articles, etc. that have been published on the platform, and each training sample contains text content, and the text content includes but is not limited to: Chinese and its punctuation marks, foreign languages ​​and their punctuation marks, and each training sample corresponds to at least two levels of category annotation results. The at least two levels of category annotation results refer to each level of category annotated for each training sample, that is, each level of category selected by each training sample when it is published.

[0039] This embodiment does not limit the specific implementation method of obtaining multiple training samples. For example, the text content of multiple published posts or articles with annotation results is selected from an associated database as multiple training samples; or, the text content in multiple posts or articles is annotated with categories manually or through an annotation model as multiple training samples; and so on. In addition, in order to balance the categories, training samples can be evenly obtained according to at least two levels of categories. For example, samples are evenly obtained according to two levels of categories, and 1,000 training samples are obtained for each level of category; or, training samples can be evenly obtained according to three or more levels of categories, for example, 1,000 training samples are obtained for each category. The selection of training samples for several levels of categories depends on the specific application situation, and this embodiment does not limit this.

[0040] In an embodiment of the present application, each training sample corresponds to a text length. In order to reduce the difference between the results of the model's category prediction for posts of different lengths, and at the same time to improve the accuracy of the trained model's category prediction results for posts of different lengths, and to improve the generalization ability of the trained model, the training samples whose text lengths are within the same length range can be divided into the same training subset according to the text lengths of multiple training samples, so as to obtain multiple training subsets, and different training subsets correspond to different length ranges, so as to select training samples within different length ranges as target training samples to train the initial category prediction model.

[0041] Optionally, according to the text lengths of multiple training samples, the training samples whose text lengths are within the same length range are divided into the same training subset to obtain multiple training subsets, including: sorting the multiple training samples according to the text lengths of the multiple training samples; and evenly dividing the sorted multiple training samples according to the preset number of subsets to obtain multiple training subsets, each training subset including the same number of training samples. Thus, it can be ensured that the multiple training samples in each training subset are within the same length range, and when selecting at least one as the target training sample, the generalization ability of the model will not be affected by the uncertainty of the length. Alternatively, the training subset corresponds to a preset number of training sample carrying capacity, and the sample carrying capacity of each training subset can be the same. If there is a situation where the training samples cannot be evenly divided, the multiple training samples can be divided into each training subset in turn according to the maximum carrying capacity of the samples, and the remaining training samples are divided into the last training subset, and the number of remaining training samples should be less than or equal to the training sample carrying capacity corresponding to the training subset. In addition, multiple training samples can also be unevenly divided.

[0042] For example, if there are 5 training subsets, including: a first training subset, a second training subset, a third training subset, a fourth training subset and a fifth training subset, and the length ranges of the 5 training subsets can be 50-60 characters, 60-70 characters, 70-80 characters, 80-90 characters, and 90-100 characters, respectively, then the training samples with a length range of 50-60 characters are divided into the first training subset, the training samples with a length range of 60-70 characters are divided into the first training subset, the training samples with a length range of 70-80 characters are divided into the first training subset, the training samples with a length range of 80-90 characters are divided into the first training subset, and the training samples with a length range of 90-100 characters are divided into the first training subset.

[0043] In an embodiment of the present application, after dividing a plurality of training samples into a plurality of training subsets, the plurality of training subsets can also be randomly sorted, and the initial category prediction model can be trained according to the target training samples in each training subset after random sorting to obtain the prediction results of at least two levels of categories corresponding to each target training sample, and the training subsets of different length ranges can be shuffled, and the target training samples are selected from each shuffled training subset to train the initial category prediction model, which can further reduce the difference between the results of the model's category prediction for posts of different lengths, and improve the accuracy of the trained model's category prediction results for posts of different lengths and the generalization ability of the model. Among them, the prediction results of at least two levels of categories are predicted by at least two levels of category prediction layers based on the semantic information extracted from the target training samples by the semantic information extraction layer.

[0044] In some embodiments, the initial category prediction model is trained according to the target training samples in each training subset after random sorting to obtain prediction results of at least two levels of categories corresponding to each target training sample, including: for each training subset after random sorting, selecting a target training sample from the training subset; inputting the target training sample into a semantic information extraction layer to extract semantic information to obtain the semantic information of the target training sample; using the pooling layer in the initial category prediction model to pool the semantic information of the target training sample to obtain the pooled semantic information of the target training sample; inputting the pooled semantic information of the target training sample into at least two levels of category prediction layers respectively to perform prediction of at least two levels of categories to obtain prediction results of at least two levels of categories corresponding to the target training sample.

[0045] Optionally, selecting a target training sample from each training subset includes: each training sample in each training subset is used as a target training sample; or, randomly selecting a training sample or part of the training samples from each training subset as the target training sample; or, selecting a training sample with the longest text length, a text with a medium text length, and a training sample with the shortest text length or a combination thereof from each training subset as the target training sample, which is not limited in this embodiment. In addition, the number of target training samples selected from each training subset may be the same or different, and the method of selecting the target training samples from each training subset may be the same or different. For example, for some training subsets, the longest text length may be selected as the target sample, and for other training subsets, the shortest text length may be selected as the training sample; or, for all training subsets, the longest text length may be selected as the target sample.

[0046] Further optionally, since the number of target training samples selected in each subset may be multiple and the lengths of the multiple training samples are within the same range, in order to further improve the generalization ability of the model, the multiple training samples contained in each target training sample may be randomly sorted, and the randomly sorted training samples may be used for model training.

[0047] It should be noted that in the following embodiments, the number of training target samples selected in each training sample is 1 as an example for detailed description.

[0048] Optionally, the target training sample contains multiple segmentations, which refer to independent and meaningful segmentations. A segmentation can be a word, phrase, symbol, number or any other segmentation that can be processed separately. For example, the text content "We are committed to providing high-quality cleaning and washing services!" includes the following segmentations: "we", "committed to", "provide", "high-quality", "cleaning and washing services" and "!". Based on this, the target training sample is input into the semantic information extraction layer in the initial category prediction model to extract semantic information to obtain the semantic information of the target training sample, including: inputting the target training sample into the semantic information extraction layer in the initial category prediction model, and based on the self-attention learning mechanism, learning the semantic information of each segmentation and the semantic association relationship between each segmentation to obtain the semantic information of the target training sample.

[0049] Among them, the self-attention learning mechanism, also known as the intra-attention mechanism, is a mechanism that allows the model to focus on the relationship between different segmentations within the sequence when processing the segmentation sequence (an independent text content) corresponding to the text content of the target training sample. The core idea of ​​the self-attention learning mechanism is to allow each segmentation in the segmentation sequence to interact directly with other segmentations in the sequence, thereby capturing the short-distance and long-distance dependencies between the segmentations within the segmentation sequence, so that the model can dynamically pay attention to different segmentations in the segmentation sequence and adjust the importance of each segmentation according to the context. Among them, short-distance dependencies refer to the mutual dependencies between segments that are close to each other in the segmentation sequence data, and long-distance dependencies refer to the mutual dependencies between segments that are far apart in the segmentation sequence data. This dependency can be semantic, grammatical, or context-based. The main features of the self-attention learning mechanism include: (1) Query vector, key vector and value vector: The self-attention learning mechanism maps each word in the input word sequence to three vectors: query (Q), key (K) and value (V). These vectors are obtained by multiplying the input word sequence with the learnable parameters (weight matrix) in the model. The query vector represents the word or position currently being processed, which needs to obtain information from other words or positions in the word sequence. In the self-attention learning mechanism, each word generates a query vector for comparison with the key vector to determine which words or positions in the sequence should be used to obtain information. The key vector determines which information or words at which positions in the word sequence are "accessible". The degree of match between the query vector and the key vector determines the degree to which the value vector is weighted and summed. In the self-attention learning mechanism, each word or position has a corresponding key vector, which together with the query vector determines the degree of influence of other words in the word sequence on the current word. The value vector contains the actual content or features of each position or word in the sequence, which will be weighted and summed according to the degree of match between the query and the key. The value vector represents the specific semantic information of each word in the word sequence, which will be integrated into the output according to the attention weight. (2) Attention score calculation: Calculate the similarity or match between the query vector and all key vectors to generate an attention score. The attention score represents the importance of each word in the word sequence to the current word. (3) Scaling and tiling: To avoid numerical problems, such as vanishing or exploding gradients caused by large dot products, the model scales the attention scores (usually by dividing by the square root of the key vector dimension) and then normalizes the scores using the Softmax function.(4) Attention weighted features: The normalized value obtained by the Softmax function is used as the weight and multiplied with the value vector to obtain the final attention weighted features. The attention weighted features integrate the information of all the words in the word sequence, and the contribution of each word is determined by its corresponding attention weight. (5) Multi-head attention: In order to capture the relationship between different subspaces in the sequence, the self-attention learning mechanism can be extended to a multi-head attention mechanism. In the multi-head attention, the model executes the above process multiple times in parallel, and each "head" may focus on different relationships or features in the word sequence. (6) Residual connection and layer normalization: The output of the self-attention will be added to the original input (residual connection), and then pass through layer normalization (Layer Normalization) to stabilize the training process and improve the gradient flow. (7) Calculation of the feedforward neural network layer (also known as the fully connected layer calculation): After the self-attention layer, there is usually a fully connected layer, which performs further nonlinear transformation on the output of the self-attention layer to strengthen the representation of the position or extract more complex features. For the detailed implementation of the self-attention learning mechanism in this application, please refer to the relevant description of the following embodiment, which will not be repeated here.

[0050] In an embodiment of the present application, through a learning process based on a self-attention learning mechanism, the semantic information of the target training sample corresponding to each segmentation in each target training sample can be obtained, and the semantic information of the target training sample corresponding to each segmentation is the same, that is, the semantic information of multiple identical target training samples is obtained. In practical applications, one of the semantic information of multiple target training samples can be selected as the output of the semantic information extraction layer. For this, special characters (without specific semantic meanings) can be added to the target training sample, and the semantic information of the target training sample corresponding to the special characters can also be obtained based on the self-attention learning mechanism, and the semantic information of the target training sample corresponding to the special characters is used as the output of the semantic information extraction layer. Based on this, the target training sample is input into the semantic information extraction layer in the initial category prediction model to extract the semantic information to obtain the semantic information of the target training sample, including: performing a special segmentation addition operation on the target training sample to obtain a target training sample containing special characters; inputting the target training sample containing special characters into the semantic information extraction layer in the initial category prediction model, based on the self-attention learning mechanism, learning the semantic information of each segmentation and the semantic association between each segmentation, so as to obtain the target semantic information corresponding to the special character as the semantic information of the target training sample.

[0051] In some embodiments, a special word segmentation adding operation is performed on a target training sample to obtain a target training sample containing special characters, including: selecting special characters. The selection of special characters depends on the specific task and model architecture. Common special characters include <cls>(Classification) and <sep>(separated), for example, in the BERT model, <cls>The token is added to the beginning of the sentence, <sep>The tag is added to the end of the sentence; word segmentation: the text content of the target training sample is divided into multiple tokens, which can be words, subwords or characters; the tokenizer converts the text content into a token sequence according to predefined rules, and the token sequence contains multiple tokens in the text content of the target training sample; adding special characters: adding special characters to the token sequence, for example, if processing a single sentence, <cls>will be added to the beginning of the sequence, <sep>It will be added to the end of the sequence; encoding (Encoding), converting multiple participles (including special characters) into numerical representations (digital form) that the model can understand. Specifically, the numerical representation of each participle in the vocabulary (including each participle and its corresponding numerical mapping relationship) can be found to input the numerical representation corresponding to each participle into the model. The numerical representation corresponding to each participle can represent the initial semantic information of the participle. The numerical representation corresponding to each participle can be implemented in the form of a matrix, which contains the numerical representation corresponding to each participle. In addition, in order to enable the model to understand the position of multiple participles in the participle sequence, position encoding can also be added to each participle in the participle sequence, which is conducive to the model capturing the order information in the sequence data.

[0052] In some embodiments, a target training sample containing special characters is input into a semantic information extraction layer in an initial category prediction model, and based on a self-attention learning mechanism, the semantic association relationship between each word segment is learned to obtain the target semantic information corresponding to the special character as the semantic information of the target training sample, including: inputting the numerical representation corresponding to the word segmentation sequence containing special characters into the semantic information extraction layer in the initial category prediction model, performing vector mapping on the numerical representation corresponding to each word segmentation in the target training sample to obtain the query vector, key vector and value vector corresponding to each word segmentation; for each word segmentation, performing matching calculations using the query vector of the word segmentation and the key vector of each word segmentation to obtain the attention weight of each word segmentation; performing a full connection calculation operation on the value vector of each word segmentation according to the attention weight of each word segmentation to obtain the target semantic information corresponding to the special character as the semantic information of the target training sample.

[0053] Optionally, the numerical representation corresponding to the word segmentation sequence containing special characters is input into the semantic information extraction layer in the initial category prediction model, and vector mapping is performed on the numerical representation corresponding to each word segmentation in the target training sample to obtain the query vector, key vector and value vector corresponding to each word segmentation, including: mapping the numerical representation of each word segmentation in the word segmentation sequence to the query vector, key vector and value vector respectively through different weight matrices, such as mapping them to the query vector, key vector and value vector respectively through three different weight matrices W^Q, W^K, and W^V.

[0054] Optionally, for each word segment, the query vector of the word segment is used to calculate the matching degree with the key vector of each word segment to obtain the attention weight of each word segment, including, for the current word segment (each word segment), the query vector of the word segment is calculated with the key vector of each word segment (including the current word segment) to obtain multiple score values, each score value represents the attention weight of the word segment providing the key vector to the current word segment. The attention weight reflects the relative importance of one word segment to another word segment in the sequence. The higher the weight, the more important the corresponding word segment is in the current context.

[0055] Further optionally, the multiple score values ​​may include positive scores and negative scores. In order to facilitate the subsequent full connection calculation, each score value may be normalized to obtain multiple normalized values, and the multiple normalized values ​​are all positive numbers. The normalization process here may be an exponential calculation operation (the score value is used as a parameter of the exponential calculation), but is not limited thereto.

[0056] Further optionally, after obtaining the attention weight of each segmentation, before performing the pooling process, a weighted sum calculation can be performed on the value vector of each segmentation according to the attention weight of each segmentation to obtain a weighted sum vector (updated representation). The weighted sum vector incorporates the segmentations at all positions in the input sequence, but the contribution of the segmentations at different positions is determined by their attention relative to the query. This means that for each segmentation in the segmentation sequence, its weighted sum vector is a weighted combination of other segmentation information associated with it in the entire segmentation sequence, and the weight is determined by the attention score (normalized). Through this process, the weighted sum vector of each segmentation not only represents the semantic information of the segmentation itself, but also represents the synthesis of the contextual semantic information of the entire segmentation sequence, which enables the model to capture long-distance dependencies within the sequence and take into account the semantic information of the entire sequence when processing the current segmentation.

[0057] Further optionally, after obtaining the weighted sum vector of each word segment, before performing the pooling process, a fully connected calculation operation can be performed on the value vector of each word segment according to the attention weight of each word segment to obtain the target semantic information corresponding to the special character as the semantic information of the target training sample, including: inputting the value vector obtained by weighted summation into the fully connected layer, the fully connected layer includes two linear transformation functions and a nonlinear activation function such as ReLU); the first linear transformation function maps the value vector to an intermediate dimension, and then applies the nonlinear activation function to perform nonlinear calculation, and finally the second linear transformation function maps the nonlinear calculation result back to the original dimension or the target dimension to output the semantic information of the target training sample corresponding to the special character, the semantic information of the target training sample integrates the context information of the entire input word segmentation sequence, and can be used as the semantic information of the target training sample.

[0058] Further optionally, the semantic information of the target training sample is a multidimensional parameter vector. Based on this, after obtaining the semantic information of the target training sample, the semantic information of the target training sample is pooled using the pooling layer in the initial category prediction model to obtain the pooled semantic information of the target training sample, including: determining the window size of the pooling window and the sliding step of the pooling window corresponding to the pooling process, and the values ​​of the window size and the sliding step are equal; based on the window size and the sliding step, for the multidimensional parameter vector, in the area covered by each pooling window, the maximum value of the parameter is selected as the output to obtain the pooled semantic information of the target training sample; or, based on the window size and the sliding step, for the multidimensional parameter vector, in the area covered by each pooling window, the average value of all parameter values ​​is calculated as the output to obtain the pooled semantic feature vector of the target training sample; or, based on the window size and the sliding step, for the multidimensional parameter vector, in the area covered by each pooling window, the sum of all parameter values ​​is calculated as the output to obtain the pooled semantic feature vector of the target training sample.

[0059] In some embodiments, the pooled semantic information of the target training sample is respectively input into at least two levels of category prediction layers to perform prediction of at least two levels of categories to obtain prediction results of at least two levels of categories corresponding to the target training sample, including: inputting the pooled semantic information of the target training sample into at least two levels of category prediction layers, analyzing the similarity between the pooled semantic information and the semantic information of multiple known categories at the corresponding category level; based on the similarity, determining the probability value of the target training sample belonging to the multiple known categories contained in each category level as the prediction result of at least two levels of categories corresponding to the target training sample.

[0060] Optionally, analyzing the similarity between the pooled semantic information and the semantic information of multiple known categories at the corresponding category level includes: calculating the similarity between the pooled semantic information and the semantic information of each known category, and the calculation method includes cosine similarity, Euclidean distance, etc. Cosine similarity evaluates their similarity by measuring the angle between two vectors, and Euclidean distance measures the straight-line distance between them. The calculation formula of cosine similarity is: Among them, A and B are the semantic information vector after pooling and the semantic information vector of the category level, "·" represents the dot product, and ||A|| and ||B|| are the moduli of the vectors, respectively; for at least two levels of category prediction layers, it is necessary to calculate the similarity between the semantic information after pooling and all known categories at each level, which can be done by iteratively comparing the semantic vectors of each category, such as constructing a similarity matrix, in which each row represents a category level, each column represents a known category, and the values ​​in the matrix represent the similarity between the semantic information after pooling and the corresponding category. According to the similarity matrix, category prediction can be performed. Usually, the category with the highest similarity will be selected as the most likely prediction result.

[0061] In the above-mentioned embodiments of the present application, in the process of self-attention learning, only the encoding process is used to realize the extraction and comparison of semantic information, and the decoding process is not involved, thereby improving the efficiency of the self-attention learning process.

[0062] In an embodiment of the present application, in order to improve the prediction accuracy of the model, the parameters of the model can also be continuously adjusted by calculating the loss function between the network layers. Therefore, the loss calculation on the same-level category can be performed according to the labeling results and prediction results of the same-level category corresponding to each target training sample to obtain the loss function on at least two levels of categories, and the parameters of each network layer are adjusted according to the loss function on at least two levels of categories. Optionally, the loss calculation on the same-level category is performed according to the labeling results and prediction results of the same-level category corresponding to each target training sample to obtain the loss function on at least two levels of categories, including: according to the labeling results and prediction results of the same-level category corresponding to each target training sample, respectively obtain the loss function on at least two levels of categories, wherein the prediction result of the category is the predicted category of the model; wherein different category levels use different loss functions for the same loss function, or different category levels use different loss functions, and the higher the category level, the higher the complexity of the loss calculation method.

[0063] In the embodiment of the present application, in order to further improve the accuracy of the model, a comprehensive loss function can also be calculated based on the loss function on at least two levels of categories, and the network parameters of the semantic information extraction layer and at least two levels of category prediction layer in the initial category prediction model can be adjusted according to the comprehensive loss function until the comprehensive loss function meets the model training termination condition to obtain an intermediate category prediction model. Among them, the model training termination condition can be that the output result of the comprehensive loss function is less than a set threshold.

[0064] In the embodiment of the present application, the semantic information extraction layers in the intermediate category prediction model can also be copied and spliced ​​to obtain a target category prediction model that includes at least a plurality of semantic information extraction layers connected in sequence and at least two-level category prediction layers, so that when performing category prediction, multiple semantic information extraction layers can be used to extract semantic information from multiple text segments of the target training sample respectively, thereby improving the efficiency of semantic information extraction and thus improving the prediction efficiency of the category prediction model. An exemplary schematic diagram of the target category prediction model is shown in FIG. Figure 1c As shown, it contains multiple semantic information extraction layers.

[0065] Further optionally, a selection mechanism for the semantic information extraction layer can be preset. Then, when performing category prediction according to the target category prediction model, a number of semantic information extraction layers that are adapted to the text length of the text content to be predicted can be adaptively selected from multiple semantic information extraction layers according to the text length of the text content to be predicted and the selection mechanism of the semantic information extraction layer to extract semantic information from the text content to be predicted, without starting the remaining semantic information extraction layers. Thus, video memory is allocated to an adapted number of semantic information extraction layers according to the actual text length of the text content to be predicted, which can save the size of video memory occupied during prediction, improve throughput, and increase prediction speed.

[0066] For example, the target category prediction model includes 5 semantic information extraction layers, and the upper limit of the text length processed by each semantic information extraction layer at a time is 500 characters. If the text length of the current text content to be predicted is 1000 characters, then according to the preset semantic information extraction layer selection mechanism and text length, two semantic information extraction layers can be selected from the 5 semantic information extraction layers and video memory can be allocated to them to support their operation, and two text fragments (each text fragment is 500 characters) are respectively input into the two semantic information extraction layer pairs to extract the semantic information of the two text fragments.

[0067] In the above embodiments of the present application, training samples in the same length range are divided into the same training subset, and training subsets of different length ranges are shuffled, and target training samples are selected from each shuffled training subset to train the initial category prediction model, rather than training the initial category prediction model according to the rule of length range from small to large or from large to small. This can not only avoid the problem of large differences in the results of category prediction for posts of different lengths made by the trained model, but also improve the accuracy of the category prediction results of the trained model for posts of different lengths, and improve the generalization ability of the trained model. In the process of model training, the loss calculation on the same-level category is performed according to the labeling results and prediction results of the same-level category corresponding to each target training sample, and the network parameters of the semantic information extraction layer and each-level category prediction layer are adjusted to improve the calculation accuracy of each network layer and the prediction accuracy of the model; further, the comprehensive loss function can be calculated according to the loss function on at least two levels of categories, and the network parameters of the semantic information extraction layer and each-level category prediction layer in the initial category prediction model are further adjusted according to the comprehensive loss function until the comprehensive loss function meets the model training termination condition to obtain the intermediate category prediction model, thereby greatly improving the accuracy of the prediction model. Further, according to the network parameters of the trained semantic information extraction layer, the semantic information extraction layer in the intermediate category prediction model can be copied and spliced ​​to obtain a target category prediction model including multiple semantic information extraction layers connected in sequence, so that when performing category prediction, multiple semantic information extraction layers can be used to extract semantic information from multiple text fragments of the target training sample respectively, thereby improving the efficiency of semantic information extraction and thus improving the prediction efficiency of the category prediction model.

[0068] The above and Figure 1a , Figure 1b and Figure 1c The relevant embodiments specifically describe the training process of the target category prediction model. Figure 2a , Figure 2b and Figure 2c A related embodiment is a specific process of performing category prediction on text content of a to-be-predicted category based on a target category prediction model.

[0069] Figure 2a The following is a flow chart of a category prediction method provided by an exemplary embodiment of the present application. Figure 2a As shown, the method includes:

[0070] 201. Obtain text content of the category to be predicted;

[0071] 202. When the length of the text content is greater than a text length threshold supported by the target category prediction model, the text content is divided into at least two text segments, and the length of each text segment is not greater than the text length threshold; the target category prediction model includes at least a plurality of semantic information extraction layers and at least two levels of category prediction layers;

[0072] 203. Inputting at least two text segments into at least two semantic information extraction layers in the target category prediction model to extract semantic features to obtain at least two semantic information; the at least two semantic information extraction layers are semantic information extraction layers selected from a plurality of semantic information extraction layers and are the same in number as the at least two text segments;

[0073] 204. Generate target semantic information corresponding to the text content according to the at least two pieces of semantic information;

[0074] 205. Input the target semantic information into at least two-level category prediction layer to predict at least two-level categories, so as to obtain at least two-level categories corresponding to the text content.

[0075] This embodiment does not limit the specific implementation method of obtaining the text content of the category to be predicted. Figure 2b The post or article to be published page shown obtains the text content filled in by the user as the text content to be predicted. Alternatively, the text content of the post or article of the category to be predicted is selected from the associated database as the text content to be predicted.

[0076] In an embodiment of the present application, the text length threshold supported by the target category prediction model actually refers to the text length threshold that a single semantic information extraction layer in the target category prediction model can process at a time. In the case where the length of the text content is greater than the text length threshold supported by the target category prediction model, that is, in the case where the length of the text content is greater than the text length threshold that a single semantic information extraction layer can process at a time, the text content of the category to be predicted can be divided into at least two text segments according to the text length threshold that each semantic information extraction layer can process at a time, the length of each text segment is not greater than the text length threshold, and the text length of each text segment can be the same or different.

[0077] Further, after obtaining at least two text segments, video memory is allocated to the same number of semantic information extraction layers according to the number of text segments to support their operation. It should be noted that, when the text length of each text segment is different, it is necessary to adaptively select a semantic information extraction layer that supports the text length of each text segment and allocate video memory to it to support its operation.

[0078] In an embodiment of the present application, each text fragment contains multiple participles; at least two text fragments are input into at least two semantic information extraction layers in the target category prediction model to extract semantic features to obtain at least two semantic information, including: performing a special participle adding operation for each text fragment to obtain at least two text fragments containing special participles according to the number of text fragments; each text fragment is input into a semantic information extraction layer with allocated video memory, and based on a self-attention learning mechanism, the semantic information of each participle in the text fragment and the semantic association relationship between each participle are learned to obtain the target semantic information corresponding to the special character as the semantic information of the text fragment to obtain at least two semantic information.

[0079] In some embodiments, each text segment containing special characters is input into a corresponding semantic information extraction layer in the target category prediction model, and based on the self-attention learning mechanism, the semantic information of each word segment and the semantic association relationship between the word segment are learned to obtain the semantic information of the text segment corresponding to each special character, including: performing vector mapping for each word segment in each text segment to obtain the query vector, key vector and value vector corresponding to each word segment in the text segment; for each word segment in the text segment, using the query vector of the word segment and the key vector of each word segment to perform matching calculation respectively to obtain the attention weight of each word segment in the text segment; performing a full connection calculation operation on the value vector of each word segment in the text segment according to the attention weight of each word segment in the text segment to obtain at least two semantic information.

[0080] In an embodiment of the present application, after obtaining at least two pieces of semantic information, target semantic information corresponding to the text content is generated based on the at least two pieces of semantic information, including: using the pooling layer in the target category prediction model to perform pooling processing on the at least two pieces of semantic information to obtain the target semantic information corresponding to the text content.

[0081] In some embodiments, the pooling layer has a pooling window, and the window size of the pooling window and the sliding step size of the pooling window are equal in value; based on at least two semantic information, target semantic information corresponding to the text content is generated, including: inputting at least two semantic information into the pooling layer in the target category prediction model, semantically splicing the at least two semantic information to obtain the spliced ​​semantic information, and the spliced ​​semantic information is a multidimensional parameter vector; based on the window size and the sliding step size, for the multidimensional parameter vector, in the area covered by each pooling window, the maximum value of the parameter is selected as output to obtain the target semantic information corresponding to the text content; or, based on the window size and the sliding step size, for the multidimensional parameter vector, in the area covered by each pooling window, the average value of all parameter values ​​is calculated as output to obtain the target semantic information corresponding to the text content; or, based on the window size and the sliding step size, for the multidimensional parameter vector, in the area covered by each pooling window, the sum of all parameter values ​​is calculated as output to obtain the target semantic information corresponding to the text content.

[0082] In an embodiment of the present application, target semantic information is input into at least two levels of category prediction layers to perform prediction of at least two levels of categories to obtain at least two levels of categories corresponding to the text content, including: inputting the target semantic information into at least two levels of category prediction layers respectively, analyzing the similarity between the target semantic information and the semantic information of multiple known categories at the corresponding category level; determining the probability value of the target training sample belonging to the multiple known categories contained in each category level based on the similarity, the probability value of each category indicating the possibility that the text content belongs to the category, and the larger the probability value, the greater the possibility; based on the probability values ​​of the multiple known categories contained in each category level, selecting the known categories whose probability values ​​are ranked before the preset ranking as the at least two levels of categories corresponding to the text content, for example, selecting the top 10 consistent categories in the probability value ranking in each level of categories as the at least two levels of categories corresponding to the text content.

[0083] It should be noted that in the training stage of the target prediction model, at least two levels of category prediction layers have learned the unique identifier (such as ID) corresponding to each category in the platform. Then, when predicting categories, the output results of at least two levels of category prediction layers can be the identifier corresponding to each known category and its probability value, and then based on the identifier, find out which category each identifier corresponds to from the known category-identifier mapping table. Alternatively, the output results of at least two levels of category prediction layers can also be each known category and its probability value. Alternatively, the output results of at least two levels of category prediction layers can also be each known category, the identifier corresponding to each known category and its probability value. Taking "home improvement service" as an example, the corresponding prediction results can be "cate": 1043, "cateName": "home improvement service", "p": 0.5791.

[0084] It should be noted that 2a, Figure 2b and Figure 2c The specific implementation methods and technical effects of each step in the relevant embodiments can be found in 1a, Figure 1b and Figure 1c The relevant descriptions of the relevant embodiments are not repeated here.

[0085] Further optionally, when the text content is content to be posted sent by the client, at least two levels of categories corresponding to the text content are sent to the client, so that the client can display it on the posting page for the user to configure the at least two levels of categories to which the text content belongs on the posting page.

[0086] Optionally, at least two levels of categories corresponding to the text content are sent to the client, including: there can be multiple predicted categories at each level, and there is an arrangement order between multiple prediction results of the same level category, and the arrangement order is determined according to the similarity between the text content and each predicted category; the top N prediction results corresponding to each level of category are selected and pushed to the client for display.

[0087] Further optionally, taking posting as an example, the user can select any category from at least two levels of predicted categories displayed on the post-to-be-posted page of the client as the category to which the text content of the post-to-be-posted belongs.

[0088] Further optionally, in response to the user's post publishing operation, the post to be published is published to the corresponding category of the corresponding platform so that other users can view the post content.

[0089] Figure 2a It is an exemplary schematic diagram of the backend corresponding to the prediction results under each level of category predicted based on the text content in the figure. The prediction results include the ID corresponding to each predicted category, the category name and the probability value of the text content belonging to the category. 2b is an exemplary schematic diagram of at least two levels of predicted categories displayed on the client's posting page. This embodiment does not limit the ranking method of at least two levels of predicted categories displayed on the posting page. For example, for categories of the same level, they can be arranged in order of the probability value belonging to each category, or they can be arranged randomly. For categories of different levels, they can be arranged in partitions.

[0090] The following is a detailed description of the training process of the cross-category recognition model and the cross-category recognition process.

[0091] In order to solve the technical problem of how to accurately identify cross-category posts from published posts, in an embodiment of the present application, a structured prompt word for each training sample is generated using the prompt word structure corresponding to the target language model. Based on the structured prompt words of the training samples, the target semantic model is used to identify and annotate whether the main semantic information of the training samples matches at least one of the corresponding primary categories under the guidance of the prompt words, and the training samples are annotated as positive samples and negative samples according to the recognition results, so as to train the initial cross-category recognition model to be trained, so as to obtain the target cross-category recognition model.

[0092] Furthermore, the trained target cross-category recognition model is used to identify whether the target content matches at least one primary category according to the main semantic information of the published target content and output the recognition result. Based on the recognition result, the target content is marked as cross-category or non-cross-category, thereby improving the accuracy of cross-category recognition.

[0093] The following is a detailed description of the cross-category recognition model training method provided in the embodiment of the present application in conjunction with the accompanying drawings. Figure 3a and Figure 3b The specific implementation methods of each step in the relevant embodiments can be found in Figure 1a , Figure 1b , Figure 1c , Figure 2a , Figure 2b , Figure 2c The relevant descriptions of the related embodiments are not repeated in this embodiment.

[0094] Figure 3a A flow chart of a cross-category recognition model training method provided by an exemplary embodiment of the present application. Figure 3a As shown, the method includes:

[0095] 301. Acquire multiple training samples, each training sample including published text content and at least one primary category selected for the text content when the text content is published;

[0096] 302. According to the prompt word structure corresponding to the target language model, convert the plurality of training samples into corresponding structured prompt words respectively, wherein the structured prompt word corresponding to each training sample at least includes the training sample and a guide instruction, wherein the guide instruction is used to instruct the target language model to identify whether the text content in the training sample matches at least one primary category according to the main semantic information of the text content in the training sample;

[0097] 303. Input the structured prompt words corresponding to each training sample into the target language model, and under the guidance of the guide instruction, identify whether the text content in the training sample matches at least one primary category according to the main semantic information of the text content in the training sample and output the recognition result;

[0098] 304. According to the recognition result output by the target language model, each training sample is labeled as a positive training sample or a negative training sample, and the initial cross-category recognition model is trained according to the positive training samples and the negative training samples to obtain the target cross-category recognition model.

[0099] In an embodiment of the present application, in order to obtain a cross-category recognition model with high cross-category recognition accuracy, the initial cross-category recognition model can be trained to obtain a target cross-category recognition model. Specifically, multiple training samples can be obtained, and the initial cross-category recognition model can be trained using the multiple training samples. Among them, the multiple training samples can be posts, articles, etc. published by the platform, each training sample includes the published text content and at least one primary category selected for the text content when the text content is published, and the text content includes but is not limited to: Chinese and its punctuation marks, foreign languages ​​and their punctuation marks.

[0100] This embodiment does not limit the specific implementation method of obtaining multiple training samples. For example, multiple samples containing published text content and at least one first-level category selected for the text content when the text content is published are selected from an associated database as multiple training samples; or multiple samples containing text content and at least one first-level category corresponding to the text content are generated manually or by a model as multiple training samples.

[0101] In order to further improve the generalization ability and accuracy of the cross-category recognition model, multiple training samples can be labeled, and the multiple training samples can be labeled as positive training samples and negative training samples. The initial cross-category recognition model can be trained using the positive training samples and negative training samples to obtain a target cross-category recognition model with strong generalization ability and high cross-category recognition accuracy. Among them, positive training samples refer to training samples labeled as non-cross-category, and non-cross-category means that the text content matches at least one of its corresponding categories. Negative training samples refer to training samples labeled as cross-category, and non-cross-category refers to training samples whose text content does not match at least one of its corresponding categories.

[0102] In an embodiment of the present application, a target language model can be used to annotate multiple training samples. In order to further improve the training ability of multiple training samples for the initial cross-category recognition model, a prompt word structure corresponding to the target language training model is introduced, and the multiple training samples are respectively converted into corresponding structured prompt words according to the prompt word structure corresponding to the target language model. The structured prompt words corresponding to each training sample at least include the training sample and a guide instruction, and the guide instruction is used to instruct the target language model to identify whether the text content in the training sample matches at least one primary category according to the main semantic information of the text content in the training sample.

[0103] In some embodiments, the multiple training samples are converted into corresponding structured prompt words according to the prompt word structure corresponding to the target language model, including: obtaining the prompt word structure and adapted guide instructions corresponding to the target language model, adding the text content of each training sample and its adapted guide instructions to the prompt word structure to obtain structured prompt words corresponding to the multiple training samples.

[0104] Figure 3b is an exemplary prompt word structure corresponding to a training sample, wherein "{item, content}" is the text content of the training sample, "{item, cateName}" is at least one first-level category corresponding to the training sample, and "Does the text content of this post match the post category?" is the guiding instruction corresponding to the training sample.

[0105] In an embodiment of the present application, after obtaining the structured prompt words corresponding to the training samples, the structured prompt words corresponding to each training sample can be input into the target language model. Under the guidance of the guide instructions, the text content in the training sample is identified according to the main semantic information of the text content in the training sample to see whether it matches at least one first-level category and the recognition result is output.

[0106] Further optionally, in order to further improve the generalization ability and accuracy of the cross-category recognition model, the structured prompt words corresponding to each training sample may also include: background information, which is used to describe the affiliation and / or non-affiliation relationship between a specific secondary category and a specific primary category, such as Figure 3c The background information (background knowledge) shown in is: "Installation and repair are not wholesale purchases", "Cutting does not belong to wholesale", etc., among which installation, repair, and cutting belong to specific secondary categories, and wholesale purchases and wholesale belong to specific primary categories. In addition, when publishing text content, at least two levels of categories can be selected for the text content. Furthermore, the guidance instructions can also include the following content "The main content matches and is judged as a match (for some posts that only add a small number of category-related words but the main content does not match, it will be judged as a mismatch)", so as to more accurately guide the model to make cross-category predictions.

[0107] Based on this, in some embodiments, the structured prompt words corresponding to each training sample are input into the target language model, and under the guidance of the guiding instructions, the main semantic information of the text content in the training sample is used to identify whether the text content in the training sample matches at least one first-level category and output the recognition result, including: the structured prompt words corresponding to each training sample are input into the target language model, and under the guidance of the guiding instructions, according to the main semantic information and background information of the text content in the training sample, whether the text content in the training sample matches at least two-level categories, and output the recognition result.

[0108] In some embodiments, the structured prompt words corresponding to each training sample are input into the target language model, and under the guidance of the guide instruction, whether the text content in the training sample matches at least two levels of categories is identified according to the main semantic information and background information of the text content in the training sample, and the recognition result is output, including: inputting the structured prompt words corresponding to each training sample into the target language model, performing semantic learning on the guide instruction and the background information in the structured prompt words to obtain the semantic information of the guide instruction and the semantic information of the background information; performing semantic relevance learning on the text content in the training sample and at least two levels of categories according to the semantic information of the guide instruction and the semantic information of the background information to obtain the semantic matching degree between the text content and at least two levels of categories; wherein the target semantic matching degree includes the overall semantic matching degree between the text content and at least two levels of categories, or includes the local semantic matching degree between the text content and each level of categories; according to the semantic matching degree between the text content and at least two levels of categories, whether the text content matches the at least two levels of categories is identified, and the recognition result is output, and the recognition result can be a match or a mismatch.

[0109] Optionally, the structured prompt words corresponding to each training sample are input into the target language model, and semantic learning is performed on the guiding instructions and background information in the structured prompt words to obtain the semantic information of the guiding instructions and the semantic information of the background information, including: inputting the guiding instructions and the background information in the structured prompt words into the target language model, and based on the self-attention learning mechanism, semantic learning is performed on the guiding instructions and the background information respectively to obtain the semantic information of the guiding instructions and the semantic information of the background information.

[0110] Further optionally, the guiding instruction and background information in the structured prompt word are input into the target language model, and based on the self-attention learning mechanism, semantic learning is performed on the guiding instruction and the background information respectively to obtain the semantic information of the guiding instruction and the semantic information of the background information, including: inputting the guiding instruction and the background information in the structured prompt word into the target language model, performing vector mapping for each word segment in the guiding instruction and the background information to obtain the query vector, key vector and value vector corresponding to each word segment; for each word segment in the guiding instruction, using the query vector of the word segment and the key vector of each word segment in the guiding instruction to perform matching calculation respectively to obtain the attention weight of each word segment in the guiding instruction; performing a full-link calculation operation on the value vector of each word segment according to the attention weight of each word segment in the guiding instruction to obtain the semantic information of the guiding instruction; for each word segment in the background information, using the query vector of the word segment and the key vector of each word segment in the background information to perform matching calculation respectively to obtain the attention weight of each word segment in the background information; performing a full-link calculation operation on the value vector of each word segment according to the attention weight of each word segment in the background information to obtain the semantic information of the background information.

[0111] Optionally, based on the semantic information of the guidance instruction and the semantic information of the background information, the semantic relevance of the text content and at least two levels of categories in the training sample is learned to obtain the semantic matching degree between the text content and at least two levels of categories, including: inputting the text content and at least two levels of categories in the training sample into the target language model, and based on the self-attention learning mechanism, performing semantic learning on the text content and the at least two levels of categories respectively to obtain the semantic information corresponding to the text content and the at least two levels of categories; based on the cross-attention mechanism, learning the semantic association relationship between the semantic information corresponding to the text content, the at least two levels of categories, the guidance instruction and the background knowledge to obtain the associated semantic information between the semantic information corresponding to the text content and the at least two levels of categories; based on the associated semantic information, obtaining the target semantic matching degree between the text content and at least two levels of categories.

[0112] Further optionally, the text content and at least two levels of categories in the training sample are input into the target language model, and based on the self-attention learning mechanism, semantic learning is performed on the text content and the at least two levels of categories respectively to obtain semantic information corresponding to the text content and the at least two levels of categories, including: the text content and the at least two levels of categories are input into the target language model, and vector mapping is performed on each word in the text content and the at least two levels of categories to obtain a query vector, a key vector and a value vector corresponding to each word; for each word in the text content, the query vector of the word is used to map the query vector of the word with the guide index. The key vectors of each word in the order are respectively matched to calculate the degree of matching, so as to obtain the attention weight of each word in the text content; according to the attention weight of each word in the text content, the value vectors of each word are fully linked to obtain the semantic information of the text content; for each word in each level of at least two levels of categories, the query vector of the word is used to calculate the degree of matching with the key vectors of each word in the category of the same level to obtain the attention weight of each word in each level of categories; according to the attention weight of each word in each level of categories, the value vectors of each word are fully linked to obtain the semantic information of each level of categories.

[0113] In the embodiment of the present application, the cross-attention learning mechanism refers to the ability of the model to pay attention to the information related to one sequence in another sequence when processing a sequence, thereby enhancing the model's ability to understand the input data, that is, it is used for semantic learning between sequences. The core concepts of the cross-attention learning mechanism are briefly introduced below: (1) Three elements: In the cross-attention learning mechanism, the three elements are the query vector, the key vector and the value vector. The query vector usually comes from one sequence (for example, a question or a task that needs to be processed at present), while the key and value come from another sequence (for example, a document or contextual information). (2) Attention weight: The cross-attention learning mechanism generates attention weights by calculating the similarity or match between the query and the key. These weights indicate the extent to which each element in the query sequence should pay attention to each element in the key sequence. (3) Weighted summation: The value sequence is weighted summed using the calculated attention weights to obtain the final output. This output is a weighted combination of the query sequence and the value sequence, reflecting the correlation between the two sequences. (4) Multimodal fusion: The cross-attention learning mechanism is particularly suitable for multimodal data, such as combining image and text information, or combining speech and lip movement information. It allows the model to find parts in one modality that are related to information in another modality. Comparison between self-attention and cross-attention: The self-attention learning mechanism focuses on the relationship between different parts within the same sequence, while the cross-attention learning mechanism focuses on the relationship between different sequences. (5) Application of the cross-attention mechanism: In the cross-attention mechanism, the query vector usually represents the information that needs to be processed at present, while the key vector and value vector represent the information that needs to be referenced. This mechanism allows the query to extract relevant information from the key-value pair, thereby modeling the interaction between input sequences.

[0114] Based on the above content of the cross-attention learning mechanism, further optionally, based on the cross-attention learning mechanism, the semantic association relationship between the semantic information corresponding to the text content, at least two levels of categories, guiding instructions and background knowledge is learned to obtain the associated semantic information between the text content and the semantic information corresponding to the at least two levels of categories, including: constructing query vectors corresponding to the text content, at least two levels of categories, guiding instructions and background knowledge respectively, wherein the text content is regarded as a sequence, each level of category is regarded as a sequence, the guiding instructions are regarded as a sequence and each background knowledge is regarded as a sequence; for any one of the text content, at least two levels of categories, guiding instructions and background knowledge, the query vector of the content is used to calculate the matching degree with the key vectors of other contents to obtain the attention weight of each content; according to the attention weight corresponding to each content, a full-link calculation operation is performed on the value vectors corresponding to other contents to obtain the associated semantic information between the text content and the semantic information corresponding to the at least two levels of categories.

[0115] It should be noted that when processing multiple sequences, the methods for selecting query vectors, key vectors, and value vectors can vary depending on different application scenarios and model designs. The following are several common selection methods: (1) Single query sequence, multiple key and value sequences: In this case, the query vector comes from one sequence, while the key and value vectors can come from two or more different sequences. For example, in multimodal learning, the query vector may come from a text sequence, while the key and value vectors may come from an image sequence or other types of data sequences. (2) Multiple query sequences, single key and value sequence: In some cases, there may be multiple query sequences, while the key and value sequences come from the same input. For example, in a question-answering system that processes multiple questions and a single document, each question can be regarded as a query sequence, while the document content provides the key and value sequences. (3) Each sequence serves as a query, key, and value respectively: In some complex model designs, each sequence may play the role of query, key, and value respectively in different contexts. This design allows the model to flexibly exploit the interaction between sequences in different tasks.

[0116] It should be noted that in the cross-attention learning mechanism, when adding special characters, only one special character can be added and added to the end of multiple sequences. In the learning process based on the cross-attention learning mechanism, the sequences learn semantic information from each other to obtain the semantic information of the content contained in the prompt word template corresponding to the special character. The learning process of the cross-attention learning mechanism between specific sequences can be referred to the relevant description of the learning process between the characters in the same sequence in the self-attention learning mechanism, which will not be repeated here.

[0117] In addition, the learning process of the cross-attention learning mechanism can also be implemented through the following implementation method: each sequence is learned once from front to back according to the order of each sequence displayed in the prompt word template, that is, after obtaining the semantic information of the first-ranked sequence, the second-ranked sequence learns the semantic information of the first-ranked sequence to obtain the first semantic information after learning; further, the third-ranked sequence learns the first semantic information to obtain the second semantic information; after learning in sequence, the semantic information contained in the prompt word template corresponding to the special character is obtained.

[0118] In an embodiment of the present application, after obtaining the recognition result based on the target language model, the initial cross-category recognition model may also be trained based on positive training samples and negative training samples to obtain a target cross-category recognition model.

[0119] In some embodiments, after obtaining the recognition result based on the target language model, the initial cross-category recognition model can also be trained based on the positive training samples and the negative training samples to obtain the target cross-category recognition model, including: taking the positive training samples and the negative training samples as the samples to be recognized whose annotation results are not cross-category and the samples to be recognized whose annotation results are cross-category, respectively, to obtain a set of samples to be recognized; inputting the samples to be recognized in the set of samples to be recognized into the initial cross-category recognition model respectively, and learning the semantic relevance between the text content and at least two levels of categories in the samples to be recognized to obtain the semantic matching degree between the text content and at least two levels of categories; performing cross-category prediction on the samples to be recognized based on the semantic matching degree between the text content and at least two levels of categories to obtain a prediction result of whether the samples to be recognized are cross-category; calculating the loss function based on the annotation result and the prediction result of whether the samples to be recognized are cross-category, and when the loss function satisfies the termination condition of the model training, adjusting the network parameters of the initial cross-category recognition model until the loss function satisfies the termination condition of the model training, to obtain the target cross-category recognition model.

[0120] In the above embodiment of the present application, the structured prompt words of each training sample are generated using the prompt word structure corresponding to the target language model. Based on the structured prompt words of the training samples, the target semantic model is used to identify and annotate whether the main semantic information of the training samples matches at least one of the corresponding primary categories under the guidance of the prompt words, and the training samples are annotated as positive samples and negative samples according to the recognition results, so as to train the initial cross-category recognition model to be trained, so as to obtain the target cross-category recognition model, so as to identify whether the published text content matches at least one of the corresponding primary categories based on the target cross-category recognition model, thereby improving the accuracy of cross-category recognition.

[0121] The following is a detailed description of the cross-category identification process. Figure 4 The specific implementation methods of each step in the relevant embodiments can be found in Figure 1a , Figure 1b , Figure 1c , Figure 2a , Figure 2b , Figure 2c , Figure 3a , Figure 3b , Figure 3c The relevant descriptions of the related embodiments are not repeated in this embodiment.

[0122] Figure 4 The following is a flow chart of a cross-category identification method provided by an exemplary embodiment of the present application. Figure 4 As shown, including:

[0123] 401. Obtain the published target content and at least one primary category selected for the target content when publishing the target content;

[0124] 402. Input the target content and at least one first-level category into the target cross-category recognition model, identify whether the target content matches at least one first-level category according to the main semantic information of the target content, and output the recognition result;

[0125] 403. Mark the target content as cross-category or non-cross-category based on the recognition result;

[0126] The target cross-category recognition model is obtained by training the initial cross-category recognition model based on negative training samples whose labeled results are cross-category and positive training samples whose labeled results are not cross-category.

[0127] Optionally, when publishing text content, at least two levels of categories are selected for the text content; the target content and at least one level of categories are input into a target cross-category recognition model, and whether the target content matches the at least one level of categories is identified according to the main semantic information of the target content, and the recognition result is output, including: based on a self-attention learning mechanism, semantic learning is performed on the guide instruction and the background information respectively to obtain the semantic information corresponding to the semantic information of the guide instruction and the semantic information of the background information; based on a cross-attention learning mechanism, the semantic association relationship between the text content and the semantic information corresponding to each of the at least two levels of categories is learned to obtain the associated semantic information between the text content and the semantic information corresponding to each of the at least two levels of categories, and the associated semantic information is used to characterize the target semantic matching degree between the text content and the at least two levels of categories; wherein the associated semantic information is used to characterize the target semantic matching degree between the text content and the at least two levels of categories, and the target semantic matching degree includes the overall semantic matching degree between the text content and the at least two levels of categories, or includes the local semantic matching degree between the text content and each level of categories; according to the semantic matching degree between the text content and the at least two levels of categories, whether the text content matches the at least two levels of categories is identified, and the recognition result is output. The recognition result is: cross-category or non-cross-category, or the recognition result is: yes or no, or the recognition result is: 0 or 1, 0 means no, 1 means yes.

[0128] In the above-mentioned embodiment of the present application, the trained target cross-category recognition model is used to identify whether the target content matches at least one primary category according to the main semantic information of the published target content and output the recognition result. Based on the recognition result, the target content is marked as cross-category or non-cross-category, thereby improving the accuracy of cross-category recognition.

[0129] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, it includes: a memory 50a and a processor 50b; the memory 50a is used to store a computer program; the processor 50b is coupled to the memory 50a and is used to execute the computer program to implement the following steps:

[0130] A plurality of training samples are obtained, each of which includes published text content and at least one first-level category selected for the text content when the text content is published; the plurality of training samples are respectively converted into corresponding structured prompt words according to the prompt word structure corresponding to the target language model, the structured prompt word corresponding to each training sample includes at least the training sample and a guide instruction, the guide instruction is used to instruct the target language model to identify whether the text content in the training sample matches at least one first-level category according to the main semantic information of the text content in the training sample; the structured prompt word corresponding to each training sample is input into the target language model, and under the guidance of the guide instruction, the text content in the training sample matches at least one first-level category according to the main semantic information of the text content in the training sample and outputs a recognition result; each training sample is marked as a positive training sample or a negative training sample according to the recognition result output by the target language model, and an initial cross-category recognition model is trained according to the positive training sample and the negative training sample to obtain a target cross-category recognition model.

[0131] Optionally, at least two levels of categories are selected for the text content when publishing the text content; the structured prompt words corresponding to each training sample also include: background information, the background information is used to describe the affiliation and / or non-affiliation between a specific secondary category and a specific primary category; when the processor inputs the structured prompt words corresponding to each training sample into the target language model, under the guidance of the guide instruction, identifies whether the text content in the training sample matches at least one level of category according to the main semantic information of the text content in the training sample and outputs the recognition result, it is specifically used to: input the structured prompt words corresponding to each training sample into the target language model, under the guidance of the guide instruction, identify whether the text content in the training sample matches at least two levels of categories according to the main semantic information and background information of the text content in the training sample, and output the recognition result.

[0132] Optionally, when the processor inputs the structured prompt word corresponding to each training sample into the target language model, under the guidance of the guide instruction, identifies whether the text content in the training sample matches at least two levels of categories according to the main semantic information and background information of the text content in the training sample, and outputs the recognition result, it is specifically used to: input the structured prompt word corresponding to each training sample into the target language model, perform semantic learning on the guide instruction and the background information in the structured prompt word to obtain the semantic information of the guide instruction and the semantic information of the background information; perform semantic relevance learning on the text content in the training sample and at least two levels of categories according to the semantic information of the guide instruction and the semantic information of the background information to obtain the semantic matching degree between the text content and at least two levels of categories; wherein the target semantic matching degree includes the overall semantic matching degree between the text content and at least two levels of categories, or includes the local semantic matching degree between the text content and each level of categories; and identify whether the text content matches at least two levels of categories according to the semantic matching degree between the text content and at least two levels of categories, and output the recognition result.

[0133] Optionally, when the processor inputs the structured prompt words corresponding to each training sample into the target language model and performs semantic learning on the guiding instructions and background information in the structured prompt words to obtain the semantic information of the guiding instructions and the semantic information of the background information, it is specifically used to: input the guiding instructions and the background information in the structured prompt words into the target language model, and based on the self-attention learning mechanism, perform semantic learning on the guiding instructions and the background information respectively to obtain the semantic information of the guiding instructions and the semantic information of the background information.

[0134] Optionally, when the processor inputs the guiding instruction and background information in the structured prompt word into the target language model, and based on the self-attention learning mechanism, performs semantic learning on the guiding instruction and the background information respectively to obtain the semantic information of the guiding instruction and the semantic information of the background information, the processor is specifically used to: input the guiding instruction and the background information in the structured prompt word into the target language model, perform vector mapping for each word segment in the guiding instruction and the background information to obtain the query vector, key vector and value vector corresponding to each word segment; for each word segment in the guiding instruction, use the query vector of the word segment to compare with each word segment in the guiding instruction; The key vectors of the word segments are matched respectively to obtain the attention weight of each word segment in the guidance instruction; a full-link calculation operation is performed on the value vectors of each word segment according to the attention weight of each word segment in the guidance instruction to obtain the semantic information of the guidance instruction; for each word segment in the background information, the query vector of the word segment is used to perform a matching calculation with the key vectors of each word segment in the background information to obtain the attention weight of each word segment in the background information; a full-link calculation operation is performed on the value vectors of each word segment according to the attention weight of each word segment in the background information to obtain the semantic information of the background information.

[0135] Optionally, when the processor learns the semantic relevance between the text content and at least two levels of categories in the training sample based on the semantic information of the guidance instruction and the semantic information of the background information to obtain the target semantic matching degree between the text content and at least two levels of categories, it is specifically used to: input the text content and at least two levels of categories in the training sample into the target language model, and based on the self-attention learning mechanism, perform semantic learning on the text content and at least two levels of categories respectively to obtain the semantic information corresponding to the text content and at least two levels of categories; based on the cross-attention learning mechanism, learn the semantic association relationship between the semantic information corresponding to the text content, at least two levels of categories, guidance instructions and background knowledge to obtain the associated semantic information between the semantic information corresponding to the text content and at least two levels of categories; based on the associated semantic information, obtain the target semantic matching degree between the text content and at least two levels of categories.

[0136] Optionally, when the processor learns the semantic association relationship between the semantic information corresponding to the text content, at least two levels of categories, guiding instructions and background knowledge based on the cross-attention learning mechanism, and obtains the associated semantic information between the text content and the semantic information corresponding to the at least two levels of categories, it is specifically used to: construct query vectors corresponding to the text content, at least two levels of categories, guiding instructions and background knowledge respectively; for any one of the text content, at least two levels of categories, guiding instructions and background knowledge, use the query vector of the content and the key vectors of other contents to perform matching calculations respectively to obtain the attention weight of each content; perform a full-link calculation operation on the value vectors corresponding to other contents according to the attention weight corresponding to each content, to obtain the associated semantic information between the text content and the semantic information corresponding to the at least two levels of categories.

[0137] Optionally, when the processor trains the initial cross-category recognition model based on the positive training samples and the negative training samples to obtain the target cross-category recognition model, it is specifically used to: use the positive training samples and the negative training samples as the samples to be recognized whose annotation results are not cross-category and the samples to be recognized whose annotation results are cross-category, respectively, to obtain a set of samples to be recognized; input the samples to be recognized in the set of samples to be recognized into the initial cross-category recognition model respectively, and learn the semantic relevance between the text content and at least two levels of categories in the samples to be recognized to obtain the semantic matching degree between the text content and at least two levels of categories; perform cross-category prediction on the samples to be recognized based on the semantic matching degree between the text content and at least two levels of categories to obtain a prediction result of whether the samples to be recognized are cross-category; calculate the loss function based on the annotation result and the prediction result of whether the samples to be recognized are cross-category, and when the loss function satisfies the termination condition of the model training, adjust the network parameters of the initial cross-category recognition model until the loss function satisfies the termination condition of the model training to obtain the target cross-category recognition model.

[0138] Optionally, when the processor inputs the samples to be identified in the sample set to be identified into the initial cross-category recognition model respectively, and learns the semantic relevance between the text content and at least two levels of categories in the samples to be identified to obtain the semantic matching degree between the text content and at least two levels of categories, it is specifically used to: input the text content and at least two levels of categories in the samples to be identified into the initial cross-category recognition model, and based on the self-attention learning mechanism, perform semantic learning on the text content and at least two levels of categories respectively to obtain the semantic information of the text content and the semantic information of at least two levels of categories; based on the cross-attention learning mechanism, learn the semantic association relationship between the text content and the semantic information corresponding to each of the at least two levels of categories, and obtain the associated semantic information between the text content and the semantic information corresponding to each of the at least two levels of categories; based on the associated semantic information, obtain the target semantic matching degree between the text content and at least two levels of categories.

[0139] In the above embodiment of the present application, the structured prompt words of each training sample are generated using the prompt word structure corresponding to the target language model. Based on the structured prompt words of the training samples, the target semantic model is used to identify and annotate whether the main semantic information of the training samples matches at least one of the corresponding primary categories under the guidance of the prompt words, and the training samples are annotated as positive samples and negative samples according to the recognition results, so as to train the initial cross-category recognition model to be trained, so as to obtain the target cross-category recognition model, so as to identify whether the published text content matches at least one of the corresponding primary categories based on the target cross-category recognition model, thereby improving the accuracy of cross-category recognition.

[0140] Furthermore, the processor is further configured to perform the following steps:

[0141] Obtain the published target content and at least one first-level category selected for the target content when publishing the target content; input the target content and at least one first-level category into the target cross-category recognition model, identify whether the target content matches at least one first-level category according to the main semantic information of the target content and output the recognition result; according to the recognition result, mark the target content as cross-category or non-cross-category. The target cross-category recognition model is obtained by training the initial cross-category recognition model based on negative training samples with cross-category labeling results and positive training samples with non-cross-category labeling results.

[0142] Optionally, when publishing text content, at least two levels of categories are selected for the text content; when the processor inputs the target content and at least one level of categories into the target cross-category recognition model, identifies whether the target content matches the at least one level of categories according to the main semantic information of the target content and outputs the recognition result, it is specifically used to: based on the self-attention learning mechanism, respectively perform semantic learning on the guidance instruction and the background information to obtain the semantic information corresponding to the semantic information of the guidance instruction and the semantic information of the background information; based on the cross-attention learning mechanism, learn the semantic association relationship between the text content and the semantic information corresponding to each of the at least two levels of categories, obtain the associated semantic information between the text content and the semantic information corresponding to each of the at least two levels of categories, and the associated semantic information is used to characterize the target semantic matching degree between the text content and the at least two levels of categories; wherein the associated semantic information is used to characterize the target semantic matching degree between the text content and the at least two levels of categories, and the target semantic matching degree includes the overall semantic matching degree between the text content and the at least two levels of categories, or includes the local semantic matching degree between the text content and each level of categories; according to the semantic matching degree between the text content and the at least two levels of categories, identify whether the text content matches the at least two levels of categories, and output the recognition result.

[0143] In the above-mentioned embodiment of the present application, the trained target cross-category recognition model is used to identify whether the target content matches at least one primary category according to the main semantic information of the published target content and output the recognition result. Based on the recognition result, the target content is marked as cross-category or non-cross-category, thereby improving the accuracy of cross-category recognition.

[0144] In addition, the processor can also execute 1a, Figure 1b , Figure 1c , Figure 2a , Figure 2b , Figure 2c The steps in the relevant embodiments will not be repeated here.

[0145] Furthermore, if Figure 5 As shown, the server also includes: a communication component 50c, a display 50d, a power component 50e, an audio component 50f and other components. Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 Components shown.

[0146] The detailed implementation and beneficial effects of the electronic device provided in the embodiments of the present application have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0147] The exemplary embodiments of the present application further provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is caused to implement the steps in the above-mentioned method embodiments.

[0148] An exemplary embodiment of the present application further provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the processor is enabled to implement the steps in the above-mentioned method embodiments.

[0149] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0150] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0151] The above-mentioned display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0152] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0153] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0154] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage, etc.) containing computer-usable program code.

[0155] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0156] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0157] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0158] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0159] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0160] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (Phase-change Random Access Memory, PRAM), static random access memory (SRAM), dynamic random access memory (Dynamic Random Access Memory, DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory, etc.

[0161] (EEPROM), flash memory or other memory technology, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0162] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, commodity or device including the element.

[0163] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.< / sep> < / cls> < / sep> < / cls> < / sep> < / cls>

Claims

1. A cross-category recognition model training method, characterized in that: include: Acquire multiple training samples, each training sample including published text content and at least one primary category selected for the text content when the text content is published; According to the prompt word structure corresponding to the target language model, the plurality of training samples are respectively converted into corresponding structured prompt words, wherein the structured prompt word corresponding to each training sample at least includes the training sample and a guide instruction, wherein the guide instruction is used to instruct the target language model to identify whether the text content in the training sample matches at least one primary category according to the main semantic information of the text content in the training sample; Inputting the structured prompt words corresponding to each training sample into the target language model, and under the guidance of the guide instruction, identifying whether the text content in the training sample matches at least one primary category according to the main semantic information of the text content in the training sample and outputting the recognition result; According to the recognition result output by the target language model, each training sample is labeled as a positive training sample or a negative training sample, and the initial cross-category recognition model is trained according to the positive training samples and the negative training samples to obtain the target cross-category recognition model.

2. The method according to claim 1, characterized in that When publishing the text content, at least two levels of categories are selected for the text content; the structured prompt words corresponding to each training sample also include: background information, the background information is used to describe the affiliation and / or non-affiliation relationship between a specific secondary category and a specific primary category; Inputting the structured prompt words corresponding to each training sample into the target language model, and under the guidance of the guide instruction, identifying whether the text content in the training sample matches at least one primary category according to the main semantic information of the text content in the training sample and outputting the recognition result, including: The structured prompt words corresponding to each training sample are input into the target language model. Under the guidance of the guiding instructions, based on the main semantic information of the text content in the training sample and the background information, it is identified whether the text content in the training sample matches at least two levels of categories, and the recognition result is output.

3. The method according to claim 2, characterized in that Inputting the structured prompt words corresponding to each training sample into the target language model, and under the guidance of the guide instruction, identifying whether the text content in the training sample matches at least two levels of categories according to the main semantic information of the text content in the training sample and the background information, and outputting the recognition result, including: Inputting the structured prompt words corresponding to each training sample into the target language model, and performing semantic learning on the guide instructions and background information in the structured prompt words to obtain the semantic information of the guide instructions and the semantic information of the background information; According to the semantic information of the guiding instruction and the semantic information of the background information, the text content and at least two levels of categories in the training sample are studied for semantic relevance to obtain the semantic matching degree between the text content and at least two levels of categories; wherein the target semantic matching degree includes the overall semantic matching degree between the text content and at least two levels of categories, or includes the local semantic matching degree between the text content and each level of categories; According to the semantic matching degree between the text content and at least two levels of categories, it is identified whether the text content matches the at least two levels of categories, and a recognition result is output.

4. The method according to claim 3, characterized in that The step of inputting the structured prompt word corresponding to each training sample into the target language model and performing semantic learning on the guide instruction and background information in the structured prompt word to obtain the semantic information of the guide instruction and the semantic information of the background information includes: The guiding instructions and background information in the structured prompt words are input into the target language model, and based on the self-attention learning mechanism, semantic learning is performed on the guiding instructions and the background information respectively to obtain the semantic information of the guiding instructions and the semantic information of the background information.

5. The method according to claim 4, characterized in that Inputting the guiding instruction and the background information in the structured prompt word into the target language model, and performing semantic learning on the guiding instruction and the background information based on the self-attention learning mechanism to obtain the semantic information of the guiding instruction and the semantic information of the background information, including: Inputting the guide instruction and the background information in the structured prompt word into the target language model, and performing vector mapping on each word segment in the guide instruction and the background information to obtain a query vector, a key vector, and a value vector corresponding to each word segment; For each word segment in the guidance instruction, a matching degree is calculated using the query vector of the word segment and the key vector of each word segment in the guidance instruction to obtain an attention weight of each word segment in the guidance instruction; a full-link calculation operation is performed on the value vector of each word segment according to the attention weight of each word segment in the guidance instruction to obtain semantic information of the guidance instruction; For each word segment in the background information, a matching degree is calculated using the query vector of the word segment and the key vector of each word segment in the background information to obtain an attention weight of each word segment in the background information; and a full-link calculation operation is performed on the value vector of each word segment according to the attention weight of each word segment in the background information to obtain semantic information of the background information.

6. The method according to claim 3, characterized in that According to the semantic information of the guiding instruction and the semantic information of the background information, the text content in the training sample and the at least two-level categories are studied for semantic relevance to obtain a target semantic matching degree between the text content and the at least two-level categories, including: Inputting the text content and at least two levels of categories in the training sample into the target language model, and performing semantic learning on the text content and the at least two levels of categories based on a self-attention learning mechanism, respectively, to obtain semantic information corresponding to the text content and the at least two levels of categories; Based on a cross-attention learning mechanism, learning the semantic association relationship between the text content, the at least two levels of categories, the guiding instructions, and the semantic information corresponding to the background knowledge, so as to obtain the associated semantic information between the text content and the semantic information corresponding to the at least two levels of categories; Based on the associated semantic information, a target semantic matching degree between the text content and at least two levels of categories is obtained.

7. The method according to claim 6, characterized in that Based on the cross-attention learning mechanism, the semantic association relationship between the semantic information corresponding to the text content, the at least two levels of categories, the guiding instructions, and the background knowledge is learned to obtain the associated semantic information between the text content and the semantic information corresponding to the at least two levels of categories, including: Constructing query vectors corresponding to the text content, the at least two levels of categories, the guiding instructions, and the background knowledge respectively; For any one of the text content, the at least two-level categories, the guiding instructions, and the background knowledge, respectively, a matching degree is calculated using the query vector of the content and the key vectors of the other contents to obtain an attention weight of each content; A full-link calculation operation is performed on the value vectors corresponding to other contents according to the attention weight corresponding to each content, so as to obtain the associated semantic information between the text content and the semantic information corresponding to each of the at least two levels of categories.

8. The method according to any one of claims 2 to 7, characterized in that: Training the initial cross-category recognition model according to the positive training samples and the negative training samples to obtain a target cross-category recognition model includes: The positive training samples and the negative training samples are respectively used as samples to be identified whose annotation results are not across categories and samples to be identified whose annotation results are across categories, so as to obtain a set of samples to be identified; Inputting the samples to be identified in the sample set to be identified into the initial cross-category recognition model respectively, and learning the semantic relevance between the text content and at least two levels of categories in the samples to be identified, so as to obtain the semantic matching degree between the text content and at least two levels of categories; Performing cross-category prediction on the sample to be identified according to the semantic matching degree between the text content and at least two levels of categories to obtain a prediction result of whether the sample to be identified is cross-category; The loss function is calculated according to whether the sample to be identified is a cross-category labeling result and a prediction result, and when the loss function meets the model training termination condition, the network parameters of the initial cross-category recognition model are adjusted until the loss function meets the model training termination condition to obtain the target cross-category recognition model.

9. The method according to claim 8, characterized in that Inputting the samples to be identified in the sample set to be identified into the initial cross-category recognition model respectively, and learning the semantic relevance between the text content and at least two levels of categories in the samples to be identified to obtain the semantic matching degree between the text content and at least two levels of categories, including: Inputting the text content and at least two levels of categories in the sample to be identified into the initial cross-category recognition model, and performing semantic learning on the text content and the at least two levels of categories respectively based on a self-attention learning mechanism to obtain semantic information of the text content and semantic information of the at least two levels of categories; Based on a cross-attention learning mechanism, learning the semantic association relationship between the text content and the semantic information corresponding to the at least two levels of categories, and obtaining the associated semantic information between the text content and the semantic information corresponding to the at least two levels of categories; Based on the associated semantic information, a target semantic matching degree between the text content and at least two levels of categories is obtained.

10. A cross-category identification method, characterized in that: include: Acquire the published target content and at least one primary category selected for the target content when the target content is published; Inputting the target content and the at least one first-level category into a target cross-category recognition model, identifying whether the target content matches the at least one first-level category according to the main semantic information of the target content, and outputting a recognition result; According to the recognition result, marking the target content as cross-category or non-cross-category; The target cross-category recognition model is obtained by training an initial cross-category recognition model based on negative training samples whose annotation results are cross-category and positive training samples whose annotation results are not cross-category.

11. The method according to claim 10, characterized in that When publishing the text content, at least two levels of categories are selected for the text content; the target content and the at least one level of categories are input into a target cross-category recognition model, and whether the target content matches the at least one level of categories according to the main semantic information of the target content is identified and a recognition result is output, including: Based on a self-attention learning mechanism, semantic learning is performed on the guidance instruction and the background information respectively to obtain semantic information corresponding to the semantic information of the guidance instruction and the semantic information of the background information; Based on the cross-attention learning mechanism, the semantic association relationship between the text content and the semantic information corresponding to the at least two levels of categories is learned to obtain the associated semantic information between the text content and the semantic information corresponding to the at least two levels of categories, and the associated semantic information is used to characterize the target semantic matching degree between the text content and the at least two levels of categories; wherein the associated semantic information is used to characterize the target semantic matching degree between the text content and the at least two levels of categories, and the target semantic matching degree includes the overall semantic matching degree between the text content and the at least two levels of categories, or includes the local semantic matching degree between the text content and each level of categories; According to the semantic matching degree between the text content and at least two levels of categories, it is identified whether the text content matches the at least two levels of categories, and a recognition result is output.

12. An electronic device, characterized in that: include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method of any one of claims 1-9 or 10-11.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to implement the steps in the method according to any one of claims 1 to 9 or 10 to 11.

14. A computer program product, characterized in that The computer program product comprises a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps of any one of the methods of claims 1-9 or 10-11.

Images