User category identification method and device, electronic equipment and storage medium

Abstract

The application provides a user category identification method and device, electronic equipment and storage medium. The method comprises the following steps: screening a plurality of to-be-analyzed texts from a historical text set corresponding to a user according to an abnormal value between a semantic vector and a semantic encoding vector of the historical text; performing semantic information extraction processing on the plurality of to-be-analyzed texts respectively to obtain corresponding to-be-analyzed semantic vectors; assigning a semantic weight value to the plurality of to-be-analyzed semantic vectors; and predicting a category of the user according to the plurality of to-be-analyzed semantic vectors and the corresponding semantic weight values. The technical scheme of the embodiment of the application filters out invalid and low-quality texts through the abnormal value between the semantic vector and the semantic encoding vector, and analyzes the plurality of to-be-analyzed texts of the user together, so as to complete accurate prediction of a real user category of the user.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a user category identification method, apparatus, electronic device, and storage medium. Background Technology

[0002] Research on user category identification is beneficial for providing different services to different categories of users. Existing user category identification schemes mostly rely on manually assigning labels to texts associated with users, then using text classification schemes to predict the category of individual texts, and finally summarizing the prediction results of all text categories associated with users, and judging the true category of the user based on the summarized data.

[0003] Clearly, the above solution can only predict based on a single text, and does not effectively utilize multiple texts corresponding to a user, thus failing to accurately predict the user category. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application provide a user category identification method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of the embodiments of this application, a user category identification method is provided, comprising: filtering multiple texts to be analyzed from a set of historical texts corresponding to a user based on outliers between the semantic vectors and semantic encoding vectors of historical texts; wherein the semantic encoding vectors are obtained by performing self-encoding processing on the semantic vectors; performing semantic information extraction processing on the multiple texts to be analyzed respectively to obtain corresponding semantic vectors to be analyzed; assigning semantic weight values ​​to the multiple semantic vectors to be analyzed; and predicting the user's category based on the multiple semantic vectors to be analyzed and the corresponding semantic weight values.

[0006] According to one aspect of the embodiments of this application, a user category identification device is provided, comprising: a preprocessing module configured to filter multiple texts to be analyzed from a set of historical texts corresponding to a user based on outliers between the semantic vectors and semantic encoding vectors of historical texts; wherein the semantic encoding vectors are obtained by performing self-encoding processing on the semantic vectors; a semantic vector acquisition module configured to perform semantic information extraction processing on the multiple texts to be analyzed respectively to obtain corresponding semantic vectors to be analyzed; a weight value allocation module configured to allocate semantic weight values ​​to the multiple semantic vectors to be analyzed; and a classification module configured to predict the user's category based on the multiple semantic vectors to be analyzed and the corresponding semantic weight values.

[0007] In one embodiment, the preprocessing module includes:

[0008] The historical semantic vector acquisition unit is configured to extract semantic information from each historical text in the historical text set to obtain the historical semantic vector corresponding to each historical text.

[0009] The historical semantic encoding vector acquisition unit is configured to perform self-encoding processing on each historical semantic vector to obtain the corresponding historical semantic encoding vector.

[0010] The first outlier acquisition unit is configured to calculate the outlier between each historical semantic vector and the corresponding historical semantic encoding vector.

[0011] The first text acquisition unit is configured to use historical texts with outliers less than a preset outlier threshold as texts to be analyzed.

[0012] In one embodiment, the preprocessing module further includes:

[0013] The concatenated text acquisition unit is configured to concatenate multiple pieces of text information associated with the user with their corresponding text names to obtain multiple concatenated texts; wherein, the text names are used to represent the topic of the corresponding text information;

[0014] The replacement unit is configured to replace text information related to a specified type of character in each concatenated text with a preset character, thereby obtaining each historical text in the historical text set.

[0015] In one embodiment, the text acquisition unit to be analyzed includes:

[0016] The first text acquisition module is configured such that if the number of texts in the historical text subset with an outlier value less than a preset outlier threshold is greater than a preset number threshold, then historical texts with an acquisition time within a preset time period are selected from the historical text subset as the first text.

[0017] The second text acquisition module is configured to randomly extract other historical texts besides the first text from a subset of historical texts as the second text; wherein the sum of the number of the first text and the second text is not greater than a preset number threshold.

[0018] The text to be analyzed is obtained from the module, which is configured to use the first and second texts as the texts to be analyzed.

[0019] In one embodiment, the preprocessing module includes:

[0020] The semantic vector acquisition unit is configured to input the historical texts in the historical text set into the language model to obtain the first semantic vector output by the first network layer segment of the language model and the second semantic vector output by the second network layer segment of the language model; wherein the output signal of the first network layer segment is used as the input signal of the second network layer segment.

[0021] The second semantic encoding vector acquisition unit is configured to perform self-encoding processing on the second semantic vector to obtain the second semantic encoding vector.

[0022] The second outlier acquisition unit is configured to calculate the outlier between the first semantic vector and the second semantic encoding vector corresponding to each historical text.

[0023] The second text acquisition unit is configured to use historical texts with outliers less than a preset outlier threshold as texts to be analyzed.

[0024] In one embodiment, the user category identification device further includes:

[0025] The first training text input module is configured to input training text into the language model to be trained.

[0026] The first training module is configured to predict the semantic vector output by the first network layer of the language model to be trained based on a preset discriminator, and train the first network layer of the language model to be trained based on the obtained prediction values ​​to obtain the trained language model.

[0027] The second training text input module is configured to input training text into the trained language model;

[0028] The second training module is configured to train the second network layer of the trained language model based on the semantic vectors output by the first network layer and the second network layer of the trained language model, thereby obtaining a language model for receiving historical text input from the historical text set.

[0029] In one embodiment, during the training of the first network layer of the language model to be trained, the discriminator is also trained; the second training module includes:

[0030] The training encoding vector acquisition unit is configured to perform autoencoding processing on the semantic vector output by the second network layer of the trained language model to obtain the training encoding vector;

[0031] The second training unit is configured to train the second network layer of the trained language model based on the training encoding vector and the semantic vector output by the first network layer of the trained language model, until the error between the first predicted value obtained by the discriminator for the semantic vector output by the first network layer and the second predicted value obtained for the training encoding vector is within a preset error threshold.

[0032] In one embodiment, the classification module includes:

[0033] The multidimensional vector acquisition unit is configured to perform weighted aggregation processing on multiple semantic vectors to be analyzed and their corresponding semantic weight values ​​to obtain the multidimensional vector corresponding to the user.

[0034] The classification unit is configured to predict the scalar obtained by mapping a multidimensional vector to determine the category to which the user belongs.

[0035] According to one aspect of the embodiments of this application, an electronic device is provided, including one or more processors; and a storage device for storing one or more computer programs, which, when executed by the one or more processors, cause the electronic device to implement the user category identification method as described above.

[0036] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a computer's processor, cause the computer to perform the user category identification method as described above.

[0037] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the user category identification method provided in the various alternative embodiments described above.

[0038] According to one aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the user category identification method as described above.

[0039] In the technical solution provided in the embodiments of this application, multiple texts to be analyzed by a user are used to predict the user category. During the prediction process, invalid and low-quality texts are filtered out by outliers between semantic vectors and semantic encoding vectors, which can improve the quality of multiple texts to be analyzed. Furthermore, by assigning semantic weight values ​​to multiple texts to be analyzed, effective semantics in multiple texts to be analyzed are filtered, thereby achieving accurate prediction of the real user category.

[0040] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0041] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0042] Figure 1 This is a schematic diagram of one implementation environment involved in this application;

[0043] Figure 2 This is a flowchart illustrating a user category identification method in an exemplary embodiment of this application;

[0044] Figure 3 This is a schematic diagram illustrating the structure of a user category identification network, as shown in an exemplary embodiment of this application.

[0045] Figure 4 yes Figure 2 A flowchart of step S210 in an exemplary embodiment shown in the illustrated example;

[0046] Figure 5 This is a schematic diagram of the structure of the first preprocessing network structure shown in an exemplary embodiment of this application;

[0047] Figure 6 This is a schematic diagram of the second preprocessing network structure shown in an exemplary embodiment of this application;

[0048] Figure 7 This is a flowchart illustrating a user category identification method in another exemplary embodiment of this application;

[0049] Figure 8 yes Figure 2 A flowchart of step S210 in an exemplary embodiment shown in the illustrated example;

[0050] Figure 9 This is a schematic diagram illustrating the structure of the third preprocessing network structure in an exemplary embodiment of this application;

[0051] Figure 10 This is a flowchart illustrating a user category identification method in another exemplary embodiment of this application;

[0052] Figure 11 This is a schematic diagram illustrating the structure of a user category identification device according to an exemplary embodiment of this application;

[0053] Figure 12 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0054] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0055] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0056] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0057] It should also be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0058] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0059] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0060] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learn-by-doing.

[0061] The language models proposed in the following examples are machine learning models that can extract semantics from text, such as BERT (Bidirectional Encoder Representation from Transformers, a language model that learns a good feature representation for words by running a self-supervised learning method on a massive corpus), RNN (Recurrent Neural Network), word2vec (a related model used to generate word vectors, which is a shallow two-layer neural network), etc., and there are no restrictions here.

[0062] The user category identification method, device, electronic device, and storage medium proposed in this application involve artificial intelligence technology, machine learning, and other technologies. These embodiments will be described in detail below.

[0063] It is understood that in the specific implementation of this application, user text information and related data such as text and tags are involved. When the embodiments of this application are applied to specific products or technologies, permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws and standards of the relevant countries and regions.

[0064] Please refer to the following first. Figure 1 , Figure 1 This is a schematic diagram of an implementation environment related to this application. The implementation environment includes a terminal 100 and a server 200, which communicate with each other via a wired or wireless network.

[0065] It should be understood that Figure 1 The number of terminal 100 and server 200 shown is merely illustrative. Any number of terminals 100 and servers 200 can be used as needed.

[0066] In some embodiments of this application, the user category identification method can be executed by server 200, and correspondingly, the user category identification device is configured in server 200. Terminal 100 is used to obtain text information associated with the user, which is related to the category that the user needs to identify. For example, when the text information is the user's complaint text information or product description information, the user's business category can be identified based on the text information; when the text information is text content uploaded by the user on the Internet, or comments left by the user under a video, the user's online social category can be identified based on the text information. Of course, this is only an example of some text information associated with the user. In other embodiments, other text information of the user can also be used to perform category identification processing on the user to obtain the category to which the user belongs. No specific limitation is made here.

[0067] Terminal 100 sends the text information associated with the user to server 200. First, server 200 processes the text information to obtain the user's historical text. Then, server 200 predicts the category to which the user belongs based on the user's historical text and sends the prediction result to terminal 100. The prediction result can be visualized and displayed through the display module built into terminal 100.

[0068] For example, after receiving text information, terminal 100 sends the text information to server 200. Server 200 processes the text information to obtain the user's historical text set. Then, based on the outliers between the semantic vectors and semantic encoding vectors of the historical texts, it selects multiple texts to be analyzed from the user's historical text set. The semantic encoding vector is obtained by self-encoding the semantic vector. The multiple texts to be analyzed are processed by semantic information extraction to obtain corresponding semantic vectors to be analyzed. Semantic weight values ​​are assigned to the multiple semantic vectors to be analyzed. The user's category is predicted based on the multiple semantic vectors to be analyzed and their corresponding semantic weight values.

[0069] In other embodiments, certain terminal devices 100 may have similar functions to server 200 to perform the user category identification method provided in the embodiments of this application.

[0070] The terminal 100 includes, but is not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft. It can be any electronic device capable of image visualization, such as smartphones, tablets, laptops, or computers; no restrictions are imposed here. The server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. Multiple servers can form a blockchain, with the server acting as a node on the blockchain. The server 200 can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms; no restrictions are imposed here either.

[0071] It should be noted that this embodiment is merely an exemplary implementation environment provided to facilitate understanding of the ideas of this application, and should not be considered as providing any limitation on the scope of use of this application.

[0072] Figure 2 This is a flowchart illustrating a user category identification method according to an exemplary embodiment, which can be applied to... Figure 1 The implementation environment shown is specifically executed by server 200. It should be understood that this method can also be used in other exemplary implementation environments and specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which this method is applicable.

[0073] In an exemplary embodiment, the method may include steps S210 to S270, which are described in detail below:

[0074] Step S210: Based on the outliers between the semantic vector and the semantic encoding vector of the historical text, select multiple texts to be analyzed from the historical text set corresponding to the user.

[0075] In this embodiment, historical text refers to text information associated with the user. Specifically, text information associated with the user can be extracted first, such as comments sent or received by the user, complaints sent or received by the user, academic articles sent by the user, and other text information subjectively created by the user, as well as text information created by others for the user.

[0076] Subsequently, the text information associated with the user can be processed using a unified template to obtain historical text under a fixed template. This unified template processing can be used to uniformly delete or replace certain types of characters in the text information, or to concatenate the topic corresponding to the text information with the text information based on the content of the text information, or to add the creation time of the text, etc. There are no specific restrictions here.

[0077] Based on the above process, a user can obtain multiple historical texts, which are then constructed into a historical text set for that user.

[0078] Of course, the historical text set corresponding to a user may contain invalid or low-quality text. In this case, invalid text can be filtered out by the outliers between the semantic vector and the semantic encoding vector of the historical text, and finally multiple texts of the user to be analyzed can be obtained.

[0079] In this embodiment, the semantic vector is the vector output by the language model from the historical text. The language model can be, for example, one of the language models listed above, and is not limited here. The semantic encoding vector is the vector obtained by performing self-encoding processing on the semantic vector, and outliers are the differences between the semantic vector and the semantic encoding vector, such as mean squared error, vector squared difference, etc., and are not specifically limited here.

[0080] In this embodiment, by using the error value between the semantic vector describing the historical text and the semantic encoding vector obtained by fitting the semantic vector with the autoencoder, which is also used to describe the semantics of the historical text, the historical text with a large error value is filtered out, so as to achieve the purpose of filtering out low-quality and invalid historical text.

[0081] Step S230: Extract semantic information from multiple texts to be analyzed to obtain corresponding semantic vectors.

[0082] In this embodiment, after filtering the historical texts in the user's historical text set, the user's text to be analyzed is obtained. Then, the user's category can be further identified based on the text to be analyzed.

[0083] In this embodiment, the process of user category identification can be as follows: Figure 3 This is implemented in the user category recognition network shown.

[0084] Specifically, the user category recognition network includes a language model, an attention structure (Self attention), and multiple discriminators (MLP). Of course, to distinguish it from the language model in step S210, the language model of the user category recognition network is referred to as the category recognition language model.

[0085] First, all the text to be analyzed for a user is simultaneously input into the category recognition language model. For a user m i It corresponds to CN i If there are 100 texts to be analyzed, then all the texts to be analyzed for that user are:

[0086] Once the text to be analyzed enters the category recognition language model, the last layer CLS (Special Classification Embedding) vector of the category recognition language model is the semantic vector to be analyzed for the text to be analyzed.

[0087] Step S250: Assign semantic weight values ​​to multiple semantic vectors to be analyzed.

[0088] In this embodiment, the multiple semantic vectors output by the category recognition language model for each text to be analyzed are also input into the attention structure, where the semantic weight value of each text to be analyzed for the user is calculated.

[0089] In one embodiment, the semantic weight value of each text to be analyzed can be calculated in the following manner:

[0090]

[0091] Among them, attention ij Let MLP be the semantic weight value of the j-th text to be analyzed for the i-th user. at It is an MLP with an attention structure.

[0092] In this embodiment, based on the richness of the semantic vector of the text to be analyzed, the richer the semantics represented by the semantic vector, the greater the semantic weight value obtained. In this way, in the subsequent user category calculation, the effective user category can be filtered according to the semantic weight value corresponding to each text to be analyzed, so that the text to be analyzed with a larger semantic weight value can be used as reference data for user category prediction.

[0093] Step S270: Predict the user's category based on multiple semantic vectors to be analyzed and their corresponding semantic weight values.

[0094] In this embodiment, by weighting and aggregating the semantic vector to be analyzed and its corresponding semantic weight values, a multidimensional vector representation of a user can be obtained, which is then processed by an MLP. m (certainly, Figure 3 MLP m Previously, an embedding layer could exist to process the semantic vector to be analyzed and its corresponding semantic weight values. Figure 3 The text does not explicitly illustrate that the embedding layer (understood as a deep learning structure for processing classification vectors) maps to a scalar, and finally, the sigmoid function is used to obtain the user's performance across different user categories (i.e., during training). Figure 3 The predicted probability of the user category is determined by the user category target label set during the training of the user category recognition network shown in the figure (and the user category target label is a manually verified category target label).

[0095] In one embodiment, user categories are calculated in the following manner:

[0096]

[0097] Among them, y i ′ is the predicted value for the i-th merchant category, MLP m For MLP used for user category identification, it transforms the weighted aggregation vector into a vector corresponding to the number of user categories (the number of user category tags mentioned above). The sigmoid function is a common S-shaped function in biology, also known as the S-shaped growth curve.

[0098] In this embodiment, valid texts to be analyzed are first filtered out based on outliers between the semantic vectors and semantic encoding vectors of historical texts. This reduces the amount of text processing in subsequent user category identification and improves the processing efficiency of the texts to be analyzed and the accuracy of user category identification. At the same time, multiple texts to be analyzed for a single user are analyzed simultaneously, and semantic weight values ​​are assigned to multiple texts to be analyzed. Texts with higher semantic weight values ​​have richer semantics and better reflect the user's characteristics. Thus, valid semantics in multiple texts to be analyzed can be filtered out through semantic weight values, thereby completing the effective prediction of the user's true user category and providing reference data for the analysis of texts and users.

[0099] Of course, in other embodiments, after the category recognition language model outputs the semantic vector to be analyzed, it can also be further analyzed through... Figure 3 MLP (Mediator-Based Logic Provider) c Predict the semantic vector to be analyzed to obtain the text category corresponding to the text to be analyzed.

[0100] Specifically, the semantic vector to be analyzed is processed by MLP c Mapped to dimensions and the number of text categories (i.e., training) Figure 3 The number of text category tags set during the training of the user category recognition network shown in the figure, and the text category tags are the same vectors (and these text category tags are manually verified category tags). Then, the softmax function is used to convert them into probabilities, and each bit of the vector corresponds to the probability of each text category. The following is the process of calculating the text category of the text to be analyzed when the user category recognition model is BERT (a language model) in one embodiment:

[0101]

[0102] Where i is the user sequence number, j is the sequence number of the text to be analyzed, and t′ ij Let c be the predicted probability of the text category for the j-th text to be analyzed corresponding to the i-th user. ijLet $\frac{i}{j}$ be the text to be analyzed corresponding to the $i$-th user, $\exp$ be an exponential function with base $e$, $BERT$ be a language model, $softmax$ be a normalized exponential function in deep learning, and $MLP$ be a language model. c It is a discriminator.

[0103] After outputting the text categories of each text to be analyzed, text analysis can be performed on each text.

[0104] In this embodiment, for Figure 3 The user category recognition network in the paper also proposes a training method that is completed through the processes of text category prediction and user category prediction.

[0105] First, input the training text of users with user class target tags. Figure 3 In the structure shown, each training text should have a corresponding text class target tag. For MLP c The text categories of each text to be analyzed output can be used to obtain the text category and the corresponding text category target pairing MLP based on the predicted text category. c And a category recognition language model is trained. The loss function here can be set as ComplaintLoss, that is, ComplaintLoss is the sum of the losses for each category recognition language model. Figure 3 MLP c And the loss function for training the category recognition language model:

[0106]

[0107] Where, loss im Let T be the loss function for the m-th training text corresponding to the i-th user, and let crossentropy be the cross-entropy function. im Let T' be the vector consisting of the prediction results for all training texts. im Let X be the vector composed of text class target tags for the training text, X' be the vector composed of all text class target tags, and X' be the vector composed of all prediction results. k For a single text class target tag, x x ′ represents the predicted category value for a single text, k represents the number of categories for the text label, and ComplaintLoss represents the training MLP value. c And the loss function for training the category recognition language model, J l This represents the set of training texts for the labeled text class target tags.

[0108] Subsequently, the semantic vectors of the training text are passed through an attention structure to reach the MLP. m This yields the predicted user category value for the user corresponding to the training text. Then, the MLP can be applied based on the predicted user category value and the user category target.m Attention structure and Figure 3 The middle layer is used to draw the embedding layer for training, which is a method for training MLP. m Attention structure and Figure 3 The loss function used for training the embedding layer is set as MerchantLoss.

[0109]

[0110] Among them, y i For the target signature of the user class of the i-th user, y i ′ represents the predicted user category value for the i-th user.

[0111] Of course, to ensure that the GPU memory does not overflow during training, the number of training texts loaded each time cannot be too large. In cases where there are too many training texts for some users, simple sampling can be performed, which is equivalent to discarding some data. For example, in order to make full use of the data under limited hardware resources, it is necessary to sample the training data during training. Specifically, if the number of training texts selected for each user in each iteration is greater than the preset number, then the most recent training texts of the users are taken out first, and then random sampling is performed from the remaining training texts. Finally, no more than the preset number of training texts are used for training.

[0112] The embodiment proposed Figure 3 The user category recognition network shown is a multi-task model, which simultaneously performs text category prediction and user category prediction, and the two tasks share a portion of the network structure. In this case, the outputs of both tasks are used to train the structure in the user category recognition network, thereby improving the training effect of the user category recognition network and making the user category recognition network more accurate when used.

[0113] To ensure continued training using a multi-task parallel structure, this embodiment... Figure 3 The structure in the code, and the final loss function used, is:

[0114] Loss=MerchantLoss+αComplaintLoss

[0115] Regarding the selection of parameter α, a relatively large value can be arbitrarily set in the early stage of training. As training progresses, ComplaintLoss will converge first, and the network's prediction effect on each training text will also tend to stabilize. At this point, α can be appropriately lowered and training can continue. Subsequently, by continuously lowering α, the prediction effects of MerchantLoss and user categories will approach stability, thus completing the training.

[0116] Figure 4 yes Figure 2 The flowchart of step S210 in the illustrated embodiment is shown in an exemplary embodiment. Figure 4 As shown, in an exemplary embodiment, step S210, which involves selecting multiple texts to be analyzed from the user's historical text set based on outliers between the semantic vector and the semantic encoding vector of the historical text, may include steps S410 to S470, as detailed below:

[0117] Step S410: Extract semantic information from each historical text in the historical text set to obtain the historical semantic vector corresponding to each historical text.

[0118] In this embodiment, please refer to Figure 5 The first preprocessing network structure processes the historical text sets of each user.

[0119] Specifically, the historical texts in the historical text set are input into the language model in the first preprocessing network structure, and thus the historical semantic vectors of each historical text in the historical text set can be obtained.

[0120] Step S430: Perform autoencoding on each historical semantic vector to obtain the corresponding historical semantic encoded vector.

[0121] After a historical text passes through the language model in the first preprocessing network structure, the resulting historical semantic vectors will reach the autoencoder structure. After autoencoder processing, the historical semantic encoded vectors corresponding to each historical semantic vector will be obtained.

[0122] Step S450: Calculate the outlier values ​​between each historical semantic vector and its corresponding historical semantic encoding vector.

[0123] In this embodiment, the difference between the historical semantic vector and the historical semantic encoding vector corresponding to a historical text is used as the outlier of the historical text, thereby filtering the text to be analyzed from the historical text set based on the outlier.

[0124] In one specific embodiment, the outlier is the mean squared error between the historical semantic vector and the historical semantic encoding vector. Specifically, the outlier can be calculated in the following way:

[0125] MSE(emb1,emb2)=∑(emb1-emb2) 2

[0126] Dv = MSE(emb1) n ,emb2 n )

[0127] Where emb1 is the historical semantic vector of the historical text, emb2 is the historical semantic encoding vector corresponding to the historical text, Dv is the outlier of the nth historical text, and MSE(emb1,emb2) is the mean squared error between vectors emb1 and emb2. n For the historical semantic vector of the nth historical text, emb2 n This is the historical semantic encoding vector for the nth historical text.

[0128] Step S470: Select historical texts with outliers less than a preset outlier threshold as texts to be analyzed.

[0129] After calculating the outliers, the outliers of a user's historical texts can be sorted by numerical value. Historical texts with values ​​greater than or equal to a preset outlier threshold are filtered out, and historical texts with outliers less than the outlier threshold are left as the texts to be analyzed for that user.

[0130] Of course, this abnormal threshold can be an empirical parameter or it can be set adaptively based on the amount of historical text for different users. If it is necessary to control the amount of historical text for each user to a certain value, then different abnormal thresholds can be set based on the amount of historical text for different users.

[0131] In one embodiment, for Figure 5 The training of the first preprocessing network can be achieved by first training the language model of the first preprocessing network, that is, by using the predicted values ​​of historical semantic vectors after passing through the MLP and the text class target tags of historical texts as loss functions to train the semantic model of the first preprocessing network and the MLP. Then, this state is maintained, that is, the parameters of the semantic model of the first preprocessing network and the MLP remain unchanged in subsequent training processes. The autoencoder structure is then trained by using the mean squared error between the training semantic vector and the training semantic encoding vector output by the first preprocessing network after passing through the training text. Since the mean squared error between the training semantic vector and the training semantic encoding vector is a specific value, a stopping condition for training also needs to be set. At this time, during the training process of the autoencoder structure, the predicted values ​​of the training semantic vector and the training semantic encoding vector are also obtained through the MLP. When the difference between the two predicted values ​​is less than a preset error threshold, the training is completed. This error threshold can be set by empirical parameters.

[0132] In this embodiment, the training stopping condition is determined by the error threshold. If the difference between the predicted value obtained for the training semantic vector and the predicted value obtained for the training semantic encoding vector is greater than or equal to the error threshold, it proves that there is a large semantic performance deviation between the training semantic encoding vector and the training semantic vector, that is, the autoencoder structure needs to be trained further. If the difference between the predicted value obtained for the training semantic vector and the predicted value obtained for the training semantic encoding vector is less than the error threshold, it proves that there is a small semantic performance deviation between the training semantic encoding vector and the training semantic vector, that is, the autoencoder structure can fit the training semantic vector, and at this time, the training of the autoencoder structure can be stopped.

[0133] In another embodiment, it can also be achieved through Figure 6 The second preprocessing structure shown is used to calculate outliers between historical semantic vectors and their corresponding historical semantic encoded vectors.

[0134] Specifically, when calculating outliers, the same historical text is input into the two pre-trained language models. The historical semantic vector output by one language model is also entered into the autoencoder structure to obtain the historical encoded vector of the historical text.

[0135] Subsequently, the vector difference between the historical semantic vector output by a language model and the historical semantic encoded vector output by the autoencoder structure is used as an outlier.

[0136] for Figure 6 The training process of the second preprocessing structure shown is as follows: First, train the language model and MLP whose output vectors do not have an autoencoder structure. That is, train using the predicted values ​​of the semantic vectors output by the language model and the real labels of the training text. Then, keep the parameters of the language model unchanged, and train the other language model plus autoencoder structure using the mean squared error between the training semantic vectors output by the language model and the training semantic encoding vectors output by the other language model plus autoencoder structure. Training is complete when the difference between the predicted values ​​of the training semantic vectors output by the MLP for the trained language model and the predicted values ​​of the training semantic encoding vectors output by the other language model plus autoencoder structure is less than a preset error threshold.

[0137] In this embodiment, after the text to be analyzed is selected, to prevent subsequent users' texts from being analyzed... Figure 3 The structure calculates memory overflow and can further filter the text to be analyzed to obtain the final text to be analyzed for user category identification.

[0138] Specifically, if the number of texts in the historical text subset with outliers less than a preset outlier threshold is greater than a preset quantity threshold, then historical texts with acquisition times within a preset time period are selected from the historical text subset as the first text; other historical texts besides the first text are randomly selected from the historical text subset as the second text; wherein the sum of the number of the first text and the second text is equal to or less than a preset quantity threshold; the first text and the second text are used as the texts to be analyzed.

[0139] In this embodiment, the anomaly threshold and the number threshold can be obtained through empirical parameters. The anomaly threshold serves as a condition for filtering the text to be analyzed, that is, it is used to distinguish between low-quality text and high-quality text in the historical text set. When the anomaly value of the historical text is greater than or equal to the anomaly threshold, it indicates that the semantic features between the historical semantic vector and the historical semantic encoding vector of the historical text are significantly different. That is, the semantic features obtained by the historical text under different feature extraction processes are different, and the semantics of the historical text are unclear, which is low-quality text. On the other hand, if the anomaly value is less than the anomaly threshold, it indicates that the semantic features between the historical semantic vector and the historical semantic encoding vector of the historical text are relatively small. That is, the semantic features obtained by the historical text after semantic feature extraction are roughly the same as the semantic features obtained by the historical text after semantic feature extraction and then after autoencoding. The historical text is high-quality text.

[0140] This threshold is generally based on user category identification networks (such as...). Figure 3 The number of texts that can be processed at one time, and the user category recognition network will not experience memory overflow or overfitting when processing this number of texts.

[0141] In this embodiment, by performing auto-encoding on the historical semantic vector and then fitting it, a historical language encoding vector is obtained. Subsequently, outliers between the historical semantic vector and the corresponding historical semantic encoding vector can be used to filter out historical texts with high outliers, i.e. invalid and low-quality texts, ensuring the high quality of the text to be analyzed later and improving the efficiency of user category recognition.

[0142] Figure 7 This is a flowchart illustrating a user category identification method according to another exemplary embodiment. In one exemplary embodiment, the method is executed in... Figure 4 Before step S410, of course, the same process is also performed. Figure 2 Before step S210, the method may include steps S710 to S730, as detailed below:

[0143] Step S710: Concatenate the multiple text messages associated with the user with their corresponding text names to obtain multiple concatenated text messages.

[0144] In this embodiment, multiple text messages associated with the user are first extracted. These text messages can be written by the user or written by others based on the user's characteristics. No specific restrictions are imposed here.

[0145] The text title corresponds to the subject of the text information, such as the title of an article; or the subject described by the text information; or when the text information is a complaint, the text title is the corresponding product or service being complained about, etc. There are no specific restrictions here.

[0146] Then, the text information can be concatenated with the corresponding text names. For example, by using the symbol "&", you can get concatenated text such as "Product A & for some unknown reason, 648 yuan was automatically deducted".

[0147] Step S730: Replace the text information related to the specified type of characters in each concatenated text with preset characters to obtain each historical text in the historical text set.

[0148] For certain characters in concatenated text, semantic analysis is unnecessary during text processing. For example, in the concatenated text obtained above, numeric characters have no analytical meaning, so preset characters can be used to replace them, that is, all numeric characters in the concatenated text can be replaced with specified preset characters.

[0149] Of course, the above-specified character type of numeric characters is just an example. Other character types are also possible. The specific type can be set according to the type of historical text and whether certain characters in the historical text need to be semantically analyzed. No specific restrictions are imposed here.

[0150] In this embodiment, by performing unified preprocessing on the text information associated with the user, historical text with the same structure can be obtained, thereby improving the efficiency of subsequent historical text data processing.

[0151] Figure 8 yes Figure 2 The flowchart of step S210 in the illustrated embodiment is shown in an exemplary embodiment. Figure 8 As shown, in an exemplary embodiment, step S210, which involves selecting multiple texts to be analyzed from the user's historical text set based on outliers between the semantic vector and the semantic encoding vector of the historical text, may include steps S810 to S870, as detailed below:

[0152] Step S810: Input the historical texts in the historical text set into the language model to obtain the first semantic vector output by the first network layer of the language model and the second semantic vector output by the second network layer of the language model.

[0153] refer to Figure 9 As an example, a third preprocessing network structure is used to process the historical text collections of each user to obtain multiple texts to be analyzed. The third preprocessing network structure includes a language model, an autoencoder structure, and a discriminator.

[0154] In this embodiment, the first network layer segment is multiple network layers in the language model of the third preprocessing network structure, and the second network layer segment is multiple network layers in the language model of the third preprocessing network structure. The output signal of the first network layer segment serves as the input signal of the second network layer segment. For example, in one embodiment, the first network layer segment can be layers 1-3 of the language model, and the second network layer segment can be layers 4-6 of the language model. Of course, the above is only an example. In other embodiments, the first network layer segment and the second network layer segment can also be other layers of the language model.

[0155] In this embodiment, the user's historical text is input into the language model of the third preprocessing network, thereby obtaining the first semantic vector output by the first network layer and the second semantic vector output by the second network layer of the language model.

[0156] Step S830: Perform autoencoding on the second semantic vector to obtain the second semantic encoded vector.

[0157] The output of the second network layer of the language model of the third preprocessing network is connected to the input of the autoencoder structure. Therefore, the second semantic vector will be input to the autoencoder for autoencoding processing, which yields the second semantic encoded vector corresponding to the second semantic vector.

[0158] Step S850: Calculate the outlier between the first semantic vector and the second semantic encoding vector corresponding to each historical text.

[0159] In this embodiment, the calculation of outliers between the first semantic vector and the second semantic encoding vector can be referred to... Figure 4 The calculation process shown in the embodiment uses the vector difference between two vectors, such as the mean square error, as an outlier.

[0160] Step S870: Select historical texts with outliers less than a preset outlier threshold as texts to be analyzed.

[0161] Referring to step S470, in this embodiment, the abnormal values ​​of the user's historical text are sorted by numerical value, and historical text with values ​​greater than or equal to a preset abnormal threshold is filtered out, leaving historical text with abnormal values ​​less than the abnormal threshold as the user's text to be analyzed.

[0162] In this embodiment, semantic vectors of different segments in a language model are used, and the semantic vectors of the next segment are autoencoded. This autoencoding process involves compressing and restoring the semantic vectors of the next segment and fitting them to the output of the previous segment. This allows for the comparison of outliers between two vectors within a language model, filtering out invalid and low-quality historical texts, and obtaining semantically rich text to be analyzed.

[0163] Figure 10 This is a flowchart illustrating a user category identification method according to another exemplary embodiment. In one exemplary embodiment, the method is executed in... Figure 8 Before step S810, the method may include steps S1010 to S1070, as detailed below:

[0164] Step S1010: Input the training text into the language model to be trained.

[0165] This embodiment shows the... Figure 9 The training process of the third preprocessing structure is shown.

[0166] In one embodiment, the training text for text-based target labeling is input to... Figure 9 The training structure shown is used by the language model to be trained. The language model receives the training text and obtains semantic vectors in each network layer of the language model to be trained.

[0167] Step S1030: Based on the preset discriminator, predict the semantic vector output of the first network layer of the language model to be trained, and train the first network layer of the language model to be trained according to the obtained prediction value to obtain the trained language model.

[0168] First, only the first training semantic vector output by the first network layer segment of the training text is obtained, and this semantic vector is then processed... Figure 9 The preset discriminator makes predictions, thereby obtaining the predicted value corresponding to the first training semantic vector. Then, the first network layer of the language model to be trained and the preset discriminator are trained using the predicted value corresponding to the first training semantic vector, the text class target tag of the corresponding training text, and the predicted value.

[0169] The loss function for training the first network layer and the preset discriminator can be set as follows:

[0170] y n = sigmoid(MLP(BERT1(x) i )))

[0171] CE(y n ,y n ′)=-y n logy′n -(1-y i )log((1-y n ′))

[0172]

[0173] Where, x n For the nth training text, y n ′ is the discriminator for x n The predicted value obtained from the first training semantic vector, CE(y) n ,y n ′) is y n y n The cross-entropy between y' and y', where CE is the cross-entropy function, and y' n For x n The text class target tag, loss1 is the loss function for training the first network layer and the preset discriminator, batch_size is the training sample size, and BERT1 is the first network layer of the language model.

[0174] Step S1050: Input the training text into the trained language model.

[0175] In this embodiment, after the training of the first network layer and the preset discriminator converges, the first network layer and the preset discriminator are frozen to obtain the trained language model and the trained discriminator.

[0176] Then, the training text is input into the trained language model. At this time, the first network layer will output the first training semantic vector for the training text, and the first network layer will also output the second training semantic vector for the training text.

[0177] The first training semantic vector is also fed into the second network layer for training, thereby obtaining the second training semantic vector.

[0178] Step S1070: Based on the semantic vectors output by the first network layer of the trained language model and the semantic vectors output by the second network layer of the trained language model, train the second network layer of the trained language model to obtain a language model for receiving historical text input from the historical text set.

[0179] In this embodiment, the semantic vector output by the second network layer of the trained language model is autoencoded to obtain a training encoding vector. Based on the training encoding vector and the semantic vector output by the corresponding first network layer of the trained language model, the second network layer of the trained language model is trained until the error between the first predicted value obtained by the discriminator for the semantic vector output by the first network layer and the second predicted value obtained for the training encoding vector is within a preset error threshold.

[0180] Specifically, the first training semantic vector output by the first network layer of the trained language model and the second training semantic vector output by the second network layer of the trained language model are further processed by the autoencoder structure to be trained to fit the output of the first network layer of the trained language model and obtain the second training semantic encoding vector.

[0181] Subsequently, the second network layer and autoencoder structure of the trained language model can be trained based on the mean square error between the first training semantic vector and the second training semantic encoding vector of the same training text.

[0182] Since the mean square error of the two vectors is a specific value, a condition is also needed to stop training the second network layer and the autoencoder structure.

[0183] In this embodiment, since the second training semantic encoding vector is the result of fitting the second training semantic vector to be close to the first training semantic vector, the encoder can also make a prediction for the second training semantic encoding vector to obtain a prediction value. The error between the prediction value obtained by the encoder for the first training semantic vector and the prediction value obtained for the second training semantic encoding vector can be used to determine the condition for stopping training.

[0184] The error threshold in this embodiment is... Figure 4 Similarly, the error threshold is used as a stopping condition for the training of the second network layer and the autoencoder structure. When the error between the first and second predicted values ​​is greater than or equal to the error threshold, it proves that the autoencoder structure has not fitted the second training semantic vector to be close to the first training semantic vector, that is, the second network layer and the autoencoder structure still need to be trained. When the error between the first and second predicted values ​​is less than the error threshold, it can be regarded as that the output of the second network layer in this state can be fitted to the output of the first network layer through the autoencoder structure in this state, that is, it proves that the training is complete.

[0185] The predicted value for the second training semantic encoding vector can be calculated in the following way:

[0186] y n" = sigmoid(MLP(AE(BERT2(x n ))))

[0187] Among them, y n "For the discriminator targeting x" n The predicted value obtained from the second training semantic encoding vector, where AE is the autoencoder and BERT2 is the second network layer of the language model.

[0188] Then, you can use y n "and y n The difference between the two values ​​is used as an indicator for stopping training of the second network layer and the autoencoder structure.

[0189] In this embodiment, since the output of the second network layer enters the autoencoder structure, the input and output dimensions of the autoencoder structure are the same as the output dimension of the CLS layer of the language model. For example, if the language model is BERT, the output dimension is the same, which is 768.

[0190] In one specific embodiment, the process of fitting the output of the second network layer using the autoencoder structure is as follows: the number of hidden layers in the autoencoder structure are 768, 256, 64, 256, and 768, respectively. The input data is compressed from 768 dimensions to 64, and finally restored to 768 dimensions, retaining the main information. That is, the output result of the first network layer is fitted by compression and restoration.

[0191] This embodiment proposes a training method for the third preprocessing network, which first trains the first network layer and encoder, and then trains the second network layer and autoencoder structure, thereby achieving a solution for filtering and processing historical texts in a language model.

[0192] In one specific embodiment, this user category identification method can be used for merchant category identification. The historical text, the text to be analyzed, or the training text is constructed from the complaint text information associated with the merchant, and the text name is the product name corresponding to the complaint text information. In this way, the merchant's historical text can be obtained, thereby enabling the identification of the merchant's category. Figures 2 to 10 The method shown completes the category identification of merchants.

[0193] In this embodiment, when... Figure 3 , Figure 5 , Figure 6 as well as Figure 9 When training the model, the text-based target tag can be the basic information of the complained party corresponding to each complaint, such as the handling plan, or the penalty information of the merchant. Then, the 01 tag value of each complaint corresponds to whether the merchant has not been penalized or has been penalized. The tag can be a binary tag or a multi-class tag.

[0194] Figure 11 This is a schematic diagram illustrating the structure of a user category identification device according to an exemplary embodiment. Figure 11 As shown, in an exemplary embodiment, the user category identification device includes:

[0195] The preprocessing module 1110 is configured to filter out multiple texts to be analyzed from the user's historical text set based on outliers between the semantic vector and the semantic encoding vector of the historical text; wherein the semantic encoding vector is obtained by performing auto-encoding processing on the semantic vector;

[0196] The semantic vector acquisition module 1130 is configured to extract semantic information from multiple texts to be analyzed to obtain the corresponding semantic vectors to be analyzed.

[0197] The weight allocation module 1150 is configured to allocate semantic weight values ​​to multiple semantic vectors to be analyzed.

[0198] Classification module 1170 is configured to predict the user's category based on multiple semantic vectors to be analyzed and their corresponding semantic weight values;

[0199] The user category identification device proposed in this embodiment can predict user categories.

[0200] In one embodiment, the preprocessing module 1110 includes:

[0201] The historical semantic vector acquisition unit is configured to extract semantic information from each historical text in the historical text set to obtain the historical semantic vector corresponding to each historical text.

[0202] The historical semantic encoding vector acquisition unit is configured to perform self-encoding processing on each historical semantic vector to obtain the corresponding historical semantic encoding vector.

[0203] The first outlier acquisition unit is configured to calculate the outlier between each historical semantic vector and the corresponding historical semantic encoding vector.

[0204] The first text acquisition unit is configured to use historical texts with outliers less than a preset outlier threshold as texts to be analyzed.

[0205] In one embodiment, the preprocessing module 1110 further includes:

[0206] The concatenated text acquisition unit is configured to concatenate multiple pieces of text information associated with the user with their corresponding text names to obtain multiple concatenated texts; wherein, the text names are used to represent the topic of the corresponding text information;

[0207] The replacement unit is configured to replace text information related to a specified type of character in each concatenated text with a preset character, thereby obtaining each historical text in the historical text set.

[0208] In one embodiment, the text acquisition unit to be analyzed includes:

[0209] The first text acquisition module is configured such that if the number of texts in the historical text subset with an outlier value less than a preset outlier threshold is greater than a preset number threshold, then historical texts with an acquisition time within a preset time period are selected from the historical text subset as the first text.

[0210] The second text acquisition module is configured to randomly extract other historical texts besides the first text from a subset of historical texts as the second text; wherein the sum of the number of the first text and the second text is not greater than a preset number threshold.

[0211] The text to be analyzed is obtained from the module, which is configured to use the first and second texts as the texts to be analyzed.

[0212] In one embodiment, the preprocessing module 1110 includes:

[0213] The semantic vector acquisition unit is configured to input the historical texts in the historical text set into the language model to obtain the first semantic vector output by the first network layer segment of the language model and the second semantic vector output by the second network layer segment of the language model; wherein the output signal of the first network layer segment is used as the input signal of the second network layer segment.

[0214] The second semantic encoding vector acquisition unit is configured to perform self-encoding processing on the second semantic vector to obtain the second semantic encoding vector.

[0215] The second outlier acquisition unit is configured to calculate the outlier between the first semantic vector and the second semantic encoding vector corresponding to each historical text.

[0216] The second text acquisition unit is configured to use historical texts with outliers less than a preset outlier threshold as texts to be analyzed.

[0217] In one embodiment, the user category identification device further includes:

[0218] The first training text input module is configured to input training text into the language model to be trained.

[0219] The first training module is configured to predict the semantic vector output by the first network layer of the language model to be trained based on a preset discriminator, and train the first network layer of the language model to be trained based on the obtained prediction values ​​to obtain the trained language model.

[0220] The second training text input module is configured to input training text into the trained language model;

[0221] The second training module is configured to train the second network layer of the trained language model based on the semantic vectors output by the first network layer and the second network layer of the trained language model, thereby obtaining a language model for receiving historical text input from the historical text set.

[0222] In one embodiment, during the training of the first network layer of the language model to be trained, the discriminator is also trained; the second training module includes:

[0223] The training encoding vector acquisition unit is configured to perform autoencoding processing on the semantic vector output by the second network layer of the trained language model to obtain the training encoding vector;

[0224] The second training unit is configured to train the second network layer of the trained language model based on the training encoding vector and the semantic vector output by the first network layer of the trained language model, until the error between the first predicted value obtained by the discriminator for the semantic vector output by the first network layer and the second predicted value obtained for the training encoding vector is within a preset error threshold.

[0225] In one embodiment, the classification module includes:

[0226] The multidimensional vector acquisition unit is configured to perform weighted aggregation processing on multiple semantic vectors to be analyzed and their corresponding semantic weight values ​​to obtain the multidimensional vector corresponding to the user.

[0227] The classification unit is configured to predict the scalar obtained by mapping a multidimensional vector to determine the category to which the user belongs.

[0228] It should be noted that the user category identification device provided in the above embodiments and the user category identification method provided in the above embodiments belong to the same concept. The specific way in which each module and unit performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0229] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the user category identification method provided in the above embodiments.

[0230] Figure 12 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0231] It should be noted that, Figure 12 The computer system 1200 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0232] like Figure 12 As shown, the computer system 1200 includes a Central Processing Unit (CPU) 1201, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 1202 or programs loaded from storage portion 1208 into Random Access Memory (RAM) 1203. The RAM 1203 also stores various programs and data required for system operation. The CPU 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An Input / Output (I / O) interface 1205 is also connected to the bus 1204.

[0233] The following components are connected to I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to I / O interface 1205 as needed. Removable media 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1210 as needed so that computer programs read from them can be installed into storage section 1208 as needed.

[0234] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1209, and / or installed from removable medium 1211. When the computer program is executed by central processing unit (CPU) 1201, it performs various functions defined in the system of this application.

[0235] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0236] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0237] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0238] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the user category identification method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0239] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the user category identification method provided in the various embodiments described above.

[0240] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A user category identification method, characterized in that, include: Based on outliers between the semantic vector and the semantic encoding vector of historical texts, multiple texts to be analyzed are selected from the historical text set corresponding to the user; wherein, the semantic encoding vector is obtained by performing auto-encoding processing on the semantic vector; The semantic information of the multiple texts to be analyzed is extracted and processed to obtain the corresponding semantic vectors to be analyzed. Assign semantic weight values ​​to multiple semantic vectors to be analyzed; Predict the user's category based on the multiple semantic vectors to be analyzed and their corresponding semantic weight values; The step of selecting multiple texts to be analyzed from the user's historical text set based on outliers between the semantic vector and the semantic encoding vector of the historical text includes: The historical texts in the historical text set are respectively input into the language model to obtain the first semantic vector output by the first network layer segment of the language model and the second semantic vector output by the second network layer segment of the language model; wherein, the output signal of the first network layer segment is used as the input signal of the second network layer segment; The second semantic vector is subjected to auto-encoding processing to obtain the second semantic encoded vector; Calculate the outliers between the first semantic vector and the second semantic encoding vector corresponding to each historical text; Historical texts with outliers less than a preset outlier threshold are used as the text to be analyzed.

2. The method according to claim 1, characterized in that, Before performing semantic information extraction processing on the multiple texts to be analyzed to obtain the corresponding semantic vectors to be analyzed, the method further includes: The multiple text messages associated with the user are concatenated with their corresponding text names to obtain multiple concatenated texts; wherein, the text names are used to represent the theme of the corresponding text messages; Replace the text information related to the specified type of characters in each concatenated text with preset characters to obtain each historical text in the historical text set.

3. The method according to claim 1, characterized in that, The step of using historical text with outliers less than a preset outlier threshold as the text to be analyzed includes: If the number of texts in the historical text subset with an outlier value less than the preset outlier threshold is greater than the preset number threshold, then historical texts whose acquisition time is within the preset time period are selected from the historical text subset as the first text. Randomly select other historical texts besides the first text from the historical text subset as the second text; wherein the sum of the number of the first text and the second text is not greater than the preset number threshold. The first text and the second text are used as the text to be analyzed.

4. The method according to claim 1, characterized in that, The method further includes: Input the training text into the language model to be trained; Based on a preset discriminator, the semantic vector output by the first network layer of the language model to be trained is predicted, and the first network layer of the language model to be trained is trained based on the obtained prediction value to obtain the trained language model. The training text is input into the trained language model; Based on the semantic vectors output by the first network layer of the trained language model and the semantic vectors output by the second network layer of the trained language model, the second network layer of the trained language model is trained to obtain a language model for receiving historical text input from the historical text set.

5. The method according to claim 4, characterized in that, During the training process of the first network layer of the language model to be trained, the discriminator is also trained; the second network layer of the language model is trained based on the semantic vector output by the first network layer of the trained language model and the semantic vector output by the second network layer of the trained language model to obtain a language model for receiving historical text input from the historical text set, including: The semantic vector output by the second network layer of the trained language model is subjected to autoencoding to obtain the training encoded vector; Based on the training encoding vector and the semantic vector output by the first network layer of the trained language model, the second network layer of the trained language model is trained until the error between the first predicted value obtained by the discriminator for the semantic vector output by the first network layer and the second predicted value obtained for the training encoding vector is within a preset error threshold.

6. The method according to claim 1, characterized in that, The step of predicting the user's category based on the plurality of semantic vectors to be analyzed and the corresponding semantic weight values ​​includes: The multiple semantic vectors to be analyzed and their corresponding semantic weight values ​​are weighted and aggregated to obtain the multidimensional vector corresponding to the user. The scalar obtained by mapping the multidimensional vector is predicted to determine the category to which the user belongs.

7. A user category identification device, characterized in that, include: The preprocessing module is configured to filter out multiple texts to be analyzed from the user's historical text set based on outliers between the semantic vector and the semantic encoding vector of the historical text; wherein the semantic encoding vector is obtained by performing autoencoding processing on the semantic vector; The semantic vector acquisition module is configured to extract semantic information from the multiple texts to be analyzed to obtain the corresponding semantic vectors to be analyzed. The weight allocation module is configured to assign semantic weight values ​​to multiple semantic vectors to be analyzed. The classification module is configured to predict the user's category based on the multiple semantic vectors to be analyzed and their corresponding semantic weight values. The preprocessing module is further configured to: input the historical texts in the historical text set into the language model to obtain the first semantic vector output by the first network layer of the language model and the second semantic vector output by the second network layer of the language model; wherein the output signal of the first network layer is used as the input signal of the second network layer; perform autoencoding processing on the second semantic vector to obtain the second semantic encoding vector; calculate the outlier between the first semantic vector and the second semantic encoding vector corresponding to each historical text; and select the historical texts with outlier values ​​less than a preset outlier threshold as the texts to be analyzed.

8. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more computer programs that, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the processor of a computer, cause the computer to perform the method described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images