User classification method based on artificial intelligence
Through the automatic classification method of user demand information based on CNN, the problem of user demand information classification in the prior art is solved, efficient short text classification and semantic relationship expression are realized, and training efficiency and classification effect are improved.
Patent Information
- Application Number
- CN202510057084.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-14
- Publication Date
- 2025-02-21
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art is difficult to effectively handle the sparse characteristics and short lengths of user demand information, especially in obtaining sufficient high-quality labeled data, which cannot meet the growing user demand information classification needs.
A method of automatic classification of user demand information based on convolutional neural network (CNN) is proposed, and the automatic classification of user information is realized through the combination of data acquisition, data preprocessing, word vector conversion and user classification CNN network. Specific steps include data collection, data preprocessing, word vector conversion and the construction of user classification CNN network, and use accuracy, recall and F1 indicators to evaluate the classification effect.
This method is more suitable for short text classification, improves training efficiency, can better express semantic relationships, realizes end-to-end automatic classification, and does not require manual extraction of text features, achieving better classification results.
Smart Images
Figure CN119493867A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of artificial intelligence classification, and in particular relates to a user classification method based on artificial intelligence. Background Art
[0002] With the development of the Internet and the explosive growth of new media such as audio and video, the amount of user data has begun to show an exponential growth; how to efficiently integrate, manage and make full use of user information has become the goal of various industries; among them, user classification plays a decisive role in connecting the upstream and downstream. The upstream task is to perform good preprocessing of user data, and the downstream task uses the classified user information for preference processing and personalized recommendations.
[0003] In recent years, the vigorous development of artificial intelligence technology has led to innovations in user classification tasks, optimizing user classification tasks in terms of timeliness, manpower and material resources, etc., which can be roughly divided into three categories: rule-based methods, machine learning-based methods, and deep learning-based methods.
[0004] Rule-based approach: Through expert knowledge or manual analysis of data, a series of rules are established for classification. They are highly interpretable and easy to understand and debug, such as decision trees and association rule learning. They are highly interpretable: the rules are easy to understand and explain, which can help users understand the classification process. Rule-based methods are easy to implement and can usually be implemented in simple programming languages. They have low development costs and are easy to maintain. They can be modified and updated based on new information and needs. They are difficult to handle complex data: rules are difficult to capture complex relationships and patterns in data, but rule-based methods have high maintenance costs. As the number of rules increases, the maintenance cost will also increase. They are also difficult to adapt to changes: rules are difficult to adapt to changes in data and new situations.
[0005] Machine learning based methods: Learning the relationship between features and categories from training data and building classification models can handle complex and dynamic data with high accuracy and robustness, such as Naive Bayes, Support Vector Machines, K Nearest Neighbors, etc. Machine learning-based methods can learn more complex features and have made breakthrough progress in many tasks. However, machine learning-based methods require a large amount of training data and computing resources and have poor interpretability.
[0006] Deep learning based methods: By using deep neural networks to automatically learn features and build classification models, more complex features can be learned, and breakthroughs have been made in many tasks, such as convolutional neural networks (CNN), recurrent neural networks (RNN), Transformer, etc.
[0007] User demand information is semi-structured or unstructured data with sparse features and short length. The current manual annotation method is time-consuming and requires rich historical data as support when training the model. It cannot meet the growing requirements for classifying a large amount of user demand information. In actual user information collection, it is often difficult to obtain enough high-quality annotated data, especially when the amount of user information is too large and messy. Therefore, we propose a novel automatic classification method for user demand information based on convolutional neural network (CNN). Summary of the invention
[0008] The present invention proposes a user classification method based on artificial intelligence, which solves the problems in the prior art.
[0009] The technical solution of the present invention is implemented as follows: a user classification method based on artificial intelligence, characterized in that the specific steps include: S1: Data collection: Data collection is implemented. Internal data is obtained through system export, database query and questionnaire survey. External data is obtained through web crawlers, API interface calls, on-site collection, cooperative authorization or purchased data channels. S2: Data preprocessing: preliminary cleaning of the original text to remove irrelevant information in order to achieve text analysis and processing; S3: Word vector conversion: Through training, each word after word segmentation is converted into a dense vector that can be understood by a computer to represent the similarity between words; S4: User classification CNN network: mainly composed of input layer, convolution layer, pooling layer, attention layer, fully connected layer and a classifier; S5: Classification results: The accuracy, recall and F1 indicators are used to evaluate the effectiveness of the multi-class classification model.
[0010] As a preferred embodiment, the data collection includes the following steps: S11 preliminary processing: data cleaning, data conversion and data integration; S12 Storage and Security Management: Select appropriate relational or non-relational databases, implement data encryption, access control, and backup and recovery strategies; S13 Update and Maintenance: Establish a regular update mechanism, monitor data quality and provide real-time feedback and adjustments.
[0011] As a preferred implementation, the data preprocessing includes the following steps: S21 text cleaning: remove tags, special characters, and spaces to make the text neat; S22 Text normalization: After completing text cleaning, the text needs to be formatted in a unified manner, including uppercase and lowercase conversion, text segmentation, and part-of-speech tagging; S23 text processing: Based on text normalization, the text is further processed by denoising, normalization and standardization; S24 Text dictionary construction: Build a dictionary containing vocabulary in related fields. During the dictionary construction process, words need to be screened, sorted and weighted.
[0012] As a preferred implementation, the word vector conversion includes two training modes, specifically, a continuous bag-of-words mode and a skip-graph mode.
[0013] As a preferred implementation, the user classification CNN network comprises the following steps: S41 Input layer: For the input layer, set is the word vector of the i-th word in the sentence, then the expression of a sentence of length n is: ; In formula (1), ⊕ represents a connection operation, and the word The connection operation is usually used express; S42 convolution layer: The convolution layer is an h×k dimensional convolution kernel , which slides on the word vector matrix of a sentence and calculates a higher-level feature vector; where the high-level feature vector ci is composed of the convolution layer composed of the convolution kernel w of the sentence and the word vector generated; ; In formula (2), b is the bias, f is the nonlinear activation function, and Relu is used as the activation function. The introduction of nonlinear activation function can accurately describe the abstract features. The filter is applied to the sentence. Generate a feature map for each possible word; ; S43 pooling layer: The pooling layer mainly includes maximum pooling and average pooling; the maximum pooling is selected, and the feature map expression after pooling is: ; The dimensionality is reduced while retaining significant feature vectors. At the same time, the pooling layer can also process feature maps of different lengths and retain their most critical features. S44 Attention Layer: A multi-head self-attention mechanism is used. Different heads can focus on features of different scales. Some heads focus on local features, while others focus on global features. Each head can observe the input sequence from different angles and then fuse the information from these perspectives to reduce the possibility of information loss. The multi-head attention mechanism can be expressed by the following formula: The calculation can be expressed as: ; ; In formulas (5) and (6), Q, K, and V are the linear transformation results of query, key, and value, respectively, which are obtained by inputting the sequence And the corresponding weight matrix , , , multiplied together, is the dimension of the key vector, which is used to scale the dot product to prevent the value from being too large during the gradient descent process. Combining all the heads, we get: ; in, is an output weight matrix used to convert the output of all heads into a single vector that can be used for subsequent calculations or as the output of the model; S45 fully connected layer: to achieve integration and weight update for classification, the input layer of the fully connected layer is: ; S46 classifier: Use softmax classifier to convert the classification results into probability values between [0,1]. The category with a higher probability is the category to which the text will be assigned.
[0014] As a preferred implementation, the three indicators in the classification results are 97.3%, 96.7% and 97.6% respectively.
[0015] After adopting the above technical solution, the beneficial effects of the present invention are: 1. More suitable for short text classification: By selecting the size of the convolution kernel, the local context semantics can be analyzed, which is more suitable for the characteristics of short texts.
[0016] 2. Improve training efficiency: CNN has the characteristic of weight sharing, which can improve training efficiency.
[0017] 3. Better expression of semantic relationships: The word vectors trained by Word2vec can better express the semantic relationship between words.
[0018] 4. End-to-end automatic classification: No need to manually extract text features, good classification results can be obtained by training weights. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0020] Figure 1 It is a flow chart of the overall technical route of the present invention; Figure 2 This is a structural block diagram of data preprocessing of the present invention; Figure 3 It is a structural block diagram of the continuous word bag mode and the skip-graph mode of the present invention; Figure 4 It is a text classification structure diagram of the CNN network of the present invention; DETAILED DESCRIPTION
[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example
[0022] like Figure 1-4 As shown, a user classification method based on artificial intelligence includes the following specific steps: S1: Data collection: Data collection is implemented. Internal data is obtained through system export, database query and questionnaire survey. External data is obtained through web crawlers, API interface calls, on-site collection, cooperative authorization or purchased data channels. S2: Data preprocessing: preliminary cleaning of the original text to remove irrelevant information in order to achieve text analysis and processing; S3: Word vector conversion: Through training, each word after word segmentation is converted into a dense vector that can be understood by a computer to represent the similarity between words; S4: User classification CNN network: mainly composed of input layer, convolution layer, pooling layer, attention layer, fully connected layer and a classifier; S5: Classification results: The accuracy, recall and F1 indicators are used to evaluate the effectiveness of the multi-class classification model.
[0023] Furthermore, data collection includes the following steps: S11 Preliminary processing: data cleaning (removing duplications, correcting errors, filling gaps), data conversion (format unification, standardization) and data integration (multi-source data fusion); S12 Storage and Security Management: Select appropriate relational or non-relational databases, implement data encryption, access control, and backup and recovery strategies; S13 Update and Maintenance: Establish a regular update mechanism, monitor data quality and provide real-time feedback and adjustments.
[0024] Furthermore, data preprocessing includes the following steps: S21 text cleaning: remove tags, special characters, and spaces to make the text neat; S22 Text Normalization: After completing text cleaning, the text needs to be formatted in a unified manner, including uppercase and lowercase conversion, text segmentation, and part-of-speech tagging, which helps understand the role and meaning of words in sentences; S23 Text processing: Based on text normalization, the text is further processed for denoising, normalization and standardization to facilitate format unification, eliminate ambiguity and other processing to ensure the reliability of text analysis results; S24 Text dictionary construction: Build a dictionary containing vocabulary in related fields. During the dictionary construction process, words need to be screened, sorted and weighted.
[0025] Furthermore, word vector conversion includes two training modes, namely the continuous bag of words mode and the skip-graph mode. The continuous bag of words mode is a popular word embedding learning method. It predicts a word by considering the context of the word. Specifically, the model takes the context of a word as input, and then sums or averages the vectors of these words, passes through a hidden layer, and finally predicts the central word. The advantage of this method is that it can capture the semantic information of the word, and due to its simple structure, the training speed is relatively fast, which is suitable for processing large-scale corpora; the skip-graph mode is another efficient word embedding learning method. Its core idea is to predict the surrounding context words through a central word. In contrast to the continuous bag of words mode, the skip-graph mode takes the central word as input and tries to predict multiple context words at the same time. This method can effectively learn the distributed representation of vocabulary and maintain good performance even in a larger context window. The skip-graph mode is particularly effective for processing low-frequency words and capturing complex vocabulary relationships, but it usually requires more computing resources because it needs to predict each context word. The skip-graph mode is widely used in the field of natural language processing, especially in scenarios that require high-precision word vectors.
[0026] Furthermore, the user classification CNN network includes the following steps: S41 Input layer: For the input layer, set is the word vector of the i-th word in the sentence, then the expression of a sentence of length n is: ; In formula (1), ⊕ represents a connection operation, and the word The connection operation is usually used express; S42 convolution layer: The convolution layer is an h×k dimensional convolution kernel , which slides on the word vector matrix of a sentence and calculates a higher-level feature vector; where the high-level feature vector ci is composed of the convolution layer composed of the convolution kernel w of the sentence and the word vector generated; ; In formula (2), b is the bias, f is the nonlinear activation function, and Relu is used as the activation function. The introduction of nonlinear activation function can accurately describe the abstract features. The filter is applied to the sentence. Generate a feature map for each possible word; ; During the convolution process, attention should be paid to the processing of edges. If the boundary of the text vector is exceeded during the sliding process, zeros are added to ensure that the edge information is not lost. Although the feature vector obtained after convolution is greatly reduced compared with the original input vector, its dimension is high and the features are not obvious. Therefore, it is necessary to use a pooling layer to optimize the feature map for dimensionality reduction.
[0027] S43 pooling layer: The pooling layer mainly includes maximum pooling and average pooling; the maximum pooling is selected, and the feature map expression after pooling is: ; While reducing the dimension, significant feature vectors are retained. At the same time, the pooling layer can also process feature maps of different lengths and retain their most critical features. The convolution process requires the use of convolution kernels of three different dimensions, namely 3, 4, and 5, to extract different abstract features. S44 Attention Layer: A multi-head self-attention mechanism is used. Different heads can focus on features of different scales. Some heads focus on local features, while others focus on global features. Each head can observe the input sequence from different angles and then fuse the information from these perspectives to reduce the possibility of information loss. The multi-head attention mechanism can be expressed by the following formula: The calculation can be expressed as: ; ; In formulas (5) and (6), Q, K, and V are the linear transformation results of query, key, and value, respectively. They are obtained by inputting And the corresponding weight matrix , , , multiplied together, is the dimension of the key vector, which is used to scale the dot product to prevent the value from being too large during the gradient descent process. Combining all the heads, we get: ; in, is an output weight matrix used to convert the output of all heads into a single vector that can be used for subsequent calculations or as the output of the model; S45 fully connected layer: to achieve integration and weight update for classification, the input layer of the fully connected layer is: ; S46 classifier: Use softmax classifier to convert the classification results into probability values between [0,1]. The category with a higher probability is the category to which the text will be assigned.
[0028] Furthermore, the three indicators in the classification results are 97.3%, 96.7% and 97.6% respectively. The experimental results show that the framework proposed in the present invention has good effect and practical significance for the automatic classification of users.
[0029] It is worth mentioning that among the traditional statistical classification methods, logistic regression and K-nearest neighbor (KNN) are sufficiently effective, especially when the amount of data is not large and feature engineering is easy to implement; rule engine: for some scenarios with clear business logic and clear rules, a rule-based classification method can be adopted instead of relying on machine learning; knowledge graph: knowledge graph can be used to assist classification, and graph algorithms can be used to analyze the relationship between entities, which can be used as an alternative solution.
[0030] An AI-based user classification method is widely used in multiple fields, including targeted advertising and customer relationship management in marketing, personalized recommendations and customer churn prediction in e-commerce, credit scoring and fraud detection in financial services, sentiment analysis and network influence assessment in social media, disease prediction and personalized treatment in healthcare, personalized teaching and resource allocation in education, customer service optimization in transportation and logistics, energy consumption analysis and demand response in energy management, public service optimization and welfare distribution in government services, as well as game design and film and television recommendations in the field of entertainment and gaming. These applications all reflect the great potential of AI in understanding and serving users.
[0031] In the description of the present invention, it should be understood that the terms "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a mechanical connection or an electrical connection, or it can be the internal communication of two elements, it can be a direct connection, or it can be an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.
[0032] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A user classification method based on artificial intelligence, characterized in that: The specific steps include: S1: Data collection: Data collection is implemented. Internal data is obtained through system export, database query and questionnaire survey. External data is obtained through web crawlers, API interface calls, on-site collection, cooperative authorization or purchased data channels. S2: Data preprocessing: preliminary cleaning of the original text to remove irrelevant information in order to achieve text analysis and processing; S3: Word vector conversion: Through training, each word after word segmentation is converted into a dense vector that can be understood by a computer to represent the similarity between words; S4: User classification CNN network: mainly composed of input layer, convolution layer, pooling layer, attention layer, fully connected layer and a classifier; S5: Classification results: The accuracy, recall and F1 indicators are used to evaluate the effectiveness of the multi-class classification model.
2. The user classification method based on artificial intelligence according to claim 1, characterized in that: The data collection comprises the following steps: S11 preliminary processing: data cleaning, data conversion and data integration; S12 Storage and Security Management: Select appropriate relational or non-relational databases, implement data encryption, access control, and backup and recovery strategies; S13 Update and Maintenance: Establish a regular update mechanism, monitor data quality and provide real-time feedback and adjustments.
3. The user classification method based on artificial intelligence according to claim 1 is characterized in that: The data preprocessing comprises the following steps: S21 text cleaning: remove tags, special characters, and spaces to make the text neat; S22 Text normalization: After completing text cleaning, the text needs to be formatted in a unified manner, including uppercase and lowercase conversion, text segmentation, and part-of-speech tagging; S23 text processing: Based on text normalization, the text is further processed by denoising, normalization and standardization; S24 Text dictionary construction: Build a dictionary containing vocabulary in related fields. During the dictionary construction process, words need to be screened, sorted and weighted.
4. The user classification method based on artificial intelligence according to claim 1 is characterized in that: The word vector conversion includes two training modes, namely, continuous bag-of-words mode and skip-graph mode.
5. The user classification method based on artificial intelligence according to claim 1 is characterized in that: The user classification CNN network includes the following steps: S41 Input layer: For the input layer, set is the word vector of the i-th word in the sentence, then the expression of a sentence of length n is: ； In formula (1), ⊕ represents a connection operation, and the word The connection operation is usually used express; S42 convolution layer: The convolution layer is an h×k dimensional convolution kernel , which slides on the word vector matrix of a sentence and calculates a higher-level feature vector; where the high-level feature vector ci is composed of the convolution layer composed of the convolution kernel w of the sentence and the word vector generated; ； In formula (2), b is the bias, f is the nonlinear activation function, and Relu is used as the activation function. The introduction of nonlinear activation function can accurately describe the abstract features. The filter is applied to the sentence. Generate a feature map for each possible word; ； S43 pooling layer: The pooling layer mainly includes maximum pooling and average pooling; the maximum pooling is selected, and the feature map expression after pooling is: ； The dimensionality is reduced while retaining significant feature vectors. At the same time, the pooling layer can also process feature maps of different lengths and retain their most critical features. S44 Attention Layer: A multi-head self-attention mechanism is used. Different heads can focus on features of different scales. Some heads focus on local features, while others focus on global features. Each head can observe the input sequence from different angles and then fuse the information from these perspectives to reduce the possibility of information loss. The multi-head attention mechanism can be expressed by the following formula: The calculation can be expressed as: ； ； In formulas (5) and (6), Q, K, and V are the linear transformation results of query, key, and value, respectively, which are obtained by inputting the sequence And the corresponding weight matrix , , , multiplied together, is the dimension of the key vector, which is used to scale the dot product to prevent the value from being too large during the gradient descent process. Combining all the heads, we get: ； in, is an output weight matrix used to convert the output of all heads into a single vector that can be used for subsequent calculations or as the output of the model; S45 fully connected layer: to achieve integration and weight update for classification, the input layer of the fully connected layer is: ； S46 classifier: Use the softmax classifier to convert the classification results into probability values between [0,1]. The category with a higher probability is the category to which the text will be assigned.
6. The user classification method based on artificial intelligence according to claim 1 is characterized in that: The three indicators in the classification results are 97.3%, 96.7% and 97.6% respectively.