A police case text classification method based on CBERT-MDPCNN model

Abstract

The application discloses a police case text classification method based on a CBERT-MDPCNN model, adopts a GPT4 large language model for police case text data enhancement, obtains a text feature matrix through a word embedding layer, and converts the text feature matrix into a text feature vector capable of being processed by MDPCNN through a BERT pre-training model, and finally inputs the MDPCNN multi-head classifier model to realize a police case text classification task. The application can quickly classify police case texts, improve the accuracy of the model in classifying police case texts, greatly reduce the requirement of model training on the sample amount of police case text data, has certain model generalization capability, and is suitable for police case texts such as network-related police cases and non-network-related police cases.

Description

Technical Field

[0001] This invention relates to the field of police report text classification, specifically to a police report text classification problem based on the CBERT-MDPCNN model. Background Technology

[0002] Over the past decade, with the rapid development of computer and artificial intelligence technologies, the application of computer intelligence technology in modern police informatization and intelligentization has become increasingly important. In the public security system, efficient classification of police report text data is crucial for improving case retrieval and investigation efficiency, and also plays a key supporting role in in-depth analysis and discovery of crime patterns. However, faced with tens of thousands of new alarm messages every day, traditional manual classification methods are often inefficient, easily influenced by subjective factors, and contain a certain degree of uncertainty. Therefore, there is an urgent need for a method that can quickly and accurately classify police report texts intelligently to help dispatchers understand and handle reports more quickly and accurately, improve the response speed and efficiency of the public security system, and provide more targeted data support to relevant departments, thereby better maintaining social order and public safety. Therefore, in-depth research on the problem of police report text classification has significant practical significance and application prospects.

[0003] Current research on police report text classification is still in its early stages, covering findings on traffic police reports, criminal cases, and hierarchical classification. With the rapid development of the internet, new types of cybercrime are on the rise, causing significant financial losses to victims through new internet technologies and platforms. These new cybercrimes are characterized by diverse methods and complex charges, posing significant challenges to prevention and combating them. Subdividing these cases can aid in case management and investigation. For example, romance scams are often perpetrated by the same person playing multiple roles; therefore, categorizing romance scams can help police focus their investigations on similar cases, leading to more leads. Thus, a model suitable for police report text classification is needed.

[0004] In general text classification, methods have evolved from traditional machine learning to deep learning based on neural networks, and are now widely used in information mining fields such as spam classification, sentiment analysis, news topic classification, and identification of key elements in judicial cases. Among these, Wang Qian et al. proposed a news topic text classification method based on the multi-head attention pooling mechanism of RoBERTaRCNN, and verified the model's feasibility in news classification tasks through ablation experiments. Mao Xingliang et al. proposed a key case element identification method that integrates global and local information, and tested it on a publicly available judicial dataset, effectively improving the accuracy of element entity classification. Zhengxuan Wu et al. combined structured self-attention weighted encoding of semantics with sentiment analysis, and demonstrated through two tests that this method encodes rich semantics and matches human interpretation of semantics. Chen Zhiqun et al. addressed the problem that traditional models cannot identify word polysemy by proposing a Weibo comment sentiment analysis method based on BERT and bidirectional LSTM, and conducted comparative experiments using Sina Weibo comments, showing that the accuracy of this method is higher than other mainstream sentiment analysis models.

[0005] In terms of police report text classification, in recent years, the public security sector has utilized big data technology to build business platforms for various police departments. The convergence of diverse data information has enabled more efficient dispatching and command in areas such as public security, investigation, and traffic. The data collected by these business systems includes a massive amount of police report text information, which provides crucial data support for intelligence analysis and case investigation. However, facing such a vast amount of police report text data, enabling machines to "learn" intelligent recognition of police report texts is one of the effective methods to improve the speed of police report processing and reduce the manual processing procedures for call takers. Wang Mengxuan et al. built a text classification system for 110 call texts applied to police report descriptions and proposed an improved convolutional recurrent neural network model, effectively improving the accuracy of case classification. Li Yunxuan et al. proposed an automatic processing method for traffic police report data based on multi-task transfer learning, achieving automatic processing of key information, types, and semantics in traffic police reports. Qiu Kaikai et al. proposed a text classification method based on ERNIE-SA-DPCNN for case texts related to new types of cybercrime, and demonstrated through comparative experiments that it outperforms other models in classifying case texts related to new types of cybercrime. Ayinga et al. proposed a text classification method based on the GRU-Glove algorithm for cybercrime cases, and conducted experiments using a publicly available Chinese word segmentation dataset, showing that this method effectively improves the accuracy and efficiency of classifying complex texts with novel vocabulary. Wang Yue et al. addressed the problem of low accuracy in identifying key entity information in police reports by proposing a named entity recognition method based on BERT for police report texts. Using a publicly available dataset for training and testing, they effectively improved the recognition accuracy.

[0006] The aforementioned studies have focused too much on the effectiveness of the models themselves, failing to adequately adapt the characteristics of police report texts to the models. Police report texts are written by police officers based on the descriptions and answers provided by the callers, following certain norms. They are objective and concise in expression. Furthermore, the varying frequencies of these cases result in text data that is denser, less coherent, and more imbalanced than text data obtained from internet sources such as Weibo and news. This leads to gradient explosion and difficulties in extracting key features. Summary of the Invention

[0007] Based on the above needs, according to the case classification rules, new types of cybercrime cases are further subdivided into nine common categories, such as credit card fraud and impersonation of public officials, in order to provide more targeted guidance and support for case investigation and crackdown.

[0008] This invention provides a method for classifying police report text based on the CBERT-MDPCNN model, which includes the following steps:

[0009] Step 1: Obtain police report texts from historical time periods as training data for the model;

[0010] Step 2: Preprocess and screen the descriptive information of the acquired police report text;

[0011] Step 3: Classify the preprocessed police report texts by category and label each category with the corresponding category label; the classification includes a coarse three-category classification and a finer eleven-category classification.

[0012] Step 4: For the classified police report text data, use the GPT4 language model to perform data augmentation to increase the amount of police report text data for the few sample categories;

[0013] Step 5: Extract semantic information from the police report text using a BERT pre-trained model that includes a word embedding layer and a Transformer layer to obtain a text feature matrix. The word embedding layer is used to generate sentence embedding word vectors, and the Transformer layer is used to generate word vector codes. The intermediate word vector codes output by the Transformer layer are summed to the word vector codes output by the BERT pre-trained model, and the model is then pre-trained using MLM with the police report text data to generate a CBERT pre-trained model.

[0014] Step 6: Transform the text feature matrix output by the CBERT pre-trained model into a text feature vector with multiple convolutional kernels that MDPCNN can process; use the word embedding layer as the backbone network, and construct the MDPCNN multi-head classifier model through a branch network composed of an eleven-class pyramid pooling structure and a three-class pyramid pooling structure. The CBERT pre-trained model and the MDPCNN multi-head classifier model constitute the CBERT-MDPCNN police text classifier model.

[0015] Step 7: Divide the enhanced police report text data from Step 4 into a training dataset and a validation dataset, initialize the model parameters, and iteratively train the CBERT-MDPCNN police report text classifier model;

[0016] Step 8: Obtain the police report text data that needs to be classified. After preprocessing in Step 2, input it into the CBERT-MDPCNN police report text classifier model trained in Step 7, and output the detailed classification results.

[0017] The beneficial effects of this invention are mainly reflected in:

[0018] The method of this invention includes a data augmentation step based on the GPT4 model, which constructs corresponding system prompt words, enabling the large language model to automatically generate text data of the corresponding category, greatly increasing the amount of data for the few sample categories, and facilitating the model to fully learn the features of various types of samples.

[0019] This invention proposes an MLM method to fine-tune the Transformer layer of a BERT pre-trained model, thereby improving the performance of the pre-trained model on natural language processing tasks in the field of police report text. It also constructs a BERT aggregation operator to enhance the multi-layer expressive power of the pre-trained model.

[0020] This invention, based on the unique binary structure of the DPCNN classifier, constructs the MDPCNN multi-head classifier model using a text region embedding layer as the backbone model and a pyramid structure as the branch network. This ensures the balance of training data while making fuller use of the original data. A multi-task emphasis operator is introduced into the MDPCNN multi-task classifier model, enabling the model to better recognize various text types in the eleven-class classification task.

[0021] To alleviate the gradient explosion problem caused by the high density and incoherence of police report text information, this invention selects the smoother Mish function as the activation function, which helps the model better capture key text features and improves the model's classification performance.

[0022] In order to better adapt to the training requirements of the Transformer architecture during the training process of the BERT pre-trained model, this invention adopts the BERTAdam dynamic learning rate method. Based on AdamW, it introduces a warmup learning rate decay strategy, removes the weight decay of all bias terms of the model, and thus achieves faster speed and better results.

[0023] In summary, this invention can quickly classify police incident texts, improve the accuracy of the model in classifying police incident texts, greatly reduce the requirements for the amount of police incident text data samples for model training, and has a certain model generalization ability. It is applicable to police incident texts involving the Internet and those not involving the Internet. Attached Figure Description

[0024] Figure 1 This refers to the overall architecture of the CBERT-MDPCNN model;

[0025] Figure 2 It is a data augmentation implementation process based on a large language model;

[0026] Figure 3 This is a graph showing the changes in loss values ​​during MLM fine-tuning training;

[0027] Figure 4 This is a comparison chart of the accuracy distribution of the comparative experiments;

[0028] Figure 5 This is a comparison chart of the accuracy of the training set in the comparative experiment. Detailed Implementation

[0029] The present invention will now be described in detail with reference to the accompanying drawings.

[0030] This embodiment discloses a method for classifying police report text based on the CBERT-MDPCNN model, such as... Figure 1 As shown, the main steps include the following:

[0031] Step 1: Obtain the police report texts from the 110 emergency call platform of a local public security bureau over a period of time.

[0032] Step 2: Screen the description information of the obtained alarm texts, check whether the alarm category is consistent with the alarm description, delete duplicate alarms and invalid alarms, retain valid alarms, and then use regular expression matching and wildcard matching to mask unnecessary information.

[0033] Step 3: After screening and analyzing the police reports, they are divided into three categories: non-internet-related cases, internet-related cases, and other cases. Non-internet-related cases refer to crimes that do not involve the internet or network technology; internet-related cases refer to crimes that involve the internet or network technology; and other cases refer to internet-related cases excluding those already classified. Furthermore, for internet-related cases, drawing parallels with traditional property security, a more detailed classification method is proposed, further dividing internet-related police reports into nine categories: credit card fraud, ordinary identity fraud, impersonation of public officials fraud, romance fraud, investment and financial fraud, fraudulent transaction fraud, fraud involving illegal activities, theft through information technology, and extortion. These categories are then used as data tags.

[0034] Step 4: For the categorized police report text data, such as... Figure 2 As shown in the flowchart, this process illustrates the complete workflow of text augmentation using GPT-4. First, the system calculates the number of text categories to initialize the augmentation configuration and determines the amount of data to be added based on the set parameter n3. Next, the system selects a specific text category H to be augmented and samples several data entries from that category as examples, including the category name and description. This information is input into the "cue word generator" to generate specific cue words, which are then passed to GPT-4 to generate augmented text data. This generated data is expanded or modified based on the initial text to obtain the augmented text. Users can influence the number and length of the generated results by inputting two parameters: the number of text entries n1 and the minimum text length n2, to meet personalized needs. System prompts and user prompts handle the tasks of data initialization, selection, and parameter setting, respectively.

[0035] Step 5: Extract semantic information from the police report text using a BERT pre-trained model that includes a word embedding layer and a Transformer layer to obtain a text feature matrix. The word embedding layer is used to generate sentence embedding word vectors, and the Transformer layer is used to generate word vector codes. The intermediate word vector codes output by the Transformer layer are summed to the word vector codes output by the BERT pre-trained model, and the model is then pre-trained using MLM with the police report text data to generate a CBERT pre-trained model.

[0036] The CBERT pre-trained model is constructed as follows:

[0037] 1) Obtain the word vectors embedded in the sentence.

[0038] The input data for the word embedding layer is data augmented police report text data, which consists of police report text sentences and corresponding classification labels. In the word embedding layer, the word embedding vector of each police report text sentence is calculated by adding the word embedding vector, the sentence embedding vector and the position embedding vector.

[0039] 2) Use the Transformer layer to generate word vector encoding.

[0040] The Transformer layer utilizes a multi-head attention layer, employing multiple attention mechanisms to focus on different positions in the input sequence, associating the vectors at each position to generate a new set of vector representations. A feedforward neural network sublayer then performs a non-linear transformation on the output of the multi-head attention sublayer. Residual links and layer normalization are used to connect layers, accelerating model convergence. Simultaneously, the Model-Modulated Language (MLM) method is used for fine-tuning, randomly replacing words in the model with [MASK] tags and having the model predict the replaced words. This enhances the model's consideration of contextual word relationships when dealing with domain-specific text, improving its performance on natural language processing tasks in the field of police reports.

[0041] 3) Introducing the BERT aggregation operator to improve its multi-level expressive power; after adding the aggregation operator, BERT is denoted as CBERT. The aggregation operator can be represented as:

[0042]

[0043] Where O is the final output of the BERT pre-trained model after adding the operator, i.e., the text feature matrix, and T j is the word vector encoding output by the j-th Transformer layer in the BERT pre-trained model, where n is the total number of Transformer layers in the BERT pre-trained model.

[0044] Step Six: Using the text region embedding layer as the backbone network, the text feature matrix output by CBERT is transformed into a multi-kernel text feature vector that MDPCNN can process. After two layers of equal-length convolutions, the vector is input into a pyramid structure to form a branch network, thus constructing the MDPCNN multi-head classifier model. The CBERT pre-trained model and the MDPCNN multi-head classifier model constitute the CBERT-MDPCNN police report text classifier model.

[0045] In a specific embodiment of the present invention, step six is ​​as follows:

[0046] 1) A non-monotonic and smooth Mish function is used in the backbone network of MDPCNN to improve the expressive power of the model, as shown in the following formula:

[0047] f2(x)=xtanh(ln(1+e x (2)

[0048] Where x is the input value of the model, and f2(x) is the output value after passing through the activation function.

[0049] 2) A branch network consisting of two pyramid pooling structures performs max pooling on the text feature vector output by the backbone network, halving the length of the text feature vector. A loop is constructed until the text feature vector of each convolutional kernel is compressed to 1×1. During this process, residual connections are used to add the text feature vectors before and after the convolutional block, thus alleviating the gradient vanishing problem during model training. Subsequently, a multi-task balancing operator is introduced to integrate the data from the two branch networks to improve the model's classification performance. The specific formula for the multi-task balancing operator is as follows:

[0050]

[0051] Where R is the output tensor of the MDPCNN multi-head classifier model, A is the output tensor of the eleven-class pyramid structure, and B is the output tensor of the three-class pyramid structure. These are the variable parameters optimized using gradient descent.

[0052] 3) The output tensor of the MDPCNN multi-head classifier model is transformed through a fully connected layer to output the scores of the corresponding classes. Then, the Softmax function is used to convert these scores into probability functions to obtain the eleven-class classification results. The Softmax function is as follows:

[0053]

[0054] Where y i For each node's output, m is the number of output nodes, which is the number of categories.

[0055] Step 7: Divide the enhanced text data into training and validation datasets according to a certain ratio, initialize model parameters such as learning rate, and iteratively train the current police text classification model based on CBERT-MDPCNN.

[0056] To achieve faster speed and better results, and to better adapt to the training requirements of Transformer architectures such as BERT, the CBERT-MDPCNN police text classifier model uses BERTAdam during training. Based on AdamW, it introduces a warmup learning rate decay strategy and removes the weight decay of all bias terms in the model. The formula is as follows:

[0057]

[0058] Where v is the momentum vector and s is the adaptive learning rate parameter. β1 and β2 are the gradients of the weights, and their values ​​are close to 1; equations (7) and (8) are the bias corrections for the first-order moment estimation of the gradient; η0 is the initial value of the learning rate, and η t The learning rate after implementing the warmup strategy, where decay is the decay factor; The weights to be updated are ε, which is a very small value, and λ is the regularization coefficient.

[0059] Step 8: Obtain the police report text data that needs to be classified, preprocess it, and then input it into the trained CBERT-MDPCNN-based police report text classification model to output the classification results.

[0060] To evaluate the classification method of this invention, the classification results output by the model are compared and analyzed, and evaluation is carried out by comparative experiments and parametric experiments.

[0061] 1) To make the performance evaluation of the model comparable, the present invention set up a relevant experimental environment, deployed on the Ubuntu-20.04.1 system, using NVIDIA A6000 45G GPU, and writing code using Python 3.9.18. Detailed information about the experimental environment is shown in Table 1.

[0062] Table 1 Experimental Environment

[0063]

[0064] 2) This invention uses top-1 and top-3 accuracy (Acc), precision (P), recall (R), and F1 score to evaluate the model. The specific formula is as follows:

[0065]

[0066] Where TP represents the number of correctly identified entities, TN represents the number of incorrectly identified entities, FP represents the number of other tags identified as this tag, and FN represents the number of this tag identified as other tags.

[0067] For multi-class text classification tasks, it is also necessary to examine the performance of classifiers under different classes. The macro-weighted average is calculated using the evaluation function `classification-report` in the sklearn module, and denoted as the macro precision (P0). w Macro recall (R) w ) and macro F1 value (F w The calculation formulas are shown in equations (15), (16), and (17):

[0068]

[0069] Where ω i P i and R i Let i = 1, 2, ..., n be the weight, precision, and recall of each category, respectively.

[0070] Because model training involves a degree of randomness, to eliminate the influence of experimental result variance and improve experimental stability, this invention, in addition to selecting the optimal values ​​of relevant indicators from all similar parameter tuning experiments, also calculates the average accuracy of the top 10% of accuracy scores in similar parameter tuning experiments, specifically denoted as Acc. 10％ .

[0071] 3) To make the BERT pre-trained model more closely match the police report text dataset, this invention uses the MLM method to fine-tune the BERT pre-trained model. The BERT pre-trained model used in subsequent comparisons is the fine-tuned model. The relevant experimental parameter settings are shown in Table 2.

[0072] Table 2 MLM Fine-tuning Experimental Parameters

[0073]

[0074] Where max_seq_length is the maximum input length of each sentence, train_epoches is the number of training rounds, batch_size is the number of input sentences in each iteration, and mlm_probability sets the proportion of [MASK] words in each sentence. The loss function used in training is negative log-likelihood estimation, as shown in Equation (18).

[0075]

[0076] Among them, L MLM Here is the loss value, where N is the number of words masked, and w is the number of words masked. i C is the i-th masked word. i It is the context of the i-th masked word, P(w i |C i ) is the probability distribution predicted by the model.

[0077] To illustrate the effectiveness of MLM fine-tuning pre-training, this invention plots a graph showing the change in MLM fine-tuning training loss values, as shown below. Figure 3 As shown. By Figure 3It can be seen that after fine-tuning through MLM pre-training, the loss value of the BERT pre-trained model for the police incident text dataset decreased from 2.2 to around 1.2, indicating an improvement in the word-filling prediction ability of the BERT pre-trained model in the police incident text domain. Through MLM fine-tuning training, the BERT pre-trained model can learn knowledge of police incident texts on the basis of existing NLP knowledge, thereby better adapting to downstream police incident text classification tasks and improving classification accuracy.

[0078] 4) To verify the classification performance of the CBERT-MDPCNN model, it was compared with the following benchmark models under the same experimental conditions: BERT-FC: Uses BERT to extract global semantic information and adds a fully connected layer (FC) for classification. ERNIE-FC: Uses ERNIE to extract global semantic information and adds a fully connected layer (FC) for classification. BERT-RCNN: Uses BERT to extract global semantic information and uses a residual network (RCNN) to process local contextual information and classify. BERT-RNN: Uses BERT to extract global semantic information and uses an RNN to process local contextual information and classify. BERT-LSTM: Uses BERT to extract global semantic information and adds an LSTM layer to process local contextual information and classify.

[0079] 5) The parameter experiments in this invention mainly test the performance of different models under different hyperparameters by setting different learn_rate and batch_size. This invention selects the learning rate (learn_rate) and the number of input model samples (batch_size) as variable hyperparameters. Seven learn_rate values ​​are selected: 0.00003, 0.0009, 0.0007, 0.0005, 0.0003, 0.0001, and 0.003; five batch_size values ​​are selected: 16, 32, 64, 128, and 256. A total of 35 parameter experiments are conducted for each model in pairs.

[0080] In order to allocate computational resources more reasonably during the training of each model, the number of epochs for each group of experiments in this invention follows equation (19).

[0081]

[0082] Based on preliminary experiments, the Hidden Layer Size was uniformly set to 768, and the Pad_s size was uniformly set to 21. For BERT-RNN, BERT-RCNN: Dropout = 0.1, Conv Kernel Size: 2x3x4, Activation: ReLU; for CBERT-MDPCNN: Conv Kernel Num = 250, Activation: Mish / ReLU, Adam Warmup = 0.5.

[0083] The experiment can produce a comparative chart of the accuracy distribution of the comparative experiments, such as... Figure 4 As shown in the figure, the horizontal axis represents the parameter Acc. top1 The experiment sorts the values ​​in ascending order, with the ordinate representing Acc. top1 Therefore, the values ​​in this graph are monotonically increasing for each model, which can reflect the experimental Acc values ​​of each model parameter to a certain extent. top1 Distribution. Because ERNIE-FC performs poorly in this figure, affecting the display of other models, it is not shown.

[0084] from Figure 4 The results show that BERT-RCNN and BERT-RNN models have higher overall accuracy compared to BERT-LSTM and BERT-FC. The accuracy distribution curve of CBERT-MDPCNN surpasses BERT-RCNN at the 3rd position in ascending order, and remains high at the same position thereafter. This indicates that the CBERT-MDPCNN model not only has high accuracy in police text classification tasks but also has good robustness to hyperparameters.

[0085] Based on the aforementioned experimental tests, according to the Acc of each model top1 The optimal values ​​for the hyperparameters are shown in Table 3.

[0086] Table 3 Comparative Experiment Optimal Parameter Settings

[0087]

[0088] 6) Using the optimal parameters obtained from the comparative experiment in Table 3, conduct another comparative experiment to obtain the optimal comparative experiment index, as shown in Table 4.

[0089] Table 4 Comparison of Optimal Comparison Experimental Indicators

[0090]

[0091] As shown in Table 4, BERT-FC significantly outperforms ERNIE-FC; therefore, the pre-trained models used in the other control group models are all BERT pre-trained models. Furthermore, CBERT-MDPCNN, except for its lower accuracy compared to BERT-RCNN, outperforms other BERT-based models in all other metrics, particularly in Accuracy. top1 Acc 10％ P w R w F w All metrics showed improvement, validating the effectiveness of the CBERT-MDPCNN model in the eleven-category police report text classification task.

[0092] Figure 5 The graph shows the training accuracy of different models on the validation set for an eleven-class classification task. As can be seen from the enlarged portion of the graph, although the CBERT-MDPCNN model exhibits some fluctuations, its overall validation set accuracy during training is higher than that of the BERT-RNN and BERT-RCNN models shown below. This demonstrates that the CBERT-MDPCNN model performs better on the validation set during training.

[0093] Table 5. CBERT-MDPCNN Test Set Indicators for Each Category of Police Report Text

[0094]

[0095] In this table, bold text indicates an index greater than or equal to 90, while italics and underlined text indicate an index less than or equal to 80.

[0096] Table 5 shows the three main metrics for each category in the CBERT-MDPCNN test set of police report texts. Analysis of the table reveals that the model performs well in the following categories: credit card fraud, impersonation of public officials, romance scams, fraud involving illegal activities, theft via information technology, and extortion. The text features of these categories may be more pronounced. Due to limitations in training data, for categories such as investment and financial fraud, fraudulent transactions, and others, the amount of training data can be further increased to improve the model's capabilities.

[0097] Furthermore, the model has high accuracy but low recall for non-internet-related cases, indicating that the model is very accurate in predicting non-internet-related cases, but misses some positive examples. The model's prediction for this category is relatively conservative, which may be due to the imbalance between the amount of non-internet-related cases and internet-related cases in the eleven-category dataset.

[0098] Overall, on the police report text test set, CBERT-MDPCNN performed better in more categories than in others, indicating that CBERT-MDPCNN has good predictive ability for various types of text data.

[0099] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for classifying police report text based on the CBERT-MDPCNN model, characterized in that, Includes the following steps: Step 1: Obtain police report texts from historical time periods as training data for the model; Step 2: Preprocess and screen the descriptive information of the acquired police report text; Step 3: Classify the preprocessed police report texts by category and label each category with the corresponding category label; the classification includes a coarse three-category classification and a finer eleven-category classification. Step 4: For the classified police report text data, use the GPT4 language model to perform data augmentation to increase the amount of police report text data for the few sample categories; Step 5: Extract semantic information from the police report text using a BERT pre-trained model that includes a word embedding layer and a Transformer layer to obtain a text feature matrix. The word embedding layer is used to generate sentence embedding word vectors, and the Transformer layer is used to generate word vector codes. The intermediate word vector codes output by the Transformer layer are summed to the word vector codes output by the BERT pre-trained model, and the model is then pre-trained using MLM with the police report text data to generate a CBERT pre-trained model. Step 6: Transform the text feature matrix output by the CBERT pre-trained model into a text feature vector with multiple convolutional kernels that MDPCNN can process; use the word embedding layer as the backbone network, and construct the MDPCNN multi-head classifier model through a branch network composed of an eleven-class pyramid pooling structure and a three-class pyramid pooling structure. The CBERT pre-trained model and the MDPCNN multi-head classifier model constitute the CBERT-MDPCNN police text classifier model. A branch network consisting of two pyramid pooling structures performs max pooling on the text feature vector output by the backbone network, halving the length of the text feature vector. A loop is constructed until the text feature vector of each convolutional kernel is compressed to 1×1. During this process, residual connections are used to add the text feature vectors before and after the convolutional block, thus alleviating the gradient vanishing problem during model training. Subsequently, a multi-task balancing operator is introduced to integrate the data from the two branch networks to improve the model's classification performance. The specific formula for the multi-task balancing operator is as follows: ； in Output tensors for the MDPCNN multi-head classifier model. The output tensor of the eleven-category pyramid structure. For the output tensor of the three-class pyramid structure, These are the variable parameters optimized using gradient descent. Step 7: Divide the enhanced police report text data from Step 4 into a training dataset and a validation dataset, initialize the model parameters, and iteratively train the CBERT-MDPCNN police report text classifier model; Step 8: Obtain the police report text data that needs to be classified. After preprocessing in Step 2, input it into the CBERT-MDPCNN police report text classifier model trained in Step 7, and output the detailed classification results.

2. The police report text classification method according to claim 1, characterized in that, Step two includes: checking whether the alarm category matches the alarm description, deleting duplicate alarms and invalid alarms, retaining valid alarm text, and then using regular expression matching and wildcard matching to mask unnecessary information.

3. The police report text classification method according to claim 1, characterized in that, In step three, the police reports are first divided into three categories: non-internet-related cases, internet-related cases, and other cases, and labeled with three-category tags. Then, internet-related cases are further subdivided into nine subcategories: credit card fraud, ordinary identity fraud, impersonation of public officials fraud, emotional fraud, investment and financial fraud, fraudulent transaction fraud, fraud involving illegal activities, theft through information means, and extortion, and labeled with eleven-category tags.

4. The police report text classification method according to claim 1, characterized in that, Step four includes: constructing corresponding prompt words according to different alarm text categories, handing them over to the prompt word generator for processing, allowing the GPT4 language model to automatically generate alarm text data of the corresponding category, and then generating and enhancing relevant alarm text data in a loop by recognizing a small number of sample categories, and finally obtaining enhanced alarm text data, thereby increasing the amount of alarm text data of a small number of sample categories.

5. The police report text classification method according to claim 1, characterized in that, Step five is achieved through the following sub-steps: (5-1) Obtain the word vectors embedded in the sentence; The input data for the word embedding layer is data augmented police report text data, which consists of police report text sentences and corresponding classification labels. In the word embedding layer, the word embedding vector of each police report text sentence is calculated by adding the word embedding vector, the sentence embedding vector and the position embedding vector. (5-2) Use the Transformer layer to generate word vector encoding; The Transformer layer utilizes a multi-head attention sublayer, employing multiple attention mechanisms to focus on the position of the word vectors embedded in the input sentence. It associates the vectors at each position in the word vectors embedded in the input sentence to generate a new set of vector representations. Then, a feedforward neural network sublayer performs a non-linear transformation on the output of the multi-head attention sublayer to obtain intermediate word vector encodings. Residual links and layer normalization are used to connect layers to accelerate the convergence speed of the Transformer layer. (5-3) Introducing the BERT aggregation operator to improve its multi-level expressive power, BERT is denoted as CBERT after adding the aggregation operator; the aggregation operator is represented as: ； in This refers to the final output of the BERT pre-trained model after adding the operator, i.e., the text feature matrix. For the BERT pre-trained model, the first The word vector encoding output by the Transformer layer. This represents the total number of Transformer layers in the BERT pre-trained model.

6. The police report text classification method according to claim 5, characterized in that, In step (5-2), the MLM method is used to fine-tune the Transformer layer. Words in the input sentence of the Transformer layer are randomly replaced with [MASK] tags, and the Transformer layer is asked to predict the replaced words. The consideration of the contextual word association is strengthened when the Transformer layer corresponds to the domain text.

7. The police report text classification method according to claim 1, characterized in that, Step six is ​​achieved through the following sub-steps: (7-1) A non-monotonic and smooth Mish function is used in the backbone network of MDPCNN to improve the expressive power of the model. The formula is as follows: ； in For the input values ​​of the model, This is the output value after passing through the activation function; (7-2) The output tensor of the MDPCNN multi-head classifier model is transformed through a fully connected layer to output the scores of the corresponding classes. Then, the Softmax function is used to transform these scores into probability functions to obtain the eleven-class classification results. The Softmax function is as follows: ； in The output of each node, This represents the number of output nodes, i.e., the number of categories.

8. The police report text classification method according to claim 1, characterized in that, In step seven, the BERTAdam optimizer was used during the training of the CBERT-MDPCNN police text classifier model. The BERTAdam optimizer, based on AdamW, introduces a warmup learning rate decay strategy and removes the weight decay of all bias terms in the model.

Images