Data processing method and device, electronic equipment and computer readable storage medium
By concatenating and training positive and negative sample data, a text validity classification model is generated, which solves the problem of sample acquisition difficulty and improves the accuracy and recall of the model, especially performing well in traffic intelligence discrimination.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING BAIDU NETCOM SCI & TECH CO LTD
- Filing Date
- 2022-03-17
- Publication Date
- 2026-06-02
AI Technical Summary
Obtaining a large amount of high-quality sample data is quite difficult, making it hard to train a high-quality model.
By acquiring multiple sets of first sample data, the positive and negative sample data are concatenated respectively. Based on the identification information of the concatenated text, multiple sets of second sample data are generated. Then, a text validity classification model is generated by training a pre-trained language model and classifier.
It enables the acquisition of a large number of training samples with a small amount of manually labeled data, improving the accuracy and recall of text classification models, especially significantly improving the model's discrimination accuracy in traffic intelligence discrimination.
Smart Images
Figure CN114662489B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of data processing technology, and in particular to a data processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology
[0002] A well-trained model can be used to automatically classify large amounts of text data. However, in related technologies, obtaining a large number of high-quality samples is required to achieve a well-trained model. Acquiring a large number of high-quality samples is quite challenging in these technologies. Summary of the Invention
[0003] The present invention discloses a data processing method, apparatus, electronic device, and computer-readable storage medium to address the problem in the related art that it is difficult to obtain a large number of high-quality samples.
[0004] According to one aspect of this disclosure, a data processing method is provided, comprising: acquiring multiple sets of first sample data, wherein the first sample data includes positive sample data and negative sample data, wherein the positive sample data includes positive sample text and first identification information for identifying positive samples as valid text, and the negative sample data includes negative sample text and second identification information for identifying negative sample text as invalid text; concatenating the positive sample text of the positive sample data in the multiple sets of first sample data and the negative sample text of the negative sample data respectively to obtain multiple sets of concatenated text; obtaining multiple sets of second sample data based on the multiple sets of concatenated text and the concatenation identification information for identifying whether the multiple sets of concatenated text are valid; and obtaining target sample data based on the multiple sets of first sample data and the multiple sets of second sample data.
[0005] Optionally, acquiring multiple sets of first sample data includes: acquiring target text data; performing text segmentation on the target text data to acquire multiple sets of segmented text; acquiring the length of the multiple sets of segmented text and determining the segmented text with a length less than a predetermined length threshold as the target segmented text; and determining sample data in the multiple sets of first sample data based on the target segmented text and the identification information used to identify whether the target segmented text is valid text.
[0006] Optionally, based on multiple sets of concatenated text and concatenation identification information used to identify whether multiple sets of concatenated text are valid, multiple sets of second sample data are obtained, including: when the positive sample text includes traffic information text, determining that the concatenation identification information is valid text for multiple sets of concatenated text; and obtaining multiple sets of second sample data based on multiple sets of concatenated text and concatenation identification information that identifies multiple sets of concatenated text as valid text.
[0007] Optionally, the above method also includes: using target sample data for machine training to obtain a text validity classification model.
[0008] Optionally, the text validity classification model is obtained by using target sample data for machine training, including: inputting sample text from the target sample data into a pre-trained language model to obtain a predicted probability for identifying whether the target sample data is valid text; inputting the predicted probability into a classifier to obtain a classification result; and training the pre-trained language model based on the classification result and the identification information for identifying whether the sample text is valid text to obtain the text validity classification model.
[0009] Optionally, the threshold used in the classifier to determine that the sample text is invalid text is greater than 50%.
[0010] Optionally, based on the classification results and the identification information used to identify whether the sample text is valid, a pre-trained language model is trained to obtain a text validity classification model, including: if the classification results indicate that the sample text is invalid, determining that the sample text is invalid; inputting the invalid sample text into a template filter to determine whether the invalid sample text contains template statements; if the invalid sample text contains template statements, determining that the invalid sample text is valid and obtaining a corrected classification result; and training the pre-trained language model based on the corrected classification result and the identification information used to identify whether the sample text is valid to obtain a text validity classification model.
[0011] According to another aspect of this disclosure, a data processing apparatus is provided, comprising: a first acquisition module, configured to acquire multiple sets of first sample data, wherein the first sample data includes positive sample data and negative sample data, wherein the positive sample data includes positive sample text and first identification information for identifying positive samples as valid text, and the negative sample data includes negative sample text and second identification information for identifying negative sample text as invalid text; a splicing module, configured to splice the positive sample text of the positive sample data in the multiple sets of first sample data and the negative sample text in the negative sample data, respectively, to obtain multiple sets of spliced text; a second acquisition module, configured to obtain multiple sets of second sample data based on the multiple sets of spliced text and the splicing identification information for identifying whether the multiple sets of spliced text are valid; and a third acquisition module, configured to obtain target sample data based on the multiple sets of first sample data and the multiple sets of second sample data.
[0012] Optionally, the first acquisition module includes: a first acquisition unit for acquiring target text data; a text segmentation unit for segmenting the target text data to acquire multiple groups of segmented text; a first determination unit for acquiring the length of the multiple groups of segmented text and determining the segmented text whose length is less than a predetermined length threshold as the target segmented text; and a second determination unit for determining sample data from multiple groups of first sample data based on the target segmented text and identification information used to identify whether the target segmented text is valid text.
[0013] Optionally, the second acquisition module includes: a third determining unit, used to determine that the splicing identification information is multiple sets of spliced text as valid text when the positive sample text includes traffic information text; and a second acquisition unit, used to obtain multiple sets of second sample data based on the multiple sets of spliced text and the splicing identification information that identifies the multiple sets of spliced text as valid text.
[0014] Optionally, the device further includes a fourth acquisition module for machine training using target sample data to obtain a text validity classification model.
[0015] Optionally, the fourth acquisition module includes: a third acquisition unit, used to input sample text from the target sample data into a pre-trained language model to obtain a predicted probability used to identify whether the target sample data is valid text; a fourth acquisition unit, used to input the predicted probability into a classifier to obtain a classification result; and a fifth acquisition unit, used to train the pre-trained language model based on the classification result and the identification information used to identify whether the sample text is valid text to obtain a text validity classification model.
[0016] Optionally, the threshold used in the classifier to determine that the sample text is invalid text is greater than 50%.
[0017] Optionally, the fifth acquisition unit includes: a first determination subunit, used to determine that the sample text is invalid if the classification result indicates that the sample text is invalid; a second determination subunit, used to input the invalid sample text into a template filter to determine whether the invalid sample text contains template statements; a first acquisition subunit, used to determine the invalid sample text as valid sample text if the invalid sample text contains template statements, and to obtain a corrected classification result; and a second acquisition subunit, used to train a pre-trained language model based on the corrected classification result and the identification information used to identify whether the sample text is valid, to obtain a text validity classification model.
[0018] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the methods described above.
[0019] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method according to any of the preceding claims.
[0020] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods described above.
[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0022] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:
[0023] Figure 1 This is a flowchart of a data processing method according to an embodiment of the present disclosure;
[0024] Figure 2 This is a flowchart of a model training method according to an embodiment of the present disclosure;
[0025] Figure 3 This is a schematic diagram of the text validity classification model structure according to an embodiment of the present disclosure;
[0026] Figure 4 This is a structural block diagram of a data processing apparatus according to an embodiment of the present disclosure;
[0027] Figure 5 This is a block diagram of an electronic device for a data processing method according to an embodiment of the present disclosure. Detailed Implementation
[0028] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0029] Terminology Explanation
[0030] BERT, Bidirectional Encoder Representation from Transformers, is a pre-trained language representation model, a machine model used for text classification.
[0031] CNN stands for Convolutional Neural Networks.
[0032] Binary classification indicates that there are only two categories in a classification task.
[0033] Softmax-activate-function is the activation function in a neural network model.
[0034] In this embodiment of the disclosure, a data processing method is provided. Figure 1 This is a flowchart of a data processing method according to the first embodiment of this disclosure, such as... Figure 1 As shown, the data processing method may include the following steps.
[0035] Step S101: Obtain multiple sets of first sample data, wherein the first sample data includes positive sample data and negative sample data. The positive sample data includes positive sample text and first identification information for identifying positive samples as valid text. The negative sample data includes negative sample text and second identification information for identifying negative sample text as invalid text.
[0036] Step S102: Concatenate the positive sample text of the positive sample data and the negative sample text of the negative sample data in the multiple sets of first sample data to obtain multiple sets of concatenated text.
[0037] In some embodiments, there are multiple methods for concatenating positive sample text data and negative sample text data. For example, one positive sample text and one negative sample text can be concatenated, multiple positive sample texts and one negative sample text can be concatenated, or one positive sample text and multiple negative sample texts can be concatenated.
[0038] Step S103: Based on multiple sets of concatenated text and concatenation identifier information used to identify whether multiple sets of concatenated text are valid, multiple sets of second sample data are obtained respectively.
[0039] Specifically, multiple sets of concatenated text can be labeled using concatenation identifiers to determine whether they are valid, thus obtaining multiple sets of second sample data. There are several methods for labeling multiple sets of concatenated text using concatenation identifiers; for example, multiple sets of concatenated text can be labeled as valid, or multiple sets of concatenated text can be labeled as valid.
[0040] Step S104: Based on multiple sets of first sample data and multiple sets of second sample data, obtain the target sample data.
[0041] It is important to understand that the target sample data is used to train the machine learning model, thereby obtaining a trained machine learning model. In some alternative embodiments, the target data includes first sample data, and second sample data consisting of concatenated positive and negative sample text from the first sample data, with multiple concatenated text sets having validity flags added to them.
[0042] The above method involves acquiring multiple sets of first sample data, concatenating the positive sample text from the positive sample data and the negative sample text from the negative sample data within these sets of first sample data, resulting in multiple sets of concatenated text. Based on these concatenated texts and the concatenation identifiers used to determine the validity of the text, multiple sets of second sample data are obtained. Finally, the target sample data is derived from both the first and second sample data. In other words, multiple sets of second sample data are obtained from multiple sets of first sample data, and the target sample data is obtained from the first and second sample data, thus enhancing the sample data. Therefore, by acquiring multiple sets of target sample data containing validity identifiers from a small amount of first sample data, this method solves the problem of the high difficulty in obtaining a large number of high-quality samples in related technologies.
[0043] In some optional embodiments, the method for obtaining multiple sets of first sample data may include the following steps: obtaining target text data; performing text segmentation on the target text data to obtain multiple sets of segmented text; obtaining the length of the multiple sets of segmented text and determining the segmented text with a length less than a predetermined length threshold as the target segmented text; and determining the sample data in the multiple sets of first sample data based on the target segmented text and the identification information used to identify whether the target segmented text is valid text.
[0044] In one embodiment, there are multiple methods for segmenting target text data. For example, the target text data can be segmented by dividing it into segments. Specifically, the target text data can be segmented according to the events included in the target text data to obtain multiple segments of text, where each segment of text includes one event. Another example is segmenting the target text data by dividing it into forwarding and commenting methods. For instance, a piece of target text data includes text data generated at a first moment, and text data generated at a second moment related to forwarding, commenting, or supplementing the text data generated at the first moment. The target text data can be segmented by dividing it into forwarding and commenting methods to obtain the text data generated at the first moment and the text data related to forwarding, commenting, or supplementing the text data generated at the second moment.
[0045] The above method involves segmenting the target text data to obtain multiple segments, filtering these segments based on their length, and identifying segments shorter than a predetermined length threshold along with their corresponding identifiers as sample data within the first set of sample data. By filtering segments exceeding the predetermined length threshold, the usability of the sample data within the first set of sample data is improved.
[0046] In some optional embodiments, multiple sets of second sample data are obtained based on multiple sets of concatenated text and concatenation identification information used to identify whether the multiple sets of concatenated text are valid. The method may include: when the positive sample text includes traffic information text, determining that the concatenation identification information indicates that the multiple sets of concatenated text are valid text; and obtaining multiple sets of second sample data based on the multiple sets of concatenated text and the concatenation identification information indicating that the multiple sets of concatenated text are valid text.
[0047] The above method concatenates valid and invalid traffic information texts, and marks the concatenated text as valid. Thus, multiple sets of sample texts can be obtained from a small number of traffic information sample texts, thereby enhancing the sample data.
[0048] In some optional embodiments, positive and negative sample texts from multiple sets of first sample data are concatenated, and multiple sets of second sample data are obtained based on the concatenated texts. Then, target sample data is obtained based on the first and second sample data, and machine training is performed using the target sample data to obtain a text validity classification model.
[0049] The above method involves concatenating positive and negative sample texts from multiple sets of first sample data to obtain multiple sets of second sample data. Then, machine learning is performed based on the first and second sample data to obtain a text validity classification model. Thus, a large amount of target sample data can be obtained from a small amount of manually labeled first sample data. Using this large amount of target sample data to train the machine model and obtain the text validity classification model solves the problem in related technologies of lacking methods for training text classification models without requiring excessive manually labeled data.
[0050] In some optional embodiments, positive and negative sample texts from multiple sets of first sample data are concatenated to obtain multiple sets of second sample data. Target sample data is then obtained based on the first and second sample data. After obtaining the target sample data, the sample text in the target sample data is input into a pre-trained language model to obtain a predicted probability used to identify whether the target sample data is valid text. The predicted probability is then input into a classifier to obtain a classification result. Based on the classification result and the identification information used to identify whether the sample text is valid text, the pre-trained language model is trained to obtain a text validity classification model.
[0051] The above method involves obtaining the predicted probability of whether the target sample data is valid text through target sample data and a pre-trained language model. This predicted probability is then input into a classifier to obtain the classification result. Based on the classification result and the identification information in the target sample data, the pre-trained model is trained to obtain a text validity classification model. During pre-training, the pre-trained language model masters the relationships between individual words and words in different contexts. The resulting text validity classification model can identify keywords and their relationships with other words in the context, and classify text validity based on these relationships. Compared to related technologies that classify text validity solely based on keywords, the model provided in this embodiment has a higher classification accuracy. Using target data containing first sample data and second sample data obtained by concatenating positive and negative sample text from the first sample data, the pre-trained language model and classifier are trained. This achieves the goal of obtaining a large amount of training sample data using a small amount of labeled data, and training the text validity classification model based on this large amount of training sample data.
[0052] In one embodiment, the pre-trained language model includes a BERT model. Downstream of the [CLS] layer in the BERT model's output layer is a linear layer for dimensionality reduction. This linear layer can be a 128-dimensional (128-dim) linear layer or a linear layer of other dimensions. Following this linear layer is a 2-dimensional (2-dim) output layer. Binary classification is performed on the output of the output layer to obtain the text classification result. Several types of functions can be used for binary classification; for example, the softmax activation function can be used.
[0053] In some alternative embodiments, the threshold used in the classifier to determine that sample text is invalid text is greater than 50%.
[0054] In related technologies, the threshold used by the classifier to determine that sample text is invalid is typically set to 50%. In this optional embodiment, setting the threshold to greater than 50% can improve the recall rate of the text validity classification model. In one optional embodiment, to improve the recall rate of the text validity classification model, the threshold is set to 90%. In another optional embodiment, the threshold can also be set to other values greater than 50%.
[0055] Optionally, a method for training a pre-trained language model to obtain a text validity classification model based on the classification results and the identification information used to identify whether the sample text is valid may include the following steps: if the classification results indicate that the sample text is invalid, determine that the sample text is invalid; input the invalid sample text into a template filter to determine whether the invalid sample text contains template statements; if the invalid sample text contains template statements, determine that the invalid sample text is valid and obtain a corrected classification result; and train the pre-trained language model based on the corrected classification result and the identification information used to identify whether the sample text is valid to obtain a text validity classification model.
[0056] The above method determines whether invalid samples in the target sample data contain template statements. If invalid sample text contains template statements, it is identified as valid sample text, thus correcting the classification results. The model is then trained based on the corrected results to obtain a text effective classification model. This improves the recall rate of the text effective classification model.
[0057] Based on the above embodiments and optional embodiments, an optional implementation method is provided, which is described in detail below. It should be noted that in this optional implementation method, the text data in the first sample data is used as traffic information data, the pre-trained language model is used as the BERT model, and the binary classification function is used as the softmax function for illustration.
[0058] Traffic news, as a crucial source of information on real-world changes, presents information characterized by its predictability, high reliability, and large volume. Predictability means that traffic news often provides clear indications of changes before they actually occur, such as: "The Great Wall Expressway will prohibit cars from entering starting tonight at 8:00 PM, with the reopening date to be determined." This information itself informs about impending events in advance, and since it is usually provided through public accounts and official websites, its reliability is high, possessing a high degree of confidence. The information provided by traffic news is crucial and essential for map-related applications.
[0059] The internet contains a vast amount of traffic information, and the corresponding textual data is enormous. Machine learning is needed to automatically assess the validity of this information, determining whether it meets the user's specific needs. Valid traffic information typically includes details such as road closures and traffic restrictions. Once valid traffic information is obtained, it can be applied to traffic-related applications.
[0060] In related technologies, keyword-triggered models are used to determine the validity of traffic intelligence. This involves using the most frequent words in valid traffic intelligence reports as keywords, and then using the keyword-triggered model to determine if the traffic intelligence report contains these keywords. If it does, the report is considered valid. However, this model has the following problems: it doesn't analyze the context during traffic intelligence processing, resulting in low accuracy. For example, in traffic intelligence reporting road closures, the words "closed" and "blocked" are usually high-frequency words, making them keywords. Therefore, when a traffic intelligence report contains "closed" or "blocked," the keyword-triggered model might classify it as valid. However, intelligence reports containing words like "closed" or "blocked" are not necessarily valid reports containing road closure information.
[0061] Alternatively, Bayesian or CNN network models can be used to determine the validity of traffic information. CNN models are non-fine-tuned neural network models. The accuracy of these two models in determining the validity of traffic information is slightly higher than that of keyword-triggered models, but the accuracy still does not meet user needs. Specifically, the accuracy of the Bayesian model is approximately 22.38%, and the accuracy of the CNN model is approximately 27.27%.
[0062] Using Bayesian or CNN-based models to determine the validity of traffic intelligence is more accurate than using keyword-triggered models because both models incorporate contextual information to some extent. However, both models rely on large amounts of training data. While large-scale traffic intelligence data is readily available, clearly labeled traffic intelligence is difficult to obtain, requiring extensive manual annotation, which increases the cost of model training.
[0063] In view of this, the present disclosure provides a data processing method that obtains a large amount of training sample data through a small amount of manually labeled training sample data. The model is then trained using this training sample data to obtain a text validity classification model. This text validity classification model includes a pre-trained language model and a classifier, requiring only a small amount of training sample data to complete the training. When the model identifies a keyword, it further determines whether the keyword's target is the object the user needs. For example, when a piece of information contains the keyword "block," the model, after identifying the keyword "block," will further identify what the "blocked" object is. If the identified "blocked" object is not a road, the traffic information is determined to be invalid.
[0064] The data processing method disclosed in this embodiment includes the following steps:
[0065] Retrieve traffic intelligence data (equivalent to the target text data in the aforementioned implementation) from the database, which contains the results of the operators' work. The results of the operators' work refer to the marking of the traffic intelligence text in the traffic intelligence data by the operators themselves to indicate whether the text is valid.
[0066] The intelligence text is segmented by methods such as segmentation and forwarding to obtain multiple segments of intelligence text (equivalent to the segmented text in the aforementioned embodiment); segments of intelligence text whose length exceeds a predetermined length threshold are removed, thereby achieving the cleaning of the multiple segments of intelligence text.
[0067] For multi-segment intelligence texts that have undergone data cleaning, multiple sets of spliced texts are obtained by splicing the valid and invalid intelligence texts one-to-one. These spliced texts are then marked as valid, thereby achieving data augmentation and obtaining more training sample data (equivalent to the target sample data in the aforementioned embodiment).
[0068] A portion of the multiple training sample data obtained through data augmentation is used as the training set, and the remainder is used as the validation set to train the model, thereby obtaining an intelligence validity discrimination model (equivalent to the text validity classification model in the aforementioned embodiment); the intelligence validity discrimination model is used to predict the online traffic intelligence data to obtain the validity prediction results of the traffic intelligence data.
[0069] Figure 3 This is a schematic diagram of the intelligence validity judgment model structure according to an embodiment of this disclosure. (Refer to...) Figure 3As shown, the intelligence validity discrimination model structure in this embodiment includes a BERT model, a 128-dimensional linear layer connected downstream of the [CLS] layer in the output layer of the BERT model, a 2-dimensional output layer following the 128-dimensional linear layer, and finally, a softmax activation function is used to perform binary classification on the output layer results to obtain the prediction result of whether the traffic intelligence is valid or invalid.
[0070] When processing traffic intelligence using an intelligence validity discrimination model, the [CLS] and [SEP] identifiers are first added to the beginning and end of the traffic intelligence text, respectively. A tokenizer is then used to segment the text into tokens (words or characters), obtaining the token_id (word ID) corresponding to each token, and adding the corresponding position vector position_id (position ID). These tokens are then input into the BERT model, and finally output to the [CLS] layer in the BERT model's output layer. The output of the [CLS] layer is taken, and this 768-dimensional output vector is connected to the designated downstream network layer (i.e., the aforementioned 128-dimensional linear layer), and then output to the final output layer (i.e., the aforementioned 2-dimensional output layer). The output value of the output layer is processed through a softmax activation function to obtain the probability that the input intelligence is "valid" or "invalid." (Continue referring to...) Figure 3 As shown, according to the model's output, the probability of traffic intelligence being invalid is 67.3%, and the probability of traffic intelligence being valid is 32.7%.
[0071] It's important to understand that the BERT model supports a maximum text length of 512 characters. Excluding the [CLS] and [SEP] characters at the beginning and end of the text, the maximum text length supported by the BERT model is 510 characters. In practical applications, except for cases where multiple pieces of information are combined into a single intelligence report, most effective road closure intelligence reports are less than 300 characters long. Therefore, when segmenting traffic intelligence texts into segments and forwarding them separately, multiple segments of intelligence text should be obtained. A predetermined length threshold of 300 characters can be set; that is, when filtering the data, segments of intelligence text longer than 300 characters will be removed.
[0072] Because the intelligence discrimination model prioritizes recall, the probability threshold for classifying intelligence as invalid was increased during model deployment. Normally, a probability greater than 50% is sufficient to classify any category. However, in the intelligence validity discrimination model, to ensure high recall, the threshold is set to only classify traffic intelligence as invalid when the probability of an invalid category exceeds 90%. In other words, the model only considers traffic intelligence invalid when it is highly certain it is invalid; otherwise, it is treated as valid intelligence.
[0073] To further enhance the model's recall, traffic information identified as invalid by the intelligence validity discrimination model undergoes further screening. Specifically, it checks whether the invalid information contains strict template content (equivalent to template statements in the aforementioned embodiments), such as "Toll station A is closed" or "Cars are prohibited from entering highway section B," etc. If the invalid information contains sub-paragraphs that conform to the strict template content, it is post-processed and judged as valid information. This corrects the output of the intelligence discrimination model and improves its recall rate.
[0074] The following provides a detailed description of the accuracy of the intelligence validity discrimination model (BERT model) output using the embodiments of this disclosure. Table 1 shows the accuracy of the intelligence validity discrimination model, as well as the accuracy of the Bayes model, SVM model (Support Vector Machines), CNN model, and Combine model (aggregation model) outputs when using the intelligence validity discrimination model of the embodiments of this disclosure, assuming a recall rate of 100%.
[0075] Table 1
[0076]
[0077] Referring to the comparison results shown in Table 1, under the premise that the recall rate is 100%, the accuracy of the information validity discrimination model output by the embodiment of this disclosure is much higher than that of other models, and its accuracy is about 65% higher than the average accuracy of other models.
[0078] The following is a detailed description of the visualization analysis of the intelligence validity discrimination model in the embodiments of this disclosure.
[0079] To enhance the interpretability of intelligence validity assessment models, it's not enough to simply know that the model can identify valid intelligence; it's also necessary to understand why it can. The greatest advantage of the transformer model within the BERT model lies in its self-attention mechanism. Therefore, the visualization analysis of intelligence validity assessment models primarily focuses on visualizing the attention mechanism within the 12 layers and 12 heads of the BERT model, thereby analyzing why the model can identify valid intelligence.
[0080] Based on the analysis of the attention distribution of the BERT model using a valid piece of intelligence selected from the intelligence database, the text content of the valid intelligence is: "[Toll Station Closed] At 19:37 on December 25th, the entrance to Yantan Station on the S66 Longhan Expressway (Zilong section) is closed due to a traffic accident." When the intelligence discrimination model judges the validity of this intelligence, the model's attention stays on the keywords "Yantan Toll Station," "entrance," and "closed." The brain mechanism understands that this is text content describing "a road being closed," therefore, this is a valid text intelligence about road closure.
[0081] The following example uses the traffic information "[Station Closed] At 19:37 on December 25th, the entrance to Yantan Station on the S66 Longhan Expressway (Zilong section) is closed due to a traffic accident" to illustrate how the fine-tuned BERT model pays attention to these text messages. The BERT model has 12 layers. When analyzing attention, we start from the layer closest to the input of the BERT model (layer 1) and proceed sequentially to the layer closest to the output (layer 12).
[0082] The first to fourth layers of the BERT model, closest to the input, are defined as shallow layers. In the first layer, [CLS] primarily focuses on two special characters, [CLS] and [SEP], paying little attention to intermediate tokens. Specifically, the first layer focuses on the special characters [CLS] at the beginning and [SEP] at the end of the intelligence text. [SEP] contains no semantic information. Therefore, in the shallow transformer layers, the BERT model only focuses on the vector information of the special character [CLS] itself and the vector information of [SEP]. The product of the q matrix (query matrix) of [CLS] and the k matrix (key matrix) of [CLS] (or [SEP]) carries the most weight. The reason a particular token might focus on the semantically insensitive [SEP] is because the BERT model considers that token to be unrelated to any other tokens, thus concentrating all its attention on the semantically insensitive [SEP].
[0083] The fifth to eighth layers of the BERT model, closest to the input, are defined as the middle layers. In these middle layers, the [CLS] layer doesn't just focus on its own vector information but also pays more attention to the text content, indicating that [CLS] begins to attempt to extract keyword information from the text. For example, the [CLS] classification layer starts to focus on the phrase "station closed" in the text and considers these words to be the most crucial words in the text.
[0084] The 10th to 12th layers of the BERT model, closest to the input, are defined as deep layers. As the layers deepen, the [CLS] layer loses its bias. The [CLS] layer of the 12th layer allocates some attention to almost every token in the sentence. Although different headers may show a preference for one or two tokens, overall, compared with the shallow and medium layers, the deep layers show a significantly reduced preference for other keywords.
[0085] Based on the visualization analysis of the shallow, middle and deep layers of the BERT model, the shallow layer of the BERT model focuses more on the vector of the [CLS] itself, the middle layer of the BERT model tends to focus more on keyword information and has the ability to identify key information, while the deep layer of the BERT model does not have a particularly strong preference for other tokens, and its attention to all tokens is not much different.
[0086] According to embodiments of this disclosure, an apparatus for implementing the above-described data processing method is also provided. Figure 4 This is a structural block diagram of a data processing apparatus according to an embodiment of the present disclosure, with reference to... Figure 4As shown, the device includes a first acquisition module 401, a splicing module 402, a second acquisition module 403, and a third acquisition module 404. The device will be described in detail below.
[0087] The first acquisition module 401 is used to acquire multiple sets of first sample data, wherein the first sample data includes positive sample data and negative sample data. The positive sample data includes positive sample text and first identification information for identifying positive samples as valid text. The negative sample data includes negative sample text and second identification information for identifying negative sample text as invalid text. The splicing module 402 is connected to the first acquisition module 401 and is used to splice the positive sample text of the positive sample data and the negative sample text of the negative sample data in the multiple sets of first sample data to obtain multiple sets of spliced text. The second acquisition module 403 is connected to the splicing module 402 and is used to obtain multiple sets of second sample data based on the multiple sets of spliced text and the splicing identification information for identifying whether the multiple sets of spliced text are valid. The third acquisition module 404 is connected to the second acquisition module 403 and is used to obtain target sample data based on the multiple sets of first sample data and the multiple sets of second sample data.
[0088] It should be noted that the first acquisition module 401, splicing module 402, second acquisition module 403, and third acquisition module 404 mentioned above correspond to steps S101 to S104 in the data processing method. The multiple modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments.
[0089] As an optional embodiment, the first acquisition module 401 includes: a first acquisition unit for acquiring target text data; a text segmentation unit for segmenting the target text data to acquire multiple groups of segmented text; a first determination unit for acquiring the length of the multiple groups of segmented text and determining the segmented text whose length is less than a predetermined length threshold as target segmented text; and a second determination unit for determining sample data in the multiple groups of first sample data based on the target segmented text and identification information used to identify whether the target segmented text is valid text.
[0090] As an optional embodiment, the second acquisition module 402 includes: a third determining unit, configured to determine that the splicing identification information is valid text when the positive sample text includes traffic information text; and a second acquisition unit, configured to obtain the multiple sets of second sample data based on the multiple sets of spliced text and the splicing identification information that identifies the multiple sets of spliced text as valid text.
[0091] As an optional embodiment, the data processing apparatus further includes: a fourth acquisition module, used for machine training using the target sample data to obtain a text validity classification model.
[0092] As an optional embodiment, the fourth acquisition module includes: a third acquisition unit, used to input sample text from the target sample data into a pre-trained language model to obtain a predicted probability for identifying whether the target sample data is valid text; a fourth acquisition unit, used to input the predicted probability into a classifier to obtain a classification result; and a fifth acquisition unit, used to train the pre-trained language model based on the classification result and the identification information for identifying whether the sample text is valid text to obtain the text validity classification model.
[0093] As an optional embodiment, the threshold used in the classifier to determine that the sample text is invalid text is greater than 50%.
[0094] As an optional embodiment, the fifth acquisition unit includes: a first determining subunit, configured to determine that the sample text is invalid if the classification result indicates that the sample text is invalid; a second determining subunit, configured to input the invalid sample text into a template filter to determine whether the invalid sample text contains template statements; a first acquisition subunit, configured to determine the invalid sample text as valid sample text and acquire a corrected classification result if the invalid sample text contains template statements; and a second acquisition subunit, configured to train the pre-trained language model based on the corrected classification result and the identification information used to identify whether the sample text is valid, to obtain the text validity classification model.
[0095] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0096] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0097] like Figure 5As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 505 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0098] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 505, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0099] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 505. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform data processing methods by any other suitable means (e.g., by means of firmware).
[0100] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0101] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0102] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0104] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0105] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.
[0106] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0107] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A data processing method, comprising: Multiple sets of first sample data are acquired. The first sample data is traffic intelligence that has been validated. The first sample data includes positive sample data and negative sample data. The positive sample data is valid traffic intelligence containing information related to road closures. The positive sample data includes positive sample text and a first identification information for identifying the positive sample as valid text. The negative sample data includes negative sample text and a second identification information for identifying the negative sample text as invalid text. The text length in the multiple sets of first sample data is less than a predetermined length threshold. The positive sample text of the positive sample data and the negative sample text of the negative sample data in the multiple sets of first sample data are concatenated to obtain multiple sets of concatenated text, wherein the multiple sets of concatenated text are obtained by concatenating at least one positive sample text and at least one negative sample text. Based on the multiple sets of spliced text and the splicing identifier information used to identify whether the multiple sets of spliced text are valid, multiple sets of second sample data are obtained respectively; Based on the multiple sets of first sample data and the multiple sets of second sample data, the target sample data is obtained; The method further includes: determining a classification result based on the target sample data, wherein the classification result is used to indicate whether the sample text in the target sample data is invalid text; if the classification result indicates that the sample text is invalid text, determining that the sample text is invalid sample text; inputting the invalid sample text into a template filter to determine whether the invalid sample text contains template statements; if the invalid sample text contains template statements, determining that the invalid sample text is valid sample text and obtaining a corrected classification result; training a pre-trained language model based on the corrected classification result and the identification information used to identify whether the sample text is valid text, to obtain the text validity classification model, wherein the text validity classification model includes the pre-trained language model and a classifier.
2. The method according to claim 1, wherein, The acquisition of multiple sets of first sample data includes: Obtain the target text data; The target text data is segmented to obtain multiple segments of text. Obtain the length of the multi-group segmented text, and determine the segmented text whose length is less than the predetermined length threshold as the target segmented text; Based on the target segmented text and the identification information used to identify whether the target segmented text is valid text, the sample data in the multiple sets of first sample data are determined.
3. The method according to claim 1, wherein, Based on the multiple sets of concatenated text and the concatenation identifier information used to identify whether the multiple sets of concatenated text are valid, multiple sets of second sample data are obtained, including: In the case where the positive sample text includes traffic information text, the splicing identification information is determined to be the multiple sets of spliced text as valid text; Based on the multiple sets of concatenated text and the concatenation identifier information that identifies the multiple sets of concatenated text as valid text, the multiple sets of second sample data are obtained.
4. The method according to claim 1, wherein, Determining the classification result based on the target sample data includes: The sample text in the target sample data is input into a pre-trained language model to obtain the predicted probability used to identify whether the target sample data is valid text; The predicted probability is input into the classifier to obtain the classification result.
5. The method according to claim 4, wherein, The threshold used in the classifier to determine that the sample text is invalid is greater than 50%.
6. A data processing apparatus, comprising: The first acquisition module is used to acquire multiple sets of first sample data, wherein the first sample data is traffic intelligence for validity labeling, the first sample data includes positive sample data and negative sample data, wherein the positive sample data is valid traffic intelligence containing information related to road closures, the positive sample data includes: positive sample text, and a first identification information for identifying the positive sample as valid text, the negative sample data includes: negative sample text, and a second identification information for identifying the negative sample text as invalid text, and the text length in the multiple sets of first sample data is less than a predetermined length threshold; The splicing module is used to splice the positive sample text of the positive sample data and the negative sample text of the negative sample data in the multiple sets of first sample data respectively to obtain multiple sets of spliced text, wherein the multiple sets of spliced text are obtained by splicing at least one positive sample text and at least one negative sample text. The second acquisition module is used to obtain multiple sets of second sample data based on the multiple sets of spliced text and splicing identification information used to identify whether the multiple sets of spliced text are valid. The third acquisition module is used to obtain target sample data based on the multiple sets of first sample data and the multiple sets of second sample data; The first acquisition module is further configured to determine a classification result based on the target sample data, wherein the classification result is used to indicate whether the sample text in the target sample data is invalid text; if the classification result indicates that the sample text is invalid text, the sample text is determined to be invalid sample text; the invalid sample text is input to a template filter to determine whether the invalid sample text contains template statements; if the invalid sample text contains template statements, the invalid sample text is determined to be valid sample text, and a corrected classification result is obtained; based on the corrected classification result and the identification information used to identify whether the sample text is valid text, a pre-trained language model is trained to obtain the text validity classification model, wherein the text validity classification model includes the pre-trained language model and a classifier.
7. The apparatus according to claim 6, wherein, The first acquisition module includes: The first acquisition unit is used to acquire target text data; The text segmentation unit is used to segment the target text data to obtain multiple groups of segmented text. The first determining unit is used to obtain the length of the multi-group segmented text and determine the segmented text whose length is less than the predetermined length threshold as the target segmented text; The second determining unit is used to determine sample data in the multiple sets of first sample data based on the target segmented text and the identification information used to identify whether the target segmented text is valid text.
8. The apparatus according to claim 6, wherein, The second acquisition module includes: The third determining unit is used to determine that the splicing identification information is the multiple sets of spliced text as valid text when the positive sample text includes traffic information text; The second acquisition unit is used to obtain the multiple sets of second sample data based on the multiple sets of concatenated text and the concatenation identification information that identifies the multiple sets of concatenated text as valid text.
9. The apparatus according to claim 6, wherein, The device further includes a fourth acquisition module, the fourth acquisition module comprising: The third acquisition unit is used to input the sample text in the target sample data into the pre-trained language model to obtain the prediction probability used to identify whether the target sample data is valid text; The fourth acquisition unit is used to input the predicted probability into the classifier and obtain the classification result.
10. The apparatus according to claim 9, wherein, The threshold used in the classifier to determine that the sample text is invalid is greater than 50%.
11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 5.
12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 5.
13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 5.