Text data processing method and device, electronic equipment and computer readable medium

Abstract

The invention discloses a text data processing method and device, electronic equipment and a computer readable medium, and relates to the technical field of natural language process.The specific implementation mode comprises the steps that a text data processing request is received, and corresponding to-be-processed text data is obtained; performing word segmentation processing on the to-be-processed text data to obtain each piece of word segmentation data; according to each piece of word segmentation data, determining a corresponding data embedding vector; calling a data enhancement architecture to generate a data embedding vector set based on the data embedding vectors; and according to the data embedding vector and the data embedding vector set, determining a corresponding data classification and outputting the data classification. Therefore, the classification accuracy of the to-be-processed text data can be improved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a text data processing method, device, electronic device and computer-readable medium. Background Art

[0002] When classifying text data, vocabulary rule-matching classification methods require significant manpower to maintain the vocabulary, leading to lags. Word vector space representations ignore the semantic information of words, resulting in high dimensionality and a tendency to cause the curse of dimensionality. Deep learning-based classification methods also suffer from significant randomness and can only retain word features, failing to preserve entire sentence features. These factors significantly impact accurate text classification, resulting in low accuracy. Summary of the Invention

[0003] In view of this, embodiments of the present application provide a text data processing method, device, electronic device, and computer-readable medium, which can solve the problem of low accuracy in existing text data classification.

[0004] To achieve the above-mentioned purpose, according to one aspect of an embodiment of the present application, a text data processing method is provided, comprising:

[0005] Receive a text data processing request and obtain the corresponding text data to be processed;

[0006] Perform word segmentation processing on the text data to be processed to obtain each word segmentation data;

[0007] According to each word segmentation data, determine the corresponding data embedding vector;

[0008] Invoke the data augmentation architecture to generate a set of data embedding vectors based on the data embedding vectors;

[0009] According to the data embedding vector and the data embedding vector set, the corresponding data classification is determined and output.

[0010] Optionally, determining a corresponding data embedding vector based on each word segmentation data includes:

[0011] Each word segmentation data is input into the prediction model to output the corresponding data embedding vector.

[0012] Optionally, before inputting each word segmentation data into the prediction model, the text data processing method further includes:

[0013] Obtain a sample data set, and perform mask preprocessing on the sample data set based on a preset vocabulary to obtain a masked sample data set;

[0014] Get the label corresponding to the data that has been mask preprocessed;

[0015] The mask sample dataset is used as the input of the initial neural network model, the corresponding label is used as the expected output of the initial neural network model, and the initial neural network model is trained to obtain a prediction model.

[0016] Optionally, generate a data embedding vector set, including:

[0017] Calling a preset data embedding vector library to determine the similarity between each preset data embedding vector in the preset data embedding vector library and the data embedding vector;

[0018] According to the similarity, the target preset data embedding vector is determined;

[0019] Based on the target preset data embedding vector, a data embedding vector set is generated.

[0020] Optionally, after generating the data embedding vector set, the text data processing method further includes:

[0021] Input the data embedding vector set into the data augmentation architecture to generate the latest data embedding vector;

[0022] Update the data embedding vector set based on the latest data embedding vector.

[0023] Optionally, determining corresponding data classification includes:

[0024] The data embedding vector and the data embedding vector set are input into the prediction model to determine the corresponding data classification based on the classification discrimination module in the prediction model.

[0025] Optionally, determining a corresponding data embedding vector based on each word segmentation data includes:

[0026] Input each word segmentation data into the prediction model to determine the correlation between each word segmentation data based on the next sentence prediction module in the prediction model;

[0027] Generate corresponding data embedding vectors based on each word segmentation data and correlation.

[0028] In addition, the present application also provides a text data processing device, comprising:

[0029] A receiving unit configured to receive a text data processing request and obtain corresponding text data to be processed;

[0030] A word segmentation data determining unit is configured to perform word segmentation processing on the text data to be processed to obtain individual word segmentation data;

[0031] A data embedding vector determining unit is configured to determine a corresponding data embedding vector based on each word segmentation data;

[0032] a data embedding vector set determining unit configured to call a data augmentation architecture to generate a data embedding vector set based on the data embedding vector;

[0033] The data classification determination unit is configured to determine and output the corresponding data classification based on the data embedding vector and the data embedding vector set.

[0034] Optionally, the data embedding vector determining unit is further configured to:

[0035] Each word segmentation data is input into the prediction model to output the corresponding data embedding vector.

[0036] Optionally, the apparatus further comprises a training unit configured to:

[0037] Obtain a sample data set, and perform mask preprocessing on the sample data set based on a preset vocabulary to obtain a masked sample data set;

[0038] Get the label corresponding to the data that has been mask preprocessed;

[0039] The mask sample dataset is used as the input of the initial neural network model, the corresponding label is used as the expected output of the initial neural network model, and the initial neural network model is trained to obtain a prediction model.

[0040] Optionally, the data embedding vector set determining unit is further configured to:

[0041] Calling a preset data embedding vector library to determine the similarity between each preset data embedding vector in the preset data embedding vector library and the data embedding vector;

[0042] According to the similarity, the target preset data embedding vector is determined;

[0043] Based on the target preset data embedding vector, a data embedding vector set is generated.

[0044] Optionally, the text data processing apparatus further includes an updating unit configured to:

[0045] Input the data embedding vector set into the data augmentation architecture to generate the latest data embedding vector;

[0046] Update the data embedding vector set based on the latest data embedding vector.

[0047] Optionally, the data classification determination unit is further configured to:

[0048] The data embedding vector and the data embedding vector set are input into the prediction model to determine the corresponding data classification based on the classification discrimination module in the prediction model.

[0049] Optionally, the data embedding vector determining unit is further configured to:

[0050] Input each word segmentation data into the prediction model to determine the correlation between each word segmentation data based on the next sentence prediction module in the prediction model;

[0051] Generate corresponding data embedding vectors based on each word segmentation data and correlation.

[0052] In addition, the present application also provides a text data processing electronic device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the one or more processors implement the text data processing method as described above.

[0053] In addition, the present application also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the text data processing method as described above is implemented.

[0054] One embodiment of the above invention has the following advantages or beneficial effects: the present application receives a text data processing request to obtain the corresponding text data to be processed; performs word segmentation processing on the text data to be processed to obtain individual word segmentation data; determines the corresponding data embedding vector based on each word segmentation data; calls a data enhancement architecture to generate a data embedding vector set based on the data embedding vector; and determines and outputs the corresponding data classification based on the data embedding vector and the data embedding vector set. This can improve the classification accuracy of the text data to be processed and improve the efficiency and accuracy of identifying illegal text data during doctor-patient communication.

[0055] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are provided to facilitate a better understanding of the present application and do not constitute an undue limitation on the present application.

[0057] Figure 1 This is a schematic diagram of the main process of a text data processing method provided according to one embodiment of the present application;

[0058] Figure 2 This is a schematic diagram of the main process of a text data processing method provided according to one embodiment of the present application;

[0059] Figure 3 This is a schematic diagram of an application scenario of a text data processing method provided according to an embodiment of the present application;

[0060] Figure 4 is a schematic diagram of main units of a text data processing device according to an embodiment of the present application;

[0061] Figure 5 is an exemplary system architecture diagram to which embodiments of the present application may be applied;

[0062] Figure 6 It is a structural diagram of a computer system of a terminal device or server suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0063] The following is an explanation of exemplary embodiments of the present application in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and they should be considered as merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that in the technical solutions disclosed herein, the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Take necessary measures for user personal information to prevent illegal access to user personal information data and maintain user personal information security, network security and national security.

[0064] At present, in the Internet medical scenario, illegal text data in the process of doctor-patient communication may have characteristics such as unclear illegal feature information and incomplete sentences, and the text classification accuracy is low, resulting in low recognition efficiency and accuracy of illegal text data in the process of doctor-patient communication.

[0065] Figure 1 This is a schematic diagram of the main process of the text data processing method provided according to an embodiment of the present application. Figure 1 As shown, the text data processing method includes:

[0066] Step S101: receiving a text data processing request and obtaining corresponding text data to be processed.

[0067] In this embodiment, the execution subject of the text data processing method (for example, it can be a server) can receive a text data processing request through a wired connection or a wireless connection. After receiving the text data processing request, the execution subject can obtain the data acquisition address carried in the request, and then jump to the data acquisition address to obtain the corresponding text data to be processed. Specifically, the text data to be processed can be, for example, text data of doctor-patient communication in an online consultation scenario, or text data of teacher-student communication in an online teaching scenario. The embodiment of the present application does not specifically limit the content of the text data to be processed. For example, the text data to be processed can be "Is there any way to contact you?"

[0068] Step S102 : performing word segmentation processing on the text data to be processed to obtain each word segmentation data.

[0069] Perform Chinese word segmentation on the text data to be processed, for example, extracting the subject, predicate, and object in the text data to be processed to obtain word segmentation data consisting of the extracted subject, predicate, and object. For example, the text data to be processed may be "Excuse me, is there any way I can contact you?", and the obtained word segmentation data may be, for example, "Excuse me," "Yes," "What," "Method," "Can," "Contact," "To," and "You."

[0070] Step S103: Determine the corresponding data embedding vector according to each word segmentation data.

[0071] Based on the word embedding method, each word segmentation data is converted into a data embedding vector. For example, "please ask", "have", "what", "method", "can", "contact", "to", and "you" are converted into data embedding vectors a1, a2, a3, a4, a5, a6, a7, and a8. After obtaining the converted data embedding vectors a1, a2, a3, a4, a5, a6, a7, and a8, the data enhancement architecture can be used to perform data enhancement on the converted data embedding vectors a1, a2, a3, a4, a5, a6, a7, and a8 based on vector similarity to obtain a data embedding vector set.

[0072] Data augmentation can be defined as calculating the similarity between a data embedding vector and other vectors to obtain a set of data embedding vectors consisting of other vectors whose similarity exceeds a preset threshold. Data augmentation allows for vector expansion based on the data embedding vector, resulting in more accurate processing of text data.

[0073] Specifically, according to each word segmentation data, a corresponding data embedding vector is determined, including: inputting each word segmentation data into a prediction model to output a corresponding data embedding vector.

[0074] For example, the prediction model may be a Bidirectional Encoder Representations from Transformers (BERT) model.

[0075] Specifically, before inputting each word segmentation data into the prediction model, the text data processing method also includes: obtaining a sample data set, performing mask preprocessing on the sample data set based on a preset vocabulary to obtain a masked sample data set; obtaining labels corresponding to the masked preprocessed data; using the masked sample data set as the input of the initial neural network model, using the corresponding labels as the expected output of the initial neural network model, and training the initial neural network model to obtain a prediction model.

[0076] The prediction model includes a masked language model (Masked Language Modeling). For example, when the text data processing request is to determine whether there are any illegal words in the text data to be processed, in order to enhance the feature information of illegal words that are not easily identified in the communication text (i.e., the text data to be processed), the strategy of randomly masking (MASK) words in the BERT model (i.e., the prediction model in the embodiment of the present application) is improved, and the illegal words are directed to be MASKed in the preprocessing stage, and the illegal words are masked using the MASK flag, so that the BERT model predicts the masked words, so as to train the BERT model and further enhance the probability of the BERT model predicting the MASKed illegal words according to the context, and finally the trained BERT model (i.e., the prediction model) can be used to predict whether the text data to be processed contains illegal words.

[0077] For example, the text data to be processed is "How can I contact you? This is my WeChat: 12345." "Contact" and "WeChat 12345" are the words that need to be masked. The words masked by the mask are illegal words. BERT is used to perform a cloze-like operation. When the BERT model can predict the illegal words masked by the mask, it means that other components in the sentence already contain the feature information of the illegal words. It also shows that the BERT model (i.e., the prediction model) can accurately predict whether the text data to be processed contains illegal words.

[0078] Step S104: calling the data enhancement architecture to generate a data embedding vector set based on the data embedding vector.

[0079] For example, a data embedding vector set can be a vector set {b1, b2, b3, b4, b5, b6, c1, c2, c3, c4, d1, d2, d3, d4} consisting of vectors b1, b2, b3, b4, b5, b6, c1, c2, c3, c4, d1, d2, d3, d4} corresponding to "I want to", "please ask", "your", "contact information", "is", and "what"; vectors c1, c2, c3, c4 corresponding to "I want to", "consult", "your", "contact number"; and vectors d1, d2, d3, d4 corresponding to "I want to", "add", and "your", "WeChat".

[0080] Specifically, after generating the data embedding vector set, the text data processing method further includes:

[0081] The data embedding vector set is input into the data augmentation architecture to generate the latest data embedding vector; and the data embedding vector set is updated based on the latest data embedding vector.

[0082] For example, conversation text data (i.e., text data to be processed) is often missing sentence components and has diverse expressions due to factors such as user input habits, which makes the accuracy of illegal text classification low. Therefore, it is necessary to diversify and enhance the text data.

[0083] In order to ensure the quality of diversified data, the embodiment of the present application adopts a data enhancement architecture (Transformer architecture) to fine-tune the BERT model to ensure that the newly generated illegal text data (i.e., the data embedding vector set of the embodiment of the present application) can retain the overall feature information of the original input real text data (i.e., the text data to be processed). At the same time, in order to diversify the newly generated illegal text data, a cyclic approach is adopted to input the word embedding vectors generated by the Transformer architecture (i.e., the data embedding vectors corresponding to each word segmentation data) into the Transformer architecture to generate batches of new illegal text data (i.e., batches of data embedding vector sets). The Transformer architecture in the embodiment of the present application is a model of an encoder-decoder structure in the field of natural language processing, including an encoder part and a decoder part. The encoder part can receive the data embedding vector obtained from the pre-trained BERT model (i.e., the prediction model in the embodiment of the present application), and obtain a vector whose similarity to the received data embedding vector is greater than a preset threshold value through the decoder part based on vector similarity calculation to generate batches of new illegal text data, i.e., batches of data embedding vector sets.

[0084] For example, the input illegal text (i.e., the text data to be processed) is first segmented into Chinese characters. The segmented data is then fed into a pre-trained BERT model to generate data embedding vectors. After the illegal text data (i.e., the text data to be processed) is embedded in BERT, it is then fed into the Transformer architecture to generate new illegal text (i.e., a set of data embedding vectors).

[0085] For the Transformer architecture of the embodiment of the present application, its input is the embedding vector of the real illegal text (i.e., the text data to be processed), and the output is the embedding vector of the new illegal text processed by the encoder and decoder. The output new illegal text (i.e., the data embedding vector set) and the input illegal text (i.e., the data embedding vector corresponding to each word segmentation data) have basically the same meaning, but are expressed in different ways. For example, the input is "Is there any way I can contact you?", and after being processed by the Transformer architecture, the output is "I would like to ask what your contact information is." The two have similar meanings, but are expressed in different ways.

[0086] Using the Transformer architecture, we encode and decode the embedding vectors of illegal text in real-world scenarios (i.e., the data embedding vectors corresponding to each word segmentation data). Simultaneously, we re-input the Transformer output into the Transformer, looping n times to generate n new pieces of text data (i.e., generating n new data embedding vectors), and maximizing the diversity of the representations of the illegal text (i.e., the data embedding vectors corresponding to each word segmentation data). Finally, we re-input all the illegal text data generated by the Transformer (i.e., the data embedding vectors obtained by looping over n words) into the pre-trained BERT model to generate the corresponding set of text embedding vectors.

[0087] Step S105: Determine and output the corresponding data classification based on the data embedding vector and the data embedding vector set.

[0088] Specifically, determining the corresponding data classification includes: inputting the data embedding vector and the data embedding vector set into the prediction model to determine the corresponding data classification based on the classification discrimination module in the prediction model.

[0089] For example, the data embedding vectors can be the vectors a1, a2, a3, a4, a5, a6, a7, and a8 corresponding to "please ask", "is there", "what", "method", "can", "contact", and "you". The data embedding vector set can be the vectors b1, b2, b3, b4, b5, and b6 corresponding to "I want to", "please ask", "your", "contact information", and "what"; the vectors c1, c2, c3, and c4 corresponding to "I want to", "consult", and "your", "contact number"; the vectors c1, c2, c3, and c4 corresponding to "I want to", "add", and "your", "WeChat"; The corresponding vectors d1, d2, d3, and d4 are composed of a vector set {b1, b2, b3, b4, b5, b6, c1, c2, c3, c4, d1, d2, d3, d4}, and the data embedding vectors a1, a2, a3, a4, a5, a6, a7, a8 and the data embedding vector set {b1, b2, b3, b4, b5, b6, c1, c2, c3, c4, d1, d2, d3, d4} are input into the prediction model to obtain the corresponding data classification, such as "violation" or "abnormal". For example, the classification and discrimination module in the prediction model detects that the input vector contains a vector that is the same or similar to the requested contact information, and therefore determines that the data classification corresponding to the input data embedding vector and the data embedding vector set is "violation" or "abnormal".

[0090] For example, the text embedding vectors in the original real scene generated by the BERT model (i.e., the data embedding vectors corresponding to each word segmentation data obtained from the text data to be processed) and the newly generated text embedding vector set (i.e., the data embedding vector set generated in the embodiment of the present application) are merged and input into a general classification and discrimination module for classification. According to the data embedding vectors obtained by the BERT model and the data embedding vector set obtained by the Transformer architecture, the illegal words can have more obvious feature information, which makes it easier for the classification and discrimination module to determine whether the sentence text in the text data to be processed is an illegal text, thereby effectively improving the detection effect of illegal communication text in online medical consultations.

[0091] This embodiment receives a text data processing request and obtains the corresponding text data to be processed; performs word segmentation on the text data to obtain individual word segmentation data; determines the corresponding data embedding vector based on each word segmentation data; invokes a data enhancement architecture to generate a data embedding vector set based on the data embedding vector; and determines and outputs the corresponding data classification based on the data embedding vector and the data embedding vector set. This improves the classification accuracy of the text data to be processed and enhances the efficiency and accuracy of identifying illegal text data during doctor-patient communication.

[0092] Figure 2 This is a schematic diagram of the main flow of a text data processing method provided according to an embodiment of the present application. Figure 2 As shown, the text data processing method includes:

[0093] Step S201: receiving a text data processing request and obtaining corresponding text data to be processed.

[0094] Step S202 : performing word segmentation processing on the text data to be processed to obtain each word segmentation data.

[0095] Step S203: Determine the corresponding data embedding vector according to each word segmentation data.

[0096] Specifically, according to each word segmentation data, the corresponding data embedding vector is determined, including:

[0097] Each word segmentation data is input into the prediction model to determine the correlation between each word segmentation data based on the next sentence prediction module in the prediction model; and a corresponding data embedding vector is generated based on each word segmentation data and the correlation.

[0098] The prediction model includes a next sentence prediction module, which is used to determine whether the feature vectors of the two sentences in the text data to be processed are similar. Since there is a strong logic and topicality between the previous and next sentences in the communication text, the BERT model is introduced to determine whether the next sentence is the next sentence of the previous sentence. At the same time, the task of determining whether the topics expressed by the two sentences belong to similar categories is added to ensure that the feature vectors of the two sentences are similar. For example, the text data to be processed is "Is there any way I can contact you? This is my WeChat: 12345". Through the analysis of the next sentence prediction module, we get "Is there any way I can contact you? <sep>This is my WeChat: 12345 <sep>".use <sep>The flag indicates the end of each sentence. The BERT model determines that the latter sentence is the context of the previous sentence and that the two sentences belong to the same topic, enabling BERT to learn the embedding vector of text data with question-answer relationship.

[0099] The above-mentioned adaptive and innovative tasks can be applied to the pre-training process of the BERT model, enabling the model to learn text data embedding vectors with the characteristics of online medical consultation and patient communication text, paving the way for the subsequent fine-tuning process.

[0100] Step S204 , calling the data enhancement framework and calling the preset data embedding vector library to determine the similarity between each preset data embedding vector in the preset data embedding vector library and the data embedding vector.

[0101] The data enhancement architecture is called to calculate the cosine similarity between each preset data embedding vector in the preset data embedding vector library and the data embedding vector corresponding to each word segmentation data of the text data to be processed based on the called preset data embedding vector library.

[0102] Step S205: Determine the target preset data embedding vector according to the similarity.

[0103] The preset data embedding vector corresponding to the calculated cosine similarity being greater than the preset threshold is determined as the target preset data embedding vector.

[0104] Step S206: Generate a data embedding vector set based on the target preset data embedding vector.

[0105] The target preset data embedding vector is a vector that has a high similarity with the data embedding vector corresponding to each word segmentation data of the text data to be processed. These vectors are aggregated to obtain a data embedding vector set.

[0106] In addition, the execution entity can also re-input these vectors with high similarity to the data embedding vectors corresponding to each word segmentation data of the text data to be processed into the data enhancement architecture, and further output similar vectors. This is repeated n times to obtain the final data embedding vector set.

[0107] Step S207: Determine and output the corresponding data classification based on the data embedding vector and the data embedding vector set.

[0108] The data embedding vector and the data embedding vector set are input into the discrimination module, so that the discrimination module determines the corresponding data classification and outputs it to the page browsed by the user through the display screen. The embodiment of the present application can improve the accuracy of the classification of the text data to be processed and improve the efficiency and accuracy of identifying illegal text data during the doctor-patient communication process.

[0109] Figure 3 This is a schematic diagram of an application scenario of a text data processing method provided by an embodiment of the present application. The text data processing method of the embodiment of the present application can be applied to the scenario of detecting whether the text data to be processed generated during the communication between doctors and patients contains illegal text. Figure 3 As shown, the embodiments of the present application involve a pre-training module using the BERT model and a text enhancement module using Transformer for text diversification.

[0110] For example, we first pre-trained the BERT model using a sample conversation text dataset. This pre-processed dataset replaced the BERT model's original random mask strategy with a directed mask strategy. We also improved the BERT model's next sentence prediction task. This data was then fed into the BERT model, allowing the model to learn the characteristic information of the offending text dataset, which is reflected in the text data's embedding vector.

[0111] Then, the illegal text data in the real doctor-patient communication scenario is processed with Chinese word segmentation and then input into the pre-trained BERT model to generate the corresponding text data embedding vector.

[0112] After passing through the judgment logic, it enters the Transformer module, and through a loop, it enhances the text embedding vectors in the original real scene, and then batch generates a diverse set of embedding vectors for illegal text data. This is then input into the BERT model and the vector set is fine-tuned so that the prediction of illegal words in the enhanced text data can adapt to different forms of expression.

[0113] Finally, the illegal text data embedding vectors in the real scene and the enhanced illegal text data embedding vector set are all input into the discrimination module, which can make a more comprehensive and effective judgment on the text data corresponding to the processed text data and improve the effect of illegal text detection.

[0114] This embodiment of the application improves the random mask strategy in the BERT model by masking keywords, sentimental words, and custom offending words in the dataset, thereby enhancing the feature information of this part. This allows text without obvious offending words to have its offending feature information visible. The Transformer architecture is introduced to perform diversified processing on real text data, thereby simulating text data with missing sentence components and diverse expressions, thereby enhancing the text data and making the BERT model's processing capabilities more robust.

[0115] In the embodiments of the present application, BERT: an emerging text pre-training model in the field of NLP. Transformer: a model with an encoder-decoder structure in the field of NLP. Mask Language Modeling: the masked language model prediction task, which refers to using a masking token to mask the tokens in a sequence, and then trying to restore the masked tokens based on the remaining part of the sentence. Next Sentence Prediction: the next sentence prediction task, which generally refers to two sentences, and the task of judging whether the second sentence is the next sentence of the first sentence, and then obtaining the ability of sentence-level representation. Encoder: Encoder, compresses data to facilitate machine understanding. Decoder: Decoder, expands data to facilitate human understanding.

[0116] In order to solve the problems existing in the data set itself, the embodiment of the present application uses the BERT pre-training model, which can not only retain the feature information of the word itself, but also retain the feature information of the relationship between the word and the whole sentence. At the same time, its bidirectional mechanism can better express the relationship between the offending word and the context; a directional mask method is adopted to replace the offending word with a mask flag, so that the BERT model can predict the masked words, enhance the feature information of the offending word and the information that other components in the sentence contain the offending word; the Transformer module is used for loop processing, and text data with similar meaning is generated in batches from the source offending text, increasing the diversity of the BERT model input, so that the model can better learn texts with missing sentence components and diverse expressions during pre-training, and thus can achieve more accurate classification, thereby improving the recognition efficiency and accuracy of offending text data during doctor-patient communication.

[0117] Figure 4 Schematic diagram of the main units of the text data processing device according to the embodiment of the present application. Figure 4 As shown, the text data processing device 400 includes a receiving unit 401, a word segmentation data determination unit 402, a data embedding vector determination unit 403, a data embedding vector set determination unit 404 and a data classification determination unit 405.

[0118] The receiving unit 401 is configured to receive a text data processing request and obtain corresponding text data to be processed;

[0119] The word segmentation data determining unit 402 is configured to perform word segmentation processing on the text data to be processed to obtain each word segmentation data;

[0120] The data embedding vector determining unit 403 is configured to determine a corresponding data embedding vector according to each word segmentation data;

[0121] The data embedding vector set determining unit 404 is configured to call the data enhancement architecture to generate a data embedding vector set based on the data embedding vector;

[0122] The data classification determination unit 405 is configured to determine and output the corresponding data classification according to the data embedding vector and the data embedding vector set.

[0123] In some embodiments, the data embedding vector determining unit 403 is further configured to: input each word segmentation data into the prediction model to output the corresponding data embedding vector.

[0124] In some embodiments, the text data processing apparatus further includes Figure 4 The training unit not shown in the figure is configured to: obtain a sample data set, perform mask preprocessing on the sample data set based on a preset vocabulary to obtain a masked sample data set; obtain labels corresponding to the masked preprocessed data; use the masked sample data set as the input of the initial neural network model, use the corresponding labels as the expected output of the initial neural network model, and train the initial neural network model to obtain a prediction model.

[0125] In some embodiments, the data embedding vector set determination unit 404 is further configured to: call a preset data embedding vector library to determine the similarity between each preset data embedding vector in the preset data embedding vector library and the data embedding vector; determine a target preset data embedding vector based on the similarity; and generate a data embedding vector set based on the target preset data embedding vector.

[0126] In some embodiments, the text data processing apparatus further includes Figure 4 The updating unit not shown in the figure is configured to: input the data embedding vector set into the data enhancement architecture to generate the latest data embedding vector; and update the data embedding vector set based on the latest data embedding vector.

[0127] In some embodiments, the data classification determination unit 405 is further configured to: input the data embedding vector and the data embedding vector set into the prediction model to determine the corresponding data classification based on the classification discrimination module in the prediction model.

[0128] In some embodiments, the data embedding vector determination unit 403 is further configured to: input each word segmentation data into the prediction model to determine the correlation between each word segmentation data based on the next sentence prediction module in the prediction model; generate a corresponding data embedding vector based on each word segmentation data and the correlation.

[0129] It should be noted that the text data processing method and the text data processing device of the present application have corresponding relationships in terms of specific implementation contents, so the repeated contents will not be described again.

[0130] Figure 5 An exemplary system architecture 500 is shown to which the text data processing method or text data processing apparatus according to the embodiments of the present application can be applied.

[0131] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0132] Users can use terminal devices 501, 502, and 503 to interact with server 505 via network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, and 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0133] The terminal devices 501 , 502 , and 503 may be various electronic devices having a text data processing screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0134] Server 505 can be a server that provides various services, such as a background management server (only as an example) that supports text data processing requests submitted by users using terminal devices 501, 502, and 503. The background management server can receive text data processing requests and obtain the corresponding text data to be processed; perform word segmentation processing on the text data to be processed to obtain each word segmentation data; determine the corresponding data embedding vector based on each word segmentation data; call the data enhancement architecture to generate a data embedding vector set based on the data embedding vector; determine the corresponding data classification and output it based on the data embedding vector and the data embedding vector set. This can improve the classification accuracy of the text data to be processed and improve the efficiency and accuracy of identifying illegal text data during medical communication.

[0135] It should be noted that the text data processing method provided in the embodiment of the present application is generally executed by the server 505 , and accordingly, the text data processing device is generally set in the server 505 .

[0136] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0137] Reference below Figure 6 , which shows a structural diagram of a computer system 600 of a terminal device suitable for implementing an embodiment of the present application. Figure 6 The terminal device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0138] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the computer system 600 are also stored in the RAM 603. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0139] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, and the like; an output section 607 including displays such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN card or a modem. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. Removable media 611, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 610 as needed, so that computer programs read therefrom can be installed into the storage section 608 as needed.

[0140] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from a removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present application are executed.

[0141] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can include, for example, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] The units involved in the embodiments described in this application can be implemented by software or hardware. The units described can also be set in a processor. For example, it can be described as: a processor includes a receiving unit, a word segmentation data determination unit, a data embedding vector determination unit, a data embedding vector set determination unit, and a data classification determination unit. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0144] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device receives a text data processing request and obtains corresponding text data to be processed; performs word segmentation processing on the text data to be processed to obtain each word segmentation data; determines the corresponding data embedding vector based on each word segmentation data; calls the data enhancement architecture to generate a data embedding vector set based on the data embedding vector; determines the corresponding data classification and outputs it based on the data embedding vector and the data embedding vector set.

[0145] According to the technical solution of the embodiment of the present application, the classification accuracy of the text data to be processed can be improved.

[0146] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.< / sep> < / sep> < / sep>

Claims

1. A text data processing method, characterized in that: include: Receive a text data processing request and obtain corresponding text data to be processed; Performing word segmentation processing on the text data to be processed to obtain each word segmentation data; Determine a corresponding data embedding vector according to each word segmentation data; Invoking a data augmentation architecture to generate a set of data embedding vectors based on the data embedding vectors; According to the data embedding vector and the data embedding vector set, a corresponding data classification is determined and output.

2. The method according to claim 1, characterized in that The step of determining the corresponding data embedding vector according to each word segmentation data includes: The individual word segmentation data are input into the prediction model to output the corresponding data embedding vector.

3. The method according to claim 2, characterized in that Before inputting each word segmentation data into the prediction model, the method further includes: Acquire a sample data set, and perform mask preprocessing on the sample data set based on a preset vocabulary to obtain a masked sample data set; Obtain the label corresponding to the data that has been subjected to mask preprocessing; The mask sample data set is used as the input of the initial neural network model, the corresponding label is used as the expected output of the initial neural network model, and the initial neural network model is trained to obtain a prediction model.

4. The method according to claim 1, characterized in that The generating of the data embedding vector set comprises: Calling a preset data embedding vector library to determine the similarity between each preset data embedding vector in the preset data embedding vector library and the data embedding vector; Determining a target preset data embedding vector according to the similarity; Based on the target preset data embedding vector, a data embedding vector set is generated.

5. The method according to claim 1, characterized in that After generating the data embedding vector set, the method further includes: Inputting the data embedding vector set into the data augmentation framework to generate an updated data embedding vector; The data embedding vector set is updated based on the latest data embedding vector.

6. The method according to claim 1, characterized in that The determining of the corresponding data classification includes: The data embedding vector and the data embedding vector set are input into a prediction model to determine a corresponding data classification based on a classification discrimination module in the prediction model.

7. The method according to claim 1, characterized in that The step of determining the corresponding data embedding vector according to each word segmentation data includes: Inputting each of the word segmentation data into a prediction model to determine the correlation between each of the word segmentation data based on a next sentence prediction module in the prediction model; Generate a corresponding data embedding vector based on the individual word segmentation data and the association degree.

8. A text data processing device, characterized in that: include: A receiving unit, configured to receive a text data processing request and obtain corresponding text data to be processed; A word segmentation data determination unit is configured to perform word segmentation processing on the text data to be processed to obtain each word segmentation data; A data embedding vector determining unit is configured to determine a corresponding data embedding vector according to each word segmentation data; a data embedding vector set determining unit, configured to call a data enhancement architecture to generate a data embedding vector set based on the data embedding vector; The data classification determination unit is configured to determine and output the corresponding data classification according to the data embedding vector and the data embedding vector set.

9. The device according to claim 8, characterized in that The data embedding vector determining unit is further configured to: The individual word segmentation data are input into the prediction model to output the corresponding data embedding vector.

10. The device according to claim 9, characterized in that The apparatus further comprises a training unit configured to: Acquire a sample data set, and perform mask preprocessing on the sample data set based on a preset vocabulary to obtain a masked sample data set; Obtain the label corresponding to the data that has been subjected to mask preprocessing; The mask sample data set is used as the input of the initial neural network model, the corresponding label is used as the expected output of the initial neural network model, and the initial neural network model is trained to obtain a prediction model.

11. An electronic device for processing text data, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

12. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images