Data processing method and system based on machine learning

Abstract

The invention relates to the technical field of government affair data processing, in particular to a data processing method and system based on machine learning. When an attention mechanism is adopted to perform text feature extraction on a bidirectional long-short-term memory network model, a traditional attention mechanism weight calculation formula is improved; the word frequency is introduced as a calculation index of the weight, so that the assignment of the weight of the word in the attention mechanism focusing process is more accurate; meanwhile, the fusion weight of the word is determined according to the occurrence frequency, the effect of the key word in the text vector can be effectively highlighted, and the correlation between the word with the low occurrence frequency and the central thought of the government affair system is relatively weak, so that the extraction time of the text vector can be effectively prolonged by adopting the fixed weight to carry out word segmentation fusion; according to the text characteristics of the government affair system, the segmented words are fused in a segmented weight fusion mode when the segmented words are fused, so that the extraction accuracy of the text vector is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of government data processing, and in particular to a data processing method and system based on machine learning. Background Art

[0002] In recent years, with the development of computer technology, the level of information construction in various industries has also improved. Government departments have paid more and more attention to the government's information construction due to data security and intelligent government processing. Obviously, the government's information construction must rely on electronic information and digital network technology. As a key link in the government's information construction, the realization of e-government business is not simply to migrate traditional government management affairs and related data from paper to the Internet, but to use Internet technology to give them a second life. Government-related management affairs need to reorganize the organizational structure and reengineer the business process on the Internet. In simple terms, it is necessary to reshape the business in an information-based way; and the data generated by the government in management and operation also needs to be analyzed and used more intelligently on the basis of reintegration and storage.

[0003] In the government and medium and large enterprises, the information that currently exists and is continuously generated is mainly text information. How to organize text information itself is a complex and time-consuming process. Therefore, how to obtain valuable information for users from a large amount of complex text information is the core goal of the text mining field. Text mining is an emerging field that covers a variety of technologies. It can use computer processing technology to extract valuable information and knowledge from text data, and use the extracted knowledge to better organize information for further use. This process is similar to the process of human learning knowledge and applying it. Its implementation technologies include data mining technology, information retrieval, machine learning, natural language processing (NLP), computational linguistics, linear geometry, probability theory, etc. The most direct way to express text data is language. Everything can be expressed through language. Government and enterprise text data is a form of expression of natural language. From this perspective, natural language processing is the most suitable means to achieve communication between government text data and computers.

[0004] As people continue to explore deep learning, the difficulties in the field of natural language processing have also been continuously broken through. Through natural language processing, communication between people and machines can be achieved. In the field of text analysis, NLP has made great contributions. By combining NLP with text analysis, it can help governments and enterprises obtain more important information in government and enterprise text big data, thereby generating huge data value. There are technical solutions for government data processing based on machine learning in the prior art. For example, the Chinese invention patent (CN115905516A) discloses a government data identification method and system. First, the table information text of the table file of government data is obtained, and the table information text includes the name information text; then the name information text of the table file is segmented to obtain several candidate content tags, and the weight of each candidate content tag is calculated based on the TF-IDF algorithm; finally, based on the weight of each candidate content tag, the content tag of the government data is identified.

[0005] However, in the prior art, when using deep learning models to process government text data, there is a problem of low processing accuracy, which in turn affects the classification accuracy. Summary of the invention

[0006] In view of this, the present invention proposes a data processing method and system based on machine learning, which are used to improve the accuracy of processing government text data using a deep learning model.

[0007] In order to achieve the above purpose, a data processing method based on machine learning is provided.

[0008] S1: Obtain text data to be processed;

[0009] S2: performing a preprocessing operation on the text data to be processed;

[0010] S3: extracting the text vector of the text data to be processed; S3 specifically comprises:

[0011] S3.1: Using the CBOW model to extract the word segmentation of the text data to be processed;

[0012] S3.2: fusing the word segmentations to generate a text vector of the text data to be processed;

[0013] S4: extracting features of the text data to be processed using a machine learning algorithm;

[0014] The machine learning algorithm is a bidirectional long short-term memory neural network with an attention mechanism. The weight formula of the word calculated by the attention mechanism is:

[0015]

[0016] In the formula, a ij is the degree of attention of word i to word j, e ij is the influence of word j on the current word i, T x is the set of input words, e jk is the influence of word k on the current word i; f i is the frequency of word i.

[0017] Preferably, in S1, the text data to be processed may be petition text data or proposal text data.

[0018] Preferably, in S2, the preprocessing operation includes special character processing, word segmentation, and stop word deletion operations.

[0019] Preferably, the word segmentation method is a maximum matching method.

[0020] Preferably, a stop word list is used to remove stop words in the text data to be processed.

[0021] Preferably, in S3.1, the CB0W model includes an Embedding layer, a Reshape layer, a Dot layer, a Dense layer, and a Model layer, wherein the Embedding layer is used to map vocabulary indexes to word vectors, the Reshape layer is used to adjust the shape of the word vectors for dot product calculations, the Dot layer is used to calculate the dot product of the target word and the context word, the Dense layer is the output layer, and the sigmoid activation function is used to convert the dot product results into probabilities, and the Model layer is used to define and compile the model, and specify the input, output, loss function, and optimizer.

[0022] Preferably, in S3.2, the word segmentations are fused by means of segmented weight fusion, specifically: the occurrence frequency of the word segmentations is counted; if the occurrence frequency of the word segmentations is greater than a first threshold, the occurrence frequency of the word segmentations is used as the fusion weight of the word segmentations; otherwise, a fixed weight is used as the fusion weight of the word segmentations.

[0023] Preferably, the first threshold is 0.05 and the fixed weight is 0.01.

[0024] Preferably, in S4, the bidirectional long short-term memory neural network (BiLSTM) consists of an input layer, a forward LSTM layer, a backward LSTM layer and an output layer.

[0025] According to another aspect of the present invention, a data processing system based on machine learning is provided, wherein the system adopts the above-mentioned data processing method based on machine learning, and the system comprises:

[0026] A text acquisition module, used to acquire text data to be processed;

[0027] A preprocessing module, used for performing a preprocessing operation on the text data to be processed;

[0028] A text vector extraction module, used for extracting the text vector of the text data to be processed;

[0029] The machine learning algorithm module is used to extract the features of the text data to be processed by using a machine learning algorithm. The advantages and beneficial effects of the present invention are:

[0030] When the attention mechanism is used to extract text features from a bidirectional long short-term memory network model, the present invention improves the traditional attention mechanism weight calculation formula and introduces word frequency as a weight calculation indicator, thereby making the weight assignment of words more accurate during the attention mechanism focusing process.

[0031] At the same time, determining the fusion weight of the word according to the frequency of occurrence can effectively highlight the role of key words in the text vector, while words with lower frequency of occurrence have relatively weak correlation with the central idea of ​​the government system. Therefore, using fixed weights for word segmentation fusion can effectively improve the extraction time of the text vector. This embodiment uses segmentation weight fusion to fuse the word segmentations according to the text characteristics of the government system, so as to improve the extraction accuracy of the text vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the description of the present invention or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 A flowchart of a data processing method based on machine learning provided by an embodiment of the present invention;

[0034] Figure 2 A flowchart of extracting text vectors of the text data to be processed provided by an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of a CBOW model provided in an embodiment of the present invention;

[0036] Figure 4 A schematic diagram of a bidirectional long short-term memory neural network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] As attached Figure 1 As shown, a data processing method based on machine learning comprises the following steps:

[0039] S1: Obtain text data to be processed;

[0040] Acquiring text data to be processed based on the text to be processed, the text data to be processed being the data form of the text to be processed input into the electronic device. In this embodiment, data processing is performed on government data, and specifically, the text data to be processed may be petition text data or proposal text data;

[0041] S2: performing a preprocessing operation on the text data to be processed;

[0042] Among them, the preprocessing operation is to convert the text data to be processed into data in a form that can be understood by a computer. For massive government big data, words or symbols that have no effect on the processing results are removed, such as "Dear XX Leader", etc. This will make the content of the text to be processed more concise and clear, avoid wasting computing resources in the training stage, and make the processing effect more effective.

[0043] The preprocessing operation includes special character processing, word segmentation, and stop word deletion.

[0044] Chinese text data often contains many characters that have no practical meaning to the text itself, such as spaces and characters such as ",",",",";", etc. Therefore, these special characters need to be filtered out;

[0045] For languages ​​of different countries, the means of word segmentation are different to varying degrees. In English text, words are separated by spaces, which are easy to extract; in Chinese text, there are no inherent separation marks between vocabulary phrases, which are difficult to distinguish. Therefore, word segmentation techniques such as maximum matching method must be adopted to clean the original text data; the main principle of maximum matching method is to segment single word strings and then compare them with the vocabulary. If it is a word, it is recorded, otherwise, by adding or reducing a single word, the comparison continues, and it is terminated until there is only one single word left; in this embodiment, the word segmentation method is maximum matching method;

[0046] Stop words refer to words that often appear in Chinese texts but have no actual meaning, such as "的" (de), "就" (jiu), "呀" (ya), "啊" (a), etc. Appropriately reducing stop words can help increase the density of keywords and improve the efficiency of text processing at the same time; in this embodiment, a stop word list is used to remove the stop words from the text data to be processed.

[0047] S3: Extract the text vector of the text data to be processed;

[0048] A text vector represents the text to be processed as numerous vectors that can express the semantics of the text. The vectorized text can be recognized and calculated by a text classification model;

[0049] In this embodiment, as shown in the appendix Figure 2 it is shown that the specific steps of S3 are as follows:

[0050] S3.1: Use the CBOW (Continuous Bag-of-Words) model to extract the word segments of the text data to be processed;

[0051] As shown in the appendix Figure 3 it is shown that the CBOW model includes an Embedding layer, a Reshape layer, a Dot layer, a Dense layer, and a Model layer. Among them, the Embedding layer is used to map the vocabulary index to a word vector, the Reshape layer is used to adjust the shape of the word vector for dot product calculation, the Dot layer is used to calculate the dot product of the target word and the context words, the Dense layer is the output layer, using the sigmoid activation function to convert the dot product result into a probability, and the Model layer is used to define and compile the model, specifying the input, output, loss function, and optimizer;

[0052] S3.2: Fuse the word segments to generate the text vector of the text data to be processed;

[0053] Among them, the word segments are fused in a segmented weight fusion manner, specifically: count the occurrence frequency of the word segments. If the occurrence frequency of the word segment is greater than the first threshold, then use the occurrence frequency of the word segment as the fusion weight of the word segment. Otherwise, use a fixed weight as the fusion weight of the word segment;

[0054] In this embodiment, the first threshold is 0.05, and the fixed weight is 0.01.

[0055] In this step, the text of the government affairs system generally has fixed writing standards and writing methods. For words with higher frequency of occurrence, they generally have a strong correlation with the central idea of ​​the government affairs system. Therefore, determining the fusion weight of the word according to the frequency of occurrence can effectively highlight the role of the key words in the text vector, while the words with lower frequency of occurrence have a relatively weak correlation with the central idea of ​​the government affairs system. Therefore, using fixed weights for word segmentation fusion can effectively improve the extraction time of the text vector. According to the text characteristics of the government affairs system, this embodiment uses segmentation weight fusion to fuse the word segmentations when performing word segmentation fusion to improve the extraction accuracy of the text vector.

[0056] S4: extracting features of the text data to be processed using a machine learning algorithm;

[0057] In this step, the machine learning algorithm is a bidirectional long short-term memory neural network with an attention mechanism. The bidirectional long short-term memory neural network (BiLSTM) is a special recurrent neural network (RNN) that can process sequence data. BiLSTM considers both forward and backward information at each time step, solving the limitation that the traditional long short-term memory neural network (LSTM) can only process unidirectional information. Figure 4 As shown, the bidirectional long short-term memory neural network (BiLSTM) consists of an input layer, a forward LSTM layer, a backward LSTM layer and an output layer, wherein w i is the output parameter of each layer. The forward layer processes information from the beginning to the end of the sequence, while the backward layer processes information from the end to the beginning of the sequence. In this way, BiLSTM can better understand the context information of the sequence, thereby improving the performance of the model. The BiLSTM model has a wide range of applications in the field of natural language processing (NLP);

[0058] Among them, the bidirectional long short-term memory neural network cannot judge the importance of words in the entire text when extracting the features of the text data to be processed. Therefore, in this embodiment, an attention mechanism is introduced to assign weights to the text output features of each node of the bidirectional long short-term memory neural network, so that the key text features get more weights, which is beneficial to the subsequent text classification;

[0059] Specifically, the weight formula of the word calculated by the attention mechanism is:

[0060]

[0061] In the formula, a ij is the degree of attention of word i to word j, e ij is the influence of word j on the current word i, T x is the set of input words, ejk is the influence of word k on the current word i; f i is the frequency of word i;

[0062] In this formula, the traditional formula for calculating weights of the attention mechanism is improved, and word frequency is introduced as a calculation indicator of weights, so that the assignment of word weights in the process of focusing the attention mechanism is more accurate.

[0063] Embodiment 2: This embodiment includes a data processing system based on machine learning. The system adopts a data processing method based on machine learning in Embodiment 1. The system includes:

[0064] A text acquisition module, used to acquire text data to be processed;

[0065] A preprocessing module, used for performing a preprocessing operation on the text data to be processed;

[0066] A text vector extraction module, used for extracting the text vector of the text data to be processed;

[0067] The machine learning algorithm module is used to extract features of the text data to be processed using a machine learning algorithm.

[0068] Embodiment 3, this embodiment includes a computer-readable storage medium, on which a data processing program is stored, and the data processing program is executed by a processor. Embodiment 1 is a data processing method based on machine learning.

[0069] It will be appreciated by those skilled in the art that the embodiments herein may be provided as methods, devices (equipment), or computer program products. Therefore, this article may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Including but not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by a computer, etc. In addition, it is well known to those of ordinary skill in the art that communication media generally contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0070] This document is described with reference to flowcharts and / or block diagrams of methods, apparatuses (devices) and computer program products according to the embodiments of this document. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0071] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0072] The embodiments and / or implementation methods described above are only used to illustrate the preferred embodiments and / or implementation methods of the technology of the present invention, and are not intended to impose any formal restrictions on the implementation methods of the technology of the present invention. Any person skilled in the art may make some changes or modifications to other equivalent embodiments without departing from the scope of the technical means disclosed in the content of the present invention, but they should still be regarded as the technology or embodiments that are essentially the same as the present invention. Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described here, and that various obvious changes, readjustments and substitutions can be made to those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention is described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data processing method based on machine learning, characterized in that: The method comprises the following steps: S1: Obtain text data to be processed; S2: performing a preprocessing operation on the text data to be processed; S3: extracting the text vector of the text data to be processed; The S3 is specifically as follows: S3.1: extracting word segments of the text data to be processed by using the CBOW model; S3.2: fusing the word segments to generate a text vector of the text data to be processed; S4: extracting features of the text data to be processed using a machine learning algorithm; The machine learning algorithm is a bidirectional long short-term memory neural network with an attention mechanism. The weight formula of the word calculated by the attention mechanism is: In the formula, a ij is the degree of attention of word i to word j, e ij is the influence of word j on the current word i, T x is the set of input words, e jk is the influence of word k on the current word i; f i is the frequency of word i.

2. A data processing method based on machine learning according to claim 1, characterized in that: In S1, the text data to be processed may be petition text data or proposal text data.

3. The data processing method based on machine learning according to claim 1, characterized in that: In S2, the preprocessing operation includes special character processing, word segmentation, and stop word deletion operations.

4. The data processing method based on machine learning according to claim 3, characterized in that: The word segmentation method is the maximum matching method.

5. The data processing method based on machine learning according to claim 3, characterized in that: A stop word list is used to remove stop words in the text data to be processed.

6. The data processing method based on machine learning according to claim 1, characterized in that: In S3.1, the CB0W model includes an Embedding layer, a Reshape layer, a Dot layer, a Dense layer, and a Model layer, wherein the Embedding layer is used to map vocabulary indexes to word vectors, the Reshape layer is used to adjust the shape of the word vectors for dot product calculations, the Dot layer is used to calculate the dot product of the target word and the context word, the Dense layer is the output layer, and the sigmoid activation function is used to convert the dot product results into probabilities, and the Model layer is used to define and compile the model, and specify the input, output, loss function, and optimizer.

7. The data processing method based on machine learning according to claim 1, characterized in that: In S3.2, the word segmentations are fused by means of segmented weight fusion, specifically: the occurrence frequency of the word segmentations is counted; if the occurrence frequency of the word segmentations is greater than a first threshold, the occurrence frequency of the word segmentations is used as the fusion weight of the word segmentations; otherwise, a fixed weight is used as the fusion weight of the word segmentations.

8. The data processing method based on machine learning according to claim 7, characterized in that: The first threshold is 0.05, and the fixed weight is 0.

01.

9. The data processing method based on machine learning according to claim 1, characterized in that: In S4, the bidirectional long short-term memory neural network consists of an input layer, a forward LSTM layer, a backward LSTM layer and an output layer.

10. A data processing system based on machine learning, characterized in that: The system adopts a data processing method based on machine learning according to any one of claims 1 to 9, and the system comprises: A text acquisition module, used to acquire text data to be processed; A preprocessing module, used for performing preprocessing operations on the text data to be processed; The text vector extraction module is used to extract the text vector of the text data to be processed; the machine learning algorithm module is used to extract the features of the text data to be processed by adopting a machine learning algorithm.

Images