Semantic parsing method, system, device and readable storage medium
Through the deep learning model combined with BERT and bidirectional GRU models, the integrated learning of semantic analytical methods and entity recognition is realized, solving the accuracy of information matching in massive data in the public security field, and achieving rapid and convenient access from massive data The effect of matching the relevant data.
Patent Information
- Application Number
- CN202110073434.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-20
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-01-20
AI Technical Summary
The existing semantic analysis methods cannot accurately find matching information from massive data sources and cannot meet the business needs of the public security field.
Using a deep learning model, combining BERT and bidirectional GRU models, the integrated learning of intention recognition and entity recognition is achieved by building a mapping table and objective function, and colloquial questions are analyzed and related data in the database are matched.
It realizes convenient, accurate and fast matching of relevant data that meets the intention from massive data, improving the accuracy and efficiency of data matching.
Smart Images

Figure CN112765993B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of language processing technology, and specifically relates to a semantic parsing method, system, device and readable storage medium. Background Art
[0002] At present, the public security field has initially established an application system based on big data that basically covers its business scenarios, and has generated a huge amount of business data in the actual use process. These data play a vital role in the research and judgment of the public security and daily work. However, the existing semantic parsing methods cannot meet the service demand of accurately finding matching information from massive data sources. Therefore, how to develop a new semantic parsing method to overcome the above problems is a direction that technicians in this field need to study. Summary of the invention
[0003] The object of the present invention is to provide a semantic parsing method that can parse spoken questions, convert them into database language, and match data that meets its intention.
[0004] A semantic parsing method comprises the following steps:
[0005] Step 1: Collect and clean the data to be processed;
[0006] Step 2: Build a deep learning model, input the data to be processed into the deep learning model, extract elements from the data to be processed, and obtain the data category of the data to be processed;
[0007] Step 3: Match the mapping table corresponding to the data category, input the extracted elements into the mapping table for mapping, and obtain the semantic logic corresponding to the data to be processed;
[0008] Step 4: Match the parsed data that conforms to the semantic logic in the database and output the parsed data.
[0009] Preferably, in the above semantic parsing method: step 1 comprises the following steps:
[0010] Step 11: Collect data to be processed;
[0011] Step 12: Remove duplicate values from the data to be processed based on Python or ETL tools;
[0012] Step 13: Remove outliers from the data to be processed based on Python or ETL tools;
[0013] Step 14: Remove useless values from the data to be processed based on Python or ETL tools;
[0014] Step 15: Construct a mapping table and establish a corresponding relationship between data classification and the mapping table.
[0015] Preferably, in the above semantic parsing method: the deep learning model in step 2 includes a serial BERT model and a bidirectional GRU model;
[0016] The objective function of the deep learning model is:
[0017]
[0018] The ultimate goal of learning is to maximize p(y|x), where p(y r |x) is the loss function corresponding to intent recognition, and the cross entropy loss function is used to classify the entire sentence; The loss function corresponding to entity recognition uses the cross entropy loss function to extract and recognize a single word. The objective function combines intent recognition and entity recognition in the form of a product, ensuring that the model not only maximizes the accuracy of intent recognition, but also maximizes the accuracy of entity recognition during the learning process.
[0019] In order to implement the above semantic analysis method, the present application also discloses a semantic parsing system, and its technical solution is as follows:
[0020] A semantic parsing system, comprising: a preprocessing module, a learning module, a mapping module and a parsing module;
[0021] The preprocessing module is used to collect and clean the data to be processed;
[0022] The learning module is used to construct a deep learning model, input the data to be processed into the deep learning model, extract elements from the data to be processed, and obtain the data category of the data to be processed;
[0023] The mapping module is used to match a mapping table corresponding to the data category, input the extracted elements into the mapping table for mapping, and obtain the semantic logic corresponding to the data to be processed;
[0024] The parsing module is used to match parsing data that conforms to the semantic logic in the database and output the parsing data.
[0025] The present application also discloses an electronic device, which includes: a processor and a memory, wherein the memory is used to store programs; the processor and the memory are connected to call the program stored in the memory and execute the above-mentioned semantic parsing method.
[0026] The present application further discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is run by a computer, the above-mentioned semantic parsing method is executed.
[0027] Compared with the prior art, the technical solution of this application has the following advantages:
[0028] The BERT model uses a self-encoding method to integrate the vector information of other characters into its own vector information when updating each character vector, and performs weighted averaging, so that each character not only contains its own semantic information, but also presents different semantic information in different contexts by learning the semantic information of the context. At the same time, in view of the sequential characteristics of Chinese sentences, a GRU model that can retain sequence information is introduced to encode the output of the BERT model at a deeper level. At the same time, in order to alleviate the problem of duration dependence, a bidirectional GRU model is adopted, that is, a GRU is performed once in the forward and reverse directions, and the forward and reverse results are spliced as the final output result. At the same time, by setting a specific loss function for the serial BERT model and the bidirectional GRU model, the loss of intent recognition and entity recognition is integrated and combined in the encoding process and learned in the same model, avoiding the error propagation problem that may be generated after the serial combination of intent recognition and entity recognition. The present invention uses a deep learning method to parse unstructured natural language questions, which ultimately enables the public security to match relevant data that meets its intentions more conveniently, accurately and quickly from a large number of data sources. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] The present application is further described in detail below with reference to the accompanying drawings and specific implementation methods:
[0030] Figure 1 It is a workflow diagram of the semantic parsing method in the present invention;
[0031] Figure 2 It is a module block diagram of the semantic parsing system in the present invention;
[0032] Figure 3 It is a module block diagram of the electronic device in the present invention. DETAILED DESCRIPTION
[0033] In order to more clearly illustrate the technical solution of the present application, further description will be given below in conjunction with various embodiments.
[0034] like Figure 1 Example 1 is shown:
[0035] A semantic parsing method comprises the following steps:
[0036] Step 1: Collect and clean the data to be processed;
[0037] Specifically, the collection and cleaning of the data to be processed includes the following steps:
[0038] Step 11: Collect data to be processed;
[0039] Step 12: Remove duplicate values from the data to be processed;
[0040] Step 13: Remove outliers from the data to be processed;
[0041] Step 14: Remove useless values from the data to be processed;
[0042] Step 15: Build a mapping table and establish the corresponding relationship between data classification and mapping table
[0043] Step 2: Build a deep learning model, input the data to be processed into the deep learning model, extract elements from the data to be processed, and obtain the data category of the data to be processed;
[0044] Wherein, the deep learning model includes a serial BERT model and a bidirectional GRU model;
[0045] The objective function of the deep learning model is:
[0046]
[0047] The ultimate goal of learning is to maximize p(y|x), where p(y r |x) is the loss function corresponding to intent recognition, and the cross entropy loss function is used to classify the entire sentence; The loss function corresponding to entity recognition uses the cross entropy loss function to extract and recognize a single word. The objective function combines intent recognition and entity recognition in the form of a product, ensuring that the model not only maximizes the accuracy of intent recognition, but also maximizes the accuracy of entity recognition during the learning process.
[0048] Among them, special elements such as ID cards and mobile phone numbers are identified by pattern matching because they have strong regular information. Specifically: by collecting the regularity of elements and formulating relevant rules that conform to the regularity, the element types that conform to the relevant rules are extracted.
[0049] Step 3: Match the mapping table corresponding to the data category, input the extracted elements into the mapping table for mapping, and obtain the semantic logic corresponding to the data to be processed;
[0050] Step 4: Match the parsed data that conforms to the semantic logic in the database and output the parsed data.
[0051] Embodiment 2: Perform semantic analysis on natural language questions.
[0052] First, data collection was carried out. The collected data included: "Missing people in Shanghai", "Missing people in Shanghai", "Hello", and "Missing people in the United States". After removing duplicates, anomalies, and useless values from the collected data using tools such as Python, "Missing people in Shanghai" was extracted as a useful sample and classified into the missing population table.
[0053] The words in the obtained useful samples are translated into vectors E0, E1, ..., E6 through the BERT model. And the E0, E1, ..., E6 are encoded into T0, T1, ..., T6 through the BERT model. Here, E0, E1, ..., E6 and T0, T1, ..., T6 are both 768-dimensional digital vectors.
[0054] Specifically: The BERT model includes an embedding layer and a transformer layer. The embedding layer maps the input question to a word vector, a position code, and a sentence code, and then adds the three vectors bit by bit and feeds them into the transformer layer. After entering the transformer layer, it first passes through self-attention to encode each word vector, then passes through a residual network and batch normalization, and then passes through a feedforward neural network plus a residual network and batch normalization. After the first and second steps are repeated 12 times, T0, T1, ..., T6 are output.
[0055] After the bidirectional GRU model, T0, T1, ..., T6 are encoded into M0, M1, ..., M6.
[0056] Among them, for the t-th input x t With its previous memory unit h t-1 , which is updated as follows:
[0057] z t =σ(W z ·[h t-1 ,x t ]),
[0058] r t =σ(W r ·[h t-1 ,x t ]),
[0059]
[0060]
[0061] Among them, W is the neuron parameter matrix after learning.
[0062] At the same time, in order to achieve the integrated learning of intent recognition and entity recognition, this invention modifies the objective function of the model into the following format:
[0063]
[0064] The ultimate goal of learning is to maximize p(y|x), where p(y r |x) is the loss function corresponding to intent recognition, and the cross entropy loss function is used to classify the entire sentence; The loss function corresponding to entity recognition uses the cross entropy loss function to extract and recognize a single word. The objective function combines intent recognition and entity recognition in the form of a product, ensuring that the model not only maximizes the accuracy of intent recognition, but also maximizes the accuracy of entity recognition during the learning process.
[0065] The process of M0, M1, ..., M6 is divided into two parts. One part is to classify the questions after splicing through a feedforward neural network and map the questions to the 'missing persons table'; the other part is to directly classify M0, M1, ..., M6 to generate the category of each word, so as to extract the elements contained therein.
[0066] Resource Mapping:
[0067] For the 'location': 'Shanghai' extracted from the useful sample "missing people in Shanghai", map it to 'missing location': 'Shanghai'
[0068] Generate a logical expression, and the output logical expression is: (missing address: Shanghai) in missing population table;
[0069] The missing persons table is as follows:
[0070] Name Date of Birth Native Place Disappearance location Zhang San 1999 / 01 / 01 Shanghai Xuhui District, Shanghai Li Si 1999 / 01 / 01 Beijing Songjiang District, Shanghai Wang Wu 1999 / 01 / 01 Nanjing Beijing Zhao Liu 1999 / 01 / 01 Shanghai Shenzhen
[0071] The final exported query results are:
[0072] Person 1: {Name: Zhang San, Date of Birth: 1999 / 01 / 01, Place of Birth: Shanghai, Place of Disappearance: Xuhui District, Shanghai}
[0073] Person 2: {Name: Li Si, Date of Birth: 1999 / 01 / 01, Place of Birth: Shanghai, Place of Disappearance: Xuhui District, Shanghai}
[0074] like Figure 2 Example 3 is shown:
[0075] A semantic parsing system includes: a preprocessing module 1, a learning module 2, a mapping module 3 and a parsing module 4.
[0076] The preprocessing module 1 is used to collect and clean the data to be processed; the learning module 2 is used to build a deep learning model, input the data to be processed into the deep learning model, extract elements from the data to be processed, and obtain the data category of the data to be processed; the mapping module 3 is used to match the mapping table corresponding to the data category, input the extracted elements into the mapping table for mapping, and obtain the semantic logic corresponding to the data to be processed; the parsing module 4 is used to match the parsed data that conforms to the semantic logic in the database, and output the parsed data.
[0077] like Figure 3 Example 4 is shown:
[0078] An electronic device, which can be, but is not limited to, a personal computer, a tablet computer, etc.
[0079] The electronic device 100 includes a processor 110 and a memory 120 .
[0080] It should be noted that Figure 3 The components and structures of the electronic device 100 shown are merely exemplary and non-limiting. The electronic device 100 may also have other components and structures as required.
[0081] The processor 110, the memory 120, and other components that may be present in the electronic device 100 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, the processor 110 and the memory 120 may be electrically connected to each other via one or more communication buses or signal lines.
[0082] The memory 120 is used to store the program corresponding to the semantic parsing method as described above or the semantic parsing system as described above. Optionally, when a semantic parsing system is stored in the memory 120, the system includes at least one software function module that can be stored in the memory 120 in the form of software or firmware. Optionally, the software function module can also be solidified in the operating system of the electronic device 100.
[0083] The processor 110 is used to call the program stored in the memory and execute the semantic parsing method described above. For example: Step 1: Collect and clean the data to be processed; Step 2: Build a deep learning model, input the data to be processed into the deep learning model, extract elements from the data to be processed, and obtain the data category of the data to be processed; Step 3: Match the mapping table corresponding to the data category, input the extracted elements into the mapping table for mapping, and obtain the semantic logic corresponding to the data to be processed; Step 4: Match the logical expression that conforms to the semantic logic in the database, and output the logical expression. The method disclosed in any embodiment of the present application can be applied to the processor 110 or implemented by the processor 110.
[0084] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. In the above-mentioned embodiments provided in this application, the system and method can also be implemented in other ways. The system embodiment described above is merely schematic, for example, each box in the flowchart or block diagram in the accompanying drawings can represent a module, a program segment or a part of a code, and a part of the module, program segment or code contains one or more executable instructions for implementing a specified logical function. At the same time, each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0085] In addition, if the semantic parsing method is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or partly contribute to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory, random access memory, disk or optical disk and other media that can store program code.
[0086] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. The protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A semantic parsing method, characterized in that: The steps include: Step 1: Collect and clean the data to be processed; The step 1 comprises the following steps: Step 11: Collect data to be processed; Step 12: Remove duplicate values from the data to be processed based on Python or ETL tools; Step 13: Remove outliers from the data to be processed based on Python or ETL tools; Step 14: Remove useless values from the data to be processed based on Python or ETL tools; Step 15: Construct a mapping table and establish a corresponding relationship between data classification and the mapping table; Step 2: Build a deep learning model, input the data to be processed into the deep learning model, extract elements from the data to be processed, and obtain the data category of the data to be processed; The deep learning model in step 2 includes a serial BERT model and a bidirectional GRU model; The objective function of the deep learning model is: The ultimate goal of learning is to maximize , is the loss function corresponding to intent recognition, and the cross entropy loss function is used to classify the entire sentence; The loss function corresponding to entity recognition uses the cross entropy loss function to extract and recognize a single word; The objective function combines intent recognition and entity recognition in the form of a product, ensuring that the model not only maximizes the accuracy of intent recognition, but also maximizes the accuracy of entity recognition during the learning process. Step 3: Match the mapping table corresponding to the data category, input the extracted elements into the mapping table for mapping, and obtain the semantic logic corresponding to the data to be processed; Step 4: Match the parsed data that conforms to the semantic logic in the database and output the parsed data.
2. A semantic parsing system, characterized in that: include: Preprocessing module, learning module, mapping module and parsing module; The preprocessing module is used to collect and clean the data to be processed; The collecting and cleaning of the data to be processed comprises the following steps: Step 11: Collect data to be processed; Step 12: Remove duplicate values from the data to be processed based on Python or ETL tools; Step 13: Remove outliers from the data to be processed based on Python or ETL tools; Step 14: Remove useless values from the data to be processed based on Python or ETL tools; Step 15: Construct a mapping table and establish a corresponding relationship between data classification and the mapping table; The learning module is used to construct a deep learning model, input the data to be processed into the deep learning model, extract elements from the data to be processed, and obtain the data category of the data to be processed; The deep learning model includes a serial BERT model and a bidirectional GRU model; The objective function of the deep learning model is: The ultimate goal of learning is to maximize , is the loss function corresponding to intent recognition, and the cross entropy loss function is used to classify the entire sentence; The loss function corresponding to entity recognition uses the cross entropy loss function to extract and recognize a single word; The objective function combines intent recognition and entity recognition in the form of a product, ensuring that the model not only maximizes the accuracy of intent recognition, but also maximizes the accuracy of entity recognition during the learning process. The mapping module is used to match a mapping table corresponding to the data category, input the extracted elements into the mapping table for mapping, and obtain the semantic logic corresponding to the data to be processed; The parsing module is used to match parsing data that conforms to the semantic logic in the database and output the parsing data.
3. An electronic device, characterized in that: It includes a processor and a memory, the memory is used to store programs; the processor and the memory are connected to call the program stored in the memory and execute the semantic parsing method as claimed in claim 1.
4. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is run by a computer, the semantic parsing method as claimed in claim 1 is executed.
