An intent recognition method, system, device and medium for government data

Abstract

The application discloses a kind of for government affair data's intention identification method, system, equipment and medium, it is related to computer information processing technical field, the method includes: step S1, obtain the search text for government affair, the search text includes entity word and non-entity word;Step S2, the search text is carried out entity identification, obtains the entity word set and non-entity word set corresponding to the search text;Step S3, according to the non-entity word set and the government affair database cluster established in advance, determine the target index corresponding to the search text;Step S4, according to the entity word set and the target index, determine the target entity corresponding to the search text;Step S5, according to the entity word set, the target index and the target entity, determine the intention text corresponding to the search text.The application improves the identification accuracy of search text, applicable to government affair field.

Description

Technical Field

[0001] This invention relates to the field of computer information processing technology, and specifically to a method, system, device, and medium for intent recognition of government data. Background Technology

[0002] There are many professional terms in the field of government affairs. The database tables of government affairs database contain a large amount of government affairs data. The data formats of government affairs data are diverse, and the dimensions for identifying government affairs data have reached more than 3,500.

[0003] Intent recognition, as the name suggests, is about determining the meaning and desired action of user input text. Currently, in the government sector, intent recognition based on database tables mainly falls into three categories: First, intent recognition methods based on a single semantic analysis model. This method directly converts the user's input question into an SQL statement to retrieve results. However, this method requires a large amount of training data, and the cost of labeling the data in the database tables is high, making it not the preferred method for intent recognition of government data. Second, intent recognition methods based on neural network models. This method trains an end-to-end model, inputs the user's input question into the model, and the model directly generates the query results for the input question. For intent recognition of government data, this type of method is not mature enough, and the accuracy of the query results is not high. Third, intent recognition methods based on answer ranking. This method assumes that the user's input question must exist in a database table. By scoring and ranking different database tables, the database table with the highest score is selected as the final answer. However, government data involves many fields and data formats, and existing intent recognition methods based on answer ranking are no longer sufficient to meet the intent recognition needs of the government sector. Summary of the Invention

[0004] The technical problem this invention aims to solve is that existing intent recognition methods have low accuracy in recognizing search text related to government affairs. To address this problem, this invention provides an intent recognition method, system, device, and medium for government data.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0006] An intent recognition method for government data includes:

[0007] Step S1: Obtain search text for government affairs, the search text including entity words and non-entity words;

[0008] Step S2: Perform entity recognition on the search text to obtain the entity word set and non-entity word set corresponding to the search text;

[0009] Step S3: Based on the non-entity word set and the pre-established government database cluster, determine the target indicator corresponding to the search text. The government database cluster includes multiple databases, and the government database cluster stores database tables of government data in multiple fields. Each database table of government data in a certain field corresponds to one database. Each database table includes multiple columns of government data, and each column of government data corresponds to a field. For each field, the field is the identifier of the corresponding column. The field includes an indicator field and a non-indicator field. The target indicator is an indicator field.

[0010] Step S4: Determine the target entity corresponding to the search text based on the entity word set and the target indicator;

[0011] Step S5: Determine the intent text corresponding to the search text based on the entity word set, the target metric, and the target entity.

[0012] The beneficial effects of this invention are as follows: Starting from the characteristics of government data, the search text is first identified by entity recognition to obtain a set of entity words and a set of non-entity words. Then, index matching is performed based on the set of non-entity words (corresponding to step S3) to obtain the target index. Then, the target entity is determined based on the target index. Finally, the intent text corresponding to the search text is determined according to the set of entity words, the target index, and the target entity. Each entity and non-entity contained in the intent text has a corresponding field in the government database cluster, which improves the recognition accuracy of the search text and solves the problem of low recognition accuracy of existing intent recognition methods. It is applicable to the government field.

[0013] Based on the above technical solution, the present invention can be further improved as follows.

[0014] Furthermore, in step S1, the search text is determined through the following steps:

[0015] Get the original text;

[0016] The original text is cleaned to remove colloquial words and redundant special characters, resulting in cleaned text.

[0017] If the length of the cleaned text is greater than a preset text length threshold, the cleaned text is truncated to obtain a short text, and the short text is standardized to obtain the search text.

[0018] If the length of the cleaned text is less than or equal to the text length threshold, then the cleaned text is standardized to obtain the search text.

[0019] The beneficial effects of adopting the above-mentioned further scheme are: by removing redundant symbols and words from the original text, a cleaned text is obtained, and then the cleaned text is standardized to complete the rewriting of the original text, thereby obtaining the search text, which facilitates subsequent entity recognition of the search text.

[0020] Furthermore, the entity word set includes multiple entity words, and the entity word set is determined through the following steps:

[0021] The first entity words contained in the search text are identified by pre-constructed time and number category recognition rules to obtain a first entity word set. For each first entity word, the first entity word represents an entity with attributes including at least one of time or number.

[0022] The second entity words contained in the search text are identified by a pre-built entity detection model to obtain a set of second entity words. For each second entity word, the second entity word represents an entity whose attributes include at least one of location, organization name, or person name.

[0023] The entity word set includes the first entity word set and the second entity word set.

[0024] The beneficial effects of adopting the above-mentioned further solution are: in the statistical needs of government data, it is necessary to include statistics on time, numbers, locations, organization names and personal names. This method uses time and number category recognition rules and entity detection models to perform entity recognition on the search text, which is efficient and accurate.

[0025] Furthermore, in step S3, determining the target indicator corresponding to the search text based on the set of non-entity words and the pre-established government database cluster includes:

[0026] Step A1: Using a pre-built automatic similar text generation model, determine the text matrix corresponding to the non-entity word set and the two-dimensional indicator matrix corresponding to the preset indicator dataset. The indicator dataset is a set of indicator fields contained in each database table of the government database cluster. Each element in the two-dimensional indicator matrix corresponds to a similar text generated by the automatic similar text generation model for the indicator field. Each row of the two-dimensional indicator matrix corresponds to a similar text set formed by all similar texts corresponding to the indicator field. For each indicator field, the similar text set corresponding to the indicator field is transformed into a one-dimensional indicator matrix corresponding to the indicator field.

[0027] Step A2: For each indicator field in the indicator dataset, determine the first similarity between the one-dimensional indicator matrix corresponding to the indicator field and the text matrix. Based on each first similarity, determine whether there are any undetermined indicators in the indicator dataset whose first similarity is greater than a preset first similarity threshold.

[0028] Step A3: If there are any undetermined indicators, then determine the target indicator based on the first similarity corresponding to each undetermined indicator; if there are no undetermined indicators, then proceed to step A4.

[0029] Step A4: Segment the non-entity word set to obtain non-entity word segmentation results, which include multiple phrases; concatenate the multiple phrases to generate short sentences corresponding to the search text, where each short sentence is text obtained by concatenating at least one of the phrases.

[0030] Step A5: For each of the short sentences, determine the target indicator based on the second similarity between the short sentence and each of the indicator fields.

[0031] The beneficial effects of adopting the above-mentioned further scheme are as follows: based on the calculated first similarity and first similarity threshold, each indicator field is divided into undetermined indicators and non-undetermined indicators (non-undetermined indicators are indicator fields that do not belong to undetermined indicators). Then, based on the divided undetermined indicators or non-undetermined indicators, the target indicator is determined. This invention completes the matching of target indicators by threshold diversion, which solves the problem that the recognition accuracy of the correct target indicator is low due to the existence of a large number of highly similar indicator fields in government data.

[0032] Further, step S4 includes:

[0033] Based on the entity word set and the target indicator, the remaining text set in the search text is determined, wherein the remaining text set is the text set in the search text other than the entity word set and the target indicator;

[0034] Based on the target metrics, the target database table and target database corresponding to the search text are determined. The target database table is a database table corresponding to the target metrics, and the target database is the database corresponding to the target database table.

[0035] Based on the remaining text set, the target database table, and the target database, the target entity corresponding to the search text is determined.

[0036] The beneficial effects of adopting the above-mentioned further solution are: In the government database cluster, the dimensions and corresponding business logic of each database table are not completely the same, and there are cases where the same indicator field corresponds to multiple database tables. By determining the target database table and target database corresponding to the search text based on the target indicator, the recognition accuracy of the search text is improved.

[0037] Furthermore, for each of the fields, the field type is divided into enumeration type and non-enumeration type, and the field value corresponding to the field of the field type is an enumeration value;

[0038] The step of determining the target database table and target database corresponding to the search text based on the target indicators includes:

[0039] If at least two of the multiple database tables contain the target indicator, the database tables containing the target indicator are designated as pending database tables. For each pending database table, the target database table and the target database are determined based on the similarity between the remaining text set and each enumerated value contained in the pending database table.

[0040] If only one of the multiple database tables contains the target indicator, then that database table is taken as the target database table, and the database corresponding to the target database table is taken as the target database.

[0041] Specifically, for each of the pending database tables, determining the target database table and the target database based on the similarity between the remaining text set and each enumerated value contained in the pending database table includes:

[0042] For each of the pending database tables, a comparison value corresponding to the pending database table is determined based on the number of matches corresponding to each enumeration value in the pending database table. The comparison value is the sum of the number of matches corresponding to each enumeration value in the pending database table. For each match, the match is the number of third similarities that satisfy the preset comparison conditions among all third similarities corresponding to the enumeration value. The third similarity corresponding to the enumeration value is the similarity between the enumeration value and the remaining text in the remaining text set.

[0043] The target database and the target database table are determined based on the comparison value corresponding to each of the database tables.

[0044] The beneficial effects of adopting the above-mentioned further solution are as follows: In the government database cluster, the dimensions and corresponding business logic of each database table are not completely the same, and there are cases where the same indicator field corresponds to multiple database tables. By determining the target database and target database table based on the third similarity corresponding to each enumerated value in the undetermined database table, the recognition accuracy of the search text is improved.

[0045] Further, determining the target entity corresponding to the search text based on the remaining text set, the target database table, and the target database includes:

[0046] Step B1: Based on the pre-constructed target domain dictionary, the remaining text set is segmented to obtain the remaining fine-grained entity segmentation results. The remaining fine-grained entity segmentation results include fine-grained entities. The target domain dictionary is the domain dictionary corresponding to the target database.

[0047] Step B2: For each fine-grained entity, determine whether there is a correspondence between the fine-grained entity and each field in the target database table according to the target domain dictionary. If there is a correspondence between the fine-grained entity and the fields in the target database table, then the fine-grained entity is taken as the target entity; if there is no correspondence between each fine-grained entity and each field in the target database table, then proceed to step B3.

[0048] Step B3: For each fine-grained entity, if the fine-grained entity meets the preset entity expansion condition, then the fine-grained entity is expanded to obtain the expanded entity corresponding to the fine-grained entity.

[0049] Step B4: For each extended entity, an extended matrix corresponding to the extended entity and a two-dimensional field matrix corresponding to the target database table are determined using a pre-constructed similar text automatic generation model. Each element in the two-dimensional field matrix corresponds to a similar text generated by the similar text automatic generation model for the target field. Each row of the two-dimensional field matrix corresponds to a set of similar texts formed by all similar texts corresponding to the target field. The target field is a field contained in the target database table. For each target field, the set of similar texts corresponding to the target field is transformed into a one-dimensional field matrix corresponding to the target field.

[0050] Step B5: For each of the extended entities, determine the fourth similarity corresponding to the extended entity, where the fourth similarity is the similarity between the extended matrix corresponding to the extended entity and the one-dimensional field matrix;

[0051] Step B6: Determine the target entity based on the fourth similarity corresponding to each of the extended entities.

[0052] The beneficial effects of adopting the above-mentioned further solution are as follows: government data contains a large number of obscure and specialized words. When it is determined from the target domain dictionary that there is no corresponding relationship between all fine-grained entities and each field in the target database table, entity expansion is performed to obtain extended entities. Based on the extended entities and the fields in the target database table, entity links are performed to infer the target entity. This achieves automated reasoning of the target entity in the search text and efficient processing of the search text.

[0053] To address the aforementioned technical problems, the present invention also provides an intent recognition system for government data, comprising:

[0054] The data acquisition module is used to acquire search text for government affairs, including entity words and non-entity words;

[0055] An entity recognition module is used to perform entity recognition on the search text to obtain a set of entity words and a set of non-entity words corresponding to the search text.

[0056] The target indicator determination module is used to determine the target indicator corresponding to the search text based on the non-entity word set and the pre-established government database cluster. The government database cluster includes multiple databases, and the government database cluster stores database tables of government data in multiple fields. Each database table of government data in a certain field corresponds to one database. Each database table includes multiple columns of government data, and each column of government data corresponds to a field. For each field, the field is the identifier of the corresponding column. The field includes an indicator field and a non-indicator field. The target indicator is an indicator field.

[0057] The target entity determination module is used to determine the target entity corresponding to the search text based on the entity word set and the target index;

[0058] The intent text determination module is used to determine the intent text corresponding to the search text based on the entity word set, the target metric, and the target entity.

[0059] To address the aforementioned technical problems, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intent recognition method for government data as described above.

[0060] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the intent recognition method for government data as described above. Attached Figure Description

[0061] Figure 1 This is a flowchart illustrating the intent recognition method for government data in this invention.

[0062] Figure 2 This is a schematic diagram of the associated information table corresponding to the indicator fields in this invention;

[0063] Figure 3 This is a schematic diagram illustrating the results of word segmentation, entity expansion, and entity linking based on the remaining text set in this invention.

[0064] Figure 4 This is a schematic diagram of the intent recognition system for government data in this invention. Detailed Implementation

[0065] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0066] Example 1

[0067] like Figure 1 As shown, this embodiment provides an intent recognition method for government data, including:

[0068] Step S1: Obtain search text for government affairs, the search text including entity words and non-entity words;

[0069] Step S2: Perform entity recognition on the search text to obtain the entity word set and non-entity word set corresponding to the search text;

[0070] Step S3: Based on the non-entity word set and the pre-established government database cluster, determine the target indicator corresponding to the search text. The government database cluster includes multiple databases, and the government database cluster stores database tables of government data in multiple fields. Each database table of government data in a certain field corresponds to one database. Each database table includes multiple columns of government data, and each column of government data corresponds to a field. For each field, the field is the identifier of the corresponding column. The field includes an indicator field and a non-indicator field. The target indicator is an indicator field.

[0071] Step S4: Determine the target entity corresponding to the search text based on the entity word set and the target indicator;

[0072] Step S5: Determine the intent text corresponding to the search text based on the entity word set, the target metric, and the target entity.

[0073] In this invention, a database table may have multiple indicator fields; the target entity corresponds to the fields contained in the database table.

[0074] In step S1, the search text is determined through the following steps:

[0075] Get the original text;

[0076] The original text is cleaned to remove colloquial words and redundant special characters, resulting in cleaned text.

[0077] If the length of the cleaned text is greater than a preset text length threshold, the cleaned text is truncated to obtain a short text, and the short text is standardized to obtain the search text.

[0078] If the length of the cleaned text is less than or equal to the text length threshold, then the cleaned text is standardized to obtain the search text.

[0079] In this embodiment, the short text is standardized to obtain the search text, including:

[0080] If the short text contains time text, for each time text, the time text is treated as the text to be confirmed. It is determined whether the text to be confirmed is in a standard time format. If not, the text to be confirmed is converted into a standard time format to obtain a standardized time text. The text to be confirmed in the short text is then replaced with the standardized time text to obtain the search text.

[0081] In this embodiment, "short text containing time text" means that the short text contains time information, and "standard time format" refers to the data format of the data type stored in the government database cluster that is of the time type. The method of standardizing the cleaned text to obtain the search text is the same as the method of standardizing the short text to obtain the search text, and the similarities will not be repeated.

[0082] In step S2, the entity word set includes multiple entity words, and the entity word set is determined through the following steps:

[0083] The first entity word contained in the search text is identified by pre-constructed time and number category recognition rules to obtain a first entity word set. For each first entity word, the first entity word represents an entity whose attribute includes at least one of time or number. In this embodiment, the time and number category recognition rules adopt existing automatic recognition rules for recognizing time and numbers.

[0084] The second entity words contained in the search text are identified by a pre-built entity detection model to obtain a set of second entity words. For each second entity word, the second entity word represents an entity whose attributes include at least one of location, organization name, or person name. In this embodiment, the entity detection model adopts the existing BERT+CRF model.

[0085] The entity word set includes the first entity word set and the second entity word set. In this embodiment, the entity words corresponding to the first entity word are stored in the entity word set according to the position order of the first entity word in the search text.

[0086] In step S2, the set of non-entity words is determined based on the search text and the set of entity words. In this embodiment, the set of non-entity words is determined through the following steps:

[0087] The search text is converted into a string to obtain the target string;

[0088] For each entity word in the entity word set, the entity word is converted into a string to obtain entity characters, and the entity characters in the target string are replaced with a specified delimiter (e.g., "&&", space);

[0089] The target string is segmented according to the specified delimiter to obtain multiple first substrings. Each first substring is treated as a non-entity word and stored in the non-entity word set according to its position in the target string. The non-entity word set is represented as a = {a1, a2, ..., a...} i}, where i represents the total number of the non-entity words, a m (1≤m≤i) represents the m-th non-entity word in the set of non-entity words.

[0090] In step S3, determining the target indicator corresponding to the search text based on the non-entity word set and the pre-established government database cluster includes:

[0091] Step A1: Using a pre-built automatic similar text generation model, determine the text matrix corresponding to the non-entity word set and the two-dimensional indicator matrix corresponding to the preset indicator dataset. The indicator dataset is the set of indicator fields contained in each database table of the government database cluster. Each element in the two-dimensional indicator matrix corresponds to a similar text generated by the automatic similar text generation model for the indicator field. Each row element in the two-dimensional indicator matrix (i.e., each dimension of the data in the two-dimensional indicator matrix) corresponds to a similar text set formed by all similar texts corresponding to the indicator field. For each indicator field, transform the similar text set corresponding to the indicator field into a one-dimensional indicator matrix corresponding to the indicator field. Each element in the one-dimensional indicator matrix corresponds to a similar text generated by the automatic similar text generation model for the indicator field.

[0092] Step A2: For each indicator field in the indicator dataset, determine the first similarity between the one-dimensional indicator matrix corresponding to the indicator field and the text matrix. Based on each first similarity, determine whether there are any undetermined indicators in the indicator dataset whose first similarity is greater than a preset first similarity threshold. In this embodiment, the cosine similarity algorithm is used to calculate the cosine similarity between the one-dimensional indicator matrix and the text matrix. The calculated cosine similarity is used as the first similarity between the one-dimensional indicator matrix and the text matrix. That is, the semantic matching between each indicator field and the non-entity word set is completed by the cosine similarity algorithm. The value of the first similarity threshold is 0.8.

[0093] Step A3: If there are any pending indicators, then determine the target indicator based on the first similarity corresponding to each pending indicator; if there are no pending indicators, then proceed to step A4; in this embodiment, if there are any pending indicators, the indicator field corresponding to the largest first similarity is taken as the target indicator.

[0094] Step A4: Segment the non-entity word set to obtain non-entity word segmentation results, which include multiple phrases; concatenate the multiple phrases to generate short sentences corresponding to the search text, where each short sentence is text obtained by concatenating at least one of the phrases; in this embodiment, the word segmentation operation on the non-entity word set is performed using the word segmentation tool jieba, and the concatenation principle for generating the short sentences from the multiple phrases is: concatenate the phrases according to their position order in the search text;

[0095] Step A5: For each phrase, determine the target indicator based on the second similarity between the phrase and each indicator field. In this embodiment, an edit distance algorithm is used to calculate the edit distance between the phrase and the indicator field. The calculated edit distance is used as the second similarity between the phrase and the indicator field. If there is a second similarity greater than the first similarity threshold, the indicator field corresponding to the largest second similarity is taken as the target indicator; if there is no second similarity greater than the first similarity threshold, it indicates that the government database does not store data corresponding to the search text.

[0096] Figure 2 This is a table of association information for the aforementioned indicator fields, obtained through manual analysis. Figure 2 In this context, DynamicIndicatorDefineID represents the ID of the indicator field, IndicatorName represents the name of the indicator field, SourceID is the ID of the database table to which the indicator field belongs, TableName represents the table name of the database table to which the indicator field belongs, IndicatorCondition represents the calculation condition corresponding to the indicator field, and OutputAgg represents the calculation rule corresponding to the indicator field.

[0097] In this embodiment, the automatic similar text generation model adopts the SimBERT model. Specifically, it takes the non-entity word set as input and outputs the text matrix through the SimBERT model; it also takes the indicator dataset as input and outputs the two-dimensional indicator matrix through the SimBERT model. The indicator fields contained in each database table of the government database cluster are manually set, and the indicator dataset is represented as b = {b1, b2, ..., b...} j}, j represents the total number of the indicator fields, b n 1≤n≤j represents the nth indicator field in the indicator dataset;

[0098] Each element in the text matrix corresponds to a similar text generated by the similar text automatic generation model for the non-entity word. Each row of the text matrix corresponds to a similar text set formed by all similar texts corresponding to the non-entity word. For each non-entity word, the similar text set corresponding to the non-entity word is transformed into a one-dimensional text matrix corresponding to the non-entity word. The text matrix can be represented as follows:

[0099]

[0100] The one-dimensional text matrix obtained after the transformation of the m-th (1≤m≤i) non-entity word in the non-entity word set by the similar text automatic generation model is represented as follows:

[0101] a m,new =[a m1 a m2 a m3 … a mk ]

[0102] The two-dimensional index matrix can be represented as:

[0103]

[0104] The one-dimensional index matrix obtained after the nth (1≤n≤j) index field in the index dataset is transformed by the similar text automatic generation model is represented as follows:

[0105] b n,new =[b n1 b n2 b n3 … b nk ].

[0106] Step S4 includes:

[0107] Based on the entity word set and the target indicator, the remaining text set in the search text is determined, wherein the remaining text set is the text set in the search text other than the entity word set and the target indicator;

[0108] Based on the target metrics, the target database table and target database corresponding to the search text are determined. The target database table is a database table corresponding to the target metrics, and the target database is the database corresponding to the target database table.

[0109] Based on the remaining text set, the target database table, and the target database, the target entity corresponding to the search text is determined.

[0110] In this embodiment, the remaining text set is determined through the following steps:

[0111] The search text is converted into a string to obtain the target string;

[0112] For each entity word in the entity word set, the entity word is converted into a string to obtain entity characters, and the entity characters in the target string are replaced with a specified decomposition character;

[0113] The target indicator is converted into a string to obtain indicator characters, and the indicator characters in the target string are replaced with a specified decomposition character;

[0114] The target string is split according to the specified delimiter to obtain multiple second substrings. Each second substring is used as the remaining text and stored in the remaining text set according to the position of the second substring in the target string.

[0115] For each of the fields, the field type is divided into enumeration type and non-enumeration type, and the field value corresponding to the field of the field type is the enumeration value;

[0116] The step of determining the target database table and target database corresponding to the search text based on the target indicators includes:

[0117] If at least two of the multiple database tables contain the target indicator, the database tables containing the target indicator are designated as pending database tables. For each pending database table, the target database table and the target database are determined based on the similarity between the remaining text set and each enumerated value contained in the pending database table.

[0118] If only one of the multiple database tables contains the target indicator, then that database table is taken as the target database table, and the database corresponding to the target database table is taken as the target database.

[0119] For each of the pending database tables, determining the target database table and the target database based on the similarity between the remaining text set and each enumerated value contained in the pending database table includes:

[0120] For each of the pending database tables, a comparison value is determined based on the matching number corresponding to each enumeration value in the pending database table. The comparison value is the sum of the matching numbers corresponding to each enumeration value in the pending database table. For each matching number, the matching number is the number of third similarities among all third similarities corresponding to the enumeration value that satisfy a preset comparison condition. The third similarity corresponding to the enumeration value is the similarity between the enumeration value and the remaining text in the remaining text set. In this embodiment, the enumeration values ​​in each of the database tables can be manually set, and the cosine similarity algorithm is used to calculate the similarity between the enumeration value and the remaining text in the remaining text set. The comparison condition is that the third similarity is greater than a preset second similarity threshold.

[0121] The target database and the target database table are determined based on the comparison value corresponding to each of the database tables. In this embodiment, the database table corresponding to the largest comparison value is designated as the target database table, and the database to which the target database table belongs is designated as the target database.

[0122] The step of determining the target entity corresponding to the search text based on the remaining text set, the target database table, and the target database includes:

[0123] Step B1: Based on the pre-constructed target domain dictionary, the remaining text set is segmented to obtain the remaining fine-grained entity segmentation results. The remaining fine-grained entity segmentation results include fine-grained entities. The target domain dictionary is the domain dictionary corresponding to the target database. In this embodiment, the target domain dictionary is loaded and the remaining text set is segmented using the segmentation tool jieba.

[0124] Step B2: For each fine-grained entity, based on the target domain dictionary, determine whether there is a correspondence between the fine-grained entity and each field in the target database table. If there is a correspondence between the fine-grained entity and the fields in the target database table, then the fine-grained entity is designated as the target entity; if there is no correspondence between each fine-grained entity and each field in the target database table, then proceed to step B3. In this embodiment, for each fine-grained entity, if the target domain dictionary contains association information between the fine-grained entity and a field in the target database table, then it is determined that there is a correspondence between the fine-grained entity and the field in the target database table.

[0125] Step B3: For each fine-grained entity, if the fine-grained entity meets the preset entity expansion condition, then the fine-grained entity is expanded to obtain the expanded entity corresponding to the fine-grained entity.

[0126] Step B4: For each extended entity, an extended matrix corresponding to the extended entity and a two-dimensional field matrix corresponding to the target database table are determined using a pre-constructed similar text automatic generation model. Each element in the two-dimensional field matrix corresponds to a similar text generated by the similar text automatic generation model for the target field. Each row of the two-dimensional field matrix corresponds to a set of similar texts formed by all similar texts corresponding to the target field. The target field is a field contained in the target database table. For each target field, the set of similar texts corresponding to the target field is transformed into a one-dimensional field matrix corresponding to the target field.

[0127] Step B5: For each of the extended entities, determine the fourth similarity corresponding to the extended entity, where the fourth similarity is the similarity between the extended matrix corresponding to the extended entity and the one-dimensional field matrix;

[0128] Step B6: Determine the target entity based on the fourth similarity score corresponding to each of the extended entities. In this embodiment, the extended entity corresponding to the highest fourth similarity score is taken as the target entity.

[0129] In this embodiment, the entity expansion condition is that the length L of the fine-grained entity satisfies 2≤L≤4. The entity expansion of the fine-grained entity to obtain the expanded entity corresponding to the fine-grained entity includes: concatenating the fine-grained entity with the fine-grained entity in the search text that is adjacent to and follows the fine-grained entity to obtain the expanded entity corresponding to the fine-grained entity; for each expanded entity, the method for determining the fourth similarity corresponding to the expanded entity is the same as the method for determining the first similarity between the one-dimensional index matrix corresponding to the index field and the text matrix, and the similarities will not be repeated.

[0130] This method, after the user inputs the original text (i.e., the user's search question), first cleans the original text to obtain cleaned text. Based on this cleaned text, the corresponding search text is obtained, completing the question rewriting. Then, entity recognition is performed on the search text. After removing entity words from the search text, indicator matching (corresponding to step S3 above, indicator matching includes semantic matching, similarity matching, and enumeration value hit number matching) is performed to obtain the target indicator. Next, based on the entity words and target indicator, the remaining text set is determined. Based on the remaining text set, the target database table and target database are determined, and the domain dictionary corresponding to the target database is loaded. The remaining text set is then segmented to obtain the corresponding remaining fine-grained entity segmentation results. Based on the remaining fine-grained entity segmentation results, entity matching (corresponding to step B2 above), entity expansion (corresponding to step B3 above), and entity linking (corresponding to steps B4 and B5 above) are performed to obtain the target entity, thereby determining the intent text corresponding to the user's original text. Based on the entity words, target indicator, and target entity contained in the intent text, the intent recognition result corresponding to the intent text can be queried in the corresponding target database.

[0131] For example, given the user-inputted original text "How many outstanding backbone teachers are there in Yunyan District, Guizhou Province?", the original text is first rewritten to obtain the search text "Number of outstanding backbone teachers in Yunyan District, Guizhou Province". Entity recognition is then performed on this search text to obtain the entity words "Guizhou Province" and "Yunyan District". Next, the corresponding text matrix is ​​determined using the similarity text automatic generation model. Based on this text matrix, index matching is performed to obtain the corresponding target index "Number of Teachers". The first similarity between this target index and the text matrix is ​​0.995. Based on the remaining text set "Outstanding Backbone Teachers at the County Level", the corresponding target database table and target database are determined. The target domain dictionary corresponding to this target database is "New Staff Graph". The results of word segmentation, entity expansion, and entity linking operations on the remaining text set "Outstanding Backbone Teachers at the County Level" are as follows: Figure 3 As shown, the target entity (corresponding to) is obtained. Figure 3 The "link dimension" in the text refers to "backbone teachers at the county level and above"; based on "Guizhou Province", "Yunyan District", "backbone teachers at the county level and above" and "number of teachers", the intended text corresponding to the original text is determined, namely "number of backbone teachers at the county level and above in Yunyan District, Guizhou Province".

[0132] Example 2

[0133] Similar in principle to the intent recognition method for government data described in Embodiment 1 above, this embodiment provides an intent recognition system for government data, such as... Figure 4 As shown, it includes:

[0134] The data acquisition module is used to acquire search text for government affairs, including entity words and non-entity words;

[0135] An entity recognition module is used to perform entity recognition on the search text to obtain a set of entity words and a set of non-entity words corresponding to the search text.

[0136] The target indicator determination module is used to determine the target indicator corresponding to the search text based on the non-entity word set and a pre-established government database cluster. The government database cluster includes multiple databases, and the government database cluster stores database tables of government data in multiple fields. Each database table of government data in a certain field corresponds to one database. Each database table includes multiple columns of government data, and each column of government data corresponds to a field. For each field, the field is the identifier of the corresponding column. The field includes indicator fields and non-indicator fields, and the target indicator is an indicator field. For each field, the field type is divided into enumeration type and non-enumeration type. The field value corresponding to the field type of the field is an enumeration value.

[0137] The target entity determination module is used to determine the target entity corresponding to the search text based on the entity word set and the target index;

[0138] The intent text determination module is used to determine the intent text corresponding to the search text based on the entity word set, the target metric, and the target entity.

[0139] The system also includes a search text determination module, which comprises:

[0140] The text acquisition unit is used to acquire raw text;

[0141] The text cleaning unit is used to clean the original text to remove colloquial words and redundant special symbols contained in the original text, so as to obtain cleaned text.

[0142] The first standardization unit is used to truncate the cleaned text to obtain a short text if the length of the cleaned text is greater than a preset text length threshold, and then standardize the short text to obtain the search text.

[0143] The second standardization unit is used to standardize the cleaned text if the length of the cleaned text is less than or equal to the text length threshold, so as to obtain the search text.

[0144] The system also includes an entity word set determination module, which includes:

[0145] The first entity word determination unit is used to identify the first entity words contained in the search text through pre-constructed time and number category recognition rules, and obtain the first entity word set. For each first entity word, the first entity word represents an entity whose attributes include at least one of time or number.

[0146] The second entity word determination unit is used to identify the second entity words contained in the search text through a pre-built entity detection model to obtain a set of second entity words. For each second entity word, the second entity word represents an entity whose attributes include at least one of location, organization name, or person name.

[0147] The entity word set includes the first entity word set and the second entity word set.

[0148] The target indicator determination module, when determining the target indicator corresponding to the search text based on the non-entity word set and a pre-established government database cluster, includes:

[0149] The first processing unit is used to determine, through a pre-constructed similar text automatic generation model, the text matrix corresponding to the non-entity word set and the two-dimensional indicator matrix corresponding to the preset indicator dataset. The indicator dataset is a set of indicator fields contained in each database table of the government database cluster. Each element in the two-dimensional indicator matrix corresponds to a similar text generated by the similar text automatic generation model for the indicator field. Each row element in the two-dimensional indicator matrix corresponds to a similar text set formed by all similar texts corresponding to the indicator field. For each indicator field, the similar text set corresponding to the indicator field is transformed into a one-dimensional indicator matrix corresponding to the indicator field.

[0150] The second processing unit is used to determine, for each indicator field in the indicator dataset, a first similarity between the one-dimensional indicator matrix corresponding to the indicator field and the text matrix, and to determine, based on each first similarity, whether there are any undetermined indicators in the indicator dataset whose first similarity is greater than a preset first similarity threshold.

[0151] The third processing unit is used to determine the target indicator based on the first similarity corresponding to each of the undetermined indicators if the undetermined indicators exist; and to execute the fourth processing unit if the undetermined indicators do not exist.

[0152] The fourth processing unit is used to segment the non-entity word set to obtain non-entity word segmentation results, the non-entity word segmentation results including multiple phrases; and to concatenate the multiple phrases to generate short sentences corresponding to the search text, each short sentence being text obtained by concatenating at least one of the phrases.

[0153] The fifth processing unit is used to determine a target indicator for each of the short sentences based on a second similarity between the short sentence and each of the indicator fields.

[0154] The target entity determination module includes:

[0155] The remaining text set determination unit is used to determine the remaining text set in the search text based on the entity word set and the target indicator, wherein the remaining text set is the text set in the search text other than the entity word set and the target indicator;

[0156] The target database and table determination unit is used to determine the target database table and target database corresponding to the search text based on the target indicators, wherein the target database table is a database table corresponding to the target indicators, and the target database is the database corresponding to the target database table;

[0157] The target entity determination unit is used to determine the target entity corresponding to the search text based on the remaining text set, the target database table, and the target database.

[0158] Specifically, the target database and table determination unit, when determining the target database table and target database corresponding to the search text based on the target indicators, is used for:

[0159] If at least two of the multiple database tables contain the target indicator, the database tables containing the target indicator are designated as pending database tables. For each pending database table, the target database table and the target database are determined based on the similarity between the remaining text set and each enumerated value contained in the pending database table.

[0160] If only one of the multiple database tables contains the target indicator, then that database table is designated as the target database table, and the database corresponding to the target database table is designated as the target database.

[0161] Specifically, for each of the pending database tables, the target database and table determination unit, when determining the target database table and target database based on the similarity between the remaining text set and each enumerated value contained in the pending database table, is used for:

[0162] For each of the pending database tables, a comparison value corresponding to the pending database table is determined based on the number of matches corresponding to each enumeration value in the pending database table. The comparison value is the sum of the number of matches corresponding to each enumeration value in the pending database table. For each match, the match is the number of third similarities that satisfy the preset comparison conditions among all third similarities corresponding to the enumeration value. The third similarity corresponding to the enumeration value is the similarity between the enumeration value and the remaining text in the remaining text set.

[0163] The target database and the target database table are determined based on the comparison value corresponding to each of the database tables.

[0164] The target entity determination unit, when determining the target entity corresponding to the search text based on the remaining text set, the target database table, and the target database, includes:

[0165] The first subunit is used to segment the remaining text set according to the pre-constructed target domain dictionary to obtain the remaining fine-grained entity segmentation results. The remaining fine-grained entity segmentation results include fine-grained entities, and the target domain dictionary is the domain dictionary corresponding to the target database.

[0166] The second subunit is used to determine, for each fine-grained entity, whether there is a correspondence between the fine-grained entity and each field in the target database table according to the target domain dictionary. If there is a correspondence between the fine-grained entity and the fields in the target database table, then the fine-grained entity is taken as the target entity; if there is no correspondence between each fine-grained entity and each field in the target database table, then the third subunit is executed.

[0167] The third subunit is used to perform entity expansion on each fine-grained entity if the fine-grained entity meets the preset entity expansion conditions, so as to obtain the expanded entity corresponding to the fine-grained entity.

[0168] The fourth subunit is used to determine, for each extended entity, the extended matrix corresponding to the extended entity and the two-dimensional field matrix corresponding to the target database table through a pre-constructed similar text automatic generation model. Each element in the two-dimensional field matrix corresponds to a similar text generated by the similar text automatic generation model for the target field. Each row element in the two-dimensional field matrix corresponds to a similar text set formed by all similar texts corresponding to the target field. The target field is a field contained in the target database table. For each target field, the similar text set corresponding to the target field is transformed into a one-dimensional field matrix corresponding to the target field.

[0169] The fifth subunit is used to determine the fourth similarity corresponding to each extended entity, wherein the fourth similarity is the similarity between the extended matrix corresponding to the extended entity and the one-dimensional field matrix;

[0170] The sixth subunit is used to determine the target entity based on the fourth similarity corresponding to each of the extended entities.

[0171] Example 3

[0172] To address the aforementioned technical problems, this embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the intent recognition method for government data as described in Embodiment 1.

[0173] Example 4

[0174] To address the aforementioned technical problems, this embodiment provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the intent recognition method for government data as described in Embodiment 1.

[0175] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0176] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0177] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for intent recognition of government data, characterized in that, include: Step S1: Obtain search text for government affairs, the search text including entity words and non-entity words; Step S2: Perform entity recognition on the search text to obtain the entity word set and non-entity word set corresponding to the search text; Step S3: Based on the non-entity word set and the pre-established government database cluster, determine the target indicator corresponding to the search text. The government database cluster includes multiple databases, and the government database cluster stores database tables of government data in multiple fields. Each database table of government data in a certain field corresponds to one database. Each database table includes multiple columns of government data, and each column of government data corresponds to a field. For each field, the field is the identifier of the corresponding column. The field includes an indicator field and a non-indicator field. The target indicator is an indicator field. The step of determining the target indicator corresponding to the search text based on the set of non-entity words and the pre-established government database cluster includes: Step A1: Using a pre-built automatic similar text generation model, determine the text matrix corresponding to the non-entity word set and the two-dimensional indicator matrix corresponding to the preset indicator dataset. The indicator dataset is a set of indicator fields contained in each database table of the government database cluster. Each element in the two-dimensional indicator matrix corresponds to a similar text generated by the automatic similar text generation model for the indicator field. Each row of the two-dimensional indicator matrix corresponds to a similar text set formed by all similar texts corresponding to the indicator field. For each indicator field, the similar text set corresponding to the indicator field is transformed into a one-dimensional indicator matrix corresponding to the indicator field. Step A2: For each indicator field in the indicator dataset, determine the first similarity between the one-dimensional indicator matrix corresponding to the indicator field and the text matrix. Based on each first similarity, determine whether there are any undetermined indicators in the indicator dataset whose first similarity is greater than a preset first similarity threshold. Step A3: If there are any undetermined indicators, then determine the target indicator based on the first similarity corresponding to each undetermined indicator; if there are no undetermined indicators, then proceed to step A4. Step A4: Segment the non-entity word set to obtain non-entity word segmentation results, which include multiple phrases; concatenate the multiple phrases to generate short sentences corresponding to the search text, where each short sentence is text obtained by concatenating at least one of the phrases. Step A5: For each of the short sentences, determine the target indicator based on the second similarity between the short sentence and each of the indicator fields; Step S4: Determine the target entity corresponding to the search text based on the entity word set and the target indicator; Step S5: Determine the intent text corresponding to the search text based on the entity word set, the target metric, and the target entity.

2. The method according to claim 1, characterized in that, In step S1, the search text is determined through the following steps: Get the original text; The original text is cleaned to remove colloquial words and redundant special characters, resulting in cleaned text. If the length of the cleaned text is greater than a preset text length threshold, the cleaned text is truncated to obtain a short text, and the short text is standardized to obtain the search text. If the length of the cleaned text is less than or equal to the text length threshold, then the cleaned text is standardized to obtain the search text.

3. The method according to claim 1, characterized in that, The entity word set includes multiple entity words, and the entity word set is determined through the following steps: The first entity words contained in the search text are identified by pre-constructed time and number category recognition rules to obtain a first entity word set. For each first entity word, the first entity word represents an entity with attributes including at least one of time or number. The second entity words contained in the search text are identified by a pre-built entity detection model to obtain a set of second entity words. For each second entity word, the second entity word represents an entity whose attributes include at least one of location, organization name, or person name. The entity word set includes the first entity word set and the second entity word set.

4. The method according to any one of claims 1 to 3, characterized in that, Step S4 includes: Based on the entity word set and the target indicator, the remaining text set in the search text is determined, wherein the remaining text set is the text set in the search text other than the entity word set and the target indicator; Based on the target metrics, the target database table and target database corresponding to the search text are determined. The target database table is a database table corresponding to the target metrics, and the target database is the database corresponding to the target database table. Based on the remaining text set, the target database table, and the target database, the target entity corresponding to the search text is determined.

5. The method according to claim 4, characterized in that, For each of the fields, the field type is divided into enumeration type and non-enumeration type, and the field value corresponding to the field of the field type is an enumeration value; The step of determining the target database table and target database corresponding to the search text based on the target indicators includes: If at least two of the multiple database tables contain the target indicator, the database tables containing the target indicator are designated as pending database tables. For each pending database table, the target database table and the target database are determined based on the similarity between the remaining text set and each enumerated value contained in the pending database table. If only one of the multiple database tables contains the target indicator, then that database table is taken as the target database table, and the database corresponding to the target database table is taken as the target database. Specifically, for each of the pending database tables, determining the target database table and the target database based on the similarity between the remaining text set and each enumerated value contained in the pending database table includes: For each of the pending database tables, a comparison value corresponding to the pending database table is determined based on the number of matches corresponding to each enumeration value in the pending database table. The comparison value is the sum of the number of matches corresponding to each enumeration value in the pending database table. For each match, the match is the number of third similarities that satisfy the preset comparison conditions among all third similarities corresponding to the enumeration value. The third similarity corresponding to the enumeration value is the similarity between the enumeration value and the remaining text in the remaining text set. The target database and the target database table are determined based on the comparison value corresponding to each of the database tables.

6. The method according to claim 4, characterized in that, The step of determining the target entity corresponding to the search text based on the remaining text set, the target database table, and the target database includes: Step B1: Based on the pre-constructed target domain dictionary, the remaining text set is segmented to obtain the remaining fine-grained entity segmentation results. The remaining fine-grained entity segmentation results include fine-grained entities. The target domain dictionary is the domain dictionary corresponding to the target database. Step B2: For each fine-grained entity, determine whether there is a correspondence between the fine-grained entity and each field in the target database table according to the target domain dictionary. If there is a correspondence between the fine-grained entity and the fields in the target database table, then the fine-grained entity is taken as the target entity; if there is no correspondence between each fine-grained entity and each field in the target database table, then proceed to step B3. Step B3: For each fine-grained entity, if the fine-grained entity meets the preset entity expansion condition, then the fine-grained entity is expanded to obtain the expanded entity corresponding to the fine-grained entity. Step B4: For each extended entity, an extended matrix corresponding to the extended entity and a two-dimensional field matrix corresponding to the target database table are determined using a pre-constructed similar text automatic generation model. Each element in the two-dimensional field matrix corresponds to a similar text generated by the similar text automatic generation model for a target field. Each row of the two-dimensional field matrix corresponds to a similar text set formed by all similar texts corresponding to a target field. The target field is a field contained in the target database table. For each target field, the similar text set corresponding to the target field is transformed into a one-dimensional field matrix corresponding to the target field. Step B5: For each of the extended entities, determine the fourth similarity corresponding to the extended entity, where the fourth similarity is the similarity between the extended matrix corresponding to the extended entity and the one-dimensional field matrix; Step B6: Determine the target entity based on the fourth similarity corresponding to each of the extended entities.

7. An intent recognition system for government data, characterized in that, include: The data acquisition module is used to acquire search text for government affairs, including entity words and non-entity words; The entity recognition module is used to perform entity recognition on the search text to obtain the entity word set and non-entity word set corresponding to the search text; The target indicator determination module is used to determine the target indicator corresponding to the search text based on the non-entity word set and the pre-established government database cluster. The government database cluster includes multiple databases, and the government database cluster stores database tables of government data in multiple fields. Each database table of government data in a certain field corresponds to one database. Each database table includes multiple columns of government data, and each column of government data corresponds to a field. For each field, the field is the identifier of the corresponding column. The field includes an indicator field and a non-indicator field. The target indicator is an indicator field. The step of determining the target indicator corresponding to the search text based on the set of non-entity words and the pre-established government database cluster includes: Step A1: Using a pre-built automatic similar text generation model, determine the text matrix corresponding to the non-entity word set and the two-dimensional indicator matrix corresponding to the preset indicator dataset. The indicator dataset is a set of indicator fields contained in each database table of the government database cluster. Each element in the two-dimensional indicator matrix corresponds to a similar text generated by the automatic similar text generation model for the indicator field. Each row of the two-dimensional indicator matrix corresponds to a similar text set formed by all similar texts corresponding to the indicator field. For each indicator field, the similar text set corresponding to the indicator field is transformed into a one-dimensional indicator matrix corresponding to the indicator field. Step A2: For each indicator field in the indicator dataset, determine the first similarity between the one-dimensional indicator matrix corresponding to the indicator field and the text matrix. Based on each first similarity, determine whether there are any undetermined indicators in the indicator dataset whose first similarity is greater than a preset first similarity threshold. Step A3: If there are any undetermined indicators, then determine the target indicator based on the first similarity corresponding to each undetermined indicator; if there are no undetermined indicators, then proceed to step A4. Step A4: Segment the non-entity word set to obtain non-entity word segmentation results, which include multiple phrases; concatenate the multiple phrases to generate short sentences corresponding to the search text, where each short sentence is text obtained by concatenating at least one of the phrases. Step A5: For each of the short sentences, determine the target indicator based on the second similarity between the short sentence and each of the indicator fields; The target entity determination module is used to determine the target entity corresponding to the search text based on the entity word set and the target index. The intent text determination module is used to determine the intent text corresponding to the search text based on the entity word set, the target metric, and the target entity.

8. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the intent recognition method for government data as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the intent recognition method for government data as described in any one of claims 1 to 6.

Images