Sentence mining method and device, electronic equipment and storage medium

Abstract

A sentence mining method and device, electronic equipment and storage medium are related to the technical field of electronic equipment. The method comprises: obtaining a plurality of to-be-mined corpora (S110), performing double-sequence alignment on the plurality of to-be-mined corpora to obtain a plurality of general sentence patterns corresponding to the plurality of to-be-mined corpora (S120), and filtering the plurality of general sentence patterns to select general sentence patterns meeting specified standards as standard sentence patterns from the plurality of general sentence patterns (S130). The method obtains general sentence patterns by performing double-sequence alignment on to-be-mined corpora, and then filters the general sentence patterns to obtain standard sentence patterns, so as to quickly and conveniently obtain standard sentence patterns from to-be-mined corpora for processing.

Description

Technical Field

[0001] This application relates to the field of electronic equipment technology, and more specifically, to a sentence pattern mining method, apparatus, electronic device, and storage medium. Background Technology

[0002] In actual internet business, we often encounter a large amount of formatted information. How to effectively process and analyze this structured information through general sentence pattern mining has become one of the focuses of many natural language processing researchers. Summary of the Invention

[0003] In view of the above problems, this application proposes a sentence pattern mining method, apparatus, electronic device, and storage medium to solve the above problems.

[0004] In a first aspect, embodiments of this application provide a sentence pattern mining method, the method comprising: acquiring multiple corpora to be mined; performing double sequence alignment on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined; filtering the multiple general sentence patterns, and selecting general sentence patterns that meet specified standards from the multiple general sentence patterns as standard sentence patterns.

[0005] Secondly, embodiments of this application provide a sentence pattern mining device, the device comprising: a corpus acquisition module for acquiring multiple corpora to be mined; a general sentence pattern acquisition module for performing double sequence alignment on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined; and a standard sentence pattern acquisition module for filtering the multiple general sentence patterns and selecting general sentence patterns that meet specified standards from the multiple general sentence patterns as standard sentence patterns.

[0006] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory is coupled to the processor, the memory stores instructions, and when the instructions are executed by the processor, the processor performs the above-described method.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, which can be invoked by a processor to execute the above-described method.

[0008] The sentence pattern mining method, apparatus, electronic device, and storage medium provided in this application embodiment acquire multiple corpora to be mined, perform double sequence alignment on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined, filter the multiple general sentence patterns, and select the general sentence patterns that meet the specified criteria as standard sentence patterns. Thus, by performing double sequence alignment on the corpora to be mined to obtain general sentence patterns, and then filtering the general sentence patterns to obtain standard sentence patterns, standard sentence patterns can be obtained from the corpora to be mined quickly and conveniently for processing. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating a sentence pattern mining method provided in one embodiment of this application is shown;

[0011] Figure 2 A flowchart illustrating a sentence pattern mining method provided in yet another embodiment of this application is shown;

[0012] Figure 3 This diagram illustrates the sentence inclusion relationship between multiple general sentence patterns provided in the embodiments of this application;

[0013] Figure 4 This application shows Figure 2 The flowchart of step S240 of the sentence pattern mining method shown is as follows;

[0014] Figure 5 A flowchart illustrating a sentence pattern mining method provided in another embodiment of this application is shown;

[0015] Figure 6 This application shows Figure 5 The flowchart of step S330 of the sentence pattern mining method shown is illustrated.

[0016] Figure 7 This application shows Figure 6 The flowchart of step S332 of the sentence pattern mining method is shown;

[0017] Figure 8 A flowchart illustrating a sentence pattern mining method provided in another embodiment of this application is shown;

[0018] Figure 9 This application shows Figure 8The flowchart of step S440 of the sentence pattern mining method is shown;

[0019] Figure 10 A flowchart illustrating a sentence pattern mining method provided in yet another embodiment of this application is shown;

[0020] Figure 11 A block diagram of the sentence pattern mining device provided in an embodiment of this application is shown;

[0021] Figure 12 A block diagram of an electronic device for performing the sentence pattern mining method according to an embodiment of this application is shown;

[0022] Figure 13 A storage unit for storing or carrying program code implementing the sentence pattern mining method according to an embodiment of the present application is shown. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0024] In recent years, with the rapid development of artificial intelligence (AI) technologies, more and more applications have been implemented, such as computer vision (CV) and natural language processing (NLP), which have greatly improved people's lives. In particular, the recent enthusiasm of researchers for NLP has led to rapid advancements in related language models. For example, Google's Transformer model based on a pure attention mechanism and the BERT (bidirectional encoder representations from transformers) model based on the Transformer model are recent research achievements. In actual internet business, we often encounter a large amount of formatted user information. How to effectively process this structured information through general sentence pattern mining, thereby facilitating the analysis of corresponding downstream NLP tasks (such as intelligent customer service, community question answering, short text classification, etc.), has become one of the focuses of many NLP researchers.

[0025] Generally, current sentence structure mining methods can be divided into the following two categories:

[0026] (1) Manual mining of regular expressions: By manually analyzing formatted data, we find the common format of related sentences and generate regular expressions for downstream NLP tasks.

[0027] (2) Based on large-scale language models: use a large amount of corpus for training, and obtain embedded expressions of relevant fixed sentence patterns through training with a large-scale language model (such as BERT).

[0028] The inventors discovered that while manually mining regular expressions to summarize relevant sentence patterns can guarantee accuracy, data in scenarios like intelligent customer service and community Q&A exhibits a long-tail distribution, making it difficult to effectively mine many special sentence patterns. Furthermore, the sheer volume of data is massive, making it time-consuming and labor-intensive. For short text classification scenarios based on large-scale language models, where some sentence patterns' domain categories depend solely on the entity part (e.g., what is [entity]? Who is [entity]?), and [entity] is diverse and variable, classification of such questions cannot be well handled using neural network-based language models. Therefore, the aim is to mine relevant sentence patterns and address these issues by integrating sentence patterns with [entity] validation. However, neural network-based language models have extremely high experimental costs and long computation cycles, making them unsuitable for small and medium-sized enterprises with large corpora that require rapid iteration and deployment.

[0029] To address the aforementioned problems, the inventors, through long-term research, discovered and proposed the sentence pattern mining method, apparatus, electronic device, and storage medium provided in the embodiments of this application. This method obtains general sentence patterns by performing double sequence alignment on the corpus to be mined, and then filters these general sentence patterns to obtain standard sentence patterns, thus quickly and conveniently obtaining standard sentence patterns from the corpus to be mined for processing. The specific sentence pattern mining method will be described in detail in subsequent embodiments.

[0030] Please see Figure 1 , Figure 1 This diagram illustrates a flowchart of a sentence pattern mining method provided in one embodiment of this application. The sentence pattern mining method obtains general sentence patterns by performing double sequence alignment on the corpus to be mined, and then filters the general sentence patterns to obtain standard sentence patterns, thereby quickly and conveniently obtaining standard sentence patterns from the corpus to be mined for processing. In a specific embodiment, the sentence pattern mining method is applied to, for example... Figure 11 The sentence pattern mining device 200 and the electronic device 100 equipped with the sentence pattern mining device 200 are shown. Figure 12 The following will use an electronic device as an example to illustrate the specific process of this embodiment. The electronic device used in this embodiment may include a mobile terminal, tablet computer, desktop computer, wearable electronic device, etc., and is not limited thereto. The following will focus on... Figure 1 The process shown will be explained in detail. The sentence pattern mining method may specifically include the following steps:

[0031] Step S110: Obtain multiple data points to be mined.

[0032] In this embodiment, multiple pieces of corpus data to be mined can be obtained. In some implementations, these multiple pieces of corpus data can be obtained from community Q&A, from short texts, or partly from community Q&A and partly from short texts, etc., and are not limited here.

[0033] In some implementations, multiple pieces of data to be mined can be obtained from a server, for example, from community Q&A or short texts recorded on the server. Multiple pieces of data to be mined can also be obtained from other electronic devices, for example, from community Q&A or short texts recorded on other electronic devices. When multiple pieces of data to be mined are obtained from a server or other electronic devices, they can be obtained from the server or other electronic devices through a wireless network or data network.

[0034] In some implementations, taking the acquisition of multiple data points from community Q&A as an example, one can obtain "In which country does the Chestnut-breasted White-faced Warbler live?" as data to be acquired from community Q&A, or "In which country is Alvin a city?" as data to be acquired from community Q&A, etc., without limitation.

[0035] Step S120: Perform double sequence alignment on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined.

[0036] In this embodiment, after acquiring multiple datasets to be mined, pairwise alignment can be performed on the datasets to obtain multiple general sentence patterns corresponding to the datasets. Pairwise alignment is a field of bioinformatics research. Its research method involves designing targeted and efficient algorithms to compare two DNA or protein sequences, finding the maximum similarity match between them, and thus determining whether they are homologous. In this embodiment, pairwise alignment is used to process multiple datasets to obtain the maximum similarity matching sentence patterns among the datasets, i.e., multiple general sentence patterns corresponding to the datasets. This allows for sentence pattern learning transfer by introducing pairwise alignment algorithms from bioinformatics, enabling sentence pattern matching at the byte level and avoiding errors caused by semantic segmentation errors and human spelling mistakes in traditional segmentation methods. In some implementations, after acquiring multiple datasets, pairwise alignment can be performed on each dataset to obtain multiple general sentence patterns corresponding to the datasets.

[0037] For example, taking multiple datasets including "Which country does the Chestnut-breasted White-faced Warbler live in?" and "Which country is Alvin a city in?" as examples, performing a double sequence alignment on the datasets "Which country does the Chestnut-breasted White-faced Warbler live in?" and "Which country is Alvin a city in?" yields a common sentence structure for multiple datasets: (.+?) is (.+?) of which country's (.+?). As another example, taking multiple datasets including "How long does it take to get from Chengdu to Beijing by train?" and "How long does it take to get from Chengdu to Beijing by plane?" as examples, performing a double sequence alignment on the datasets "How long does it take to get from Chengdu to Beijing by train?" and "How long does it take to get from Chengdu to Beijing by plane?" yields a common sentence structure for multiple datasets: How long does it take to get from Chengdu to Beijing? (.+?)

[0038] Step S130: Filter the multiple general sentence patterns and select the general sentence patterns that meet the specified criteria as standard sentence patterns.

[0039] In this process, performing double-sequence alignment on multiple corpora to be mined typically extracts a large number of general sentence patterns. Therefore, a quantitative mechanism can be used to mine sentence patterns with specific concrete meanings and a certain degree of generalization ability. In this embodiment, after obtaining multiple general sentence patterns through double-sequence alignment on multiple corpora to be mined, these general sentence patterns can be filtered to select those that meet specified criteria as standard sentence patterns. These standard sentence patterns can refer to those with specific concrete meanings and a certain degree of generalization ability. Quantitative indicators are then used to measure the generalization degree and concrete meaning of the standard sentence patterns, making the standard sentence patterns mined from multiple corpora to be mined more accurate.

[0040] In some implementations, general sentence pattern filtering rules can be pre-set and stored. After obtaining multiple general sentence patterns corresponding to multiple corpora to be mined, these general sentence patterns can be filtered based on the general sentence pattern filtering rules to select general sentence patterns that meet specified criteria as standard sentence patterns. As one approach, after obtaining multiple general sentence patterns corresponding to multiple corpora to be mined, it can be sequentially determined whether the multiple general sentence patterns satisfy the general sentence pattern filtering rules. Based on the determination results, general sentence patterns that meet the specified criteria can be selected as standard sentence patterns. Specifically, general sentence patterns whose determination results indicate that they satisfy the general sentence pattern filtering rules can be identified as meeting the specified criteria, i.e., as standard sentence patterns. General sentence patterns whose determination results indicate that they do not satisfy the general sentence pattern filtering rules can be identified as not meeting the specified criteria, i.e., as non-standard sentence patterns.

[0041] One embodiment of this application provides a sentence pattern mining method that obtains multiple corpora to be mined, performs double sequence alignment on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined, filters the multiple general sentence patterns, and selects the general sentence patterns that meet the specified criteria as standard sentence patterns. Thus, by performing double sequence alignment on the corpora to be mined to obtain general sentence patterns, and then filtering the general sentence patterns to obtain standard sentence patterns, standard sentence patterns can be obtained from the corpora to be mined quickly and conveniently for processing.

[0042] Please see Figure 2 , Figure 2 A flowchart illustrating a sentence pattern mining method provided in another embodiment of this application is shown. The following will focus on... Figure 2 The process shown will be explained in detail. The sentence pattern mining method may specifically include the following steps:

[0043] Step S210: Obtain multiple data points to be mined.

[0044] Step S220: Perform double sequence alignment on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined.

[0045] For a detailed description of steps S210-S220, please refer to steps S110-S120, which will not be repeated here.

[0046] Step S230: Obtain the sentence inclusion relationship between the multiple general sentence patterns, and obtain the sentence complexity of each general sentence pattern among the multiple general sentence patterns.

[0047] In this embodiment, after obtaining multiple general sentence patterns, the sentence pattern inclusion relationship between the multiple general sentence patterns can be obtained. In some implementations, after obtaining multiple general sentence patterns, the sentence pattern inclusion relationship between the multiple general sentence patterns can be obtained based on the sample coverage of the multiple general sentence patterns. Specifically, after obtaining multiple general sentence patterns, the parent-child node relationship can be divided based on the sample coverage of the multiple general sentence patterns. The sentence pattern with the largest coverage is set as the parent node. Based on the sample coverage of the remaining general sentence patterns from largest to smallest, child nodes at different levels from top to bottom are divided. That is, the parent node has the greatest generalization ability, but it does not have a certain concrete meaning. The generalization ability of child nodes at different levels from top to bottom decreases sequentially, but their concrete meaning increases sequentially.

[0048] Please see Figure 3 , Figure 3 This diagram illustrates the sentence structure inclusion relationships among multiple general sentence structures provided in embodiments of this application. For example... Figure 3 As shown, several general sentence patterns include: general sentence pattern S0, general sentence pattern S1, and general sentence pattern S2. General sentence structure General sentence structure General sentence structure General sentence structure ...among which, the general sentence pattern S0 covers the general sentence pattern General sentence structure and general sentence structure General sentence structure S1 covers general sentence structure General sentence structure and general sentence structure General sentence structure Covering common sentence patterns and common sentence patterns General sentence structure Covering common sentence patterns and common sentence patterns General sentence structure Covering common sentence patterns Therefore, the general sentence patterns S0 and S1 can be identified as parent nodes, and the general sentence patterns can be defined as parent nodes. General sentence structure General sentence structure General sentence structure General sentence structure ...Identified as a child node.

[0049] In this embodiment, after obtaining multiple general sentence patterns, the sentence complexity of each general sentence pattern can be obtained. The higher the sentence complexity of a general sentence pattern, the more complex and concrete the general sentence pattern is; conversely, the lower the sentence complexity of a general sentence pattern, the simpler and less concrete the general sentence pattern is. In some implementations, this can be based on... To obtain the syntactic complexity of each of several general sentence patterns, where n represents the number of times the general sentence pattern is segmented, and t represents the number of characters in each segment of the general sentence pattern. For example, the syntactic complexity of the general sentence pattern "(.+?) is (.+?) from which country (.+?)" is...

[0050] Step S240: Filter the multiple general sentence patterns based on the sentence pattern inclusion relationship between the multiple general sentence patterns and the sentence pattern complexity of each general sentence pattern, and select the general sentence patterns that meet the specified criteria from the multiple general sentence patterns as standard sentence patterns.

[0051] The sentence inclusion relationship among multiple general sentence patterns can be used to reflect the generalization ability of each general sentence pattern, and the sentence complexity of each general sentence pattern can be used to reflect its concrete meaning. Therefore, in this embodiment, after obtaining the sentence inclusion relationship and the sentence complexity of each general sentence pattern, the multiple general sentence patterns can be filtered based on these relationships to select general sentence patterns that meet specified criteria as standard sentence patterns. It is understood that the general sentence patterns selected from the multiple general sentence patterns that meet specified criteria can have a certain generalization ability and a certain concrete meaning, depending on the requirements.

[0052] In some implementations, if the requirement is to select generalized sentence patterns with strong generalization ability and weak concrete meaning, multiple generalized sentence patterns can be filtered based on the sentence pattern inclusion relationship between multiple generalized sentence patterns and the sentence pattern complexity of each generalized sentence pattern, so as to select generalized sentence patterns with large sample coverage and low sentence pattern complexity as standard sentence patterns.

[0053] In some implementations, if the requirement is to select generalized sentence patterns with weak generalization ability and strong concrete meaning, multiple generalized sentence patterns can be filtered based on the sentence pattern inclusion relationship between multiple generalized sentence patterns and the sentence pattern complexity of each generalized sentence pattern, so as to select generalized sentence patterns with small sample coverage and large sentence pattern complexity as standard sentence patterns.

[0054] In some implementations, if the requirement is to select generalized sentence patterns with a certain generalization ability and concrete meaning, multiple generalized sentence patterns can be filtered based on the sentence pattern inclusion relationship between them and the sentence pattern complexity of each generalized sentence pattern. This allows for the selection of generalized sentence patterns whose sentence pattern inclusion relationship with other generalized sentence patterns meets a first specified criterion and whose sentence pattern complexity meets a second specified criterion, thus becoming the standard sentence pattern. The first specified criterion can be pre-set and stored as the basis for judging the sentence pattern inclusion relationship between a certain generalized sentence pattern and other generalized sentence patterns. Therefore, after obtaining the sentence pattern inclusion relationship between a certain generalized sentence pattern and other generalized sentence patterns, this relationship can be compared with the first specified criterion to determine whether the sentence pattern inclusion relationship meets the first specified criterion. The second specified standard can be pre-set and stored as the basis for judging the sentence complexity of each general sentence. Therefore, after obtaining the sentence complexity of each general sentence, the sentence complexity of each general sentence can be compared with the second specified standard to determine whether the sentence complexity of each general sentence meets the second specified standard.

[0055] Please see Figure 4 , Figure 4 This application shows Figure 2 The flowchart shown illustrates step S240 of the sentence pattern mining method. The following will focus on... Figure 4 The process shown will be described in detail, and the method may specifically include the following steps:

[0056] Step S241: Based on the sentence inclusion relationship among the multiple general sentence patterns, obtain the graph in-degree of each general sentence pattern among the multiple general sentence patterns.

[0057] In this embodiment, after obtaining the sentence inclusion relationships among multiple general sentence patterns, the graph in-degree of each general sentence pattern can be obtained based on these relationships. In some implementations, after obtaining the sentence inclusion relationships among multiple general sentence patterns, the graph in-degree of each general sentence pattern can be obtained based on these relationships. Among them, graph in-degree This reflects, to some extent, the generalization ability of this common sentence pattern, such as Figure 3 As shown, common sentence patterns among multiple common sentence patterns in-degree of graph Common sentence patterns among multiple common sentence patterns in-degree of graph Explanation of general sentence patterns Its generalization ability is better than that of general sentence patterns. It has strong generalization ability.

[0058] Step S242: Filter the multiple general sentence patterns based on the graph in-degree and complexity of each general sentence pattern, and select the general sentence pattern that meets the specified criteria from the multiple general sentence patterns as the standard sentence pattern.

[0059] In this context, the graph in-degree of each of the multiple general sentence patterns can be used to reflect its generalization ability, and the sentence complexity of each of the multiple general sentence patterns can be used to reflect its concrete meaning. Therefore, in this embodiment, after obtaining the graph in-degree and sentence complexity of each general sentence pattern, the multiple general sentence patterns can be filtered based on these two parameters to select general sentence patterns that meet the specified criteria as standard sentence patterns. It is understood that the general sentence patterns selected from the multiple general sentence patterns that meet the specified criteria can have a certain generalization ability and a certain concrete meaning, depending on the requirements.

[0060] In some implementations, if the requirement is to select generalized sentence patterns with strong generalization ability and weak concrete meaning, multiple generalized sentence patterns can be filtered based on the graph in-degree and sentence complexity of each generalized sentence pattern to select generalized sentence patterns with large graph in-degree and small sentence complexity as standard sentence patterns.

[0061] In some implementations, if the requirement is to select generalized sentence patterns with weak generalization ability and strong concrete meaning, multiple generalized sentence patterns can be filtered based on the graph in-degree and sentence complexity of each generalized sentence pattern to select generalized sentence patterns with small graph in-degree and large sentence complexity as standard sentence patterns.

[0062] In some implementations, if the requirement is to select generalized sentence patterns with a certain degree of generalization and concrete meaning, multiple generalized sentence patterns can be filtered based on their graph in-degree and sentence complexity. This allows for the selection of generalized sentence patterns whose graph in-degree meets a third specified criterion and whose sentence complexity meets a second specified criterion, thus serving as the standard sentence pattern. The third specified criterion can be pre-set and stored as the basis for determining the graph in-degree of a generalized sentence pattern. Therefore, after obtaining the graph in-degree of a generalized sentence pattern, it can be compared with the third specified criterion to determine whether the graph in-degree of the generalized sentence pattern meets the third specified criterion.

[0063] In some implementations, a specified graph in-degree can be pre-set and stored. This specified graph in-degree serves as the criterion for determining the graph in-degree of each general sentence pattern. When the graph in-degree of a general sentence pattern is greater than the specified graph in-degree, it can be determined that the graph in-degree of the general sentence pattern meets a third specified criterion; when the graph in-degree of a general sentence pattern is not greater than the specified graph in-degree, it can be determined that the graph in-degree of the general sentence pattern does not meet the third specified criterion. Similarly, a specified complexity can be pre-set and stored. This specified complexity serves as the criterion for determining the complexity of each general sentence pattern. When the complexity of a general sentence pattern is greater than the specified complexity, it can be determined that the complexity of the general sentence pattern meets a second specified criterion; when the complexity of a general sentence pattern is not greater than the specified complexity, it can be determined that the complexity of the general sentence pattern does not meet the second specified criterion. Therefore, in this embodiment, based on the aforementioned specified graph in-degree and specified complexity, general sentence patterns with graph in-degree greater than the specified graph in-degree and sentence complexity greater than the specified complexity can be selected from multiple general sentence patterns as standard sentence patterns. This ensures that the obtained standard sentence patterns have a certain generalization ability and a certain concrete meaning.

[0064] Another embodiment of this application provides a sentence pattern mining method that obtains multiple corpora to be mined, performs double sequence alignment on the multiple corpora to obtain multiple general sentence patterns, obtains the sentence pattern inclusion relationship between the multiple general sentence patterns, and obtains the sentence pattern complexity of each general sentence pattern. Based on the sentence pattern inclusion relationship between the multiple general sentence patterns and the sentence pattern complexity of each general sentence pattern, the multiple general sentence patterns are filtered, and general sentence patterns that meet the specified criteria are selected as standard sentence patterns. Compared to Figure 1 The sentence pattern mining method shown in this embodiment filters multiple general sentence patterns by obtaining the sentence pattern inclusion relationship between multiple general sentence patterns and the sentence pattern complexity of each general sentence pattern in order to obtain standard sentence patterns and improve the accuracy of the obtained standard sentence patterns.

[0065] Please see Figure 5 , Figure 5 A flowchart illustrating a sentence pattern mining method provided in another embodiment of this application is shown. The following will focus on... Figure 5 The process shown will be explained in detail. The sentence pattern mining method may specifically include the following steps:

[0066] Step S310: Obtain multiple data points to be mined.

[0067] For a detailed description of step S310, please refer to step S110, which will not be repeated here.

[0068] Step S320: Obtain the sequence type of each piece of data to be mined from the multiple pieces of data to be mined.

[0069] In some implementations, double sequence alignment may include global alignment and local alignment. Global alignment aligns each remaining part of each common sentence and is typically used when the sequence types are similar or the sequence lengths are roughly the same. In this embodiment, global alignment may be the Needleman-Wunsch algorithm based on dynamic programming. Local alignment is more suitable for sequences that are not very similar. In this embodiment, local alignment may be the Smith-Waterman algorithm.

[0070] In this embodiment, in order to select a more suitable method from global comparison and local comparison to perform double sequence alignment on multiple corpora to be mined, the sequence type of each corpus to be mined can be obtained.

[0071] Step S330: Based on the sequence type of each piece of corpus to be mined, determine the processing method for performing double sequence alignment on the multiple pieces of corpus to be mined.

[0072] In this embodiment, after obtaining the sequence type of each piece of corpus to be mined, the processing method for performing double sequence alignment on multiple pieces of corpus to be mined can be determined based on the sequence type of each piece of corpus to be mined. In some embodiments, after obtaining the sequence type of each piece of corpus to be mined, the processing method for performing double sequence alignment on multiple pieces of corpus to be mined can be determined from global alignment and local alignment based on the sequence type of each piece of corpus to be mined.

[0073] Please see Figure 6 , Figure 6 This application shows Figure 5 The flowchart shown is a diagram of step S330 of the sentence pattern mining method. The following will focus on... Figure 6 The process shown will be described in detail, and the method may specifically include the following steps:

[0074] Step S331: Based on the sequence type of each piece of corpus to be mined, obtain the sequence similarity between the multiple pieces of corpus to be mined.

[0075] In some implementations, after obtaining the sequence type of each piece of data to be mined, the sequence similarity between multiple pieces of data can be obtained based on the sequence type of each piece of data. Alternatively, after obtaining the sequence type of each piece of data to be mined, the sequence types of multiple pieces of data to be mined can be matched to obtain the sequence similarity between them.

[0076] Step S332: Based on the sequence similarity between the multiple corpora to be mined, determine the processing method for performing double sequence alignment on the multiple corpora to be mined from the global alignment and the local alignment.

[0077] In some implementations, after obtaining the sequence similarity between multiple corpora to be mined, the processing method for performing double sequence alignment on the multiple corpora to be mined can be determined from global alignment and local alignment based on the sequence similarity between the multiple corpora to be mined. That is, based on the sequence similarity between the multiple corpora to be mined, it is determined that global alignment is used as the processing method for performing double sequence alignment on the multiple corpora to be mined, or local alignment is used as the processing method for performing double sequence alignment on the multiple corpora to be mined.

[0078] Please see Figure 7 , Figure 7 This application shows Figure 6 The flowchart shown illustrates step S332 of the sentence pattern mining method. The following will focus on... Figure 7 The process shown will be described in detail, and the method may specifically include the following steps:

[0079] Step S3321: When the sequence similarity between the multiple corpora to be mined is greater than a specified similarity, the global comparison is determined as a processing method for performing bi-sequence comparison on the multiple corpora to be mined.

[0080] Since global alignment aligns each remaining part of each sequence, it is usually applied to situations where the sequence types are similar or the sequence lengths are roughly the same. Therefore, in this embodiment, when the sequence similarity between multiple corpora to be mined is greater than a specified similarity, global alignment can be determined as a processing method for double sequence alignment of multiple corpora to be mined.

[0081] Step S3322: When the sequence similarity between the multiple corpora to be mined is not greater than a specified similarity, the local alignment is determined as a processing method for performing double sequence alignment on the multiple corpora to be mined.

[0082] Since local alignment is more suitable for situations where sequence types are not very similar, in this embodiment, when the sequence similarity between multiple corpora to be mined is not greater than a specified similarity, local alignment can be determined as the processing method for performing double sequence alignment on multiple corpora to be mined.

[0083] Step S340: Perform double sequence alignment on the multiple corpora to be mined based on the processing method to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined.

[0084] Step S350: Filter the multiple general sentence patterns and select the general sentence patterns that meet the specified criteria as standard sentence patterns.

[0085] For a detailed description of steps S340-S350, please refer to steps S120-S130, which will not be repeated here.

[0086] In another embodiment of this application, a sentence pattern mining method is provided, which involves obtaining multiple corpora to be mined, obtaining the sequence type of each corpus to be mined, determining a processing method for double sequence alignment of the multiple corpora based on the sequence type of each corpus to be mined, performing double sequence alignment of the multiple corpora to be mined based on the processing method, obtaining multiple general sentence patterns corresponding to the multiple corpora to be mined, filtering the multiple general sentence patterns, and selecting general sentence patterns that meet specified criteria as standard sentence patterns. Compared to... Figure 1 The sentence pattern mining method shown in this embodiment determines the double sequence alignment method based on the type of each piece of corpus to be mined, so as to improve the accuracy of the obtained general sentence patterns.

[0087] Please see Figure 8 , Figure 8A flowchart illustrating a sentence pattern mining method provided in another embodiment of this application is shown. The following will focus on... Figure 8 The process shown will be explained in detail. The sentence pattern mining method may specifically include the following steps:

[0088] Step S410: Obtain multiple data points to be mined.

[0089] Step S420: Perform double sequence alignment on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined.

[0090] Step S430: Filter the multiple general sentence patterns and select the general sentence patterns that meet the specified criteria as standard sentence patterns.

[0091] For a detailed description of steps S410-S430, please refer to steps S110-S130, which will not be repeated here.

[0092] Step S440: Output the standard sentence structure.

[0093] In some implementations, after obtaining standard sentence patterns, these patterns can be output to serve subsequent downstream NLP tasks. Based on this, this embodiment can be used to assist in intent recognition: automatically mining high-frequency questions / phrases from users' historical question-and-answer data to help analysts / product managers quickly understand user intent and reduce manual labor costs. Based on this, this embodiment can also be used to improve the performance of text classification models: in short text classification tasks, certain sentence patterns, combined with entity information, effectively process entity-dependent classification text, serving as a prior / external knowledge embedding model. Based on this, this embodiment can also be used as answer templates for community question-and-answer tasks: in NLP question-and-answer tasks, discovering users' high-frequency question patterns allows for the preparation of targeted answer template sentence patterns (in some sensitive vertical domain question-and-answer scenarios, answers to certain questions need to be limited to a specific sentence pattern, such as financial customer service), or mining the sentence patterns of Q and A from large-scale community question-and-answer (Q, A) pairs, organizing A into answer templates for Q.

[0094] Please see Figure 9 , Figure 9 This application shows Figure 8 The flowchart shown illustrates step S440 of the sentence pattern mining method. The following will focus on... Figure 9 The process shown will be described in detail, and the method may specifically include the following steps:

[0095] Step S441: When the standard sentence pattern is an interrogative sentence pattern, obtain the standard response sentence pattern based on the standard sentence pattern.

[0096] In some implementations, the sentence format of the determined standard sentence can be identified. The sentence format may include declarative sentences, interrogative sentences, etc. In this embodiment, when the standard sentence is identified as an interrogative sentence, the standard response sentence corresponding to the standard sentence can be obtained based on the standard sentence. One standard sentence may correspond to one standard response sentence, or multiple standard response sentences, etc., which is not limited here.

[0097] Step S442: Output the standard sentence structure and the standard response sentence structure.

[0098] In some implementations, after obtaining the standard sentence structure and the standard response sentence structure, the standard sentence structure and the standard response sentence structure can be output.

[0099] Another embodiment of this application provides a sentence pattern mining method that obtains multiple pieces of corpus data to be mined, performs double sequence alignment on the multiple pieces of corpus data to obtain multiple general sentence patterns corresponding to the multiple pieces of corpus data, filters the multiple general sentence patterns, selects the general sentence patterns that meet the specified criteria from the multiple general sentence patterns as standard sentence patterns, and outputs the standard sentence patterns. Compared to Figure 1 The sentence pattern mining method shown in this embodiment also outputs standard sentence patterns for use by corresponding downstream tasks, thereby improving the accuracy of downstream task responses.

[0100] Please see Figure 10 , Figure 10 A flowchart illustrating a sentence pattern mining method provided in yet another embodiment of this application is shown. The following will focus on... Figure 10 The process shown will be explained in detail. The sentence pattern mining method may specifically include the following steps:

[0101] Step S510: Obtain the training dataset, which includes multiple corpora and multiple standard sentence patterns.

[0102] The embodiments of this application also include a training method for a sentence pattern mining model. The training of the sentence pattern mining model can be performed in advance based on the acquired training dataset. Subsequently, each time sentence pattern mining is performed, the mining process can be carried out based on the sentence pattern mining model, without the need to train the sentence pattern mining model every time a sentence pattern is mined.

[0103] In some implementations, a training dataset may be collected, which includes multiple corpora and multiple standard questions.

[0104] Step S520: Based on the training dataset, take each corpus as input data and each standard sentence pattern as output data, and train it using a machine learning algorithm to obtain a trained sentence pattern mining model.

[0105] In this embodiment, a machine learning algorithm can be used to train the training dataset to obtain a sentence pattern mining model. The machine learning algorithm used may include: neural networks, Long Short-Term Memory (LSTM) networks, threshold recurrent units, simple recurrent units, autoencoders, decision trees, random forests, feature mean classification, classification regression trees, hidden Markov models, k-Nearest Neighbor (KNN) algorithms, logistic regression models, Bayesian models, Gaussian models, and Kullback-Leibler divergence, etc. The specific machine learning algorithm is not limited.

[0106] The following section uses a neural network as an example to illustrate how to train an initial model based on a training dataset.

[0107] In the training dataset, a set of data containing lexical phrases serves as the input samples (input data) to the neural network, while a set of data containing standard sentence patterns serves as the output samples (output data). Neurons in the input layer are fully connected to neurons in the hidden layers, and neurons in the hidden layers are fully connected to neurons in the output layer, thus enabling the effective extraction of latent features at different granularities. Furthermore, the number of hidden layers can be multiple, allowing for better fitting of non-linear relationships and resulting in a more accurate sentence pattern mining model.

[0108] Understandably, the training process for a sentence pattern mining model can be completed by electronic devices or not. When the training process is not completed by electronic devices, the electronic devices can act as either direct or indirect users.

[0109] In some implementations, the sentence pattern mining model can periodically or irregularly acquire new training data to train and update the model.

[0110] Step S530: Obtain multiple data points to be mined.

[0111] Step S540: Perform double sequence alignment on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined.

[0112] Step S550: Filter the multiple general sentence patterns and select the general sentence patterns that meet the specified criteria as standard sentence patterns.

[0113] For a detailed description of steps S530-S540, please refer to steps S110-S130, which will not be repeated here.

[0114] This application provides another embodiment of a sentence pattern mining method, which obtains a training dataset including multiple corpora and multiple standard sentence patterns. Based on the training dataset, each corpus is used as input data and each standard sentence pattern as output data. A machine learning algorithm is used to train the model, resulting in a trained sentence pattern mining model. Multiple corpora to be mined are obtained, and a double-sequence alignment is performed on these corpora to obtain multiple general sentence patterns corresponding to them. These general sentence patterns are then filtered, and those conforming to specified criteria are selected as standard sentence patterns. Compared to... Figure 1 The sentence pattern mining method shown in this embodiment also collects a training dataset to train and obtain a sentence pattern mining model to mine standard sentence patterns from the corpus, so as to improve the accuracy of the obtained standard sentence patterns.

[0115] Please see Figure 11 , Figure 11 The diagram shows a block diagram of the sentence pattern mining device 200 provided in an embodiment of this application. The following will focus on... Figure 11 The block diagram shown illustrates that the sentence pattern mining device 200 includes: a corpus acquisition module 210, a general sentence pattern acquisition module 220, and a standard sentence pattern acquisition module 230, wherein:

[0116] The data acquisition module 210 is used to acquire multiple data points to be acquired.

[0117] The general sentence pattern acquisition module 220 is used to perform double sequence alignment on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined.

[0118] Furthermore, the general sentence structure acquisition module 220 includes: a sequence type acquisition submodule, a processing method determination submodule, and a general sentence structure acquisition submodule, wherein:

[0119] The sequence type acquisition submodule is used to acquire the sequence type of each piece of data to be mined from the multiple pieces of data to be mined.

[0120] The processing method determination submodule is used to determine the processing method for performing double sequence alignment on the multiple corpora to be mined based on the sequence type of each corpus to be mined.

[0121] Furthermore, the processing method determination submodule includes: a processing method determination unit, wherein:

[0122] The processing method determination unit is used to determine the processing method for performing double sequence alignment on the multiple texts to be mined based on the sequence type of each text to be mined from global alignment and local alignment.

[0123] Furthermore, the processing method determination unit includes: a sequence similarity acquisition subunit and a processing method determination subunit, wherein:

[0124] The sequence similarity acquisition subunit is used to acquire the sequence similarity between the multiple corpora to be mined based on the sequence type of each corpus to be mined.

[0125] The processing method determination subunit is used to determine the processing method for performing double sequence alignment on the multiple corpora to be mined from the global alignment and the local alignment based on the sequence similarity between the multiple corpora to be mined.

[0126] Further, the processing mode determination subunit includes: a first processing mode determination sub-subunit and a second processing mode determination sub-subunit, wherein:

[0127] The first processing method determines the sub-unit, which is used to determine the global comparison as a processing method of performing double sequence comparison on the multiple corpora to be mined when the sequence similarity between the multiple corpora to be mined is greater than a specified similarity.

[0128] The second processing method determines sub-units, which are used to determine the local comparison as a processing method for performing double sequence comparison on the multiple corpora to be mined when the sequence similarity between the multiple corpora to be mined is not greater than a specified similarity.

[0129] The general sentence pattern acquisition submodule is used to perform double sequence alignment on the multiple corpora to be mined based on the processing method to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined.

[0130] The standard sentence pattern acquisition module 230 is used to filter the multiple general sentence patterns and select the general sentence patterns that meet the specified criteria from the multiple general sentence patterns as standard sentence patterns.

[0131] Furthermore, the standard sentence structure acquisition module 230 includes: an information acquisition submodule and a standard sentence structure acquisition submodule, wherein:

[0132] The information acquisition submodule is used to obtain the sentence inclusion relationship between the multiple general sentence patterns and to obtain the sentence complexity of each general sentence pattern.

[0133] Furthermore, the information acquisition submodule includes: a sentence complexity acquisition unit, wherein:

[0134] The sentence complexity acquisition unit is used to obtain the sentence complexity based on Obtain the sentence complexity of each of the multiple general sentence structures, where n represents the number of times the general sentence structure is segmented, and t represents the number of characters in each segment of the general sentence structure.

[0135] The standard sentence pattern acquisition submodule is used to filter the multiple general sentence patterns based on the sentence pattern inclusion relationship between the multiple general sentence patterns and the sentence pattern complexity of each general sentence pattern, and select the general sentence patterns that meet the specified criteria from the multiple general sentence patterns as standard sentence patterns.

[0136] Furthermore, the standard sentence structure acquisition submodule includes: a first standard sentence structure acquisition unit, wherein:

[0137] The first standard sentence pattern acquisition unit is used to select from the plurality of general sentence patterns a general sentence pattern whose sentence pattern inclusion relationship with other general sentence patterns meets a first specified standard and whose sentence pattern complexity meets a second specified standard as a standard sentence pattern.

[0138] Furthermore, the standard sentence structure acquisition submodule includes: a graph in-degree acquisition unit and a second standard sentence structure acquisition unit, wherein:

[0139] The graph in-degree acquisition unit is used to acquire the graph in-degree of each of the multiple general sentence patterns based on the sentence pattern inclusion relationship between the multiple general sentence patterns.

[0140] The second standard sentence pattern acquisition unit is used to filter the multiple general sentence patterns based on the graph in-degree and complexity of each general sentence pattern, and select the general sentence pattern that meets the specified standard from the multiple general sentence patterns as the standard sentence pattern.

[0141] Furthermore, the second standard sentence pattern acquisition unit includes: a standard sentence pattern acquisition subunit, wherein:

[0142] The standard sentence pattern acquisition sub-unit is used to select from the multiple general sentence patterns the general sentence patterns whose graph in-degree meets the third specified criterion and whose sentence pattern complexity meets the second specified criterion as the standard sentence pattern.

[0143] Furthermore, the standard sentence pattern acquisition sub-loop includes: standard sentence pattern acquisition sub-unit, wherein:

[0144] The standard sentence pattern obtains sub-units, which are used to select general sentence patterns from the multiple general sentence patterns that have a graph in-degree greater than a specified graph in-degree and a sentence pattern complexity greater than a specified complexity as standard sentence patterns.

[0145] Furthermore, the sentence pattern mining device 200 also includes: a standard sentence pattern output module, wherein:

[0146] The standard sentence output module is used to output the standard sentence.

[0147] Furthermore, the standard sentence output module includes: a standard response sentence acquisition submodule and a standard sentence output submodule, wherein:

[0148] The standard response format acquisition submodule is used to acquire the standard response format based on the standard format when the standard format is an inquiry format.

[0149] The standard sentence output submodule is used to output the standard sentence and the standard response sentence.

[0150] Furthermore, the sentence pattern mining device 200 also includes: a training dataset acquisition module and a sentence pattern mining model training module, wherein:

[0151] The training dataset acquisition module is used to acquire the training dataset, which includes multiple corpora and multiple standard sentence patterns.

[0152] The sentence pattern mining model training module, based on the training dataset, uses each corpus as input data and each standard sentence pattern as output data, and trains it through machine learning algorithms to obtain a trained sentence pattern mining model.

[0153] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0154] In the several embodiments provided in this application, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0155] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0156] Please see Figure 12 This document illustrates a structural block diagram of an electronic device 100 provided in an embodiment of this application. The electronic device 100 can be a smartphone, tablet computer, e-reader, or other electronic device capable of running applications. The electronic device 100 in this application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications can be stored in the memory 120 and configured to be executed by one or more processors 110, and the one or more applications are configured to perform the methods described in the foregoing method embodiments.

[0157] The processor 110 may include one or more processing cores. The processor 110 connects to various parts within the electronic device 100 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and by calling data stored in the memory 120. Optionally, the processor 110 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 110 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 110 and may be implemented separately using a communication chip.

[0158] The memory 120 may include random access memory (RAM) or read-only memory (ROM). The memory 120 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the electronic device 100 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0159] Please see Figure 13 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 300 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0160] The computer-readable storage medium 300 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 300 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 300 has storage space for program code 310 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 310 may be compressed, for example, in a suitable form.

[0161] In summary, the sentence pattern mining method, apparatus, electronic device, and storage medium provided in this application embodiment acquire multiple corpora to be mined, perform double sequence alignment on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined, filter the multiple general sentence patterns, and select the general sentence patterns that meet the specified criteria as standard sentence patterns. Thus, by performing double sequence alignment on the corpora to be mined to obtain general sentence patterns, and then filtering the general sentence patterns to obtain standard sentence patterns, standard sentence patterns can be obtained quickly and conveniently from the corpora to be mined for processing.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A sentence pattern mining method, characterized in that, The method includes: Obtain multiple data points for further analysis; Double sequence alignment is performed on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined; Obtain the sentence structure inclusion relationship among the multiple general sentence structures, and obtain the sentence structure complexity of each general sentence structure among the multiple general sentence structures; Based on the sentence inclusion relationship between the multiple general sentence patterns and the sentence complexity of each general sentence pattern, the multiple general sentence patterns are filtered, and the general sentence patterns that meet the specified criteria are selected as standard sentence patterns.

2. The method according to claim 1, characterized in that, The step of selecting general sentence patterns that meet specified criteria from the multiple general sentence patterns as standard sentence patterns includes: From the multiple general sentence patterns, the general sentence pattern that satisfies the first specified criterion in its sentence pattern inclusion relationship with other general sentence patterns and the second specified criterion in its sentence pattern complexity is selected as the standard sentence pattern.

3. The method according to claim 1, characterized in that, The method filters the multiple general sentence patterns based on the sentence pattern inclusion relationship between them and the sentence pattern complexity of each general sentence pattern, selecting those that meet specified criteria as standard sentence patterns. Based on the sentence inclusion relationship among the multiple general sentence patterns, obtain the graph in-degree of each general sentence pattern among the multiple general sentence patterns; Based on the graph in-degree and complexity of each general sentence pattern, the multiple general sentence patterns are filtered, and the general sentence patterns that meet the specified criteria are selected as the standard sentence patterns.

4. The method according to claim 3, characterized in that, The step of selecting general sentence patterns that meet specified criteria from the multiple general sentence patterns as standard sentence patterns includes: From the multiple general sentence patterns, the general sentence pattern that satisfies the third specified criterion in-degree and the second specified criterion in sentence complexity is selected as the standard sentence pattern.

5. The method according to claim 4, characterized in that, The step of selecting general sentence patterns from the plurality of general sentence patterns that satisfy the third specified criterion in-degree and the second specified criterion in sentence complexity as standard sentence patterns includes: From the multiple general sentence patterns, select the general sentence patterns whose graph in-degree is greater than the specified graph in-degree and whose sentence complexity is greater than the specified complexity as the standard sentence patterns.

6. The method according to any one of claims 1-5, characterized in that, The step of obtaining the sentence complexity of each of the multiple general sentence patterns includes: based on Obtain the sentence complexity of each of the multiple general sentence patterns, where, This indicates the number of times the general sentence structure is segmented. This indicates the number of characters in each segment of the general sentence structure. This indicates the complexity of the sentence structure.

7. The method according to any one of claims 1-5, characterized in that, The step of performing a double sequence alignment on the multiple datasets to be mined to obtain multiple general sentence patterns corresponding to the multiple datasets to be mined includes: Obtain the sequence type of each piece of data in the multiple pieces of data to be mined; Based on the sequence type of each piece of corpus to be mined, a processing method for performing double sequence alignment on the multiple pieces of corpus to be mined is determined; Based on the processing method, a double sequence alignment is performed on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined.

8. The method according to claim 7, characterized in that, The step of determining the processing method for performing double sequence alignment on multiple pieces of data based on the sequence type of each piece of data to be mined includes: Based on the sequence type of each piece of corpus to be mined, the processing method for performing double sequence alignment on the multiple pieces of corpus to be mined is determined from global alignment and local alignment.

9. The method according to claim 8, characterized in that, The step of determining the processing method for performing double sequence alignment on multiple pieces of text to be mined based on the sequence type of each piece of text to be mined from global and local alignments includes: Based on the sequence type of each piece of corpus to be mined, the sequence similarity between the multiple pieces of corpus to be mined is obtained; Based on the sequence similarity between the multiple corpora to be mined, the processing method for performing double sequence alignment on the multiple corpora to be mined is determined from the global alignment and the local alignment.

10. The method according to claim 9, characterized in that, The method for determining the double sequence alignment of the multiple corpora to be mined based on the sequence similarity between the multiple corpora to be mined from the global alignment and the local alignment includes: When the sequence similarity between the multiple corpora to be mined is greater than a specified similarity, the global comparison is determined to be a two-sequence comparison process for the multiple corpora to be mined. When the sequence similarity between the multiple corpora to be mined is not greater than a specified similarity, the local alignment is determined as a processing method for performing double sequence alignment on the multiple corpora to be mined.

11. The method according to claim 8, characterized in that, The global alignment includes the Needleman–Wunsch algorithm.

12. The method according to claim 8, characterized in that, The local alignment includes the Smith-Waterman algorithm.

13. The method according to any one of claims 1-5, characterized in that, The filtering of the multiple general sentence patterns, selecting those that meet specified criteria as standard sentence patterns, includes: Output the standard sentence structure.

14. The method according to claim 13, characterized in that, The output of the standard sentence structure includes: When the standard sentence structure is an interrogative sentence structure, a standard response sentence structure is obtained based on the standard sentence structure. Output the standard sentence structure and the standard response sentence structure.

15. The method according to any one of claims 1-5, characterized in that, Before acquiring multiple pieces of corpus data to be mined, the process also includes: Obtain a training dataset, which includes multiple corpora and multiple standard sentence patterns; Based on the training dataset, each corpus is used as input data and each standard sentence pattern is used as output data. The training is performed using a machine learning algorithm to obtain a trained sentence pattern mining model.

16. A sentence structure mining device, characterized in that, The device includes: The data acquisition module is used to acquire multiple data points to be mined. A general sentence pattern acquisition module is used to perform double sequence alignment on the multiple corpora to be mined to obtain multiple general sentence patterns corresponding to the multiple corpora to be mined; The information acquisition submodule is used to acquire the sentence structure inclusion relationship between the multiple general sentence structures and to acquire the sentence structure complexity of each general sentence structure among the multiple general sentence structures; The standard sentence pattern acquisition submodule is used to filter the multiple general sentence patterns based on the sentence pattern inclusion relationship between the multiple general sentence patterns and the sentence pattern complexity of each general sentence pattern, and select the general sentence patterns that meet the specified criteria from the multiple general sentence patterns as standard sentence patterns.

17. An electronic device, characterized in that, The method includes a memory and a processor, the memory being coupled to the processor, the memory storing instructions, and the processor performing the method as described in any one of claims 1-15 when the instructions are executed by the processor.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1-15.

Images