Intended mining methods, apparatus, computer equipment and storage media

Abstract

This application relates to an intent mining method, apparatus, computer device, and storage medium. The method includes: acquiring dialogue content of the intent to be mined and mining parameters for the dialogue content specified by the front end; performing preliminary screening of the dialogue content using the mining parameters to obtain multiple statements; if the multiple statements match the expression style of a general corpus, obtaining a corresponding sentence vector that retains general semantic information for each statement; if the multiple statements match the expression style of a business corpus, obtaining multiple corresponding word segments for each statement, and obtaining a sentence vector based on the word vectors and corresponding weights of the word segments; selecting a corresponding clustering algorithm to cluster the sentence vectors corresponding to the multiple statements based on the topic scope features of the topics to which the multiple statements belong; and obtaining the intent for the dialogue content based on the clustering results. This method can improve the efficiency of intent mining.

Description

Technical Field

[0001] This application relates to the field of software technology, and in particular to an intent mining method, apparatus, computer device, and storage medium. Background Technology

[0002] As society continues to develop, various industries offer an increasing number of products, generating a large amount of unstructured data related to these products. This includes dialogue data between users and product customer service personnel, discussions between users and relevant personnel about the products, and so on. Because companies need to continuously improve these products to enhance product quality, they need to analyze this dialogue content to uncover relevant intentions and understand product-related hot topics, pain points, and so on.

[0003] Currently, companies typically use manual methods for data mining. However, due to the diverse nature of dialogue content, manually analyzing this content is inefficient. Summary of the Invention

[0004] Therefore, it is necessary to provide an intent mining method, apparatus, computer device, storage medium, and computer program product that can improve the efficiency of intent mining in response to the above-mentioned technical problems.

[0005] Firstly, this application provides an intent mining method. The method includes:

[0006] Obtain the dialogue content of the intent to be mined specified by the front end and the mining parameters for the dialogue content;

[0007] The mining parameters are used to initially filter the dialogue content, resulting in multiple statements;

[0008] If the multiple statements match the expression style of the general corpus, then for each statement, a corresponding sentence vector that retains the general semantic information is obtained;

[0009] If the multiple statements match the expression style of the business corpus, then for each statement, multiple corresponding word segments are obtained, and a sentence vector is obtained based on the word vectors and corresponding weights of the word segments;

[0010] Based on the topic scope characteristics of the topics to which the multiple statements belong, a corresponding clustering algorithm is selected to cluster the sentence vectors corresponding to the multiple statements;

[0011] The intent for the dialogue content is obtained based on the clustering results.

[0012] Secondly, this application also provides an intent-mining device. The device includes:

[0013] The preprocessing module is used to obtain the dialogue content with the intent to be mined specified by the front end and the mining parameters for the dialogue content; and to perform preliminary filtering on the dialogue content using the mining parameters to obtain multiple statements.

[0014] The vector acquisition module is used to obtain a corresponding sentence vector that retains general semantic information for each statement if the multiple statements match the expression mode of the general corpus; and to obtain multiple corresponding word segments for each statement if the multiple statements match the expression mode of the business corpus, and to obtain a sentence vector based on the word vectors and corresponding weights of the word segments.

[0015] The clustering module is used to select the corresponding clustering algorithm to cluster the sentence vectors corresponding to the multiple statements based on the topic scope features of the topics to which the multiple statements belong;

[0016] An intent determination module is used to determine the intent for the dialogue content based on the clustering results.

[0017] In one embodiment, the vector acquisition module is further configured to, if the plurality of statements match the expression of a general corpus, input the statement into a language vectorization model for each statement to obtain a sentence vector that retains general semantic information; wherein the language vectorization model is trained using the general corpus.

[0018] In one embodiment, the word segmentation includes non-keywords and keywords; the vector acquisition module is further configured to calculate a sentence vector by weighting the word vectors of the keywords and the word vectors of the non-keywords according to their respective weights; wherein the weight of the keywords is greater than the weight of the non-keywords.

[0019] In one embodiment, the clustering module is further configured to determine the topic range features of the topics to which the plurality of sentences belong; if the topic range features represent a topic range that satisfies a first condition, then a density-based clustering algorithm is used to cluster the plurality of sentence vectors; if the topic range features represent a topic range that satisfies a second condition, then a partition-based clustering algorithm is used to cluster the plurality of sentence vectors; the topic range represented by the second condition is greater than the topic range represented by the first condition.

[0020] In one embodiment, the intent determination module is further configured to determine the number of sentence vectors in each cluster of the clustering results; sort the plurality of clusters according to the number of sentence vectors corresponding to each cluster; select the plurality of target clusters with the highest ranking according to the sorting results; and obtain the intent for the dialogue content based on the plurality of target clusters.

[0021] In one embodiment, the intent determination module is further configured to determine the distance between each sentence vector and the central sentence vector in the clusters in the clustering results; select at least one target sentence vector from the clusters based on the distance; and determine the intent for the dialogue content based on the at least one target sentence vector.

[0022] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the steps of the intent mining method described above.

[0023] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, the computer program being executed by a processor through the steps of the intent mining method described above.

[0024] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that is processed by a processor to perform the steps of the intent mining method described above.

[0025] The aforementioned intent mining method, apparatus, computer equipment, storage medium, and computer program product acquire dialogue content of the intent to be mined specified by the front end and mining parameters for the dialogue content; use the mining parameters to perform preliminary screening of the dialogue content to obtain multiple statements. This eliminates the need for users to perform preliminary screening of the dialogue content; they can directly specify the intent on the client side. If the multiple statements match the expression style of a general corpus, a corresponding sentence vector retaining general semantic information is obtained for each statement. If the multiple statements match the expression style of a business corpus, multiple corresponding word segments are obtained for each statement, and a sentence vector is obtained based on the word vectors and corresponding weights of the word segments. Thus, this method can handle dialogue content with both general and business characteristics. Based on the topic to which the multiple statements belong, a corresponding clustering algorithm is selected to cluster the multiple sentence vectors. Based on the topic range characteristics of the topics to which the multiple statements belong, a corresponding clustering algorithm is selected to cluster the sentence vectors corresponding to the multiple statements, and the intent for the dialogue content is obtained based on the clustering results. Therefore, users can specify the dialogue content and mining parameters on the client side, and this method can then extract the corresponding intents without manual intervention, thus improving the efficiency of intent extraction from dialogue content. Furthermore, this method can handle dialogue content that is either general or business-specific, and can select different clustering methods based on topic range characteristics, increasing both the efficiency of intent extraction and the breadth and compatibility of its application. Attached Figure Description

[0026] Figure 1This is a diagram illustrating the application environment of the proposed mining method in one embodiment.

[0027] Figure 2 This is a flowchart illustrating the intent mining method in one embodiment;

[0028] Figure 3 This is a flowchart illustrating the intended mining method in another embodiment;

[0029] Figure 4 This is a schematic diagram illustrating the principle of an intention mining method in one embodiment;

[0030] Figure 5 This is a structural block diagram of an intended excavation device in one embodiment;

[0031] Figure 6 This is an internal structural diagram of a computer device in one embodiment;

[0032] Figure 7 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0034] The intent mining method provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 110 communicates with server 120 via a network. A data storage system can store the data that server 120 needs to process. The data storage system can be integrated onto server 120, or it can be located in the cloud or on other network servers. Terminal 110 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Server 120 can be a standalone server or a server cluster consisting of multiple servers.

[0035] Terminal 110 can serve as a front-end display page, where users specify the dialogue content for which the intent to be mined and the mining parameters for that content. Terminal 110 sends the dialogue content and mining parameters to server 120. Server 120 obtains the dialogue content and mining parameters specified by terminal 110, performs preliminary filtering of the dialogue content using the mining parameters, and obtains multiple statements. If multiple statements match the expression style of a general corpus, a corresponding sentence vector retaining general semantic information is obtained for each statement; if multiple statements match the expression style of a business corpus, multiple word segments are obtained for each statement, and sentence vectors are obtained based on the word vectors and their corresponding weights. Server 120 selects a corresponding clustering algorithm to cluster the sentence vectors corresponding to the multiple statements based on the topic range features of the topics to which they belong; based on the clustering results, the intent for the dialogue content is obtained. Server 120 returns the intent to terminal 110, which can then display the intent mined from the dialogue content on the page.

[0036] In one embodiment, server 120 may also be replaced by a terminal, and there is no limitation thereto.

[0037] In one embodiment, such as Figure 2 As shown, an intent mining method is provided. This embodiment illustrates the method applied to a server, but it is understood that the method can also be applied to a terminal, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0038] S202: Obtain the dialogue content of the intent to be mined specified by the front end and the mining parameters for the dialogue content; use the mining parameters to perform preliminary filtering of the dialogue content to obtain multiple statements.

[0039] Specifically, the user specifies the dialogue content with the intent to be mined and the mining parameters for that dialogue content on the front end. The server receives the dialogue content with the intent to be mined and the mining parameters sent by the front end, and uses the mining parameters to perform preliminary filtering of the dialogue content, resulting in multiple statements.

[0040] It's understandable that by having users specify the dialogue content and mining parameters on the front end, the server can perform initial filtering of the dialogue content and execute subsequent intent mining steps. This allows users to flexibly customize the dialogue content and mining parameters without needing to submit mining requirements to developers or modify the mining processing logic, thus improving the flexibility and convenience of intent mining.

[0041] In one embodiment, the mining parameters include keywords. The server can determine the keywords for each statement in a set of multiple statements based on the keywords specified by the mining parameters. The server can also retain statements containing keywords and filter out statements that do not contain keywords, thereby obtaining the final set of multiple statements.

[0042] In one embodiment, the mining parameters include a minimum number of dialogue turns and a maximum number of dialogue turns. If the number of dialogue turns in a conversation is less than the minimum number of dialogue turns or greater than the maximum number of dialogue turns, the server can filter out that conversation content. It is understood that the server can pre-filter out invalid or abnormal data to reduce unnecessary computing resource consumption.

[0043] In one embodiment, the mining parameters include stop words. The server can filter out stop words from the dialogue content based on the stop words specified by the mining parameters, such as meaningless stop words that may interfere with the results, like "oh," "ah," and "hehe."

[0044] In one embodiment, the mining parameters include deduplication instructions. Each statement in the dialogue content has a unique session identifier. The server can filter statements in the dialogue content that correspond to duplicate session identifiers based on the deduplication instructions specified in the mining parameters.

[0045] In one embodiment, the mining parameters include at least one of keywords, minimum number of dialogue turns, maximum number of dialogue turns, stop words, and deduplication instructions.

[0046] S204. If multiple statements match the expression style of the general corpus, then for each statement, a corresponding sentence vector that retains the general semantic information is obtained; if multiple statements match the expression style of the business corpus, then for each statement, multiple corresponding word segments are obtained, and a sentence vector is obtained based on the word vectors of the word segments and their corresponding weights.

[0047] Business corpus refers to the corpus generated during the conduct of a specific business activity. For example, in the financial industry, buying and selling funds is a business activity, and the corpus generated during this activity is business corpus. In contrast, general corpus is general corpus unrelated to the specific business activity.

[0048] Specifically, the server matches multiple statements against the expression style of the business corpus. If multiple statements match the expression style of the business corpus, then for each statement, multiple corresponding word segments are obtained, and a sentence vector is obtained based on the word vectors and corresponding weights of the word segments. If multiple statements do not match the expression style of the business corpus, the server matches multiple statements against the expression style of the general corpus. If multiple statements match the expression style of the general corpus, then for each statement, a corresponding sentence vector retaining general semantic information is obtained. It can be understood that the order of these steps—whether the multiple statements are matched against the general corpus first or against the business corpus first—is not critical.

[0049] In one embodiment, the sentence vectors that retain general semantic information are obtained based on a trained language vectorization model.

[0050] In one embodiment, word segmentation includes keywords and non-keywords, with keywords having a greater weight than non-keywords. The server calculates a sentence vector by weighting the different word segmentation weights.

[0051] S206. Based on the topic scope characteristics of the topics to which multiple statements belong, select the corresponding clustering algorithm to cluster the sentence vectors corresponding to the multiple statements.

[0052] Among them, topic scope features are used to characterize the attributes of topic scope. Different topics can have different topic scope features. For example, for the financial industry, the topic scope feature of "green finance" can be broad, while the topic scope feature of "green bonds," which belongs to "green finance," is more precise.

[0053] Specifically, the server selects the corresponding clustering algorithm to cluster the sentence vectors corresponding to the multiple statements based on the topic scope characteristics of the topics to which the statements belong.

[0054] In one embodiment, the clustering algorithm can be a partition-based clustering algorithm, a density-based clustering algorithm, a hierarchy-based clustering algorithm, a network-based clustering algorithm, and so on.

[0055] S208, Based on the clustering results, obtain the intent for the dialogue content.

[0056] Specifically, after the server obtains the clustering results, it further analyzes the clustering results to obtain the intent regarding the content of the conversation.

[0057] In one embodiment, the server can calculate the number of sentence vectors in each cluster of the clustering results, select the cluster with the most sentence vectors as the target cluster, and further obtain the intent for the dialogue content based on the target cluster.

[0058] In another embodiment, the server uses each cluster in the clustering results as a target cluster and obtains the intent for the dialogue content based on the target cluster.

[0059] In one embodiment, the server determines the sentence vector closest to the center point in the target cluster, and obtains the intent of the dialogue content based on the intent of the statement corresponding to the sentence vector.

[0060] In another embodiment, the server determines the distance between each sentence vector in the target cluster and the central sentence vector; based on the distance, at least one target sentence vector is selected to determine the intent for the dialogue content.

[0061] The aforementioned intent mining method obtains the dialogue content and mining parameters specified by the front end, representing the intent to be mined. The mining parameters are then used to initially filter the dialogue content, resulting in multiple statements. This eliminates the need for users to perform initial filtering; they can simply specify the intent on the client side. If multiple statements match the expression style of a general corpus, a sentence vector retaining general semantic information is obtained for each statement. If multiple statements match the expression style of a business corpus, multiple word segments are obtained for each statement, and sentence vectors are derived based on the word vectors and their corresponding weights. This method can handle dialogue content that is either general or business-specific. Based on the topics to which the multiple statements belong, a corresponding clustering algorithm is selected to cluster the sentence vectors. Based on the topic scope features of the topics to which the multiple statements belong, a corresponding clustering algorithm is selected to cluster the sentence vectors corresponding to the multiple statements, and the intent for the dialogue content is obtained based on the clustering results. Therefore, users can specify the dialogue content and mining parameters on the client side to execute this method and mine the corresponding intent, eliminating the need for manual mining and thus improving the efficiency of intent mining from dialogue content. Furthermore, this method can handle dialogue content that is either general or business-specific, and can select different clustering methods based on the characteristics of the topic range. This not only improves the efficiency of intent mining but also increases the breadth and compatibility of its application. Moreover, for businesses requiring operational analysis, this method can cluster massive amounts of unstructured data, extracting key information from the data. This assists operational analysts in quickly understanding user needs, identifying problems, and optimizing workflows, eliminating the need for repeated listening to recordings and significantly improving work efficiency.

[0062] In one embodiment, if multiple statements match the expression of a general corpus, obtaining a corresponding sentence vector that retains general semantic information for each statement includes: if multiple statements match the expression of a general corpus, inputting each statement into a language vectorization model to obtain a sentence vector that retains general semantic information; wherein the language vectorization model is trained using a general corpus.

[0063] Specifically, the server pre-trains a language vectorization model using a large amount of general corpus. After the language vectorization model is trained, and multiple sentences are matched with the expression of the general corpus, the server inputs each sentence into the language vectorization model, causing the language vectorization model to output sentence vectors that retain general speech information.

[0064] In this embodiment, the language vectorization model is trained using a large amount of general corpus to obtain sentence vectors that retain general semantic information. This ensures that each sentence vector retains general semantic information, meaning that each sentence vector retains feature information belonging to the same range, thereby improving the accuracy of clustering sentence vectors.

[0065] In one embodiment, word segmentation includes non-keywords and keywords; obtaining sentence vectors based on word vectors and corresponding weights includes: calculating sentence vectors by weighting the word vectors of keywords and non-keywords according to their respective weights; wherein the weight of keywords is greater than the weight of non-keywords.

[0066] Specifically, word segmentation includes non-keywords and keywords. The server calculates the weights of non-keywords and keywords. The weight of keywords is greater than the weight of non-keywords. Further, the server weights the word vectors of keywords and non-keywords according to their respective weights to obtain sentence vectors. It can be understood that increasing the weight of keywords allows the sentence vectors to retain more keyword information, thus meeting the user's specified needs. For example, if a user specifies that they want to confirm whether a customer service representative is inquiring about 5G services, then the word "5G" is much more important than other words. The server can use "5G" as a keyword, weight the word vectors to obtain the sentence vector, and increase the weight of keywords within the sentence vector.

[0067] In one embodiment, the server can obtain the weights of word segmentation by executing the TF-IDF (term frequency–inverse document frequency, a common weighting technique used in information retrieval and data mining) algorithm, i.e., word segmentation weight = TF * IDF. The server can calculate TF (Term Frequency) for each sentence rather than the entire document and increase the weight of IDF (Inverse Document Frequency). For example, taking financial-related dialogue content as an example, the TF-IDF formula can be adjusted with weights, resulting in the following optimized formula:

[0068]

[0069]

[0070] Where W represents the keyword, TFw is the term frequency of keyword W, and IDFw is the inverse text frequency index of keyword W. It can be seen that the server calculates TF (Term Frequency) for each sentence rather than the entire document, and the weight of IDFw is increased.

[0071] In this embodiment, by increasing the weight of keywords, the sentence vectors can retain more semantic information of the keywords, thereby obtaining more accurate results when clustering the sentence vectors.

[0072] In one embodiment, selecting a corresponding clustering algorithm to cluster the sentence vectors corresponding to multiple statements based on the topic features of the topics to which the multiple statements belong includes: determining the topic range features of the topics to which the multiple statements belong; if the topic range represented by the topic range features satisfies a first condition, then using a density-based clustering algorithm to cluster the multiple sentence vectors; if the topic range represented by the topic range features satisfies a second condition, then using a partition-based clustering algorithm to cluster the multiple sentence vectors; the topic range represented by the second condition is greater than the topic range represented by the first condition.

[0073] Specifically, the server determines the topic scope features of multiple sentences and evaluates these features. If the topic scope features satisfy a first condition, a density-based clustering algorithm is used to cluster the sentence vectors. If the topic scope features satisfy a second condition, a partitioning-based clustering algorithm is used. The topic scope represented by the second condition is larger than that represented by the first condition. In essence, the server selects different clustering algorithms based on the size of the topic scope to obtain more accurate clustering results for the sentence vectors.

[0074] In one embodiment, the server may select the k-means algorithm as the partition-based clustering algorithm.

[0075] In another embodiment, the server may choose the k-medoids algorithm as the partition-based clustering algorithm.

[0076] In this embodiment, different clustering algorithms are selected by judging the topic range represented by the topic range features. Since partition-based clustering algorithms are insensitive to noise and suitable for scenarios requiring coarse-grained clustering, while density-based clustering algorithms are suitable for scenarios requiring fine-grained clustering, in this embodiment, the topic range corresponding to the partition-based clustering algorithm is larger than that corresponding to the density-based clustering algorithm, thus making the clustering of sentence vectors more accurate and better meeting the expected requirements.

[0077] In one embodiment, obtaining the intent for the dialogue content based on the clustering results includes: determining the number of sentence vectors in each cluster of the clustering results; sorting multiple clusters according to the number of sentence vectors corresponding to each cluster; selecting multiple target clusters with the highest rankings according to the ranking results; and obtaining the intent for the dialogue content based on the multiple target clusters.

[0078] Specifically, the server calculates the number of sentence vectors in each cluster of the clustering results. The server then sorts the multiple clusters based on the number of sentence vectors corresponding to each cluster, and selects the top-ranked target clusters based on the sorting results. The server then obtains the intent for the dialogue content based on these multiple target clusters.

[0079] In this embodiment, the cluster with the most sentence vectors in each cluster is selected as the target cluster by calculating the number of sentence vectors in each cluster. In this way, the server completes the inter-cluster cleaning operation, removing low-quality clusters and retaining high-quality clusters. Furthermore, the intent for the dialogue content is obtained based on the high-quality clusters, making the derived intent the most accurate.

[0080] In one embodiment, obtaining the intent for the dialogue content based on the clustering results includes: determining the distance between each sentence vector in the cluster and the central sentence vector for the clusters in the clustering results; selecting at least one target sentence vector from the clusters based on the distance; and determining the intent for the dialogue content based on the at least one target sentence vector.

[0081] Specifically, the server can determine the distance between each sentence vector in a cluster and the central sentence vector for a subset or all clusters in the clustering results. Based on the distance, the server selects at least one target sentence vector from the clusters and determines the intent for the dialogue content based on at least one target sentence vector.

[0082] In one embodiment, the server can determine the distance between each sentence vector in a cluster and the central sentence vector for a target cluster (i.e., a subset of clusters) in the clustering results. Based on the distance, at least one target sentence vector is selected from the clusters.

[0083] In one embodiment, the mining parameters may specify at least one preset distance index to select target sentence vectors that conform to the preset distance index.

[0084] In one embodiment, the server may select the sentence vector with the smallest distance as the target sentence vector. This means the server retains the highest-quality sentence vector within a given class.

[0085] In one implementation, the intents received by the server are repetitive, and the server needs to perform deduplication processing to obtain the final intent for the dialogue content.

[0086] In this implementation, at least one target sentence vector is obtained through distance, and the intent regarding the dialogue content is determined based on this target sentence vector. This allows the server to flexibly adjust the strategy for acquiring the target sentence vector using distance.

[0087] In one embodiment, such as Figure 3 The diagram illustrates the flowchart of the intent mining method. Specifically, the user creates a clustering task in the clustering tool on the user's end and configures the dialogue content and corresponding mining parameters. The clustering tool sends a clustering task initiation command. This command can be sent via an HTTP (Hypertext Transfer Protocol) request. The server, acting as the backend, runs the clustering service and receives the clustering task initiation command through it. The clustering service retrieves the corresponding dialogue content and mining parameters based on the command. Further, the service performs preliminary filtering of the dialogue content based on the mining parameters, obtaining multiple statements. Data features are extracted from these statements to obtain sentence vectors. Based on the topic range features of the statements, the appropriate clustering algorithm is selected for clustering. The clustering service obtains the intent of the dialogue content based on the clustering results and returns this intent to the clustering tool on the user's end, allowing the user to view the intent related to the dialogue content on the tool's interface.

[0088] In one embodiment, such as Figure 4The diagram illustrates the principle of the intent mining method. Specifically, the server obtains the dialogue content and mining parameters specified by the front end, representing the intent to be mined. The server uses the mining parameters to initially filter the dialogue content, resulting in multiple sentences. If multiple sentences match the expression style of a general corpus, then for each sentence, the sentence is input into a language vectorization model to obtain a sentence vector that retains general semantic information; the language vectorization model is trained using a general corpus. If multiple sentences match the expression style of a business corpus, then for each sentence, multiple corresponding word segments are obtained, including non-keywords and keywords. The server calculates the sentence vector by weighting the word vectors of keywords and non-keywords according to their respective weights. The weight of keywords is greater than the weight of non-keywords. This can be understood as follows: Figure 4 Sentence vectors 1, 2, and n can be derived from a language vectorization model or from a weighted calculation based on multiple word segments, depending on the type of corpus the sentences are matched with. The server determines the topic scope features of the multiple sentences. If the topic scope features satisfy the first condition, a density-based clustering algorithm is used to cluster the sentence vectors; if the topic scope features satisfy the second condition, a partition-based clustering algorithm is used. The topic scope represented by the second condition is larger than that represented by the first condition. The clustering results contain multiple clusters, such as... Figure 4 The clustering results for sentence vectors 1 to n are clusters 1, 2, and m. The server determines the number of sentence vectors in each cluster and sorts the clusters according to the number of sentence vectors in each cluster. The server then selects the top-ranked target clusters based on the sorting results. Figure 4 The server sorts clusters 1 to m and selects target cluster 1 and target cluster k. The server can also determine the distance between each sentence vector in the target cluster and the central sentence vector. Further, based on the distance, the server selects at least one target sentence vector from the cluster. For example... Figure 4 The target sentence vector selected for target cluster 1 is the sentence vector of the first and second sentences. For example... Figure 4 The target sentence vectors selected for target cluster k are the sentence vectors of the third and fourth sentences. Further, the server determines the intent regarding the dialogue content based on at least one target sentence vector. For example... Figure 4 As shown, the server obtains the first intent based on the first and second statements, and the second intent based on the second and third statements. The server can use the first and second intents as intents for the dialogue content.

[0089] It should be understood that although the steps in the flowcharts of some embodiments of this application are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple steps or multiple stages, which are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0090] Based on the same inventive concept, this application also provides an intent mining apparatus for implementing the intent mining method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more intent mining apparatus embodiments provided below can be found in the limitations of the intent mining method described above, and will not be repeated here.

[0091] In one embodiment, such as Figure 5 As shown, an intent mining device 500 is provided, including: a preprocessing module 502, a vector acquisition module 504, a clustering module 506, and an intent determination module 508, wherein:

[0092] The preprocessing module 502 is used to obtain the dialogue content of the intent to be mined specified by the front end and the mining parameters for the dialogue content; and to use the mining parameters to perform preliminary screening on the dialogue content to obtain multiple statements.

[0093] The vector acquisition module 504 is used to obtain a corresponding sentence vector that retains general semantic information for each statement if the multiple statements match the expression mode of the general corpus; and to obtain multiple corresponding word segments for each statement if the multiple statements match the expression mode of the business corpus, and to obtain a sentence vector based on the word vectors and corresponding weights of the word segments.

[0094] The clustering module 506 is used to select the corresponding clustering algorithm to cluster the sentence vectors corresponding to the multiple statements based on the topic range characteristics of the topics to which the multiple statements belong.

[0095] The intent determination module 508 is used to obtain the intent for the dialogue content based on the clustering results.

[0096] In one embodiment, the vector acquisition module 504 is further configured to, if the plurality of statements match the expression of a general corpus, input the statement into a language vectorization model for each statement to obtain a sentence vector that retains general semantic information; wherein the language vectorization model is trained using the general corpus.

[0097] In one embodiment, the word segmentation includes non-keywords and keywords; the vector acquisition module 504 is further configured to calculate a sentence vector by weighting the word vectors of the keywords and the word vectors of the non-keywords according to their respective weights; wherein the weight of the keywords is greater than the weight of the non-keywords.

[0098] In one embodiment, the clustering module 506 is further configured to determine the topic range features of the topics to which the plurality of sentences belong; if the topic range features represent a topic range that satisfies a first condition, then a density-based clustering algorithm is used to cluster the plurality of sentence vectors; if the topic range features represent a topic range that satisfies a second condition, then a partition-based clustering algorithm is used to cluster the plurality of sentence vectors; the topic range represented by the second condition is greater than the topic range represented by the first condition.

[0099] In one embodiment, the intent determination module 508 is further configured to determine the number of sentence vectors in each cluster of the clustering results; sort the plurality of clusters according to the number of sentence vectors corresponding to each cluster; select the plurality of target clusters with the highest ranking according to the sorting results; and obtain the intent for the dialogue content based on the plurality of target clusters.

[0100] In one embodiment, the intent determination module 508 is further configured to determine the distance between each sentence vector and the central sentence vector in the clusters in the clustering results; select at least one target sentence vector from the clusters based on the distance; and determine the intent for the dialogue content based on the at least one target sentence vector.

[0101] The aforementioned intent mining device acquires the dialogue content for which the intent to be mined is specified by the front end, along with mining parameters for the dialogue content. It then uses the mining parameters to perform preliminary filtering of the dialogue content, resulting in multiple statements. This eliminates the need for users to perform preliminary filtering of the dialogue content; they can simply specify the intent on the client side. If the multiple statements match the expression style of a general corpus, a corresponding sentence vector retaining general semantic information is obtained for each statement. If the multiple statements match the expression style of a business corpus, multiple corresponding word segments are obtained for each statement, and a sentence vector is obtained based on the word vectors and corresponding weights of the word segments. This method can handle dialogue content that is either general or business-specific. Based on the topic to which the multiple statements belong, a corresponding clustering algorithm is selected to cluster the multiple sentence vectors. Based on the topic range characteristics of the topics to which the multiple statements belong, a corresponding clustering algorithm is selected to cluster the sentence vectors corresponding to the multiple statements, and the intent for the dialogue content is obtained based on the clustering results. Therefore, users can specify the dialogue content and mining parameters on the client side to execute this method to mine the corresponding intent, eliminating the need for manual mining and thus improving the efficiency of intent mining from dialogue content. Furthermore, this method can handle dialogue content that is general or business-specific, and can also select different clustering methods based on the characteristics of the topic range. While improving the efficiency of intent mining, it also increases the breadth and compatibility of the application of this method.

[0102] For specific limitations regarding the aforementioned intent mining device, please refer to the limitations of the aforementioned intent mining method above, which will not be repeated here. Each module in the aforementioned intent mining device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0103] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores dialogue content data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements an intent mining method.

[0104] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an intent mining method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0105] Those skilled in the art will understand that Figure 6 and Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0106] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0107] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0108] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0109] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0110] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0111] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An intent mining method, characterized in that, The method includes: Obtain the dialogue content of the intent to be mined specified by the front end and the mining parameters for the dialogue content; the mining parameters include stop words; The dialogue content is initially filtered using the mining parameters to obtain multiple statements, including: filtering out the stop words from the dialogue content according to the stop words specified in the mining parameters to obtain the multiple statements; If the multiple statements match the expression style of the general corpus, then for each statement, a corresponding sentence vector that retains the general semantic information is obtained; If the multiple statements match the expression style of the business corpus, then for each statement, multiple corresponding word segments are obtained, and a sentence vector is obtained based on the word vectors and corresponding weights of the word segments; wherein, the business corpus is the corpus generated in the process of carrying out specific business, and the general corpus is a general corpus unrelated to the specific business; the word segmentation includes keywords and non-keywords, and the weight of the keywords is greater than the weight of the non-keywords; Based on the topic scope features of the topics to which the multiple statements belong, a corresponding clustering algorithm is selected to cluster the sentence vectors corresponding to the multiple statements; the topic scope features are used to characterize the attributes of topic scope. The intent for the dialogue content is obtained based on the clustering results.

2. The method according to claim 1, characterized in that, If the multiple statements match the expression style of a general corpus, then for each statement, the corresponding sentence vector that retains general semantic information includes: If the multiple statements match the expression of the general corpus, then for each statement, the statement is input into the language vectorization model to obtain a sentence vector that retains general semantic information; wherein, the language vectorization model is trained using the general corpus.

3. The method according to claim 1, characterized in that, The word segmentation includes non-keywords and keywords; the process of obtaining sentence vectors based on the word vectors and corresponding weights of the segmented words includes: The sentence vector is obtained by weighting the word vectors of the keywords and the word vectors of the non-keywords according to their respective weights; wherein the weight of the keywords is greater than the weight of the non-keywords.

4. The method according to claim 1, characterized in that, The step of selecting a corresponding clustering algorithm to cluster the sentence vectors corresponding to the multiple statements based on the topic features of the topics to which the multiple statements belong includes: Determine the topic scope characteristics of the topics to which the multiple statements belong; If the topic range represented by the topic range feature satisfies the first condition, then a density-based clustering algorithm is used to cluster the multiple sentence vectors. If the topic range represented by the topic range feature satisfies the second condition, then a partition-based clustering algorithm is used to cluster the multiple sentence vectors; the topic range represented by the second condition is greater than the topic range represented by the first condition.

5. The method according to claim 1, characterized in that, The intent derived from the clustering results for the dialogue content includes: Determine the number of sentence vectors in each cluster of the clustering results; The multiple clusters are sorted according to the number of sentence vectors corresponding to each cluster; Select the top-ranked target clusters based on the sorting results; The intent for the dialogue content is obtained based on the multiple target clusters.

6. The method according to any one of claims 1 to 5, characterized in that, The intent derived from the clustering results for the dialogue content includes: For each cluster in the clustering results, determine the distance between each sentence vector in the cluster and the central sentence vector; Based on the distance, at least one target sentence vector is selected from the cluster; The intent for the dialogue content is determined based on the at least one target sentence vector.

7. An intentional excavation device, characterized in that, The device includes: A preprocessing module is used to obtain the dialogue content of the intent to be mined specified by the front end and the mining parameters for the dialogue content; the mining parameters include stop words; the dialogue content is initially filtered using the mining parameters to obtain multiple statements, including: filtering out the stop words from the dialogue content according to the stop words specified in the mining parameters to obtain the multiple statements; The vector acquisition module is used to obtain a corresponding sentence vector that retains general semantic information for each statement if the multiple statements match the expression style of a general corpus; and to obtain multiple corresponding word segments for each statement if the multiple statements match the expression style of a business corpus, and to obtain a sentence vector based on the word vectors and corresponding weights of the word segments. The business corpus is corpus generated during the implementation of a specific business, while the general corpus is general corpus unrelated to the specific business. The word segmentation includes keywords and non-keywords, with the weight of keywords being greater than the weight of non-keywords. The clustering module is used to select the corresponding clustering algorithm to cluster the sentence vectors corresponding to the multiple statements based on the topic scope features of the topics to which the multiple statements belong; the topic scope features are used to characterize the attributes of the topic scope. An intent determination module is used to determine the intent for the dialogue content based on the clustering results.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images