A data pattern mining method, device, storage medium and terminal

Abstract

The application discloses a data mode mining method and device, a storage medium and a terminal. The method comprises the following steps: obtaining and preprocessing to-be-mined data, and generating a root node; when the root node does not satisfy a preset mining termination condition, determining a target word with the maximum word frequency according to an element of the root node; performing node segmentation on the root node according to the target word with the maximum word frequency, and generating a left child node containing the target word and a right child node not containing the target word; and constructing a mode node set of the to-be-mined data according to the left child node and the right child node. Since the application takes the target word with the maximum appearance frequency as the segmentation condition by counting the appearance frequency of the word, and further generates the mode node set, the mode mining is realized, the fitting depth of the mode is controlled by adjusting the mode mining termination condition, and therefore the interpretability and reliability of the mode mining result are enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, and particularly relates to a data pattern mining method and device, a storage medium and a terminal. BACKGROUND

[0002] Pattern mining is a kind of information mining technology developed by using computers and various algorithms, and is part of data mining. So far, the idea of pattern mining has been applied to various fields such as electronic communication networks, social networks and scientific research. The research on pattern mining can find valuable information in a large amount of data, which can help researchers to better analyze the association rules, correlation, sequential patterns and anomaly detection of data, and to carry out subsequent prediction research through pattern discovery and extraction.

[0003] At present, one method of pattern mining is to use a deep learning method. The deep learning algorithm is used to extract features and classify data sets, and to mine frequent items of data, so as to mine the behavior template sequence of data. However, this method may depend on subjective selection of the algorithm, has high requirements for samples, and is prone to problems such as dimension disaster and underfitting. In addition, in the bottom-up deep learning, it is difficult to know whether the decision basis behind it is reliable, and the interpretability is relatively poor. SUMMARY

[0004] The present application provides a data pattern mining method, device, storage medium and terminal. In order to have a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This part is not a general review, nor does it determine the key / important elements or describe the protection scope of these embodiments. The only purpose is to present some concepts in a simple form as a preface to the detailed description below.

[0005] In a first aspect, the present application provides a data pattern mining method, which comprises:

[0006] acquiring and preprocessing the data to be mined to generate a root node;

[0007] when the root node does not satisfy a preset mining termination condition, determining a target word with the maximum word frequency according to the elements of the root node;

[0008] performing node segmentation on the root node according to the target word with the maximum word frequency to generate a left child node containing the target word and a right child node not containing the target word;

[0009] constructing a pattern node set of the data to be mined according to the left child node and the right child node.

[0010] Optionally, acquiring and preprocessing the data to be mined to generate a root node comprises:

[0011] obtaining data to be mined;

[0012] dividing the data to be mined into a plurality of processing individuals according to a preset rule;

[0013] segmenting data content of each processing individual to obtain a segmentation result of each processing individual;

[0014] placing the segmentation result, line number and preset pattern identifier of each processing individual as a single element in a preset node to obtain a root node.

[0015] Optionally, a target word with the maximum word frequency is determined according to the elements of the root node, including:

[0016] eliminating repeated words in the segmentation result of each element in the root node to obtain a plurality of eliminated segmentation results;

[0017] summarizing the plurality of eliminated segmentation results to obtain a summary vocabulary;

[0018] counting the word frequency of each word in the summary vocabulary and determining the target word with the maximum word frequency.

[0019] Optionally, the root node is split according to the target word with the maximum word frequency, including:

[0020] determining whether the segmentation result of each element in the root node contains the target word;

[0021] if yes, the element containing the target word is divided into a left child node containing the target word;

[0022] if no, the element not containing the target word is divided into a right child node containing the target word.

[0023] Optionally, a pattern node set of the data to be mined is constructed according to the left child node and the right child node, including:

[0024] calculating the similarity between each element in the left child node and all elements of the right child node one by one to obtain a plurality of similarities of each element in the left child node;

[0025] copying the elements with a similarity greater than a preset similarity to the right child node according to the plurality of similarities of each element in the left child node to obtain an updated left child node and an updated right child node;

[0026] constructing the pattern node set of the data to be mined according to the updated left child node and the updated right child node.

[0027] Optionally, a pattern node set of the data to be mined is constructed according to the updated left child node and the updated right child node, including:

[0028] The same word as the target word in the word segmentation result of each element in the updated left child node is retained, and the remaining words in the word segmentation result except the target word are converted into identifiers, and adjacent same identifiers are merged and processed to obtain a pattern of each element in the left child node;

[0029] Based on the pattern of each element in the left child node, elements with the same pattern are aggregated and processed, and the aggregated elements and the patterns corresponding thereto are determined as a plurality of pattern nodes corresponding to the left child node;

[0030] The updated right child node is processed to obtain a plurality of pattern nodes corresponding to the right child node;

[0031] The plurality of pattern nodes corresponding to the left child node and the plurality of pattern nodes corresponding to the right child node are determined as a pattern node set of the data to be mined.

[0032] Optionally, after the pattern node set of the data to be mined is constructed according to the left child node and the right child node, the method further includes:

[0033] When each node in the pattern node set of the data to be mined satisfies a preset mining termination condition, a pattern matching degree between each element in each node and each pattern in the pattern node set is calculated to obtain a plurality of pattern matching degrees of each element;

[0034] The pattern corresponding to the maximum pattern matching degree in the plurality of pattern matching degrees of each element is determined as a final pattern of each element.

[0035] In a second aspect, an embodiment of the present application provides a mode mining device for data, and the device includes:

[0036] A root node generation module is configured to acquire and preprocess data to be mined, and generate a root node;

[0037] A target word determination module is configured to determine a target word with the maximum word frequency according to elements of the root node when the root node does not satisfy a preset mining termination condition;

[0038] A node segmentation module is configured to perform node segmentation on the root node according to the target word with the maximum word frequency, and generate a left child node containing the target word and a right child node not containing the target word;

[0039] A pattern node set construction module is configured to construct a pattern node set of the data to be mined according to the left child node and the right child node.

[0040] In a third aspect, an embodiment of the present application provides a computer storage medium, which stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and performing the method steps described above.

[0041] In a fourth aspect, an embodiment of the present application provides a terminal, which can include a processor and a memory; wherein the memory stores a computer program, and the computer program is adapted to be loaded by the processor and execute the method steps described above.

[0042] The technical scheme provided by the embodiments of the present application can have the following beneficial effects:

[0043] In the embodiments of the present application, the mode mining device of data first acquires and pre-processes the data to be mined, generates a root node, and then when the root node does not satisfy the preset mining termination condition, determines a target word with the maximum word frequency according to the elements of the root node, secondly performs node segmentation on the root node according to the target word with the maximum word frequency to generate a left child node containing the target word and a right child node not containing the target word, and finally constructs a mode node set of the data to be mined according to the left child node and the right child node. Since the present application takes the target word with the maximum appearance frequency as the segmentation condition by counting the appearance frequency of the word, and further generates a mode node set, the mode mining is realized, and at the same time, the present application controls the fitting depth of the mode by adjusting the mode mining termination condition, thereby enhancing the interpretability and reliability of the mode mining result.

[0044] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0046] Figure 1 is a flowchart of a mode mining method of data provided by an embodiment of the present application;

[0047] Figure 2 is a schematic diagram of data conversion into a mode provided by an embodiment of the present application;

[0048] Figure 3 is a flowchart of a mode mining process of data provided by the present application;

[0049] Figure 4 is a structural schematic diagram of a mode mining device of data provided by an embodiment of the present application;

[0050] Figure 5 is a structural schematic diagram of a terminal provided by an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following description and drawings sufficiently illustrate specific embodiments of the present application to enable those skilled in the art to practice them.

[0052] It should be noted that the described embodiments are merely some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0053] The following description refers to the accompanying drawings. In the following description, same numbers in different drawings represent the same or similar elements unless otherwise represented. The implementations described in the following exemplary embodiments are not meant to represent all implementations consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0054] In the description of the present application, it should be understood that the terms "first", "second", etc. are used only for descriptive purposes and cannot be understood as indicating or implying relative importance. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, "a plurality of" means two or more, unless otherwise stated. "And / or", which describes the relationship between the associated objects, means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after are in an "or" relationship.

[0055] The present application provides a data pattern mining method, device, storage medium and terminal to solve the above-mentioned related technical problems. In the technical scheme provided by the present application, the present application takes the target word with the highest frequency as the segmentation condition by counting the frequency of the word, and further generates a pattern node set, thereby realizing pattern mining. At the same time, the present application controls the fitting depth of the pattern by adjusting the termination condition of the pattern mining, thereby enhancing the interpretability and reliability of the pattern mining result. The following exemplary embodiments are described in detail.

[0056] The following will be described in conjunction with the accompanying drawings Figure 1 -Appendix Figure 3 The data pattern mining method provided by the embodiment of the present application is described in detail. The method can be realized by relying on a computer program and can be run on a data pattern mining device based on the von Neumann system. The computer program can be integrated in an application or run as an independent tool class application.

[0057] Please refer to Figure 1 A flowchart of a data pattern mining method provided by the embodiment of the present application is shown. As Figure 1 The method of the embodiment of the present application can include the following steps:

[0058] S101, acquire and pre-process the data to be mined to generate a root node;

[0059] The data to be mined is original data that needs to be subjected to pattern mining, and the root node is a component of a tree, also called a tree root, and is the topmost node in a non-empty binary tree.

[0060] Generally, pattern mining is to extract fixed patterns hidden in complex structured and large database text information, and the fixed patterns are composed of constant strings and variable values. The fixed patterns can be understood as omitting some variables to obtain the corresponding event template. The goal of pattern mining is to convert each text data into a specific event template associated with a key parameter, and to structure the free text content, for example Figure 2 As shown in the table, the template of the data in the 3rd, 5th and 7th rows after pattern mining is the content of the 4th row, i.e., "HTTP request URL:*".

[0061] In the embodiments of the present application, when acquiring and pre-processing the data to be mined to generate a root node, first, the data to be mined is acquired, and then the data to be mined is divided into multiple processing individuals according to a preset rule. Next, the data content of each processing individual is segmented to obtain the segmentation result of each processing individual. Finally, the segmentation result, line number and preset pattern identifier of each processing individual are placed in a preset node as a single element to obtain the root node.

[0062] In one possible implementation, first, the data to be mined is read, and the read data is divided into processing individuals for pattern mining according to a preset rule. In this example, the division is performed by row, and each row is a processing individual obj, as shown in Table 1.

[0063] Table 1

[0064] Line number Data content 1 This cat 2 That cat 3 The cat is animal 4 I was here 5 I will go

[0065] Further, the data content in Table 1 is segmented according to a preset rule. For example, English can be segmented by space as a delimiter, and Chinese can be segmented by a segmentation tool. In this example, the obj line＝1 with a line number of 1 has a data content, and the segmentation result is [This, cat]; in the pattern expression of the present method, the preset pattern identifier * is used to adapt to any number of any words, and at this time, the pattern of the obj line＝1 is *, and the line number, segmentation result and pattern corresponding to each obj are placed in a node as an element to obtain a root node, as shown in Table 2.

[0066] Table 2

[0067]

[0068]

[0069] S102, when the root node does not satisfy the preset mining termination condition, determining a target word with the maximum word frequency according to elements of the root node;

[0070] The preset mining condition is used to determine whether the newly generated node satisfies the condition that needs to be processed. For example, the preset mining condition can be whether the depth of the node is less than the maximum depth of the preset pattern mining tree, or whether the number of elements of the node is greater than the minimum number of elements.

[0071] Generally, when the node satisfies the preset mining termination condition, that is, all nodes are determined not to need to be processed, the algorithm ends. Otherwise, further processing is needed.

[0072] In the embodiments of the present application, when further processing is needed, first, the repeated words in the segmentation results of each element in the root node are removed to obtain a plurality of removed segmentation results, then the plurality of removed segmentation results are summarized to obtain a summary vocabulary, and finally the word frequency of each word in the summary vocabulary is counted, and a target word with the maximum word frequency is determined.

[0073] In one possible implementation, first, the repeated words in the segmentation results of each element of the root node in Table 2 are removed, then all the segmentation results with the repeated words removed in Table 2 are summarized together, the number of occurrences of each word is counted, and the word with the maximum word frequency Token max .

[0074] Further, if there are several words with the maximum word frequency, one word can be randomly selected from the several words with the maximum word frequency as the target word, or the first word is taken as the target word at this time. After counting the words of the root node in Table 2, the statistical result is shown in Table 3. It can be seen from Table 3 that the word with the maximum word frequency at this time is cat.

[0075] Table 3

[0076]

[0077]

[0078] S103, according to the target word with the maximum word frequency, the root node is segmented to generate a left child node containing the target word and a right child node not containing the target word;

[0079] In the embodiments of the present application, when the root node is divided according to the target word with the maximum word frequency, firstly, it is judged whether the word segmentation result of each element in the root node contains the target word, if yes, the element containing the target word is divided into the left child node containing the target word, if not, the element not containing the target word is divided into the right child node containing the target word.

[0080] In a possible implementation, the word segmentation result in Table 2 is divided according to the target word with the maximum word frequency, if the word segmentation result of each element contained by each node contains the target word Token max with the maximum word frequency, the element is divided into the left child node of the node, if the word segmentation result of each element contained by each node does not contain the target word Token max with the maximum word frequency, the element is divided into the right child node. In the present example, the division result of the child nodes of the root node is shown in Table 4.

[0081] Table 4

[0082]

[0083]

[0084] S103, constructing the mode node set of the data to be mined according to the left child node and the right child node.

[0085] In a possible implementation, for the right child node in Table 4, it is judged whether the preset mining termination condition is met to continue execution until the termination condition is met. When the left child node in Table 4 is processed, the word segmentation result of all elements of the left child node is generated according to the target word Token max with the maximum word frequency, and the elements with the same mode are merged into the child node of the node. In the present example, the result of the left child node after processing is shown in Table 5.

[0086] Table 5

[0087]

[0088] For the child node in Table 5, it is judged whether the preset mining termination condition is met to continue execution until the termination condition is met. After the final running is finished, the final result of the present example is shown in Table 6.

[0089] Table 6

[0090] Line number Data content Pattern 1 This cat *cat 2 That cat *cat 3 The cat is animal *cat* 4 I was here I* 5 I will go I*

[0091] Further, through the above manner, the problem of falling into local optimum in some cases is easily caused. As shown in Table 7.

[0092] Table 7

[0093] Line number Data content 1 Cat was running 2 Cat is eating 3 Cat have a ball and I will take it to my home 4 He will take it to my home 5 They will take it to my home

[0094] Further, it can be seen that line number 3 matches both the pattern *will take it to my home* and Cat*. The result obtained by the basic method is shown in Table 8, for example:

[0095] Table 8

[0096]

[0097] Further, if the two patterns appear the same number of times, according to the principle that the more specific the pattern is, the better, line number 3 is more inclined to be attributed to the pattern *will take it to my home* than Cat*. The reason why the basic method has this local optimal problem is that in the node splitting of step 4, the element of line number 3 is only divided into the left child node.

[0098] Therefore, after the node splitting is completed, the similarity between each element of the left child node and the elements of all right child nodes is calculated. For example, the similarity sim(element 1, element 2) between 2 elements is calculated, and as long as the similarity between the element and the element of one right child node is greater than the threshold, the element is copied to the right child node. The similarity calculation method sim(element 1, element 2) used here can use but is not limited to simhash, TF-IDF and the like.

[0099] In the embodiment of the present application, when the pattern node set of the data to be mined is constructed according to the left child node and the right child node, first, the similarity between each element in the left child node and all elements of the right child node is calculated one by one to obtain multiple similarities of each element in the left child node, then the element with a similarity greater than a preset similarity is copied to the right child node according to the multiple similarities of each element in the left child node, to obtain an updated left child node and an updated right child node, and finally the pattern node set of the data to be mined is constructed according to the updated left child node and the updated right child node.

[0100] Specifically, in constructing the pattern node set of the data to be mined according to the updated left child node and the updated right child node, firstly, the same word as the target word in the word segmentation result of each element in the updated left child node is reserved, and the remaining words in the word segmentation result except the target word are converted into identifiers, and adjacent same identifiers are merged and processed to obtain the pattern of each element in the left child node, and then the elements with the same pattern are aggregated based on the pattern of each element in the left child node, and the aggregated elements and the corresponding patterns are determined as the plurality of pattern nodes corresponding to the left child node, secondly, the updated right child node is processed to obtain the plurality of pattern nodes corresponding to the right child node, and finally the plurality of pattern nodes corresponding to the left child node and the plurality of pattern nodes corresponding to the right child node are determined as the pattern node set of the data to be mined.

[0101] Further, when each node in the pattern node set of the data to be mined satisfies the preset mining termination condition, i.e. all nodes stop splitting, the mode matching degree of the elements of all nodes is calculated, and finally the mode corresponding to the largest mode matching degree is selected as the final mode of the element. Firstly, the mode matching degree between each element in each node and each mode in the pattern node set is calculated to obtain a plurality of mode matching degrees of each element, and then the mode corresponding to the largest mode matching degree in the plurality of mode matching degrees of each element is determined as the final mode of each element. For example, the element is: This cat is at home, and in 2 nodes: the mode of one node is *cat*, and the mode of the other node is *cat is at home*. At this time, the similarity between the element and the first mode is one fifth, and the similarity between the element and the other mode is four fifths. It can be seen that the mode *cat is at home* is the best mode of This cat is at home.

[0102] The matching degree formula is:

[0103] For example Figure 3 As shown, Figure 3is a mode mining process schematic block diagram of data provided by the present application, first reading data for pre-processing to generate a root node, then judge whether the root node meets the preset mode mining termination condition; if yes, calculate the mode matching degree for each element in the node, and select the mode with the largest matching degree as the mode of the element; if no, remove the duplicate and word frequency statistics of the segmentation result of each element in the node, determine the target word with the largest word frequency, secondly judge whether the target word is contained in the segmentation result of each element in the node, if yes, put the element into the left child node, otherwise put it into the right child node, after the division, calculate the similarity between each element of the left child node and all elements in the right child node, copy the elements of the left child node with similarity greater than the threshold to the right child node, finally generate and aggregate the mode of the left child node, and construct the mode node, and return the mode node and the right child node to judge whether the preset mode mining termination condition is met.

[0104] In the embodiment of the present application, the mode mining device of data first acquires and pre-processes the data to be mined, generates a root node, then when the root node does not meet the preset mining termination condition, determines the target word with the largest word frequency according to the elements of the root node, then divides the root node according to the target word with the largest word frequency to generate the left child node containing the target word and the right child node not containing the target word, and finally constructs the mode node set of the data to be mined according to the left child node and the right child node. Since the present application takes the target word with the largest appearance frequency as the division condition by counting the appearance frequency of the word, and further generates the mode node set, the mode mining is realized, and at the same time, the present application realizes the control of the fitting depth of the mode by adjusting the mode mining termination condition, thereby enhancing the interpretability and reliability of the mode mining result.

[0105] The following is an embodiment of the device of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the device embodiment of the present application, please refer to the method embodiment of the present application.

[0106] Please refer to Figure 4 , which shows the structure schematic diagram of the mode mining device of data provided by an exemplary embodiment of the present application. The mode mining device of data can be realized by software, hardware or a combination of the two to become all or part of the terminal. The device 1 includes a root node generation module 10, a target word determination module 20, a node division module 30, and a mode node set construction module 40.

[0107] The root node generation module 10 is used to acquire and pre-process the data to be mined to generate a root node;

[0108] The target word determination module 20 is used to determine the target word with the largest word frequency according to the elements of the root node when the root node does not meet the preset mining termination condition;

[0109] The node splitting module 30 is configured to split the root node according to a target word with the maximum word frequency to generate a left child node containing the target word and a right child node not containing the target word.

[0110] The mode node set construction module 40 is configured to construct a mode node set of the data to be mined according to the left child node and the right child node.

[0111] It should be noted that the data pattern mining device provided in the above embodiment is used to execute the data pattern mining method, and the above functions are only used as an example for the division of the functional modules. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above functions. In addition, the data pattern mining device and the data pattern mining method provided in the above embodiment belong to the same concept, and the implementation process is described in detail in the method embodiment, which will not be described here.

[0112] The serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0113] In the embodiments of the present application, the data pattern mining device first acquires and preprocesses the data to be mined to generate a root node, then determines a target word with the maximum word frequency according to the elements of the root node when the root node does not satisfy a preset mining termination condition, and then splits the root node according to the target word with the maximum word frequency to generate a left child node containing the target word and a right child node not containing the target word. Finally, a mode node set of the data to be mined is constructed according to the left child node and the right child node. Since the present application takes the target word with the maximum word frequency as the splitting condition by counting the word frequency, and further generates a mode node set, the pattern mining is realized, and the fitting depth of the pattern is controlled by adjusting the pattern mining termination condition, thereby enhancing the interpretability and reliability of the pattern mining result.

[0114] The present application also provides a computer readable medium having program instructions stored thereon, which, when executed by a processor, implement the data pattern mining method provided by each of the method embodiments.

[0115] The present application also provides a computer program product containing instructions, which, when executed on a computer, cause the computer to perform the data pattern mining method of each of the method embodiments.

[0116] Please refer to Figure 5 The present application provides a terminal structure schematic diagram. As shown in Figure 5As shown, the terminal 1000 can include at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, at least one communication bus 1002.

[0117] The communication bus 1002 is configured to realize the connection communication between the components.

[0118] The user interface 1003 can include a display, a camera, and can further include a standard wired interface and a wireless interface.

[0119] The network interface 1004 can include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0120] The processor 1001 can include one or more processing cores. The processor 1001 connects various parts in the entire electronic device 1000 through various interfaces and lines, executes various functions of the electronic device 1000 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Optionally, the processor 1001 can be realized in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 1001 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU is mainly used to process an operating system, a user interface, and an application program; the GPU is used to render and draw the content to be displayed on the display; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 1001, but can be realized by a separate chip.

[0121] The memory 1005 can include a random access memory (RAM) and can also include a read-only memory (ROM). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1005 can include a program storage area and a data storage area, where the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above various method embodiments, etc.; the data storage area can store data involved in the above various method embodiments, etc. The memory 1005 can also be at least one storage device located away from the aforementioned processor 1001. As shown in Figure 5 The memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a data pattern mining application.

[0122] In the terminal 1000 shown in Figure 5 In the terminal 1000 shown in

[0123] obtaining and preprocessing the data to be mined to generate a root node;

[0124] When the root node does not satisfy a preset mining termination condition, determining a target word with the maximum word frequency according to an element of the root node;

[0125] performing node splitting on the root node according to the target word with the maximum word frequency to generate a left child node containing the target word and a right child node not containing the target word;

[0126] constructing a pattern node set of the data to be mined according to the left child node and the right child node.

[0127] In one embodiment, when the processor 1001 performs obtaining and preprocessing the data to be mined to generate a root node, the processor 1001 specifically performs the following operations:

[0128] obtaining the data to be mined;

[0129] dividing the data to be mined into a plurality of to-be-processed individuals according to a preset rule;

[0130] performing word segmentation on data content of each to-be-processed individual to obtain a word segmentation result of each to-be-processed individual;

[0131] The word segmentation result, the line number, and the preset mode identifier of each individual to be processed are placed as a single element in the preset node, and a root node is obtained.

[0132] In one embodiment, when the processor 1001 determines the target word with the maximum word frequency according to the elements of the root node, the processor 1001 specifically performs the following operations:

[0133] The repeated words in the word segmentation result of each element in the root node are removed, and a plurality of removed word segmentation results are obtained.

[0134] The plurality of removed word segmentation results are summarized, and a summary vocabulary is obtained.

[0135] The word frequency of each word in the summary vocabulary is counted, and the target word with the maximum word frequency is determined.

[0136] In one embodiment, when the processor 1001 performs node segmentation on the root node according to the target word with the maximum word frequency, the processor 1001 specifically performs the following operations:

[0137] It is determined whether the word segmentation result of each element in the root node contains the target word.

[0138] If yes, the element containing the target word is divided into a left child node containing the target word.

[0139] If no, the element not containing the target word is divided into a right child node containing the target word.

[0140] In one embodiment, when the processor 1001 constructs the mode node set of the data to be mined according to the left child node and the right child node, the processor 1001 specifically performs the following operations:

[0141] The similarity between each element in the left child node and all elements of the right child node is calculated one by one, and a plurality of similarities of each element in the left child node are obtained.

[0142] According to the plurality of similarities of each element in the left child node, the element with a similarity greater than a preset similarity is copied to the right child node, and an updated left child node and an updated right child node are obtained.

[0143] The mode node set of the data to be mined is constructed according to the updated left child node and the updated right child node.

[0144] In one embodiment, when the processor 1001 constructs the mode node set of the data to be mined according to the updated left child node and the updated right child node, the processor 1001 specifically performs the following operations:

[0145] The same word as the target word in the word segmentation result of each element in the updated left child node is reserved, and the remaining words in the word segmentation result except the target word are converted into identifiers, and adjacent same identifiers are merged and processed to obtain the pattern of each element in the left child node;

[0146] The elements with the same pattern are aggregated based on the pattern of each element in the left child node, and the aggregated elements and the corresponding pattern are determined as the multiple pattern nodes corresponding to the left child node;

[0147] The updated right child node is processed to obtain the multiple pattern nodes corresponding to the right child node;

[0148] The multiple pattern nodes corresponding to the left child node and the multiple pattern nodes corresponding to the right child node are determined as the pattern node set of the data to be mined.

[0149] In an embodiment, the processor 1001, after constructing the pattern node set of the data to be mined based on the left child node and the right child node, further performs the following operations:

[0150] When each node in the pattern node set of the data to be mined satisfies a preset mining termination condition, the pattern matching degrees between each element in each node and each pattern in the pattern node set are calculated to obtain multiple pattern matching degrees of each element;

[0151] The pattern corresponding to the maximum pattern matching degree in the multiple pattern matching degrees of each element is determined as the final pattern of each element.

[0152] In the embodiments of the present application, the mode mining device of data first acquires and pre-processes the data to be mined to generate a root node, then when the root node does not satisfy a preset mining termination condition, determines a target word with the maximum word frequency according to the elements of the root node, secondly performs node segmentation on the root node according to the target word with the maximum word frequency to generate a left child node containing the target word and a right child node not containing the target word, and finally constructs a pattern node set of the data to be mined according to the left child node and the right child node. Since the present application takes the target word with the maximum appearance frequency as the segmentation condition by counting the appearance frequency of the word, and further generates a pattern node set, the mode mining is realized, and at the same time, the present application controls the fitting depth of the pattern by adjusting the mining termination condition, thereby enhancing the interpretability and reliability of the mode mining result.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the program of the mode mining of data can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory or a random access memory, etc.

[0154] The above disclosure is merely the preferred embodiments of the present application and is not intended to limit the scope of the present application. Any equivalent changes made according to the claims of the present application are still within the scope of the present application.

Claims

1. A method of pattern mining of data, characterized by, The method comprises: acquiring and preprocessing data to be mined, generating a root node; when the root node does not satisfy a preset mining termination condition, determining a target word with the maximum word frequency according to elements of the root node; performing node segmentation on the root node according to the target word with the maximum word frequency, generating a left child node containing the target word and a right child node not containing the target word; constructing a mode node set of the data to be mined according to the left child node and the right child node.

2. The method of claim 1, wherein, The acquiring and preprocessing of the data to be mined and the generation of the root node comprise: acquiring data to be mined; dividing the data to be mined into multiple processing individuals according to a preset rule; performing word segmentation on data content of each processing individual to obtain a word segmentation result of each processing individual; placing the word segmentation result, line number and preset mode identifier of each processing individual as a single element in a preset node to obtain a root node.

3. The method according to claim 1 or 2, characterized in that, The determining of the target word with the maximum word frequency according to the elements of the root node comprises: eliminating repeated words in the word segmentation result of each element in the root node to obtain multiple eliminated word segmentation results; summarizing the multiple eliminated word segmentation results to obtain a summary vocabulary; counting the word frequency of each word in the summary vocabulary and determining the target word with the maximum word frequency.

4. The method of claim 1, wherein, The node segmentation of the root node according to the target word with the maximum word frequency comprises: judging whether the target word is contained in the word segmentation result of each element in the root node; if yes, dividing the element containing the target word into a left child node containing the target word; if no, dividing the element not containing the target word into a right child node not containing the target word.

5. The method of claim 1, wherein, The construction of the mode node set of the data to be mined according to the left child node and the right child node comprises: calculating the similarity between each element in the left child node and all elements of the right child node one by one to obtain multiple similarities of each element in the left child node; copying elements with a similarity greater than a preset similarity to the right child node according to the multiple similarities of each element in the left child node to obtain an updated left child node and an updated right child node; constructing the mode node set of the data to be mined according to the updated left child node and the updated right child node.

6. The method of claim 5, wherein, The construction of the mode node set of the data to be mined according to the updated left child node and the updated right child node comprises: retaining the same word as the target word in the word segmentation result of each element in the updated left child node, converting the remaining words in the word segmentation result except the target word into identifiers, and merging and processing adjacent identical identifiers to obtain a mode of each element in the left child node; aggregating and processing elements with the same mode based on the mode of each element in the left child node, and determining the aggregated elements and the mode corresponding thereto as multiple mode nodes corresponding to the left child node; processing the updated right child node to obtain multiple mode nodes corresponding to the right child node; determining the multiple mode nodes corresponding to the left child node and the multiple mode nodes corresponding to the right child node as the mode node set of the data to be mined.

7. The method of claim 1, wherein, The method further comprises, after constructing the mode node set of the data to be mined according to the left child node and the right child node: When each node in the mode node set of the data to be mined satisfies a preset mining termination condition, calculating a mode matching degree between each element in each node and each mode in the mode node set, to obtain multiple mode matching degrees of each element; Determining a mode corresponding to a maximum mode matching degree in the multiple mode matching degrees of each element as a final mode of each element.

8. A data pattern mining device characterized by comprising: The apparatus comprises: a root node generation module configured to acquire and preprocess the data to be mined, and generate a root node; a target word determination module configured to determine a target word with a maximum word frequency according to elements of the root node when the root node does not satisfy a preset mining termination condition; a node segmentation module configured to perform node segmentation on the root node according to the target word with the maximum word frequency, to generate a left child node containing the target word and a right child node not containing the target word; a mode node set construction module configured to construct a mode node set of the data to be mined according to the left child node and the right child node.

9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, which are suitable for being loaded and executed by a processor to perform the method steps of any one of claims 1-7.

10. A terminal, characterized by comprising: comprise: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded and executed by the processor to perform the method steps of any one of claims 1-7.

Images