Data mining method and system based on artificial intelligence
By employing AI-based data mining methods, utilizing semantic analysis and behavioral pattern libraries to identify user needs, and combining data-focused analysis models to optimize processing items, along with setting up interactive feedback mechanisms, this approach solves the problems of insufficient identification of implicit needs and low efficiency in traditional data mining, achieving efficient and accurate data mining results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI ZHIENTROPY INFORMATION TECH CO LTD
- Filing Date
- 2026-02-05
- Publication Date
- 2026-05-15
AI Technical Summary
Traditional data mining methods are inadequate in identifying implicit needs, the mining results may deviate from actual business objectives, and they are inefficient. In particular, when dealing with unstructured data, the processing flow is lengthy and it is difficult to adapt to dynamic changes in data format.
Using an AI-based data mining approach, explicit and implicit needs are identified through semantic analysis and a behavioral pattern library. The data-focused analysis model prioritizes processing items and sets up an interactive feedback mechanism for dynamic optimization, forming an ordered mining execution sequence.
It improves the ability to accurately identify user needs, enhances the targeting and efficiency of data mining, responds to user needs in a timely manner, and ensures high-quality and efficient processing of mining results.
Smart Images
Figure CN122045280A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data analysis technology, and in particular to a data mining method and system based on artificial intelligence. Background Technology
[0002] In traditional data mining processes, user demand analysis often relies on manual experience or simple keyword matching, resulting in insufficient ability to identify implicit needs. For example, in financial risk control scenarios, users may only explicitly state the need to "detect abnormal transactions," but in actual business operations, a comprehensive judgment needs to be made based on multiple dimensions of implicit features such as user historical behavior patterns, device fingerprints, and geographical location. Traditional methods, lacking the support of semantic analysis and behavioral pattern libraries, struggle to construct complete user demand profiles, leading to discrepancies between mining results and actual business objectives. Traditional data mining methods typically employ offline batch processing, resulting in significant delays when dealing with real-time data streams. Taking e-commerce recommendation systems as an example, user behavior data (such as clicks, adding to cart, and dwell time) has highly time-sensitive characteristics. Traditional methods require waiting for the data to be stored in the database before triggering analysis, causing recommended content to become disconnected from the user's current interests. Furthermore, when processing unstructured data such as text reviews and images, traditional methods require pre-processing through manual annotation or feature engineering, resulting in lengthy processes that are difficult to adapt to dynamic changes in data formats, leading to low efficiency.
[0003] Therefore, it is necessary to provide a data mining method and system based on artificial intelligence to solve the above-mentioned technical problems. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention provides an artificial intelligence-based data mining method and system to solve the problems of insufficient ability to identify implicit needs, deviation between mining results and actual business objectives, and low efficiency in traditional data mining techniques.
[0005] This invention provides a data mining method based on artificial intelligence, comprising the following steps: S1. Obtain the current processing request for the data to be processed, and identify the explicit and implicit requirement sets in the processing request through semantic analysis methods and a pre-built behavioral pattern library; S2. Combining explicit and implicit needs, analyze the data mining baseline and allocate processing items for the data to be mined based on the data mining baseline; S3. Obtain the data type and analyze the data mining focus of the model based on the pre-trained data. S4. Divide the processing items into high and low priorities according to their focus, and set up an interactive feedback mechanism to obtain interactive data for high priority processing items. S5. By recognizing the feedback semantics of interactive data, adjust the high-priority processing items, and combine the adjusted high-priority processing items with the divided low-priority processing items to mine and classify the data to be processed in sequence.
[0006] Preferably, step S1 includes the following specific steps: S101. Receive the current processing request initiated by the user for the data to be processed, and use the named entity recognition or keyword extraction method in the semantic analysis method to extract the explicit target, field or operation type in the current processing request and organize it into an explicit requirement set. S102. Collect user historical requests and their subsequent operation behavior data, identify the mapping relationship between user historical requests and corresponding final execution tasks, build a behavior pattern library, use the behavior pattern library to perform similarity matching on the current processing requests, and extract the corresponding complete tasks as a set of implicit requirements if the corresponding matching degree range is identified.
[0007] Preferably, step S2 includes the following specific steps: S201. Merge the explicit demand set and the implicit demand set to generate a structured comprehensive demand set; S202. Based on the comprehensive set of requirements, key field indicators or specific business indicators are identified through semantic analysis methods to form data mining baselines. S203. Construct a keyword-processing item mapping library based on key field indicators or specific business indicators in the data mining baseline, and preliminarily allocate processing items for the data to be mined based on the keyword-processing item mapping library to form a list of processing items for the data to be mined.
[0008] Preferably, step S3 includes the following specific steps: S301. Collect historical data types of mining tasks and corresponding feedback data, train them using a neural network, and obtain a data-focused analysis model after training. S302. Automatically identify the data type to be processed, including numerical, text, or date types, and input the data into the data focus analysis model, outputting the emphasis of the current data type.
[0009] Preferably, step S4 includes the following specific steps: S401. Mark each bias and each processing item with a semantic label. If the labels overlap and exceed the preset overlap threshold, the processing item is classified as high priority. If the labels overlap and do not exceed the preset overlap threshold or do not overlap, the processing item is classified as low priority. S402. Embed interactive interfaces at the execution node of each high-priority processing item to record user feedback, correction instructions or behavioral signals in real time, forming structured interactive data.
[0010] Preferably, step S5 includes the following specific steps: S501. The obtained interaction data is semantically recognized, and the keywords and semantics in the interaction data are analyzed to form feedback semantics; S502. Based on the feedback semantics, the corresponding high-priority processing items are reconfigured, logically modified, or rearranged to generate an adjusted set of high-priority processing items. S503. The adjusted set of high-priority processing items is merged with the divided low-priority processing items in order of priority to form the final ordered mining execution sequence. Each processing item is executed in sequence according to the final ordered mining execution sequence to complete the mining and processing of the data to be processed.
[0011] An artificial intelligence-based data mining system includes: The semantic analysis module is used to obtain the current processing requests for the data to be processed, and to identify the explicit and implicit sets of requirements in the processing requests through semantic analysis methods and a pre-built behavioral pattern library. The processing item allocation module is used to combine explicit and implicit requirements, analyze the data mining baseline, and allocate processing items to the data to be mined based on the data mining baseline. The data analysis module is used to acquire data types and analyze the data mining focus of a model based on pre-trained data. The data acquisition module is used to classify processing items into high and low priorities according to their focus, and to set up an interactive feedback mechanism to obtain interactive data at the high-priority processing items. The data mining and sorting module is used to adjust high-priority processing items by recognizing the feedback semantics of interactive data, and then mine and classify the data to be processed in order according to the adjusted high-priority processing items and the divided low-priority processing items.
[0012] Compared with related technologies, the data mining method and system based on artificial intelligence provided by this invention has the following beneficial effects: This invention leverages semantic analysis and a behavioral pattern library to comprehensively and accurately grasp user needs, laying a solid foundation for data mining; it rationally allocates processing items based on comprehensive needs, improving the targeting and efficiency of mining; it utilizes a data focus analysis model to grasp the mining priorities of different types of data, enhancing mining effectiveness; it sets up an interactive feedback mechanism to dynamically optimize the mining process, responding promptly to needs and improving result quality; finally, it forms an ordered mining execution sequence according to priority, ensuring that the mining and organization work is carried out efficiently and systematically, ultimately obtaining high-quality classification and organization results, comprehensively improving the performance and user experience of data mining work. Attached Figure Description
[0013] Figure 1This is a flowchart of an artificial intelligence-based data mining method according to the present invention; Figure 2 This is a system block diagram of an artificial intelligence-based data mining system according to the present invention. Detailed Implementation
[0014] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0015] Example 1 like Figure 1 As shown, an artificial intelligence-based data mining method includes: S1. Obtain the current processing request for the data to be processed, and identify the explicit and implicit requirement sets in the processing request through semantic analysis methods and a pre-built behavioral pattern library; S2. Combining explicit and implicit needs, analyze the data mining baseline and allocate processing items for the data to be mined based on the data mining baseline; S3. Obtain the data type and analyze the data mining focus of the model based on the pre-trained data. S4. Divide the processing items into high and low priorities according to their focus, and set up an interactive feedback mechanism to obtain interactive data for high priority processing items. S5. By recognizing the feedback semantics of interactive data, adjust the high-priority processing items, and combine the adjusted high-priority processing items with the divided low-priority processing items to mine and classify the data to be processed in sequence.
[0016] In the specific implementation process, step S1 includes the following steps: S101. Receive the current processing request initiated by the user for the data to be processed, and use the named entity recognition or keyword extraction method in semantic analysis to extract the explicit target, field or operation type in the current processing request and organize it into an explicit requirement set.
[0017] Specifically, the system receives the user's original request text, identifies the time dimension, indicator fields, and operation type through a named entity recognition model, and structures the extracted results into a set of explicit requirements based on the target-field-operation triplet.
[0018] In this embodiment, the user request is: Please check the reasons for the abnormal fluctuations in inventory turnover rate this month and compare it with the data of the same period last year. The explicit requirement extraction results are: Time dimension: this month, last year; Indicator field: inventory turnover rate; Operation type: abnormal fluctuation analysis, data comparison; Output requirement: cause analysis report.
[0019] S102. Collect user historical requests and their subsequent operation behavior data, identify the mapping relationship between user historical requests and corresponding final execution tasks, build a behavior pattern library, use the behavior pattern library to perform similarity matching on the current processing requests, and extract the corresponding complete tasks as a set of implicit requirements if the corresponding matching degree range is identified.
[0020] Specifically, the system collects user historical request records, subsequent operation logs (such as customer segmentation and promotional strategy adjustments), and final task results. It uses association rule mining, such as the Apriori algorithm, to identify the mapping relationship between historical requests and final tasks. The current request is then vectorized into text (e.g., using Sentence-BERT), and the cosine similarity to historical requests in the behavior pattern library is calculated. When the similarity exceeds a preset threshold of 0.8, the complete task chain of the corresponding historical request is extracted as a latent requirement. For example, historical request 1: "Analyze Q3 2023 sales data"; subsequent tasks: customer RFM segmentation, design of discount promotional activities; final result: Q4 sales increased by 15%. Current request: "Analyze the decline in December 2023 sales"; similarity matching result: 0.82 similarity to historical request 1; latent requirement extraction: customer RFM segmentation, review of promotional activity effects.
[0021] In the specific implementation process, step S2 includes the following steps: S201. Merge the explicit demand set and the implicit demand set to generate a structured comprehensive demand set.
[0022] S202. Based on the comprehensive set of requirements, key field indicators or specific business indicators are identified through semantic analysis methods to form the basis for data mining.
[0023] Specifically, semantic analysis models such as BERT-based keyword extraction are used to identify core business terms in the requirements, such as "repurchase rate," "customer segmentation," and "association rules," as the basis for data mining.
[0024] S203. Construct a keyword-processing item mapping library based on key field indicators or specific business indicators in the data mining baseline, and preliminarily allocate processing items for the data to be mined based on the keyword-processing item mapping library to form a list of processing items for the data to be mined.
[0025] Specifically, the data mining points are transformed into specific data processing actions (such as cleaning, aggregation, clustering, visualization, etc.) to form a preliminary list of processing items. Each data mining point is traversed, and the corresponding processing item is matched by looking up the table. It should be noted that many-to-many mapping is supported, such as one data mining point may trigger multiple processing items.
[0026] In the specific implementation process, step S3 includes the following steps: S301. Collect historical data types of mining tasks and corresponding feedback data, train them using a neural network, and obtain a data-focused analysis model after training.
[0027] Specifically, records of past data mining tasks performed by users are collected, including: input data types such as numerical (sales revenue, user ratings), text (product reviews, social media posts), and date (order time, user activity periods); task objectives such as classification, clustering, association rule mining, and trend prediction; and user feedback such as correction instructions (e.g., adjusting the number of clusters, adding keyword filtering) and operational behaviors (e.g., rerunning the task multiple times, exporting specific results). Based on user feedback and task results, the core focus of each data type in the task is manually labeled. Then, data types (encoded as one-hot vectors, e.g., [1,0,0] representing numerical data), task objectives (encoded as embedding vectors), and other metadata from historical tasks (e.g., data volume, number of fields) are used as input features, and the biased category is used as the output label. A neural network is used for training to obtain a data-focused analysis model.
[0028] S302. Automatically identify the data type to be processed, including numerical, text, or date types, and input the data into the data focus analysis model, outputting the emphasis of the current data type.
[0029] In the specific implementation process, step S4 includes the following steps: S401. Mark each bias and each processing item with a semantic label. If the labels overlap and exceed the preset overlap threshold, the processing item is classified as high priority. If the labels overlap but do not exceed the preset overlap threshold or do not overlap, the processing item is classified as low priority.
[0030] S402. Embed interactive interfaces at the execution node of each high-priority processing item to record user feedback, correction instructions or behavioral signals in real time, forming structured interactive data.
[0031] Specifically, an interactive interface is embedded in the execution node of each high-priority processing item. The interface takes the form of existing natural language interaction and supports text feedback (such as inaccurate sentiment classification), operation instructions (such as adding negative comment filtering), and behavioral signals (such as rerunning a certain processing item multiple times). User feedback is mapped to structured fields, such as: feedback type: parameter correction / logic modification / order adjustment; target processing item: sentiment classification model.
[0032] In the specific implementation process, step S5 includes the following steps: S501. The obtained interaction data is semantically recognized, and the keywords and semantics in the interaction data are analyzed to form feedback semantics.
[0033] Specifically, keywords and semantics in the obtained interaction data are identified through natural language processing (NLP) or rule engines (such as regular expressions), the core intent in user feedback (such as parameter adjustment, logic correction, and order change) is extracted, and feedback semantics that can be understood by the system are generated, providing a basis for subsequent processing adjustments.
[0034] In this embodiment, during e-commerce data mining, a user requests to mine sales data for a certain type of product. Based on the previous steps, high-priority processing items were determined, one of which is mining the sales volume of this type of product in a specific region. During execution, the user provides feedback through the interface stating, "This specific region is too small; it should be expanded to several surrounding cities." Upon receiving this feedback, the system uses semantic recognition to extract the keywords "specific region," "expand," and "several surrounding cities," understanding that the user's feedback indicates a desire to expand the geographical scope of the data mining, thus forming a feedback semantic regarding adjusting the geographical scope.
[0035] S502. Based on the feedback semantics, the corresponding high-priority processing items are reconfigured, their logic is modified, or their order is rearranged to generate an adjusted set of high-priority processing items.
[0036] Specifically, for parameter adjustments: directly modify the parameters of the processing item, such as changing the time window from 7 days to 30 days, or adjusting the regional range; for logical modifications: replace or insert new processing steps, such as changing from simple sorting to weighted sorting; for order reordering: reorder the processing items according to dependencies, such as moving deduplication before aggregation calculation.
[0037] In this embodiment, during e-commerce data mining, the feedback semantics "This specific region is too small; it should be expanded to several surrounding cities" indicate that this feedback pertains to the region range parameter. Therefore, the high-priority processing item, which originally mined sales volume in a specific region, is reconfigured, changing the region range parameter from "specific region" to "several surrounding cities." Assuming there were other high-priority processing items, such as mining sales volume of this type of product within a specific time period, this adjustment forms a new set of high-priority processing items.
[0038] S503. The adjusted set of high-priority processing items is merged with the divided low-priority processing items in order of priority to form the final ordered mining execution sequence. Each processing item is executed in sequence according to the final ordered mining execution sequence to complete the mining and processing of the data to be processed.
[0039] Specifically, the adjusted set of high-priority processing items and the original low-priority processing items are sorted by priority, with higher priority items given priority and items of the same priority in their original order. Processing items are executed sequentially according to the final sequence, and intermediate results are recorded after each item is completed.
[0040] In this embodiment, in the e-commerce data mining scenario, the adjusted high-priority processing item set includes sales volume mining after expanding the geographical scope and sales volume mining within a specific time period, while low-priority processing items may include mining customer reviews for this type of product. Next, these items are merged according to priority to form a final ordered mining execution sequence. For example, sales volume mining after expanding the geographical scope is executed first, followed by sales volume mining within the specific time period, and finally customer review mining. By executing each processing item sequentially according to this sequence, the sales volume data and customer review data for this type of product within the expanded geographical scope and the specific time period are obtained, completing the mining and organization of the data to be processed.
[0041] Example 2 like Figure 2 As shown, an application such as an AI-based data mining system specifically includes: The semantic analysis module is used to obtain the current processing requests for the data to be processed, and to identify the explicit and implicit sets of requirements in the processing requests through semantic analysis methods and a pre-built behavioral pattern library. The processing item allocation module is used to combine explicit and implicit requirements, analyze the data mining baseline, and allocate processing items to the data to be mined based on the data mining baseline. The data analysis module is used to acquire data types and analyze the data mining focus of a model based on pre-trained data. The data acquisition module is used to classify processing items into high and low priorities according to their focus, and to set up an interactive feedback mechanism to obtain interactive data at the high-priority processing items. The data mining and sorting module is used to adjust high-priority processing items by recognizing the feedback semantics of interactive data, and then mine and classify the data to be processed in order according to the adjusted high-priority processing items and the divided low-priority processing items.
[0042] This invention leverages semantic analysis and a behavioral pattern library to comprehensively and accurately grasp user needs, laying a solid foundation for data mining; it rationally allocates processing items based on comprehensive needs, improving the targeting and efficiency of mining; it utilizes a data focus analysis model to grasp the mining priorities of different types of data, enhancing mining effectiveness; it sets up an interactive feedback mechanism to dynamically optimize the mining process, responding promptly to needs and improving result quality; finally, it forms an ordered mining execution sequence according to priority, ensuring that the mining and organization work is carried out efficiently and systematically, ultimately obtaining high-quality classification and organization results, comprehensively improving the performance and user experience of data mining work.
[0043] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0044] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.
[0045] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
Claims
1. A data mining method based on artificial intelligence, characterized in that, Includes the following steps: S1. Obtain the current processing request for the data to be processed, and identify the explicit and implicit requirement sets in the processing request through semantic analysis methods and a pre-built behavioral pattern library; S2. Combining explicit and implicit needs, analyze the data mining baseline and allocate processing items for the data to be mined based on the data mining baseline; S3. Obtain the data type and analyze the data mining focus of the model based on the pre-trained data. S4. Divide the processing items into high and low priorities according to their focus, and set up an interactive feedback mechanism to obtain interactive data for high priority processing items. S5. By recognizing the feedback semantics of interactive data, adjust the high-priority processing items, and combine the adjusted high-priority processing items with the divided low-priority processing items to mine and classify the data to be processed in sequence.
2. The data mining method based on artificial intelligence according to claim 1, characterized in that, The specific steps of step S1 include: S101. Receive the current processing request initiated by the user for the data to be processed, and use the named entity recognition or keyword extraction method in the semantic analysis method to extract the explicit target, field or operation type in the current processing request and organize it into an explicit requirement set. S102. Collect user historical requests and their subsequent operation behavior data, identify the mapping relationship between user historical requests and corresponding final execution tasks, build a behavior pattern library, use the behavior pattern library to perform similarity matching on the current processing requests, and extract the corresponding complete tasks as a set of implicit requirements if the corresponding matching degree range is identified.
3. The data mining method based on artificial intelligence according to claim 1, characterized in that, The specific steps of step S2 include: S201. Merge the explicit demand set and the implicit demand set to generate a structured comprehensive demand set; S202. Based on the comprehensive set of requirements, key field indicators or specific business indicators are identified through semantic analysis methods to form data mining baselines. S203. Construct a keyword-processing item mapping library based on key field indicators or specific business indicators in the data mining baseline, and preliminarily allocate processing items for the data to be mined based on the keyword-processing item mapping library to form a list of processing items for the data to be mined.
4. The data mining method based on artificial intelligence according to claim 1, characterized in that, The specific steps of step S3 include: S301. Collect historical data types of mining tasks and corresponding feedback data, train them using a neural network, and obtain a data-focused analysis model after training. S302. Automatically identify the data type to be processed, including numerical, text, or date types, and input the data into the data focus analysis model, outputting the emphasis of the current data type.
5. The data mining method based on artificial intelligence according to claim 1, characterized in that, The specific steps of step S4 include: S401. Mark each bias and each processing item with a semantic label. If the labels overlap and exceed the preset overlap threshold, the processing item is classified as high priority. If the labels overlap and do not exceed the preset overlap threshold or do not overlap, the processing item is classified as low priority. S402. Embed interactive interfaces at the execution node of each high-priority processing item to record user feedback, correction instructions or behavioral signals in real time, forming structured interactive data.
6. The data mining method based on artificial intelligence according to claim 1, characterized in that, The specific steps of step S5 include: S501. The obtained interaction data is semantically recognized, and the keywords and semantics in the interaction data are analyzed to form feedback semantics; S502. Based on the feedback semantics, the corresponding high-priority processing items are reconfigured, logically modified, or rearranged to generate an adjusted set of high-priority processing items. S503. The adjusted set of high-priority processing items is merged with the divided low-priority processing items in order of priority to form the final ordered mining execution sequence. Each processing item is executed in sequence according to the final ordered mining execution sequence to complete the mining and processing of the data to be processed.
7. A data mining system based on artificial intelligence, employing a data mining method based on artificial intelligence as described in any one of claims 1-6, characterized in that, include: The semantic analysis module is used to obtain the current processing requests for the data to be processed, and to identify the explicit and implicit sets of requirements in the processing requests through semantic analysis methods and a pre-built behavioral pattern library. The processing item allocation module is used to combine explicit and implicit requirements, analyze the data mining baseline, and allocate processing items to the data to be mined based on the data mining baseline. The data analysis module is used to acquire data types and analyze the data mining focus of a model based on pre-trained data. The data acquisition module is used to classify processing items into high and low priorities according to their focus, and to set up an interactive feedback mechanism to obtain interactive data at the high priority processing items. The data mining and sorting module is used to adjust high-priority processing items by recognizing the feedback semantics of interactive data, and then mine and classify the data to be processed in order according to the adjusted high-priority processing items and the divided low-priority processing items.