A data retrieval system with classification query function

Abstract

The present invention relates to the field of information processing technology, and specifically to a data retrieval system with a classification query function, including: a data input module, a data preprocessing module, a classifier module, an index construction module, a query processing module, a result filtering module, a result presentation module, and a system management module, wherein the data input module is used to receive a query request input by a user and a data set to be retrieved. Through the classifier module and the index construction module, it is possible to flexibly select a classification algorithm and generate classification labels, and then construct an efficient data index based on these labels. This mechanism significantly improves the efficiency of data retrieval, avoids the problem of traversing the entire data set due to lack of effective classification in the existing system, and through accurate data classification and efficient index construction, it is possible to achieve faster response time when processing large-scale data, greatly improving query efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of information processing, and in particular to a data retrieval system with a classification query function. Background Art

[0002] In the field of information processing technology, data retrieval systems are key tools for processing and querying large amounts of data. With the development and popularization of information technology, the amount of data has exploded, and the data types have become increasingly diverse, covering text, images, audio, video and other forms. These data not only come from traditional databases, but also include data collected from various channels such as the Internet, social media, and sensor networks.

[0003] With the continuous increase in data volume and the diversification of data types, existing data retrieval systems face many challenges in processing complex queries. Specifically, the retrieval efficiency of existing systems is low due to the lack of an effective classification mechanism, which requires traversing the entire data set when processing large-scale data, resulting in a long response time. The efficiency of traditional full table scans or simple index matching drops sharply in the processing of massive data. In addition, the existing system's classification capabilities are insufficient and it is unable to accurately classify data sets and generate classification labels, which affects the efficiency of subsequent query processing. Data classification is an important means to improve retrieval efficiency, but the existing system is not flexible enough in classification algorithms and model selection, and it is difficult to meet diverse classification needs. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the present invention provides a data retrieval system with a classification query function, which can effectively solve the problems of low retrieval efficiency and insufficient classification ability of the prior art.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] The present invention provides a data retrieval system with a classification query function, comprising: a data input module, a data preprocessing module, a classifier module, an index construction module, a query processing module, a result filtering module, a result presentation module, and a system management module, wherein the data input module, the data preprocessing module, the classifier module, the index construction module, the query processing module, the result filtering module, the result presentation module, and the system management module are electrically connected to a controller.

[0007] Preferably, the data input module is used to receive query requests input by users and data sets to be retrieved, and the data input module supports multiple types of data input, including text, images, and videos.

[0008] Preferably, the data preprocessing module is used to clean and format the data set to be retrieved, and the data preprocessing module processes different data types, including but not limited to word segmentation and stem extraction of text data, compression and edge detection of image data, etc.

[0009] Preferably, the classifier module is used to classify the data set to be retrieved and generate classification labels, and the classifier module supports model training and updating, including online learning and incremental learning.

[0010] Preferably, the index building module is used to build a data index according to the classification label, and the index building module adopts multiple indexing technologies, such as one or more indexing technologies in an inverted index, a forward index, and a graph index.

[0011] Preferably, the query processing module is used to match data indexes and generate preliminary query results based on the query request input by the user. The query processing module supports multiple query types, including but not limited to exact match query, fuzzy query, range query, Boolean query, natural language query, etc.

[0012] Preferably, the result filtering module is used to filter the preliminary query results to remove data that does not meet the user's query conditions. The result filtering module adopts multiple result filtering methods, and the results include but are not limited to score-based, time-based, user preference-based, context-based, etc.

[0013] Preferably, the result presentation module is used to present the filtered query results to the user in a visual form, and the result presentation module supports multiple result presentation methods, including but not limited to lists, charts, maps, images, videos, etc.

[0014] Preferably, the system management module is used to manage various configurations and operating states of the system to ensure stable operation of the system. The system management module provides a variety of system management functions, including but not limited to user management, authority management, log management, performance monitoring, system configuration, etc.

[0015] Compared with the known prior art, the technical solution provided by the present invention has the following beneficial effects:

[0016] 1. Through the classifier module and index construction module, it is possible to flexibly select classification algorithms and generate classification labels, and then build efficient data indexes based on these labels. This mechanism significantly improves the efficiency of data retrieval and avoids the problem of traversing the entire data set due to lack of effective classification in existing systems. Through precise data classification and efficient index construction, faster response time can be achieved when processing large-scale data, greatly improving query efficiency.

[0017] 2. It supports the input of multiple types of data, including text, images, videos and other formats. It also eliminates noise and invalid data through the data preprocessing module to provide high-quality input data. The system management module provides comprehensive monitoring and management functions to ensure that the system can still run stably under high load conditions, and can better adapt to the environment with increasing data volume and diversified data types. Compared with existing systems, it has higher flexibility and wide applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a flow chart of the present invention.

[0020] Figure numerals: 1. Data input module; 2. Data preprocessing module; 3. Classifier module; 4. Index building module; 5. Query processing module; 6. Result filtering module; 7. Result presentation module; 8. System management module. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] The present invention will be further described below in conjunction with the embodiments.

[0023] Example: Refer to Figure 1 A data retrieval system with classification query function includes: a data input module 1, a data preprocessing module 2, a classifier module 3, an index construction module 4, a query processing module 5, a result filtering module 6, a result presentation module 7, and a system management module 8. The data input module 1, the data preprocessing module 2, the classifier module 3, the index construction module 4, the query processing module 5, the result filtering module 6, the result presentation module 7, and the system management module 8 are electrically connected to a controller.

[0024] The data input module 1 is used to receive the query request input by the user and the data set to be retrieved. The data input module 1 supports multiple types of data input, including text, images, videos, etc., to ensure that users can easily submit queries. At the same time, the powerful input verification mechanism and batch processing function ensure the accuracy and efficiency of data input.

[0025] The data preprocessing module 2 is used to clean and format the data set to be retrieved. The data preprocessing module 2 processes different data types, including but not limited to word segmentation and stem extraction of text data, compression and edge detection of image data. The data preprocessing module 2 removes noise and invalid data through advanced cleaning algorithms, and performs unified formatting to ensure data consistency and availability, laying a solid foundation for subsequent analysis.

[0026] Classifier module 3 is used to classify the data set to be retrieved and generate classification labels. Classifier module 3 flexibly selects classification algorithms to accurately classify the data set. The generated classification labels not only improve the readability of the data, but also provide key information for index construction, thereby improving the overall retrieval efficiency. Classifier module 3 supports model training and updating, including online learning and incremental learning.

[0027] Index construction module 4 is used to construct data indexes according to classification labels. Index construction module 4 adopts efficient indexing algorithms to quickly construct data indexes according to classification labels, ensuring fast query in a big data environment, greatly improving the response speed and query performance of the system. Index construction module 4 adopts multiple indexing technologies, such as one or more indexing technologies in inverted index, forward index, and graph index.

[0028] The query processing module 5 is used to match the data index and generate preliminary query results according to the query request input by the user. The query processing module 5 parses the user query conditions and efficiently matches the constructed data index to generate preliminary query results, thereby ensuring that the user can quickly obtain the required information and improving the interactivity and user experience of the system. The query processing module 5 supports multiple query types, including but not limited to exact match query, fuzzy query, range query, Boolean query, natural language query, etc.

[0029] The result filtering module 6 is used to filter the preliminary query results and remove the data that does not meet the user's query conditions. The result filtering module 6 accurately removes invalid data according to the filtering conditions specified by the user, improves the accuracy of the query results, and ensures that the final presented data fully meets the user's needs. The result filtering module 6 adopts a variety of result filtering methods, and the results include but are not limited to score-based, time-based, user preference-based, context-based, etc.

[0030] The result presentation module 7 is used to present the filtered query results to the user in a visual form. The result presentation module 7 provides a variety of visualization tools to convert complex data into intuitive and easy-to-understand charts or reports, helping users to quickly understand and analyze query results and improve the efficiency and effectiveness of data presentation. The result presentation module 7 supports multiple result presentation methods, including but not limited to lists, charts, maps, images, videos, etc.

[0031] The system management module 8 is used to manage the various configurations and operating status of the system to ensure the stable operation of the system. The system management module 8 provides comprehensive system monitoring and management functions, including real-time monitoring of operating status, automatic adjustment of configurations, recording of operation logs, etc., to ensure that the system can operate stably under high load conditions and to ensure the long-term reliability and efficiency of the system. The system management module 8 provides a variety of system management functions, including but not limited to user management, permission management, log management, performance monitoring, system configuration, etc.

[0032] The working principle of the present invention is as follows:

[0033] In order to further improve the user experience and the ease of operation of the system, we first provide users with a friendly and intuitive interface through the data input module. This interface not only simplifies the input process of query requests, but also supports the processing of multiple data types, which greatly enhances the flexibility and wide applicability of the system. Users can easily select and input data in different formats, such as text, numbers, images or multimedia files on the interface without worrying about format incompatibility. In addition, we have also specially optimized the batch data loading function, which significantly improves the efficiency of data input and effectively reduces the time users spend waiting for data processing and loading. Through this series of improvements, users can complete data input and query operations more quickly and conveniently, greatly improving overall work efficiency and operating comfort. The specific operations are as follows:

[0034] In the system's efficient interactive interface, users can conveniently submit query requests in a variety of ways, including entering keywords, setting time ranges, and selecting specific query conditions such as data types. In order to meet the usage habits and needs of different users, we have designed a variety of input methods: users can not only use traditional keyboards to enter text information, but also upload various files containing data, and even use advanced voice recognition technology to directly input voice. On this basis, the system background receives and carefully analyzes the query requests submitted by users to ensure that every key information can be accurately identified and fully recorded. In the process of information acquisition, the system will strictly follow the instructions of the user's query request to intelligently load and aggregate relevant data sets from local storage devices, cloud databases, or a variety of other data sources. These data sets are rich and varied in form, and may include text files, detailed records in relational databases, vivid image data, and even dynamic video content. The system can be compatible and effectively processed to provide comprehensive and accurate data support. Through such a design, we have not only improved the convenience of user interaction, but also significantly broadened the application field and data processing capabilities of the system.

[0035] After receiving and parsing the user's query request, the system immediately enters the data preprocessing stage, that is, the loaded data set is cleaned and formatted through the data preprocessing module. This link is a key step to ensure the accuracy of the final retrieval results. The data preprocessing module adopts an automated and intelligent mechanism that can automatically identify and correct errors in the data, fill in gaps, and consistently format the data, thereby improving the overall data quality. This not only reduces the necessity of manual intervention and greatly improves the efficiency of data processing, but also provides purer and higher-quality input data for subsequent classification and index construction, ensuring the consistency and efficiency of the entire data processing process. Through this advanced data preprocessing technology, users can safely rely on the data provided by the system and trust the accuracy of their retrieval results and the completeness of the data. The specific operations are as follows:

[0036] In the data preprocessing module, the system will first perform a detailed preliminary check on the received data set. This check process relies on an efficient data recognition algorithm that can automatically detect and identify various types of noise in the data set, including but not limited to blank rows, invalid data, and various format errors. The presence of these noises may seriously affect the accuracy and completeness of the data, so the automatic recognition function is particularly important in this link. Once the noise is identified, the system will apply a variety of cleaning algorithms to correct and optimize the data. For example, the system can remove redundant blank rows, detect and delete duplicate data records, effectively filter out junk data, and revise format errors, thereby greatly improving the purity and quality of the data set.

[0037] After the data preprocessing process is completed, the system enters the critical classification stage, which is the responsibility of the classifier module. The classifier module provides a powerful and accurate classification function, which can help the system accurately classify the data in the data set and generate corresponding labels for each category. These classification labels are not only a microcosm of the data content, but also an important basis for subsequent data query and analysis. In order to meet the diverse data classification needs, the classifier module provides a flexible selection of classification algorithms. Users can choose the most appropriate classification algorithm according to the specific data type and business needs to ensure the accuracy and applicability of the classification results. The specific operations are as follows:

[0038] The system selects a suitable classification algorithm based on the characteristics of the preprocessed data, such as K-means clustering, decision tree, support vector machine (SVM), random forest, etc. The classification algorithm classifies the preprocessed data set and generates classification labels. These labels can be content-based labels, time-based labels or other classification standards. The system stores the generated classification labels in the database to facilitate subsequent index construction and query processing.

[0039] After the steps of data classification and label generation, the system further uses the index building module to build an efficient data index. The importance of this link lies in significantly improving the query response time and overall retrieval efficiency. The index building module uses a variety of advanced index building algorithms to adapt to different scales of data and diverse query requirements. These algorithms can dynamically adjust the optimization strategy of the index structure according to the characteristics of the data and the complexity of the query to ensure that the most accurate retrieval results are provided in the shortest time. The specific operations are as follows:

[0040] The system selects the appropriate index building algorithm based on the classification label and data volume. Common ones include B-tree index, hash index, inverted index, etc. The system uses the selected index algorithm to build data indexes according to the classification labels. The index usually contains the index key of the data and the corresponding storage location (or storage reference). The system stores the built index on a high-performance storage device (such as SSD) for fast query.

[0041] After building the data index and optimizing the query response, the system needs to further refine and optimize the query results, which involves the function of the result filtering module. The result filtering module provides flexible and diverse filter condition settings. Users can set corresponding filter criteria according to their specific needs to ensure the accuracy and relevance of the query results. This module has a wide range of application scenarios. Whether in complex business analysis or in daily data retrieval, you can accurately locate the required information through customized filter conditions. The specific operations are as follows:

[0042] The system selects appropriate filtering conditions based on the user's query conditions and preliminary query results. These conditions may include exact matching, fuzzy matching, and reclassification of data. Filtering algorithms (such as Boolean filtering, rule-based filtering, and machine learning-driven filtering) remove data that does not meet the user's query conditions and further optimize the preliminary results. The system stores the filtered query results in preparation for final presentation.

[0043] As a core functional module, the filter module provides flexible and comprehensive filter condition settings to support users to customize the accuracy and relevance of query results according to their personal or business specific needs. Users can set a variety of filter criteria, such as value range, time interval, specific keywords or other custom conditions, so as to eliminate irrelevant information in the query results and retain only the data that best meets the needs. The specific operations are as follows:

[0044] The system parses the query conditions (such as keywords, time range, classification tags, etc.) according to the query request entered by the user to ensure that the user's query intention is correctly understood. The query processing algorithm uses the constructed index to quickly match the data that meets the query conditions and generate preliminary query results. This step usually involves an efficient retrieval algorithm. The system stores the preliminary query results in the cache, waiting for subsequent filtering and presentation steps.

[0045] In the whole process of data processing and result display, the system management module plays a key role. It not only provides flexible system configuration adjustment functions to adapt to various operating environments and user needs, but also ensures that the system can run stably and efficiently through detailed system operation logs and comprehensive monitoring information. The core value of this module lies in its automated system adjustment mechanism, which can timely capture and analyze various conditions in system operation, quickly discover potential problems and make automatic adjustments, thereby ensuring that the system always maintains the best state in the complex and changeable actual use environment. At the same time, the system management module also integrates intelligent early warning and troubleshooting functions, further enhancing the system's self-healing ability and overall stability, and providing solid background support for users' daily operations. Through this comprehensive management and optimization method, the system management module greatly improves the system's availability, reliability and user satisfaction, and brings users a smoother and more efficient use experience. Its specific operations are as follows:

[0046] The system administrator adjusts the system configuration through the management interface, such as database connection parameters, classification algorithm parameters, index construction parameters, filtering conditions, etc. The system records various operation logs in detail, including administrator operations, user query records, system operation status information, etc., and monitors the operation of the system. The system automatically or manually adjusts the configuration based on the recorded monitoring information to ensure stable operation of the system. These adjustments may involve operations such as resource allocation, database optimization, and index reconstruction.

[0047] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data retrieval system with classification query function, characterized in that: The invention comprises a data input module (1), a data preprocessing module (2), a classifier module (3), an index building module (4), a query processing module (5), a result filtering module (6), a result presentation module (7), and a system management module (8); the data input module (1), the data preprocessing module (2), the classifier module (3), the index building module (4), the query processing module (5), the result filtering module (6), the result presentation module (7), and the system management module (8) are electrically connected to a controller.

2. A data retrieval system with classification query function according to claim 1, characterized in that: The data input module (1) is used to receive a query request input by a user and a data set to be retrieved. The data input module (1) supports multiple types of data input, including text, image, and video.

3. A data retrieval system with classification query function according to claim 1, characterized in that: The data preprocessing module (2) is used to clean and format the data set to be retrieved. The data preprocessing module (2) is targeted at different data types, including but not limited to word segmentation and stem extraction of text data, compression and edge detection of image data.

4. A data retrieval system with classification query function according to claim 1, characterized in that: The classifier module (3) is used to classify the data set to be retrieved and generate classification labels. The classifier module (3) supports model training and updating, including online learning and incremental learning.

5. A data retrieval system with classification query function according to claim 1, characterized in that: The index building module (4) is used to build a data index according to the classification label, and the index building module (4) adopts one or more indexing technologies among inverted index, forward index and graph index.

6. A data retrieval system with classification query function according to claim 1, characterized in that: The query processing module (5) is used to match data indexes according to the query request input by the user and generate preliminary query results. The query processing module (5) supports multiple query types, including but not limited to exact match query, fuzzy query, range query, Boolean query, and natural language query.

7. A data retrieval system with classification query function according to claim 1, characterized in that: The result filtering module (6) is used to filter the preliminary query results and remove data that does not meet the user's query conditions. The result filtering module (6) adopts multiple result filtering methods, and the results include but are not limited to those based on score, time, user preference, and context.

8. A data retrieval system with classification query function according to claim 1, characterized in that: The result presentation module (7) is used to present the filtered query results to the user in a visual form. The result presentation module (7) supports multiple result presentation modes, including but not limited to lists, charts, maps, images, and videos.

9. A data retrieval system with classification query function according to claim 1, characterized in that: The system management module (8) is used to manage various configurations and operating states of the system to ensure stable operation of the system. The system management module (8) provides a variety of system management functions, including but not limited to user management, authority management, log management, performance monitoring, and system configuration.

Images