Data self-service query system and method thereof

Abstract

The invention discloses a data self-service query system and a method thereof, which are used for solving the problems of high development and maintenance cost, poor expansibility and poor user experience in the prior art. The data self-service query system comprises a main control module, a display module, a network module, a database module and a data processing module, wherein the display module, the network module, the database module and the data processing module are connected with the main control module. The network module and the data processing module are respectively connected with the database module. Wherein at least one query request sent by the display module is received through the main control module, and each query request is sent to the network module through multi-thread concurrent processing, so that the network module obtains corresponding query data according to the query request and sends the query data to the database module for storage; and receiving the query data transmitted from the database module through the data processing module, performing data processing on the query data based on the query request, and sending a query result to the main control module, so that the main control module returns the query result to the display module.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a self-service data query system, a self-service data query method, a self-service data query device, an electronic device and a storage medium. Background Art

[0002] In the wave of digital transformation, enterprises have increasing requirements for the efficiency and accuracy of information management. As the core of enterprise operations, the production and operation management system requires fast and accurate query of internal operation information to improve decision-making efficiency and reduce risks. However, the existing model that relies on manual query is not only inefficient, but also prone to errors and difficult to adapt to rapidly changing business needs.

[0003] Traditional query devices are usually based on dedicated systems, developed in programming languages ​​such as C++ and Java, with large amounts of code and long development cycles. Once a dedicated system is developed, subsequent maintenance and functional expansion requires a lot of manpower and time, resulting in high maintenance costs. At the same time, the functional modules of existing systems are often highly coupled, making it difficult to add new functions or adjust existing functions, and have poor scalability. In addition, as the amount of data increases, the query speed and response capabilities of traditional systems are difficult to adapt to changing user needs and data growth, resulting in a poor user experience. Summary of the invention

[0004] The present invention provides a data self-service query system, a data self-service query method, a data self-service query device, an electronic device and a storage medium, which are used to solve or partially solve the technical problems of high development and maintenance costs, poor scalability and poor user experience in the existing related technologies.

[0005] The present invention provides a self-service data query system, which includes a main control module, a display module connected to the main control module, a network module, a database module and a data processing module, wherein the network module and the data processing module are respectively connected to the database module; wherein:

[0006] The main control module is used to receive at least one query request sent by the display module, and send each query request to the network module through multi-threaded concurrent processing, so that the network module can obtain corresponding query data according to the query request, and send the query data to the database module for storage;

[0007] The data processing module is used to receive the query data transmitted from the database module, perform data processing on the query data based on the query request, and send the query result to the main control module, so that the main control module returns the query result to the display module.

[0008] Optionally, the display module includes at least one editable user interface, and the display module is used to:

[0009] For each of the user interfaces, when a trigger operation for the user interface is detected, an initial user interface is loaded, the initial user interface including selectable query options and / or a text input box;

[0010] In response to a selection operation on the selectable query option and / or an edit input operation on the text input box, generating a query condition;

[0011] Performing input verification on the query condition;

[0012] If the verification fails, an error message is displayed on the user interface to guide the user to re-enter the query conditions;

[0013] If the verification is successful, a query request corresponding to the query condition is generated and the query request is sent to the main control module.

[0014] Optionally, the display module is further used for:

[0015] According to the query result, a query result interface is loaded on the user interface, data summary information is displayed, and a data query button, and / or a data query link, and / or an interactive operation button, and / or a result export button are popped up;

[0016] If it is detected that the data query button is clicked, the data detailed information corresponding to the data summary information is displayed on the query result interface;

[0017] If it is detected that the data query link is clicked, the display interface corresponding to the data query link is jumped to, and the data detailed information corresponding to the data summary information is displayed on the display interface;

[0018] If it is detected that the interactive operation button is clicked, the main control module is triggered to reprocess the information displayed on the query result interface, and refresh the display content of the query result interface based on the processing result returned from the main control module. The interactive action includes but is not limited to adjusting the sorting method and filtering the displayed content;

[0019] If it is detected that the result export button is clicked, an export format selection button is popped up on the query result interface, and in response to a selection operation on the export format selection button, a query result data file in a corresponding format is exported.

[0020] Optionally, the main control module is used to:

[0021] Upon receiving a query request sent by the display module, first determine whether the query request data corresponding to the query request is frequently queried data, the frequently queried data being data queried more than once within a preset query time period, and the frequently queried data being cached in the database module within the preset query time period;

[0022] If yes, directly retrieve the frequently queried data from the database module as the query result corresponding to the query request and return it to the display module;

[0023] If not, the query request is sent to the network module to execute the subsequent data query process.

[0024] Optionally, the data processing module is used to:

[0025] When receiving the query data transmitted from the database module, performing data preprocessing on the query data to obtain query processing data;

[0026] Analyzing and summarizing the query processing data to obtain query analysis data;

[0027] Extracting query conditions from the query request, and analyzing the user's query preference based on the query conditions;

[0028] According to the query conditions and the query preferences, the query analysis data is first sorted and then paginated to obtain the query results.

[0029] Optionally, the network module maintains a communication connection with a remote data source, and the network module is used to:

[0030] Periodically obtain the latest data from the remote data source and send the latest data to the database module;

[0031] The database module is used for:

[0032] comparing the latest data with the existing data previously stored;

[0033] If there is an updated data item in the latest data, the updated data item is used to overwrite the original data item of the existing data;

[0034] If there is a new data item in the latest data, the new data item is stored.

[0035] The present invention also provides a self-service data query method, which is used in the self-service data query system as described in any one of the above items, and the method comprises:

[0036] Receive at least one query request sent by the display module through the main control module, send each query request to the network module through multi-threaded concurrent processing, so that the network module obtains corresponding query data according to the query request, and sends the query data to the database module for storage;

[0037] The query data transmitted from the database module is received through the data processing module, data processing is performed on the query data based on the query request, and the query result is sent to the main control module, so that the main control module returns the query result to the display module.

[0038] Optionally, the display module includes at least one editable user interface, and the method further includes:

[0039] For each of the user interfaces, when the display module detects a trigger operation for the user interface, an initial user interface is loaded, wherein the initial user interface includes selectable query options and / or a text input box;

[0040] In response to a selection operation on the selectable query option and / or an edit input operation on the text input box, generating a query condition;

[0041] Performing input verification on the query condition;

[0042] If the verification fails, an error message is displayed on the user interface to guide the user to re-enter the query conditions;

[0043] If the verification is successful, a query request corresponding to the query condition is generated and the query request is sent to the main control module.

[0044] Optionally, the method further comprises:

[0045] According to the query result, the query result interface is loaded on the user interface through the display module, data summary information is displayed, and a data query button, and / or a data query link, and / or an interactive operation button, and / or a result export button are popped up;

[0046] If it is detected that the data query button is clicked, the data detailed information corresponding to the data summary information is displayed on the query result interface;

[0047] If it is detected that the data query link is clicked, the display interface corresponding to the data query link is jumped to, and the data detailed information corresponding to the data summary information is displayed on the display interface;

[0048] If it is detected that the interactive operation button is clicked, the main control module is triggered to reprocess the information displayed on the query result interface, and refresh the display content of the query result interface based on the processing result returned from the main control module. The interactive action includes but is not limited to adjusting the sorting method and filtering the displayed content;

[0049] If it is detected that the result export button is clicked, an export format selection button is popped up on the query result interface, and in response to a selection operation on the export format selection button, a query result data file in a corresponding format is exported.

[0050] Optionally, the method further comprises:

[0051] When the main control module receives the query request sent by the display module, it first determines whether the query request data corresponding to the query request is frequently queried data, the frequently queried data is data that is queried more than once within a preset query time period, and the frequently queried data is cached in the database module within the preset query time period;

[0052] If yes, directly retrieve the frequently queried data from the database module as the query result corresponding to the query request and return it to the display module;

[0053] If not, the query request is sent to the network module to execute the subsequent data query process.

[0054] Optionally, receiving the query data transmitted from the database module through a data processing module, and performing data processing on the query data based on the query request includes:

[0055] When the data processing module receives the query data transmitted from the database module, it performs data preprocessing on the query data to obtain query processing data;

[0056] Analyzing and summarizing the query processing data to obtain query analysis data;

[0057] Extracting query conditions from the query request, and analyzing the user's query preference based on the query conditions;

[0058] According to the query conditions and the query preferences, the query analysis data is first sorted and then paginated to obtain the query results.

[0059] Optionally, the network module maintains a communication connection with a remote data source, and the method further comprises:

[0060] Periodically acquiring the latest data from the remote data source through the network module, and sending the latest data to the database module;

[0061] Comparing the latest data with the existing data previously stored by the database module;

[0062] If there is an updated data item in the latest data, the updated data item is used to overwrite the original data item of the existing data;

[0063] If there is a new data item in the latest data, the new data item is stored.

[0064] The present invention also provides a data self-service query device, which is used in any of the above data self-service query methods, and the device includes:

[0065] A query request processing unit, configured to receive at least one query request sent by the display module through the main control module, and send each query request to the network module through multi-threaded concurrent processing, so that the network module can obtain corresponding query data according to the query request, and send the query data to the database module for storage;

[0066] The query data processing unit is used to receive the query data transmitted from the database module through the data processing module, perform data processing on the query data based on the query request, and send the query result to the main control module so that the main control module returns the query result to the display module.

[0067] The present invention also provides an electronic device, the device comprising a processor and a memory:

[0068] The memory is used to store program code and transmit the program code to the processor;

[0069] The processor is used to execute any of the above self-service data query methods according to the instructions in the program code.

[0070] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the self-service data query method as described in any one of the above items.

[0071] It can be seen from the above technical solutions that the present invention has the following advantages:

[0072] A data self-service query system and method are proposed. The data self-service query system includes a main control module, a display module, a network module, a database module and a data processing module connected to the main control module, and the network module and the data processing module are respectively connected to the database module. The modular design of the system enables each functional module to be independently developed, tested and maintained, which not only improves the development efficiency of the system, but also each module is independent and closely coordinated with each other, so that the system has good scalability and maintainability, and can also facilitate functional updates and replacements. Compared with traditional highly integrated systems, the flexibility and adaptability of modular design are irreplaceable, so that the technical solution of the present invention can bring excellent performance in application scenarios.

[0073] In actual application, at least one query request sent by the display module is received by the main control module, and each query request is sent to the network module through multi-threaded concurrent processing, so that the network module can obtain corresponding query data according to the query request, and send the query data to the database module for storage; the query data transmitted from the database module is received by the data processing module, and the query data is processed based on the query request, and the query result is sent to the main control module, so that the main control module returns the query result to the display module. Therefore, through the mutual cooperation of various functional modules in the system, fast and effective self-service data query can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0075] Figure 1 It is a structural diagram of a data self-service query system;

[0076] Figure 2 This is a schematic diagram of the principle of a query cache mechanism;

[0077] Figure 3 A flowchart of the steps of a self-service data query method;

[0078] Figure 4 The present invention is a structural block diagram of a data self-service query device. DETAILED DESCRIPTION

[0079] The embodiments of the present invention provide a data self-service query system, a data self-service query method, a data self-service query device, an electronic device and a storage medium, which are used to solve or partially solve the technical problems of high development and maintenance costs, poor scalability and poor user experience in the existing related technologies.

[0080] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] As an example, traditional query devices are usually based on dedicated systems, developed in programming languages ​​such as C++ and Java, with large amounts of code and long development cycles. Once a dedicated system is developed, subsequent maintenance and functional expansion requires a lot of manpower and time, and the maintenance cost is high. At the same time, the functional modules of existing systems are often highly coupled, making it difficult to add new functions or adjust existing functions, and have poor scalability. In addition, as the amount of data increases, the query speed and responsiveness of traditional systems are difficult to adapt to changing user needs and data growth, resulting in a poor user experience.

[0082] Therefore, one of the core invention points of the embodiment of the present invention is: to solve the deficiencies in the prior art, a data self-service query system and method based on Python technology is proposed.

[0083] First of all, the data self-service query system adopts a modular design, mainly including a display module, a main control module, a database module, a network module and a data processing module. The modular design of the system enables each functional module to be independently developed, tested and maintained, which not only improves the development efficiency of the system, but also makes each module independent and closely coordinated, making the system have good scalability and maintainability, and can also easily update and replace functions. Compared with traditional highly integrated systems, the flexibility and adaptability of modular design are irreplaceable, which enables the technical solution of the present invention to bring outstanding performance in application scenarios.

[0084] Secondly, based on the data self-service query system, Python technology is used to realize the intelligent data self-service query process. Python code is easy to understand and maintain, which can reduce the complexity of subsequent maintenance and reduce maintenance costs. Modular design makes it easier to add new functions or adjust existing functions, and subsequent expansion is more flexible. Through optimized data processing processes and interface interaction design, the query speed and user experience can be significantly improved.

[0085] In addition, due to the simplicity of Python language and rich third-party library support, the system provided by the embodiment of the present invention not only has low development cost, high development efficiency and high maintainability, but also has obvious market competitiveness and technical superiority in practical applications, and is difficult to be replaced by other development languages ​​or technical platforms.

[0086] On the one hand, the present invention proposes a data self-service query system based on Python technology. The system adopts a modular design and consists of multiple independent functional modules. Based on the mutual cooperation of these modules, while utilizing the efficient development capabilities of Python, combined with multi-threaded processing, database interface, data analysis and graphical user interface (GUI) technology, efficient operation, flexible expansion and easy maintenance of data self-service query are achieved.

[0087] Reference Figure 1 , showing a schematic diagram of the structure of a data self-service query system provided by an embodiment of the present invention.

[0088] Combination Figure 1 The data self-service query system 100 mainly includes a main control module 101, a display module 102 connected to the main control module 101, a network module 103, a database module 104 and a data processing module 105. The network module 103 and the data processing module 105 are connected to the database module 104 respectively.

[0089] Next, each functional module of the data self-service query system 100 is introduced in detail.

[0090] The display module 102 is the main interface for users to interact with the data self-service query system 100. The display module 102 is mainly developed based on Python's graphical user interface (GUI) library, such as Tkinter, PyQt or Kivy, which can provide an intuitive and easy-to-use user interface. It supports cross-platform graphical user interfaces and can run stably on different operating systems and hardware devices, improves the compatibility and stability of the system in various usage environments, and provides a user-friendly operating experience. In order to simultaneously process multiple query requests from different users, the display module 102 includes multiple editable user interfaces. Each user interface corresponds to a user to process the user's query request. The user interface can use a touch screen display. In actual applications, users can enter query conditions and browse query results by clicking on the screen.

[0091] In a specific implementation, the display module 102 includes at least one editable user interface. The display module 102 can be specifically used to: for each user interface, when a trigger operation on the user interface is detected, load the initial user interface, the initial user interface includes selectable query options and / or a text input box; in response to a selection operation on the selectable query options and / or an edit input operation on the text input box, generate query conditions; perform input verification on the query conditions; if the verification fails, display an error prompt message on the user interface to guide the user to re-enter the query conditions; if the verification succeeds, initialize and generate a query request corresponding to the query condition, and send the query request to the main control module 101.

[0092] Thus, through the intuitive graphical user interface, users can easily operate without additional training or guidance. The interface design adopts a responsive layout, which can adapt to display screens (display devices) of different sizes to enhance the user's operating experience. In addition, the interface can also provide multi-language support according to user needs, which is suitable for users with different language backgrounds. Thereby further expanding the scope of application of the system. Compared with the traditional dedicated interface design, the Python-based GUI design of the display module 102 in the embodiment of the present invention has strong versatility and adaptability, and is difficult to be easily replaced by other technologies.

[0093] The main control module 101 is developed based on Python language and is the core control unit of the entire system. The main control module 101 is responsible for processing user input, executing query logic, managing system resources, and coordinating the work between various modules.

[0094] The main control module 101 makes full use of Python's multi-threaded processing technology to distribute the user's query request to different threads for parallel processing. Each thread is responsible for processing different parts of the query request, such as accessing the database, filtering data, etc. In this way, not only can the query requests of multiple users be processed simultaneously, but also the processing time of each query request can be saved, and the response speed and concurrent processing capability of the system can be improved. Especially in the case of multiple users querying at the same time, the waiting time of the system can be significantly reduced, so that the system can be maintained at an efficient operation level. Further, adopting this concurrent processing mode can not only greatly optimize the user experience, but also has a high degree of complexity and uniqueness in technical implementation, is difficult to be simply replaced or bypassed, and solves the performance bottleneck problem caused by single-threaded processing in traditional query systems.

[0095] At the same time, the main control module 101 adopts a modular design, and all functions are implemented through independent modules, which is convenient for subsequent function expansion or modification. Further, the main control module 101 can also have a built-in perfect error handling and log recording mechanism, so as to capture and handle various abnormal situations, so that the system has good stability and reliability.

[0096] In a specific implementation, the main control module 101 can be mainly used to receive at least one query request sent by the display module 102, and send each query request to the network module 103 through multi-threaded concurrent processing, so that the network module 103 can obtain corresponding query data according to the query request, and send the query data to the database module 104 for storage.

[0097] The database module 104 is responsible for storing and managing various data in the system, including user information, query records, and data sources. This module uses a relational database (such as MySQL or SQLite, a widely used open source relational database management system) for data storage and management. Python interacts with the database through libraries such as SQLAlchemy, PyMySQL, or SQLite.

[0098] The efficiency of data query and access is greatly improved through optimized SQL query statements and database index design. The database module 104 adopts transaction mechanism and authority control to improve the integrity, consistency and security of data. In addition, the database module 104 also has a built-in regular data backup function to provide data recovery capabilities in unexpected situations.

[0099] At the same time, the present invention also integrates an efficient query and cache mechanism in the database module 104. That is, when a user makes a request, within a certain time period (such as the time period from the first receipt of the same query request to the current receipt of the query request as the query time period, such as 1 hour or 5 hours, or a time period set based on actual needs), the system will first check whether the requested data has been stored in the cache. If so, the cached data will be directly returned to avoid time-consuming database queries again.

[0100] For example, Figure 2 A schematic diagram showing the principles of a query cache mechanism provided by an embodiment of the present invention is shown.

[0101] In order to implement transient caching of query data, a first database 1041 may be provided in the database module 104 to store data currently obtained from a remote data source based on a query request and processed by the data processing module 105, such as Figure 2The data tables a, b, c and d in the first database 1041 are stored in the second database 1042. The second database 1042 is set to cache the data currently stored in the first database 1041, such as the data tables ad. To simplify the drawings, only the cache schematic diagram of the data table a from the first database 1041 to the second database 1042 is shown in the figure. After the data stored in the first database 1041 is cached to the second database 1042, the same data stored in the first database 1041 is deleted to continue to store subsequent new query result data. For example, after the data table a is cached to the second database 1042, the data table a stored in the first database 1041 is deleted.

[0102] Thus, through the data cache setting, when repeated query requests are encountered within a subsequent period of time (such as setting 5 hours as the query validity period), the query results can be quickly returned to improve the query efficiency. When the preset query time period of the query result ends, the relevant query data cached in the second database 1042 will be automatically cleared, thereby freeing up storage space to temporarily cache other subsequent query request data.

[0103] Thus, the database module 104 realizes fast access to the database through the SQLAlchemy or PyMySQL library, which not only supports efficient data query operations, but also can respond to frequent query requests in a short time by introducing an intelligent query cache mechanism, reducing the time of repeated queries and the pressure on the database, improving the overall query speed, and further improving the overall performance of the system. Especially in scenarios with large amounts of data or frequent queries, the query cache setting is particularly important. This close integration and optimization design of the database and the cache is difficult to find a direct alternative in the prior art.

[0104] Combined with the query cache mechanism of the database module 104, the main control module 101 can be used for: when receiving the query request sent by the display module 102, first determine whether the query request data corresponding to the query request is frequent query data. Frequent query data is data that is queried more than 1 time or more within a preset query time period. The frequent query data is cached in the database module 104 within the preset query time period; if so, directly retrieve the frequent query data from the database module 104 as the query result corresponding to the query request, and return it to the display module 102; if not, send the query request to the network module 103 to execute the subsequent data query process.

[0105] The network module 103 provides the system with the ability to connect to external networks and remote servers, supports real-time data updates and remote queries, and implements network communications through Python's Requests, Socket, and other libraries.

[0106] The network module 103 supports remote data query across the network, and users can access remote databases or other data sources through the network module 103. Through the network module 103, data interaction can be carried out in real time from remote data sources such as remote servers or API interfaces (Application Programming Interface) to obtain the latest data and store it in the local database. In order to improve the security of data transmission, the network module 103 supports HTTPS protocol (HyperText Transfer Protocol Secure) and SSL (Secure Sockets Layer) encryption to prevent data from being stolen or tampered with during transmission.

[0107] As an optional embodiment, the network module 103 maintains a communication connection with the remote data source. The network module 103 can be mainly used to: regularly obtain the latest data from the remote data source and send the latest data to the database module 104.

[0108] Based on this, the database module 104 can be mainly used to: compare the latest data with the previously stored existing data; if there are updated data items in the latest data, use the updated data items to overwrite the original data items of the existing data; if there are new data items in the latest data, store the new data items.

[0109] Thus, the network module 103 realizes secure communication and real-time data synchronization with a remote server or data source through Python. In particular, in the process of transmitting sensitive data, based on SSL encryption technology, the security and integrity of data transmission can be greatly improved. This secure communication mechanism combines the real-time and security of network communication, and is difficult to be replaced by other conventional communication schemes. It is particularly suitable for application scenarios with high requirements for data security or the need to update data in real time. Such as bank transaction inquiries, or real-time inventory inquiries in libraries, etc.

[0110] The data processing module 105 is responsible for further processing and analyzing the original query data obtained from the database module 104 to meet the user's query needs. This module is developed based on Python's Pandas, NumPy and other data analysis libraries.

[0111] First, the original query data is cleaned, formatted, and standardized to improve data consistency and accuracy. Then, according to the user's query conditions, complex data analysis and aggregation operations are performed, such as statistical calculations and trend analysis. Then, according to user needs, the query results are sorted, filtered, and displayed in pages for easy viewing and use by users.

[0112] Based on the relevant content introduced above, the data processing module 105 can be mainly used to receive query data transmitted from the database module 104, process the query data based on the query request, and send the query results to the main control module 101, so that the main control module 101 returns the query results to the display module 102.

[0113] More specifically, in conjunction with the data processing process, the data processing module 105 is mainly used for: when receiving query data transmitted from the database module 104, preprocessing the query data to obtain query processing data; analyzing and summarizing the query processing data to obtain query analysis data; extracting query conditions from the query request, and analyzing the user's query preferences based on the query conditions; sorting the query analysis data first and then paging it according to the query conditions and query preferences to obtain query results.

[0114] In the data processing module 105, by using data analysis libraries such as Pandas and NumPy of Python, the query results can be intelligently analyzed and processed in depth. The powerful functions provided by these libraries include not only basic data sorting and filtering functions, but also complex data analysis according to user needs. Such as statistical summary and trend analysis, such as statistical analysis, data cleaning and format conversion. These functions give the data self-service query system 100 powerful data processing capabilities. This data processing and analysis capability based on Python has unique efficiency and flexibility, which is difficult to replace by traditional languages ​​or technical means, improves the intelligence level of the system, and can improve user experience.

[0115] Based on the Python technology framework, DrissionPage (a Python-based web automation tool) provides a high-level API to simplify web automation and data crawling by combining Selenium (responsible for browser automation) and Requests (responsible for sending and responding to HTTP requests) libraries. It can handle browser interactions at the bottom layer, making it easier for users to operate web pages more efficiently. The simplified API also enables users to quickly get started, reduce learning costs, and allow users to customize extensions based on demand to meet specific business needs.

[0116] After obtaining the query results through data query, the display module 102 can also be used to: load the query result interface in the user interface according to the query results, display data summary information, and pop up a data query button, and / or a data query link, and / or an interactive operation button, and / or a result export button.

[0117] In one case, if it is detected that a data query button is clicked, detailed data information corresponding to the data summary information is displayed on the query result interface.

[0118] In another case, if it is detected that a data query link is clicked, the display interface corresponding to the data query link is jumped to, and the data detailed information corresponding to the data summary information is displayed on the display interface.

[0119] In another case, if a click of an interactive operation button is detected, the main control module 101 is triggered to reprocess the information displayed on the query result interface, and refresh the display content of the query result interface based on the processing result returned from the main control module 101. The interactive actions include but are not limited to adjusting the sorting method and filtering the displayed content.

[0120] In another case, if it is detected that the result export button is clicked, an export format selection button pops up on the query result interface, and in response to the selection operation on the export format selection button, the query result data file of the corresponding format is exported.

[0121] It should be pointed out that in actual applications, after obtaining the query results, the user can choose to view detailed information according to the actual query needs, or can interact with the interface in real time, and can also export the query results.

[0122] That is, when the user views the obtained query information, based on the interaction with the user graphical interface, the user can choose to perform only one of the real-time interactions, or can choose to perform two or more real-time interactions. For example, the user can view the detailed information only through the link jump, or can export the query data file after viewing the detailed information through the link jump, and so on. For each real-time interaction, the user can choose to execute it only once, or can repeat it multiple times. For example, the user can view the detailed information only once, or can click to view the detailed information repeatedly, and so on. It is understandable that the present invention is not limited to this.

[0123] In order to facilitate the stable operation of the system and the long-term storage of data, the data self-service query system 100 provided in the embodiment of the present invention also includes a storage and power module. Among them, the storage module uses a solid state drive (SSD) or other high-speed storage media, focusing on physical storage and efficient reading and writing, providing actual data storage space for the database, and efficiently and flexibly managing and operating a large number of data sources to improve the speed and reliability of data storage. The power module provides a continuous and stable power supply for the system and supports an uninterruptible power supply (UPS) to prevent data loss due to power outages.

[0124] It should be noted that, in order to enable those skilled in the art to better distinguish data of the same type but with different actual meanings, some technical features are distinguished by the first and the second in the embodiments of the present invention. The first and the second are only used for data distinction and have no other special meanings. It can be understood that the present invention is not limited to this.

[0125] In an embodiment of the present invention, a Python-based self-service data query system is provided. The system adopts a modular design, mainly including a display module, a main control module, a database module, a network module and a data processing module. The modular design of the system enables each functional module to be independently developed, tested and maintained, which not only improves the development efficiency of the system, but also each module is independent and closely coordinated with each other, so that the system has good scalability and maintainability, and can also conveniently update and replace functions. Compared with traditional highly integrated systems, the flexibility and adaptability of modular design are irreplaceable, so that the technical solution of the present invention can bring excellent performance in application scenarios.

[0126] On the other hand, in combination with the Python-based self-service data query system introduced in the above embodiment, the present invention also provides a corresponding implementation method. The method flow is intended to provide efficient and convenient self-service query services. Taking a single user query as an example, the entire workflow starts with the user inputting query conditions, and finally returns the information required by the user through a series of steps such as data processing, query, and result display. The specific implementation means are as follows:

[0127] 1. User input and initialization

[0128] When the user is ready to search, he / she inputs the search conditions through the touch screen of the display module. The content input by the user may include keywords, time range, category selection, etc. This process involves the following steps:

[0129] (1) User interface loading: When the user starts to use the data self-service query system, the display module will load the initial interface, showing the query options or input boxes available for selection. This interface is implemented based on Python's GUI library (such as Tkinter or PyQt).

[0130] (2) Input verification: After the user enters the query conditions, the system first performs a preliminary verification of the input content. For example, check whether the date format is correct, whether the keywords meet the requirements, etc. If the input content does not meet the requirements, prompt the user to re-enter through a prompt message, such as popping up a prompt message in the user interface.

[0131] (3) Query request initialization: If the verification is successful, the query request corresponding to the query condition is initialized and then passed to the main control module. The main control module will create a query request object and prepare relevant resources for subsequent query processing.

[0132] (II) Query processing

[0133] After receiving the query request, the main control module starts to process the request. This stage mainly includes the following steps:

[0134] (1) Multithreaded processing: To improve the system response speed, the main control module uses Python's multithreading technology to assign query tasks to multiple threads for parallel processing. Each thread is responsible for processing different parts of the query request, such as accessing the database, filtering data, etc.

[0135] (2) Database access: The main control module interacts with the database through the database module to perform query operations. The database module uses the SQLAlchemy or PyMySQL library to build and execute SQL query statements. The system dynamically generates query statements based on the conditions entered by the user and accesses the database through the network module to retrieve matching data.

[0136] (3) Cache mechanism: To reduce the pressure of repeated queries on the database, a query cache mechanism is designed. For frequently queried data, the system will cache it in the local database module for a period of time. If there is an identical query request within the cache validity period, the system will directly return the query result in the cache without accessing the database again.

[0137] 3. Data processing and analysis

[0138] After obtaining the query data, the data processing module further processes and analyzes the original query data in order to present the most useful information to the user. The work at this stage includes:

[0139] (1) Data preprocessing: To improve the consistency and integrity of the data, the returned query data is first preprocessed, including data cleaning, missing value processing, and format conversion.

[0140] (2) Data analysis and summary: Use Python's Pandas and NumPy libraries to analyze the processed data, such as statistical summary, trend analysis, data screening, etc. For some specific queries, the system may also conduct in-depth analysis of the data to discover the user's potential query needs.

[0141] (3) Data sorting and paging: The processed data is sorted according to the user's query conditions and preferences, and displayed on multiple pages, making it easier for users to browse page by page and find the required information.

[0142] (IV) Query result display

[0143] The data obtained after being processed by the data processing module is used as the query result, which is returned to the display module through the main control module and displayed to the user in a graphical manner through the display module. The user can perform further interactive operations through the touch screen, such as viewing detailed information, exporting results, printing results, etc. The main steps of this stage include:

[0144] (1) Loading the result interface: The display module loads the query result interface based on the query results returned by the data processing module and displays the summary information of the data. Users can view more detailed information through buttons or links on the interface.

[0145] (2) Interactive operations: When viewing query results, users may perform further operations, such as adjusting the sorting method, filtering the displayed content, etc. The display module supports dynamic interaction, and the user's operation will trigger the background reprocessing process in real time and refresh the displayed results.

[0146] (3) Result export: Users can choose to export query results to PDF, Excel, and other formats. The system generates export files through Python-related libraries (such as ReportLab and XlsxWriter) and provides download or print functions.

[0147] (V) Data synchronization and update

[0148] To ensure the real-time nature of the data, the network module periodically synchronizes and updates with remote data sources such as remote servers or other data sources. This process includes the following operations:

[0149] (1) Real-time data acquisition: Maintain a connection with remote data sources through the network module to regularly obtain the latest data updates, such as product inventory information, library new book inventory status, etc.

[0150] (2) Data synchronization and merging: Compare and merge the acquired new data with the existing data in the local database. For updated data items, overwrite the old data; for newly added data items, insert them into the database.

[0151] (3) Synchronization status monitoring: monitor the status of data synchronization in real time, record synchronization logs, and handle errors when they occur, or notify the administrator in the form of prompt messages.

[0152] Reference Figure 3 , shows a flowchart of a method for self-service data query provided by an embodiment of the present invention, the method for self-service data query is used in the self-service data query system as described above, and the method may specifically include the following steps:

[0153] Step 301, receiving at least one query request sent by the display module through the main control module, sending each query request to the network module through multi-threaded concurrent processing, so that the network module obtains corresponding query data according to the query request, and sends the query data to the database module for storage;

[0154] Step 302, receiving the query data transmitted from the database module through the data processing module, performing data processing on the query data based on the query request, and sending the query result to the main control module, so that the main control module returns the query result to the display module.

[0155] In an optional embodiment, the display module includes at least one editable user interface, and the method further includes:

[0156] For each of the user interfaces, when the display module detects a trigger operation for the user interface, an initial user interface is loaded, wherein the initial user interface includes selectable query options and / or a text input box;

[0157] In response to a selection operation on the selectable query option and / or an edit input operation on the text input box, generating a query condition;

[0158] Performing input verification on the query condition;

[0159] If the verification fails, an error message is displayed on the user interface to guide the user to re-enter the query conditions;

[0160] If the verification is successful, a query request corresponding to the query condition is generated and the query request is sent to the main control module.

[0161] In an optional embodiment, the method further includes:

[0162] According to the query result, the query result interface is loaded on the user interface through the display module, data summary information is displayed, and a data query button, and / or a data query link, and / or an interactive operation button, and / or a result export button are popped up;

[0163] If it is detected that the data query button is clicked, the data detailed information corresponding to the data summary information is displayed on the query result interface;

[0164] If it is detected that the data query link is clicked, the display interface corresponding to the data query link is jumped to, and the data detailed information corresponding to the data summary information is displayed on the display interface;

[0165] If it is detected that the interactive operation button is clicked, the main control module is triggered to reprocess the information displayed on the query result interface, and refresh the display content of the query result interface based on the processing result returned from the main control module. The interactive action includes but is not limited to adjusting the sorting method and filtering the displayed content;

[0166] If it is detected that the result export button is clicked, an export format selection button is popped up on the query result interface, and in response to a selection operation on the export format selection button, a query result data file in a corresponding format is exported.

[0167] In an optional embodiment, the method further includes:

[0168] When the main control module receives the query request sent by the display module, it first determines whether the query request data corresponding to the query request is frequently queried data, the frequently queried data is data that is queried more than once within a preset query time period, and the frequently queried data is cached in the database module within the preset query time period;

[0169] If yes, directly retrieve the frequently queried data from the database module as the query result corresponding to the query request and return it to the display module;

[0170] If not, the query request is sent to the network module to execute the subsequent data query process.

[0171] In an optional embodiment, receiving the query data transmitted from the database module through a data processing module, and performing data processing on the query data based on the query request includes:

[0172] When the data processing module receives the query data transmitted from the database module, it performs data preprocessing on the query data to obtain query processing data;

[0173] Analyzing and summarizing the query processing data to obtain query analysis data;

[0174] Extracting query conditions from the query request, and analyzing the user's query preference based on the query conditions;

[0175] According to the query conditions and the query preferences, the query analysis data is first sorted and then paginated to obtain the query results.

[0176] In an optional embodiment, the network module maintains a communication connection with the remote data source, and the method further includes:

[0177] Periodically acquiring the latest data from the remote data source through the network module, and sending the latest data to the database module;

[0178] Comparing the latest data with the existing data previously stored by the database module;

[0179] If there is an updated data item in the latest data, the updated data item is used to overwrite the original data item of the existing data;

[0180] If there is a new data item in the latest data, the new data item is stored.

[0181] As for the method embodiment, since it is basically similar to the system embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the aforementioned system embodiment.

[0182] In an embodiment of the present invention, a method for self-service data query is provided. At least one query request sent by the display module is received by the main control module, and each query request is sent to the network module through multi-threaded concurrent processing, so that the network module can obtain corresponding query data according to the query request, and send the query data to the database module for storage; the query data transmitted from the database module is received by the data processing module, the query data is processed based on the query request, and the query result is sent to the main control module, so that the main control module returns the query result to the display module. Thus, based on the data self-service query system, Python technology is used to achieve fast and effective self-service data query. Python code is easy to understand and maintain, which can reduce the complexity of subsequent maintenance and reduce maintenance costs. Modular design makes it easier to add new functions or adjust existing functions, and subsequent expansion is more flexible. And through optimized data processing flow and interface interaction design, the query speed and user experience can be significantly improved.

[0183] Reference Figure 4 , shows a structural block diagram of a data self-service query device provided by an embodiment of the present invention, the data self-service query device is used for the data self-service query method as described above, and the device may specifically include:

[0184] A query request processing unit 401 is used to receive at least one query request sent by the display module through the main control module, and send each query request to the network module through multi-threaded concurrent processing, so that the network module can obtain corresponding query data according to the query request, and send the query data to the database module for storage;

[0185] The query data processing unit 402 is used to receive the query data transmitted from the database module through the data processing module, process the query data based on the query request, and send the query result to the main control module so that the main control module returns the query result to the display module.

[0186] In an optional embodiment, the display module includes at least one editable user interface, and the device further includes:

[0187] An initial user interface loading unit, configured to load an initial user interface for each of the user interfaces when the display module detects a triggering operation for the user interface, wherein the initial user interface includes selectable query options and / or a text input box;

[0188] A query condition generating unit, configured to generate a query condition in response to a selection operation on the selectable query option and / or an edit input operation on the text input box;

[0189] An input verification unit, used to perform input verification on the query condition;

[0190] An error prompt information unit, used to display an error prompt information on the user interface when the verification fails, so as to guide the user to re-enter the query conditions;

[0191] The query request initialization unit is used to initialize and generate a query request corresponding to the query condition when the verification is successful, and send the query request to the main control module.

[0192] In an optional embodiment, the device further includes:

[0193] An information display unit, configured to load a query result interface on the user interface according to the query result through the display module, display data summary information, and pop up a data query button, and / or a data query link, and / or an interactive operation button, and / or a result export button;

[0194] A data detailed information display unit, configured to display the data detailed information corresponding to the data summary information on the query result interface when detecting that the data query button is clicked;

[0195] A data detailed information jump display unit, used to jump to a display interface corresponding to the data query link when detecting that the data query link is clicked, and to display the data detailed information corresponding to the data summary information on the display interface;

[0196] An information interaction unit, configured to trigger the main control module to reprocess the information displayed on the query result interface when detecting that the interactive operation button is clicked, and refresh the displayed content of the query result interface based on the processing result returned from the main control module, wherein the interactive action includes but is not limited to adjusting the sorting mode and filtering the displayed content;

[0197] The result export unit is used to pop up an export format selection button on the query result interface when detecting that the result export button is clicked, and export the query result data file in a corresponding format in response to the selection operation on the export format selection button.

[0198] In an optional embodiment, the device further includes a query cache unit, and the query cache unit is specifically used to:

[0199] When the main control module receives the query request sent by the display module, it first determines whether the query request data corresponding to the query request is frequently queried data, the frequently queried data is data that is queried more than once within a preset query time period, and the frequently queried data is cached in the database module within the preset query time period;

[0200] If yes, directly retrieve the frequently queried data from the database module as the query result corresponding to the query request and return it to the display module;

[0201] If not, the query request is sent to the network module to execute the subsequent data query process.

[0202] In an optional embodiment, the query data processing unit 402 includes:

[0203] a data preprocessing unit, configured to perform data preprocessing on the query data to obtain query processing data when the data processing module receives the query data transmitted from the database module;

[0204] A data analysis and summary unit, used to analyze and summarize the query processing data to obtain query analysis data;

[0205] A query extraction and analysis unit, configured to extract query conditions from the query request and analyze the user's query preference based on the query conditions;

[0206] The data sorting and paging unit is used to sort the query analysis data first and then paging according to the query conditions and the query preferences to obtain the query results.

[0207] In an optional embodiment, the network module maintains a communication connection with the remote data source, and the device further includes:

[0208] A latest data acquisition unit, used for periodically acquiring the latest data from the remote data source through the network module, and sending the latest data to the database module;

[0209] A data comparison unit, used to compare the latest data with the existing data stored previously through the database module;

[0210] A data updating unit, configured to, when there is an updated data item in the latest data, overwrite the original data item of the existing data with the updated data item;

[0211] The newly added data storage unit is used to store the newly added data item when there is a newly added data item in the latest data.

[0212] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the aforementioned method embodiment.

[0213] An embodiment of the present invention further provides an electronic device, the device comprising a processor and a memory:

[0214] The memory is used to store the program code and transmit the program code to the processor;

[0215] The processor is used to execute the self-service data query method of any embodiment of the present invention according to the instructions in the program code.

[0216] An embodiment of the present invention further provides a computer-readable storage medium, which is used to store program codes, and the program codes are used to execute the self-service data query method of any embodiment of the present invention.

[0217] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0218] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0219] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0220] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0221] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0222] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A self-service data query system, characterized in that: The self-service data query system includes a main control module, a display module connected to the main control module, a network module, a database module and a data processing module, wherein the network module and the data processing module are respectively connected to the database module; The main control module is used to receive at least one query request sent by the display module, and send each query request to the network module through multi-threaded concurrent processing, so that the network module can obtain corresponding query data according to the query request, and send the query data to the database module for storage; The data processing module is used to receive the query data transmitted from the database module, perform data processing on the query data based on the query request, and send the query result to the main control module, so that the main control module returns the query result to the display module.

2. The self-service data query system according to claim 1, characterized in that: The display module includes at least one editable user interface, and the display module is used to: For each of the user interfaces, when a trigger operation for the user interface is detected, an initial user interface is loaded, the initial user interface including selectable query options and / or a text input box; In response to a selection operation on the selectable query option and / or an edit input operation on the text input box, generating a query condition; Performing input verification on the query condition; If the verification fails, an error message is displayed on the user interface to guide the user to re-enter the query conditions; If the verification is successful, a query request corresponding to the query condition is generated and the query request is sent to the main control module.

3. The self-service data query system according to claim 2, characterized in that: The display module is also used for: According to the query result, a query result interface is loaded on the user interface, data summary information is displayed, and a data query button, and / or a data query link, and / or an interactive operation button, and / or a result export button are popped up; If it is detected that the data query button is clicked, the data detailed information corresponding to the data summary information is displayed on the query result interface; If it is detected that the data query link is clicked, the display interface corresponding to the data query link is jumped to, and the data detailed information corresponding to the data summary information is displayed on the display interface; If it is detected that the interactive operation button is clicked, the main control module is triggered to reprocess the information displayed on the query result interface, and refresh the display content of the query result interface based on the processing result returned from the main control module. The interactive action includes but is not limited to adjusting the sorting method and filtering the displayed content; If it is detected that the result export button is clicked, an export format selection button is popped up on the query result interface, and in response to a selection operation on the export format selection button, a query result data file in a corresponding format is exported.

4. The self-service data query system according to claim 1, characterized in that: The main control module is used for: Upon receiving a query request sent by the display module, first determine whether the query request data corresponding to the query request is frequently queried data, the frequently queried data being data queried more than once within a preset query time period, and the frequently queried data being cached in the database module within the preset query time period; If yes, directly retrieve the frequently queried data from the database module as the query result corresponding to the query request and return it to the display module; If not, the query request is sent to the network module to execute the subsequent data query process.

5. The self-service data query system according to claim 1, characterized in that: The data processing module is used for: When receiving the query data transmitted from the database module, performing data preprocessing on the query data to obtain query processing data; Analyzing and summarizing the query processing data to obtain query analysis data; Extracting query conditions from the query request, and analyzing the user's query preference based on the query conditions; According to the query conditions and the query preferences, the query analysis data is first sorted and then paginated to obtain the query results.

6. The self-service data query system according to any one of claims 1 to 5, characterized in that: The network module maintains a communication connection with a remote data source, and the network module is used to: Periodically obtain the latest data from the remote data source and send the latest data to the database module; The database module is used for: comparing the latest data with the existing data previously stored; If there is an updated data item in the latest data, the updated data item is used to overwrite the original data item of the existing data; If there is a new data item in the latest data, the new data item is stored.

7. A self-service data query method, characterized in that: The self-service data query method is used in the self-service data query system according to any one of claims 1 to 6, and the method comprises: Receive at least one query request sent by the display module through the main control module, send each query request to the network module through multi-threaded concurrent processing, so that the network module obtains corresponding query data according to the query request, and sends the query data to the database module for storage; The query data transmitted from the database module is received through the data processing module, data processing is performed on the query data based on the query request, and the query result is sent to the main control module, so that the main control module returns the query result to the display module.

8. A data self-service query device, characterized in that: The self-service data query device is used in the self-service data query method according to claim 7, and the device comprises: A query request processing unit, configured to receive at least one query request sent by the display module through the main control module, and send each query request to the network module through multi-threaded concurrent processing, so that the network module can obtain corresponding query data according to the query request, and send the query data to the database module for storage; The query data processing unit is used to receive the query data transmitted from the database module through the data processing module, perform data processing on the query data based on the query request, and send the query result to the main control module so that the main control module returns the query result to the display module.

9. An electronic device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the self-service data query method of claim 7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the self-service data query method described in claim 7.

Images