Client data query method and device, storage medium and electronic equipment

Abstract

The invention provides a client data query method and device, a storage medium and electronic equipment, and belongs to the technical field of data processing. The client is deployed with a client agent, the client agent is provided with a local mirror image table and a data access interface, the local mirror image table is obtained by synchronizing a real-time library of the server, and the data access interface is a local mirror image table external access interface established based on an index of the local mirror image table; the method comprises the steps of obtaining a data query request, wherein the data query request comprises a data identifier; and based on the data query request, querying in the local mirror image table through the data access interface to obtain a query result. The client agent establishes the local mirror image table, the query efficiency of real-time data of the client can be improved through the client agent technology, query is achieved through local direct addressing, the addressing time is not affected by network access, and reading lagging cannot be caused.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically to a client-side data query method, a client-side data query device, a machine-readable storage medium, and an electronic device. Background Technology

[0002] With the rapid development of high-concurrency scenarios such as the Internet of Things and industrial monitoring, the demand for low latency, high throughput, and dynamic scalability in real-time data processing systems is becoming increasingly urgent. Traditional disk-based relational databases (such as MySQL) are unable to meet millisecond-level response requirements due to I / O bottlenecks, while in-memory databases (such as Redis and InfluxDB) significantly improve performance by directly manipulating in-memory data.

[0003] However, when performing data queries, the client needs to interact with the server frequently, and network latency increases the real-time query response time (>50ms). Existing client caching technologies (such as Apache Ignite) require transmitting data one by one for full synchronization, which consumes a lot of bandwidth and takes a long time.

[0004] Therefore, existing client-side data querying suffers from low query efficiency. Summary of the Invention

[0005] The purpose of this invention is to provide a client data query method, a client data query device, a machine-readable storage medium, and an electronic device. This client data query method can improve the query efficiency of real-time client data through client proxy technology.

[0006] To achieve the above objectives, the first aspect of this application provides a client-side data query method. The client deploys a client proxy, which is configured with a local mirror table and a data access interface. The local mirror table is obtained by synchronizing a real-time database from a server. The data access interface is an external access interface for the local mirror table, built based on an index of the local mirror table. The method includes: Obtain a data query request, wherein the data query request includes a data identifier; Based on the data query request, a query is performed in the local mirror table through the data access interface to obtain the query result; The real-time library is obtained by the server through real-time acquisition of collected data, which includes data from at least one data source. Based on the collected data and a preset required data specified address, a data specified memory address is determined. The preset required data specified address is determined based on the physical address of the memory table in the memory table group corresponding to the required data. One memory table group is used to store data from one data source. Based on the data specified memory address, the collected data is written into the corresponding memory table.

[0007] In this embodiment of the application, it also includes: The client proxy generates a data synchronization request and sends the data synchronization request to the server. The server synchronizes the data from the real-time database to the local mirror table based on the data synchronization request.

[0008] In this embodiment of the application, the step of generating a data synchronization request by the client proxy and sending the data synchronization request to the server includes: The frequently accessed data table is determined by the client agent in the local mirror table; The client proxy determines whether to submit a query request to the server based on the access time of the frequently accessed data table; If it is determined that a query request will be submitted to the server, the client proxy will generate a data synchronization request and send the data synchronization request to the server.

[0009] In this embodiment of the application, the step of the client proxy determining whether to submit a query request to the server based on the access time of the frequently accessed data table includes: The client agent determines whether the difference between the access time of the frequently accessed data table and the current system time meets a preset time threshold. If the difference between the access time of the frequently accessed data table and the current system time meets a preset time threshold, the client agent determines to submit a query request to the server.

[0010] In this embodiment of the application, the data synchronization request includes the timestamp of the local mirror table; The step of the server synchronizing data from the real-time database to the local mirror table based on the data synchronization request includes: The server responds to the data synchronization request by determining whether the timestamp of the local mirror table is the same as the timestamp of the real-time database. If the timestamp of the local mirror table is determined to be different from the timestamp of the real-time database, the server synchronizes the data of the real-time database to the local mirror table based on the data synchronization request.

[0011] In this embodiment of the application, the step of generating a data synchronization request by the client proxy and sending the data synchronization request to the server includes: The client agent determines whether the synchronization time interval of the local mirror table exceeds a preset synchronization time threshold. If the synchronization time interval of the local mirror table exceeds a preset synchronization time threshold, the client agent generates a data synchronization request and sends the data synchronization request to the server.

[0012] In this embodiment of the application, after obtaining the query results, the method further includes: The client agent updates the access time of the local mirror table to the current system time.

[0013] A second aspect of this application provides a client-side data query device, wherein the client is equipped with a client proxy, the client proxy has a local mirror table and a data access interface, the local mirror table is obtained by synchronizing a real-time database from a server, and the data access interface is an external access interface for the local mirror table built based on the index of the local mirror table; the device includes: The acquisition module is used to acquire data query requests, wherein the data query requests include data identifiers; The query module is used to query the local mirror table through the data access interface based on the data query request to obtain the query result. The real-time database is obtained by the server through real-time acquisition of collected data, which includes data from at least one collection source. A designated memory address for the data is determined based on the collected data and a preset address for the required collected data. The preset address for the required collected data is determined based on the physical address of the memory table in the memory table group corresponding to the required collected data. One memory table group is used to store data from one collection source. The collected data is written to the corresponding memory table based on the designated memory address.

[0014] A third aspect of this application provides an electronic device, the electronic device comprising: At least one processor; A memory connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the above-described client data query method by executing the instructions stored in the memory.

[0015] A fourth aspect of this application provides a machine-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the aforementioned client data query method.

[0016] The above technical solution involves deploying a client-side proxy on the client side. This client-side proxy is configured with a local mirror table and a data access interface. The local mirror table is obtained by synchronizing with the server's real-time database. The data access interface is an external access interface for the local mirror table, built based on its index. The system retrieves data query requests, which include data identifiers. Based on these requests, the system queries the local mirror table through the data access interface to obtain the query results. The real-time database is obtained by the server through real-time acquisition of collected data. This collected data includes data from at least one source. Based on the collected data and a pre-defined address for the required collected data, a specified memory address is determined. This pre-defined address is based on the physical address of a memory table in a memory table group corresponding to the required collected data. One memory table group stores data from one source. Based on the specified memory address, the collected data is written to the corresponding memory table. The client-side proxy's establishment of a local mirror table improves the efficiency of real-time data queries on the client side, converting frequent, fragmented queries into single network accesses, reducing latency to <5ms, shortening full synchronization time by 80%, and reducing bandwidth usage by 70%. Since the query is performed through local direct addressing, the addressing time is not affected by network access and will not cause reading lag.

[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 The illustration shows a flowchart of a client data query method according to an embodiment of this application; Figure 2 This schematic diagram illustrates a structural block diagram of a client data query device according to an embodiment of the present application; Figure 3 The diagram illustrates the internal structure of a computer device according to an embodiment of this application.

[0019] Explanation of reference numerals in the attached figures 410 - Acquisition module; 420 - Query module; A01 - Processor; A02 - Network interface; A03 - Internal memory; A04 - Display screen; A05 - Input device; A06 - Non-volatile storage medium; B01 - Operating system; B02 - Computer program. Detailed Implementation

[0020] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0021] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0022] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0023] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0024] Please refer to Figure 1 , Figure 1 This illustration schematically shows a flowchart of a client-side data query method according to an embodiment of this application. This embodiment provides a client-side data query method whereby the client deploys a client proxy, the client proxy is configured with a local mirror table and a data access interface, the local mirror table is obtained by synchronizing a real-time database from a server, and the data access interface is an external access interface for the local mirror table built based on the index of the local mirror table; the method includes the following steps: Step 210: Obtain a data query request, wherein the data query request includes a data identifier; In this embodiment, the data query request includes the data identifier to be queried, which may be a data ID. This ID can be obtained through user input.

[0025] The real-time library is obtained by the server through real-time acquisition of collected data, which includes data from at least one data source. Based on the collected data and a preset required data specified address, a data specified memory address is determined. The preset required data specified address is determined based on the physical address of the memory table in the memory table group corresponding to the required data. One memory table group is used to store data from one data source. Based on the data specified memory address, the collected data is written into the corresponding memory table.

[0026] In this embodiment, deploying a client-side proxy is a technical strategy that directly embeds proxy logic into the client device. This proxy, acting as a local intermediary between the client and the remote service, can significantly optimize system performance and reliability by handling tasks such as network communication, data conversion, and security control. The client-side proxy can be established by accessing the server's real-time library system (i.e., the real-time library) via a network connection (which can be local) to obtain all its tables and their memory structure and data; that is, by creating a local mirror table based on the real-time library's table structure and data. In specific implementations, only the parts that need to be accessed can be selected according to the design. Then, the local mirror table is completely copied in local memory, and indexes, including global and local indexes, are created. These indexes are identical to those in the real-time library, thus enabling the creation of a local mirror table on the client-side proxy. An external access interface for the local mirror table is then established based on its indexes. The data access interface can be in C++ (or wrapped in other forms such as C#, Java, etc.). This interface is generally embedded in the client program as a dynamic library, belonging to the same process as the client program. By establishing the data access interface, the original server-side interface calls can be converted into local interface calls. Remote service API calls involve establishing a connection with the remote service, passing input parameters, waiting for the return result, parsing the obtained result data, and closing the connection. Local API calls, on the other hand, provide API functions (such as GetXXX, ReadXXX, etc.) that allow users to directly retrieve data from the local mirror table through the data access interface.

[0027] In this embodiment, the real-time library can be established on the server side or it can be established from multiple memory tables. Since the preset address of the required data to be collected is obtained based on the storage location of the required data in multiple memory table groups, that is, the required data to be collected is stored separately according to the collection source, only the corresponding collection driver of a memory table group performs write operations, and there is no competition with other programs for the table write lock, thus eliminating lock contention, thereby improving throughput, reducing latency, meeting the needs of high-concurrency scenarios, and meeting the real-time requirements of IoT, industrial monitoring and other scenarios.

[0028] During the establishment of the aforementioned real-time library, data acquisition can be achieved through a terminal acquisition program. This program can include multiple acquisition drivers, each corresponding to a thread or instance. Different drivers are used to acquire data from different sources. After the terminal acquisition program starts running, it begins acquiring data through each acquisition driver. The preset required acquisition data address contains multiple pre-defined memory addresses for data; these are the required acquisition data. A memory table group is used to store data from one acquisition source, and only the corresponding acquisition driver can access a memory table group. The acquired data can be located at the corresponding storage address within the preset required acquisition data address, thus obtaining the data's specified memory address.

[0029] In some embodiments, the process of determining the preset address of the required data to be collected includes: First, the memory table is divided according to the preset table partitioning rules to obtain multiple memory table groups. The preset table partitioning rules include at least partitioning the memory table based on the collection source of the data to be collected. Then, based on the physical address of the memory table in each memory table group, the storage address of the data to be collected corresponding to each memory table group is obtained; Finally, based on the storage address of the required data to be collected corresponding to each memory table group, the specified address of the required data to be collected is obtained.

[0030] In this embodiment, the storage location of the required data in the multiple memory table groups can be obtained by allocating corresponding storage locations for the required data in the multiple memory table groups divided according to preset table partitioning rules. The preset table partitioning rules include at least table partitioning based on the collection source of the required data. Specifically, table partitioning can be based on the collection source. Since the collection source program generally sends data serially, data from the same collection source can be divided into one table or a group of tables. The distinction of the collection source here includes geographical location, individual collection program, etc., which can be determined according to the actual situation. For example, the air conditioners in Building A and the air conditioners in Building B are divided into two tables due to their geographical separation; the air conditioners in Building A and the electricity meters in Building A are also divided into different tables because they use different terminal collection programs (different interface protocols). The required data can include data from multiple collection sources, and each collection source can have multiple data. For each data, a corresponding storage location can be allocated in the memory table of the corresponding memory table group to construct the specified address of the required data.

[0031] It should be noted that the aforementioned preset table partitioning rules can also include partitioning by basic data type and / or by data acquisition frequency. The specific rules can be determined based on actual circumstances, and this embodiment does not impose any limitations. That is, after dividing the data into multiple memory table groups according to the acquisition source, different memory tables can be further partitioned based on the basic data type of the acquisition source and / or the data acquisition frequency. For example, for switch signals and analog signals, due to their significant differences in data type and business algorithms, they can be divided into different memory tables. Data with high-frequency changes and large volumes can also be divided into different memory tables.

[0032] By dividing the memory into multiple memory table groups according to the preset table partitioning rules, the storage address of the required data for each memory table group can be accurately obtained based on the physical address of the memory table in each memory table group. This allows for the pre-allocation of storage locations for the required data. At the same time, the required data is also divided according to the data collection source, which helps the data to quickly and accurately find the corresponding specified memory address.

[0033] In some embodiments, the preset address of the required data to be collected is associated with a local index, which is obtained by indexing the data identifier of the required data to be collected, the corresponding physical address of the memory table, and the row position in the table. The step of determining the specified memory address for the data based on the collected data and a preset address for the required collected data includes: First, based on the collected data, the corresponding data identifier is determined; In this embodiment, the data identifier, or data ID, is a unique identifier for the data. It can be a string of characters and can be named according to a certain naming convention. For example, the naming convention could be: geographical location + professional system number + device type + device code + device attribute name (referring to voltage, current, temperature, etc.). The row position in the above table refers to the specific row in the memory table, which can refer to the row index. Based on the data ID of the data to be collected, the corresponding physical address of the memory table, and the row index in the table, a hash table or array can be built for the index according to the characteristics of the data to be collected, to obtain a local index. Each data ID corresponds to an index number, which is associated with the corresponding physical address of the memory table and the row index in the table. The characteristics of the data to be collected can refer to the data structure of the data to be collected. For example, in the data, a number or array represents that it comes from different devices. After obtaining the collected data, the corresponding data identifier can be obtained by parsing the collected data.

[0034] Then, based on the corresponding data identifier, the local index is used for addressing to obtain the specified memory address of the data.

[0035] In this embodiment, the table row can be directly addressed according to the corresponding data identifier and the local index to obtain the specified memory address of the data, thereby reducing the addressing time, avoiding addressing overhead, and improving transaction processing capabilities.

[0036] After determining the specified memory address for the data, the collected data can be written to the specified column of the corresponding row in the memory table. For example, the data collected by the terminal acquisition program generally includes three items: value, status (valid / invalid), and time. The corresponding variable table has three columns: Value, State, and Time. The position of the corresponding column can be calculated based on the memory area of ​​each row and column according to a fixed byte size. The data type and length of each column in each table are determined. For example, float occupies 4 bytes, double occupies 8 bytes, integers occupy 1, 8, 16, 32 to 64 bytes depending on their size, and strings occupy bytes according to the configured length, thus completing the data writing.

[0037] It should be noted that at most one acquisition program can process all tables in a memory table group. The data in this table (such as the voltage and current of an electricity meter) is collected from the device and will only be modified by this one acquisition program. Other programs will not modify it (through configuration, the core program must go through the acquisition program when modifying the data in this table, and the acquisition program will block tampering by other programs). If the user wants to modify the device data, he / she issues a modification command to the device. The acquisition program will only modify the data after the device data changes and the new data is collected.

[0038] In some embodiments, when establishing a real-time database, the server can also respond to data synchronization instructions to synchronize data in each memory table group to an external system.

[0039] In this embodiment, the aforementioned external system can refer to multiple service processes. Data synchronization of all in-memory tables can be performed among these multiple service processes to support service redundancy and service clustering. Synchronization of each table can be based on timestamps; for tables with a large number of rows, synchronization can be performed by partitioning them according to a certain number of rows.

[0040] In some embodiments, the memory table is divided into multiple regions; In this embodiment, the columns in the memory table can be divided into primary keys (including foreign keys), parameter columns, and real-time columns. The primary key is used to uniquely identify data, the parameter columns are used to configure the algorithmic behavior of the data (such as alarm behavior, limit violation behavior, etc.), and the real-time columns contain the data collected in real time and the calculation data based on it. Therefore, the memory table can be divided into three parts: primary key data area, parameter data area, and real-time data area.

[0041] Accordingly, synchronizing the data in each of the memory table groups to the external system includes: First, the synchronization area is determined based on the position of the currently changed data in the memory table of each memory table group; Then, the data in the synchronization area is synchronized to the external system.

[0042] In this embodiment, there are several existing methods for processing data in memory tables. For example, one method involves processing each cell (row and column) as a data block (memory block), one cell at a time. This leads to memory fragmentation and requires creating an index for each piece of data, resulting in a large number of indexes. Furthermore, data synchronization requires synchronizing each data cell individually, leading to low overall efficiency. Another method involves processing the entire table as a single data block. In this case, the memory table does not distinguish between parameter data and real-time data. Parameter data is relatively static (e.g., range, unit, description, alarm level, etc.), while real-time data changes frequently. Mixing these two types of data together forces the parameter data to be synchronized along with the real-time data when frequent synchronization is required, reducing synchronization efficiency and consuming network bandwidth.

[0043] By dividing the memory table into multiple regions, during data synchronization, it is determined whether there are data changes in each region. Only when data changes are found is the data in that region synchronized. In other words, based on the position of the currently changed data in each memory table group, the synchronization region is determined, and the data in that region is synchronized to the external system. This avoids memory fragmentation, facilitates fast addressing, and improves data synchronization efficiency.

[0044] In some embodiments, after successfully writing the collected data into the corresponding memory table, the method further includes: sending the historical data of the memory table to an external database for storage.

[0045] In this embodiment, the most recent data can be stored in a memory table, supporting real-time (millisecond-level) read and write operations. The memory table stores hot data in a fixed-size memory area, while cold data is archived to external storage. Here, cold data refers to historical data, and hot data refers to data currently being written to the memory table. Specifically, a write operation to the memory table can trigger a call to the storage interface. It should be noted that some insensitive data (such as continuously changing analog quantities like temperature and voltage) can use periodic changes to trigger a call to the storage interface. The storage interface caches data and calls the storage adapter interface to write to external databases such as MySQL or Oracle (or other storage media).

[0046] In practice, a single module can interface with one storage engine (or multiple storage engines simultaneously), providing tiered storage for hot and cold data, and also including initial system loading from configuration data. To avoid delays in real-time computation from external storage, secondary caching and asynchronous operations can be used for external storage.

[0047] By sending historical data from the in-memory table to an external database for storage, the in-memory computing node only processes hot data, and the external database can expand cold data storage as needed (such as MySQL, OSS, etc.), improving resource utilization by 40% and reducing hardware costs by 30%. Computing and storage resources can be expanded and scald independently, supporting rapid business expansion.

[0048] Extending existing systems (such as Redis) requires intrusive modifications to core module code, lacking modular design, resulting in long development cycles and poor compatibility. An algorithm class can be attached to each two-dimensional in-memory table. This algorithm class can be extended based on object-oriented inheritance (e.g., subclasses override onInsert and onUpdate methods), supporting custom data processing logic (e.g., alarm rules, limit calculations, etc.), achieving non-intrusive feature extensions. However, traditional feature extensions rely on single inheritance or composition patterns, lacking flexibility. For example, multiple inheritance easily leads to the "diamond inheritance" problem, while composition patterns require manual maintenance of interface call chains, increasing development complexity. To avoid coupling issues in inheritance extensions, features can be dynamically combined for each two-dimensional in-memory table using specific column data. For example, an "alarm column" and related columns can be set in the "collected variable table," automatically triggering alarm algorithms based on column values ​​(automatically determining whether to trigger an alarm based on alarm configuration).

[0049] In some embodiments, the memory table is provided with a function column, which is associated with a function module, and the function module is used to perform corresponding data processing based on the value of the function column.

[0050] In this embodiment, a "Function Flag Column" can be added to the memory table, which is the function column. This function column is associated with a functional module, and the associated algorithm can be automatically triggered based on the column value. For example, if the "ELevel1" column is a number >= 0, it means that an alarm level is triggered when the collected value is 1; otherwise, no alarm is triggered. A "Function Extension Column" can be added to the table, extending different algorithms based on the column value. For example, if the "Trans" column is "+:10", it means that the collected data is incremented by 10; if the "Trans" column is "Code:Unicode", it means that the collected data is converted to Unicode.

[0051] In practical implementation, a base class CRdbTable can be defined, providing virtual methods such as onInsert() and onUpdate(). Subclasses (such as CAITable, CDITable, etc.) can override these methods to implement business logic, thereby achieving algorithm class inheritance and extension.

[0052] It should be noted that a plug-in-style function loading method can be adopted, and the function modules can be plugged and played by configuring column mapping relationships.

[0053] By triggering different algorithms through function columns (such as column values ​​triggering alarms or calculation logic), the coupling between modules is greatly reduced, with tests showing a reduction of up to 70%. Functions can be dynamically combined according to business needs, thus achieving flexible configuration.

[0054] Step 220: Based on the data query request, perform a query on the local mirror table through the data access interface to obtain the query result.

[0055] In this embodiment, after receiving a data query request, the query is performed in the local mirror table based on the data identifier of the data query request by calling the data access interface, and the query result is obtained.

[0056] In practical implementation, as a client program, the goal is to display data that users care about in real time, such as air conditioner temperature and mode, electricity meter voltage and current, etc. How this data is distributed in the real-time database, and whether it belongs to the same table, is generally unknown (and uncertain, as it is affected by the server-side table creation mechanism) to the client. The client program typically only iterates through the data IDs to query and read data one by one. Therefore, each query request for each data ID is a fragmented query. These data IDs are scattered across different tables. The client proxy queries the remote database once to read the entire table and returns all the data. The client proxy does not aggregate fragmented queries; instead, it simply queries and returns all data tables.

[0057] For example, a partial query needs to read 10 data entries, such as meter 1, meter 2, ..., meter 10, located in rows 1 to 10 of table A. From a network call perspective, this partial query requires querying table A 10 times, while the client agent only needs to query table A once. Although the effective data volume should be the same—10 data entries—considering auxiliary data such as network query connection, input and return, and packet headers and footers, the client agent querying the entire table is more efficient.

[0058] In the above implementation process, a client proxy is deployed on the client side. The client proxy is configured with a local mirror table and a data access interface. The local mirror table is obtained by synchronizing the real-time database of the server. The data access interface is an external access interface for the local mirror table built based on the index of the local mirror table. A data query request is obtained, which includes a data identifier. Based on the data query request, a query is performed on the local mirror table through the data access interface to obtain the query result. The real-time database is obtained by the server through real-time acquisition of collected data. The collected data includes data from at least one collection source. Based on the collected data and a preset required collection data address, a specified memory address is determined. The preset required collection data address is determined based on the physical address of the memory table in the memory table group corresponding to the required collection data. One memory table group is used to store data from one collection source. Based on the specified memory address, the collected data is written to the corresponding memory table. The client proxy establishes a local mirror table, which can improve the query efficiency of real-time data on the client side, converting frequent, scattered queries into single network accesses, reducing latency to <5ms, shortening full synchronization time by 80%, and reducing bandwidth usage by 70%. Since the query is performed through local direct addressing, the addressing time is not affected by network access and will not cause reading lag.

[0059] In this embodiment, after obtaining the query results, the method further includes: updating the access time of the local mirror table to the current system time by the client agent.

[0060] In this embodiment, after obtaining the query results, it indicates that an external program has accessed the table data. Therefore, the access time of this table is updated to the current system time to leave an access record.

[0061] In some embodiments, it also includes: First, the client proxy generates a data synchronization request and sends the data synchronization request to the server. Then, the server synchronizes the data from the real-time database to the local mirror table based on the data synchronization request.

[0062] In this embodiment, the data synchronization request is used to inform the server that the real-time library needs to be synchronized. The server synchronizes the data in the real-time library to the local mirror table according to the data synchronization request, so that the data in the local mirror table is consistent with the data in the real-time library.

[0063] In some embodiments, the step of generating a data synchronization request by the client proxy and sending the data synchronization request to the server includes: The first step is for the client agent to determine the frequently accessed data table in the local mirror table; In this embodiment, the local mirror table includes multiple data tables. The frequently accessed data tables can be determined based on statistics and user habits. Specifically, if a user accesses a certain type of data, they will generally access this type of data frequently over a long period of time. This type of data is concentrated in several tables. The frequently accessed data tables can be determined based on the frequency of access to the data tables.

[0064] The second step involves the client proxy determining whether to submit a query request to the server based on the access time of the frequently accessed data table. In this embodiment, the access time of the frequently accessed data table is the time when the client agent confirms the access to the table, which can be obtained by recording the current system time each time the data table is queried.

[0065] In some embodiments, the step of the client proxy determining whether to submit a query request to the server based on the access time of the frequently accessed data table includes: First, the client agent determines whether the difference between the access time of the frequently accessed data table and the current system time meets a preset time threshold. In this embodiment, the access time of the frequently accessed data table can be the most recent access time of this table. The preset time threshold can be set in advance according to the actual situation, for example, it can be 2 seconds. The difference between the access time of the frequently accessed data table and the current system time is calculated, and then compared to see if the difference is less than the time threshold.

[0066] Then, if the difference between the access time of the frequently accessed data table and the current system time meets a preset time threshold, the client agent determines to submit a query request to the server.

[0067] In this embodiment, if the difference is less than a time threshold, it indicates that the table has recently been accessed by an external program and needs to be updated promptly. In this case, the client agent determines to submit a query request to the server. Otherwise, no update is required.

[0068] By analyzing the difference between the access time of frequently accessed data tables and the current system time, it is possible to accurately determine whether the frequently accessed data tables need to be updated in a timely manner, so as to issue query requests.

[0069] Third, if it is determined that a query request will be submitted to the server, the client proxy generates a data synchronization request and sends the data synchronization request to the server.

[0070] In this embodiment, if it is determined that a query request will be submitted to the server, the client proxy generates a data synchronization request and sends the data synchronization request to the server; otherwise, no data synchronization request is sent.

[0071] By identifying the frequently accessed data table, and based on the access time of the frequently accessed data table, it can be quickly determined whether an update is needed, and thus an accurate data synchronization request can be sent to the server for data synchronization.

[0072] In some embodiments, the step of generating a data synchronization request by the client proxy and sending the data synchronization request to the server includes: First, the client agent determines whether the synchronization time interval of the local mirror table exceeds a preset synchronization time threshold. Then, if it is determined that the synchronization time interval of the local mirror table exceeds a preset synchronization time threshold, the client agent generates a data synchronization request and sends the data synchronization request to the server.

[0073] In this embodiment, to avoid the local mirror table data becoming too outdated, a synchronization time threshold can be pre-set based on actual conditions, such as 5 seconds. Then, it is determined whether more than 5 seconds have passed without synchronization. If so, synchronization is required, and the client proxy generates a data synchronization request and sends it to the server. Otherwise, synchronization is not required. The aforementioned synchronization time interval refers to the time interval between the most recent synchronization and the current system time.

[0074] By determining whether the synchronization interval of the local mirror table exceeds the preset synchronization time threshold, it is possible to quickly determine whether data updates are needed, thus preventing the data in the local mirror table from becoming too outdated and helping to ensure the reliability of data queries.

[0075] In some embodiments, the data synchronization request includes the timestamp of the local mirror table; correspondingly, the step of synchronizing the data of the real-time library to the local mirror table by the server based on the data synchronization request includes: First, in response to the data synchronization request, the server determines whether the timestamp of the local mirror table is the same as the timestamp of the real-time database. Then, if it is determined that the timestamp of the local mirror table is different from the timestamp of the real-time library, the server synchronizes the data of the real-time library to the local mirror table based on the data synchronization request.

[0076] In this embodiment, after receiving a data synchronization request, the server needs to determine whether the timestamp of the local mirror table is the same as the timestamp of the real-time database. If they are the same, it means that the data in the local mirror table is consistent with the data in the real-time database, and synchronization is not required; otherwise, synchronization is required. Here, the timestamp refers to the change time marker for each table set by the server, obtained by recording the server time when changes are made to the real-time database. When the client proxy copies the real-time database, it also copies the timestamp to the local machine, meaning the local mirror table also has a corresponding timestamp. When the table on the server undergoes a new change, the timestamp will also change accordingly. When a data synchronization request occurs, the client proxy also sends the timestamp of the local mirror table to the server for comparison. It should be noted that the timestamp is the timestamp of each data table. Accordingly, determining whether the timestamp of the local mirror table is the same as the timestamp of the real-time database involves checking whether the timestamps of each table are the same. If the timestamp of any one table is different, the timestamp of the local mirror table is considered different from the timestamp of the real-time database; otherwise, they are considered the same.

[0077] It should be noted that, in the process of synchronizing table data from the server-side real-time database to the local machine, the synchronization process for each table can be carried out separately according to the primary key data area, parameter data area, and real-time data area (generally only the real-time data area needs to be synchronized). If the table has partitions (when the number of rows in the table is large), then the synchronization is carried out by partition.

[0078] Since the timestamp represents the time of change of the table by the server, by judging whether the timestamp of the local mirror table is the same as the timestamp of the real-time database, it can be quickly determined whether the data of the local mirror table is consistent with the data of the server, so as to quickly and accurately determine whether the data is synchronized.

[0079] The following specific examples illustrate the effectiveness of the solution: Example 1: Industrial IoT equipment monitoring system. Application scenario: Real-time collection and processing of sensor data (temperature, pressure, vibration, etc.) from 100,000 industrial devices.

[0080] Client Deployment: Hardware Configuration: Desktop Computer: 1 unit (4-core CPU / 8GB RAM); Software Configuration: Client agent, display interface program. Interface Display: Single screen displays 1 to 100 devices, data display is smooth, and the data display latency is 500ms to 1000ms (affected by the interface refresh cycle).

[0081] Example 2: Subway Integrated Monitoring System. Application Scenario: Urban rail transit subway integrated monitoring system, which collects and controls subway-related professional equipment.

[0082] Client Deployment: Hardware Configuration: Desktop Computer: 1 unit (8-core CPU / 16GB RAM); Software Configuration: Client agent, display interface program. Interface Display: Single screen displays all or most devices (tens to thousands) within a professional system (or a subsystem within a professional system). Data display is smooth with a latency of 500ms to 1000ms (affected by the interface refresh cycle).

[0083] Please refer to Figure 2 , Figure 2 This schematically illustrates a structural block diagram of a client data query device according to an embodiment of the present application. This embodiment provides a client data query device, wherein the client deploys a client agent, the client agent is configured with a local mirror table and a data access interface, the local mirror table is obtained by synchronizing a real-time database from a server, and the data access interface is an external access interface for the local mirror table established based on the index of the local mirror table; the device includes an acquisition module 410 and a query module 420, wherein: The acquisition module 410 is used to acquire a data query request, wherein the data query request includes a data identifier; The query module 420 is used to query the local mirror table through the data access interface based on the data query request to obtain the query result; wherein, the real-time library is obtained by the server through real-time acquisition of collected data, the collected data includes data from at least one collection source, and a data-specified memory address is determined based on the collected data and a preset required collection data-specified address. The preset required collection data-specified address is determined based on the physical address of the memory table in the memory table group corresponding to the required collection data. One memory table group is used to store data from one collection source; the collected data is written to the corresponding memory table based on the data-specified memory address.

[0084] The client data query device includes a processor and a memory. The acquisition module 410 and the query module 420 are stored in the memory as program units. The processor executes the program units stored in the memory to realize the corresponding functions.

[0085] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and client data queries are implemented by adjusting kernel parameters.

[0086] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0087] This invention provides a machine-readable storage medium storing a program that, when executed by a processor, implements the client data query method.

[0088] This invention provides a processor for running a program, wherein the program executes the client data query method during runtime.

[0089] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown in the figure, the computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05, and a memory (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01 and a computer program B02. The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A06. The network interface A02 is used for communication with external terminals via a network connection. When the computer program is executed by the processor A01, it implements a client data query method. The display screen A04 can be a liquid crystal display (LCD) or an e-ink display. The input device A05 can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0090] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0091] In one embodiment, the client data query device provided in this application can be implemented as a computer program, and the computer program can be implemented in the form of, for example, Figure 3 It runs on the computer device shown. The computer device's memory can store the various program modules that make up the client data query device, for example, Figure 2 The acquisition module 410 and the query module 420 are shown. The computer program, composed of these various program modules, causes the processor to execute the steps in the client data query methods of the various embodiments of this application described in this specification.

[0092] Figure 3 The computer equipment shown can be used as follows Figure 2 The acquisition module 410 in the client data query device shown executes step 210, and the query module 420 executes step 220.

[0093] This application provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. The client deploys a client agent, which is configured with a local mirror table and a data access interface. The local mirror table is obtained by synchronizing a real-time database from a server. The data access interface is an external access interface for the local mirror table built based on its index. When the processor executes the program, it implements the following steps: Obtain a data query request, wherein the data query request includes a data identifier; Based on the data query request, a query is performed in the local mirror table through the data access interface to obtain the query result; The real-time library is obtained by the server through real-time acquisition of collected data, which includes data from at least one data source. Based on the collected data and a preset required data specified address, a data specified memory address is determined. The preset required data specified address is determined based on the physical address of the memory table in the memory table group corresponding to the required data. One memory table group is used to store data from one data source. Based on the data specified memory address, the collected data is written into the corresponding memory table.

[0094] In one embodiment, it also includes: The client proxy generates a data synchronization request and sends the data synchronization request to the server. The server synchronizes the data from the real-time database to the local mirror table based on the data synchronization request.

[0095] In one embodiment, the step of generating a data synchronization request by the client proxy and sending the data synchronization request to the server includes: The frequently accessed data table is determined by the client agent in the local mirror table; The client proxy determines whether to submit a query request to the server based on the access time of the frequently accessed data table; If it is determined that a query request will be submitted to the server, the client proxy will generate a data synchronization request and send the data synchronization request to the server.

[0096] In one embodiment, the step of the client proxy determining whether to submit a query request to the server based on the access time of the frequently accessed data table includes: The client agent determines whether the difference between the access time of the frequently accessed data table and the current system time meets a preset time threshold. If the difference between the access time of the frequently accessed data table and the current system time meets a preset time threshold, the client agent determines to submit a query request to the server.

[0097] In one embodiment, the data synchronization request includes the timestamp of the local mirror table; The step of the server synchronizing data from the real-time database to the local mirror table based on the data synchronization request includes: The server responds to the data synchronization request by determining whether the timestamp of the local mirror table is the same as the timestamp of the real-time database. If the timestamp of the local mirror table is determined to be different from the timestamp of the real-time database, the server synchronizes the data of the real-time database to the local mirror table based on the data synchronization request.

[0098] In one embodiment, the step of generating a data synchronization request by the client proxy and sending the data synchronization request to the server includes: The client agent determines whether the synchronization time interval of the local mirror table exceeds a preset synchronization time threshold. If the synchronization time interval of the local mirror table exceeds a preset synchronization time threshold, the client agent generates a data synchronization request and sends the data synchronization request to the server.

[0099] In one embodiment, after obtaining the query results, the method further includes: The client agent updates the access time of the local mirror table to the current system time.

[0100] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0101] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0102] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0103] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0104] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0105] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0106] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0107] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0108] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A client-side data query method, characterized in that, The client deploys a client proxy, which is configured with a local mirror table and a data access interface. The local mirror table is obtained by synchronizing a real-time database from the server, and the data access interface is an external access interface for the local mirror table built based on its index. The method includes: Obtain a data query request, wherein the data query request includes a data identifier; Based on the data query request, a query is performed in the local mirror table through the data access interface to obtain the query result; The real-time library is obtained by the server through real-time acquisition of collected data, which includes data from at least one data source. Based on the collected data and a preset required data specified address, a data specified memory address is determined. The preset required data specified address is determined based on the physical address of the memory table in the memory table group corresponding to the required data. One memory table group is used to store data from one data source. Based on the data specified memory address, the collected data is written into the corresponding memory table.

2. The client data query method according to claim 1, characterized in that, Also includes: The client proxy generates a data synchronization request and sends the data synchronization request to the server. The server synchronizes the data from the real-time database to the local mirror table based on the data synchronization request.

3. The client data query method according to claim 2, characterized in that, The step of generating a data synchronization request by the client proxy and sending the data synchronization request to the server includes: The frequently accessed data table is determined by the client agent in the local mirror table; The client proxy determines whether to submit a query request to the server based on the access time of the frequently accessed data table; If it is determined that a query request will be submitted to the server, the client proxy will generate a data synchronization request and send the data synchronization request to the server.

4. The client data query method according to claim 3, characterized in that, The step of the client proxy determining whether to submit a query request to the server based on the access time of the frequently accessed data table includes: The client agent determines whether the difference between the access time of the frequently accessed data table and the current system time meets a preset time threshold. If the difference between the access time of the frequently accessed data table and the current system time meets a preset time threshold, the client agent determines to submit a query request to the server.

5. The client data query method according to claim 2, characterized in that, The data synchronization request includes the timestamp of the local mirror table; The step of the server synchronizing data from the real-time database to the local mirror table based on the data synchronization request includes: The server responds to the data synchronization request by determining whether the timestamp of the local mirror table is the same as the timestamp of the real-time database. If the timestamp of the local mirror table is determined to be different from the timestamp of the real-time database, the server synchronizes the data of the real-time database to the local mirror table based on the data synchronization request.

6. The client data query method according to claim 2, characterized in that, The step of generating a data synchronization request by the client proxy and sending the data synchronization request to the server includes: The client agent determines whether the synchronization time interval of the local mirror table exceeds a preset synchronization time threshold. If the synchronization time interval of the local mirror table exceeds a preset synchronization time threshold, the client agent generates a data synchronization request and sends the data synchronization request to the server.

7. The client data query method according to claim 1, characterized in that, After obtaining the query results, it also includes: The client agent updates the access time of the local mirror table to the current system time.

8. A client-side data query device, characterized in that, The client deploys a client proxy, which is configured with a local mirror table and a data access interface. The local mirror table is obtained by synchronizing a real-time database from the server, and the data access interface is an external access interface for the local mirror table built based on its index. The device includes: The acquisition module is used to acquire data query requests, wherein the data query requests include data identifiers; The query module is used to query the local mirror table through the data access interface based on the data query request to obtain the query result. The real-time database is obtained by the server through real-time acquisition of collected data, which includes data from at least one collection source. A designated memory address for the data is determined based on the collected data and a preset address for the required collected data. The preset address for the required collected data is determined based on the physical address of the memory table in the memory table group corresponding to the required collected data. One memory table group is used to store data from one collection source. The collected data is written to the corresponding memory table based on the designated memory address.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; A memory connected to the at least one processor; The memory stores instructions executable by the at least one processor, which implements the client data query method according to any one of claims 1 to 7 by executing the instructions stored in the memory.

10. A machine-readable storage medium storing instructions thereon, characterized in that, When executed by a processor, this instruction causes the processor to be configured to perform the client data query method according to any one of claims 1 to 7.

Images