Data dynamic caching method and system

Abstract

The invention provides a data dynamic caching method and system. The method comprises the following steps: determining a current corresponding target data source of data display content in a large visual screen; under the condition that it is determined that data change happens to the target data source, target change data in the current target data source is obtained according to data change state information corresponding to the target data source and a target data change strategy set bound with the target data source, and the target data change strategy set at least comprises a data change strategy; the data change state information is determined based on a change condition of data subjected to data change in the target data source, and the target change data is data subjected to data change in the target data source and the data change state information meets a data change strategy; and performing data caching processing on the target change data to refresh the data display content in the visual large screen. According to the method and the device, the efficiency and the flexibility of visualized large-screen data refreshing are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data dynamic caching method and system. Background Art

[0002] With the continuous advancement of information technology, visualization large-screen technology has penetrated into all walks of life and become an important tool for real-time display of key data and business indicators.

[0003] In daily operations, the visualization screen system needs to process a large amount of data. In order to ensure the timeliness and accuracy of the data, the server usually regularly combines multiple data sources to perform data precipitation operations. Among them, data precipitation operations are a way of realizing space for time in the development field. By systematizing and structuring a large amount of source data, a new data model is formed, which can improve the efficiency of subsequent data analysis. Subsequently, the visualization screen will periodically call the back-end interface to refresh the displayed data content to ensure real-time update of information.

[0004] However, although the related technologies can ensure the smooth refresh of the visualization screen data, the periodic configuration of data accumulation is often fixed, which means that the system cannot dynamically adjust the period of data accumulation according to business changes, which may limit the flexibility and response speed of the system to a certain extent. Therefore, a data dynamic caching method and system are urgently needed to solve the above problems. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a data dynamic caching method and system.

[0006] The present invention provides a data dynamic caching method, comprising: Determine the target data source corresponding to the data displayed on the visualization screen; In the case where it is determined that data change occurs in the target data source, the target change data in the current target data source is obtained according to the data change status information corresponding to the target data source and the target data change policy set bound to the target data source, wherein the target data change policy set includes at least a data change policy, the data change status information is determined based on the change status of the data that has undergone data change in the target data source, and the target change data is the data that has undergone data change in the target data source and the data change status information satisfies the data change policy; The target change data is subjected to data caching processing to refresh the data display content in the visualization large screen.

[0007] According to a data dynamic caching method provided by the present invention, the target data change strategy set also includes a time change strategy, and the time change strategy includes a fixed time change strategy and a fixed period change strategy, wherein the fixed time change strategy is used to trigger a corresponding strategy operation when a preset time is reached, and the fixed period change strategy is used to trigger a corresponding strategy operation at the beginning of each preset period; The target data change strategy set includes at least two of the data change strategies and one of the fixed time change strategy or the fixed period change strategy.

[0008] According to a data dynamic caching method provided by the present invention, determining the target data source corresponding to the data display content in the visualization large screen currently includes: According to the data table information or field information, obtain the target data table or target field corresponding to the data display content in the database; The target data table or the target field is used as the target data source.

[0009] According to a data dynamic caching method provided by the present invention, the steps of determining whether data change occurs in the target data source are as follows: Get binary log information in real time; If it is determined according to the binary log information that the target data source currently has data changes, each data currently having data changes is converted into a corresponding target message body; Based on the change time information of the message body, each of the target message bodies is added to the message cache queue in sequence; Based on a preset data model format, format conversion is performed on the data corresponding to each target message body in the message cache queue to obtain pending change data of the current data change; If a comparison result between the pending change data of the current data change and the pending change data before the data change corresponding to the pending change data of the current data change in the target data source meets a preset data change condition, it is determined that the target data source has a data change; When it is determined that the target data source has a data change, obtaining the target change data in the current target data source according to the data change status information corresponding to the target data source and the target data change policy set bound to the target data source, includes: Determine, from the target data change policy set, a target data change policy corresponding to the pending change data currently undergoing data change, according to a policy configuration relationship between the pending change data currently undergoing data change and the data change policy in the target data change policy set; Acquire the data change status information corresponding to the pending change data that is currently undergoing data change, wherein the data change status information at least includes data change quantity information, data change type information, and data change duration of the pending change data that is currently undergoing data change; If the data change quantity information, the data change type information and the data change duration of the pending change data currently undergoing data change meet the target data change strategy, the pending change data currently undergoing data change is determined as the target change data.

[0010] According to a data dynamic caching method provided by the present invention, if it is determined according to the binary log information that the target data source is currently experiencing data changes, each data currently experiencing data changes is converted into a corresponding target message body, including: Determine the data change type information, source table information, changed data information and change time information of each data that has changed; The target message body is constructed according to the data change type information, the belonging source table information, the change data information and the change time information.

[0011] According to a data dynamic caching method provided by the present invention, before determining the target data change policy corresponding to the pending change data currently undergoing data change from the target data change policy set according to the policy configuration relationship between the pending change data currently undergoing data change and the data change policy in the target data change policy set, the method further comprises: Obtain multiple preset data change strategy sets consisting of different data change strategies; Based on the effective time of each of the preset data change policy sets, construct a time axis corresponding to all the preset data change policy sets; The target data change strategy set is determined from the plurality of preset data change strategy sets according to the position of the current corresponding time period on the time axis.

[0012] According to a data dynamic caching method provided by the present invention, the target change data is subjected to data caching processing for refreshing the data display content in the visualization large screen, comprising: After acquiring the target change data, generating target data refresh request information; Determine whether the target data refresh request information is the data refresh request information generated for the first time in the current cycle; if it is the data refresh request information generated for the first time in the current cycle, perform data caching processing on the target change data to obtain dynamic cache data; Refreshing the data display content in the visualization large screen based on the dynamic cache data; If it is not the data refresh request information generated for the first time in the current cycle, the target data refresh request information is deleted.

[0013] According to a data dynamic caching method provided by the present invention, the method further includes: When it is determined that the visualization large screen is in the initialization startup stage, an initial service data query is performed according to the data display content to obtain initial query data; Based on the initial query data, a data precipitation operation is performed to obtain initial cache data; According to the initial cache data, the data display content in the visualization large screen in the initialization startup phase is refreshed.

[0014] The present invention also provides a data dynamic cache system, comprising: A data source determination module is used to determine the target data source corresponding to the data display content in the visualization screen; a policy execution module, for obtaining target changed data in the current target data source according to data change status information corresponding to the target data source and a target data change policy set bound to the target data source, when it is determined that data change occurs in the target data source, wherein the target data change policy set includes at least a data change policy, the data change status information is determined based on a change condition of data in which data change occurs in the target data source, and the target changed data is data in which data change occurs in the target data source and the data change status information satisfies the data change policy; The cache control module is used to cache the target change data and refresh the data display content in the visualization large screen.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above-mentioned dynamic data caching methods is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for dynamically caching data as described above is implemented.

[0017] The data dynamic caching method and system provided by the present invention, when determining that the data display content in the visualization large screen has a data change in the corresponding target data source within the current cycle, obtains the change data of the target data source and the data change status satisfies the data change policy corresponding to the target data change policy set according to the change status information of the target data source and the target data change policy set, and then constructs dynamic cache data for refreshing the data display content in the visualization large screen according to these change data, thereby improving the efficiency and flexibility of refreshing the data of the visualization large screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic diagram of the flow of the data dynamic caching method provided by the present invention; Figure 2 A schematic diagram of a data dynamic cache architecture for a large-screen visualization provided by the present invention; Figure 3 An internal logic schematic diagram of a data receiver provided by the present invention; Figure 4 A schematic diagram of the structure of the data dynamic caching system provided by the present invention; Figure 5 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] With the rapid development of information technology, visualization screens have been widely used in all walks of life to display key data and business indicators in real time. Whether in the fields of financial monitoring, public safety, traffic management or corporate operations, visualization screens have greatly improved decision-making efficiency and business insight with their intuitive and dynamic data display capabilities.

[0022] In general, the visualization screen system needs to process a huge data base, and the service often combines data sources regularly to perform data precipitation operations, and then the visualization screen system periodically calls the backend interface to refresh the data. This solution can ensure smooth refresh of the screen data, but because the cycle configuration of data precipitation is often fixed, it cannot be dynamically adjusted as the business changes.

[0023] In response to the problems existing in the above-mentioned related technologies, the present invention provides a data dynamic caching method for a large visual screen, the core purpose of which is to enhance the flexibility and response efficiency of data refresh of the large visual screen. The method first sets up a set of detailed data refresh strategy rules. When the information of the underlying data source changes, these changes will be captured and pushed to the server in real time. Subsequently, the system will intelligently evaluate the degree of change of the data according to the predefined refresh strategy. Once the amplitude of the data change reaches the preset refresh threshold, the system will automatically activate the data precipitation process and update the data in the cache. After the update is completed, the latest data will be immediately pushed to the large screen interface to ensure the real-time and accuracy of the displayed content.

[0024] Figure 1 A schematic diagram of the flow chart of the data dynamic caching method provided by the present invention, such as Figure 1 As shown, the present invention provides a data dynamic caching method, comprising: Step 101: determine the target data source corresponding to the data display content in the visualization screen.

[0025] In the present invention, the visualization screen is usually linked to multiple databases, and these data sources store rich business data. When determining the target data source, the most relevant and accurate data is screened out from these data sources based on the specific content displayed on the screen. For example, if the screen is used to display sales data, the relevant data table or field content in the sales database containing sales orders, customer information, product details and other data may be selected as the target data source. In one embodiment, if a visualization screen is built to monitor the operating status of a data center, the CPU usage of each server needs to be displayed. In this case, the "server status table" in the database can be selected as the data source, and the fields such as "server ID" and "CPU usage" in the table can be specified as the data to be extracted. Then, the system will establish corresponding data extraction and mapping rules to display the extracted CPU usage data in real time at the corresponding position on the screen.

[0026] Furthermore, when determining the data content to be displayed in the current cycle of the visualization screen, it is necessary to select the target data source based on this cycle, that is, it is necessary to filter out the most relevant and accurate data from multiple data sources. For example, if the visualization screen is currently used to display monthly sales data, select the sales database containing data such as sales orders, customer information, and product details as the target data source. In the next stage, if the display content type corresponding to the data display content changes (for example, from displaying monthly sales data to displaying warehouse inventory data), it is necessary to re-determine the data source for the next cycle; if the display content type corresponding to the data display content has not changed, continue to perform subsequent data dynamic caching steps based on the currently determined target data source.

[0027] In one embodiment, the data retrieved from the data source may be cleaned and converted (such as removing duplicate data, converting data formats, etc.), and the data may be aggregated or calculated according to business needs to form data in a standard format for subsequent data comparison and analysis.

[0028] Step 102, when it is determined that data changes occur in the target data source, obtain target change data in the current target data source according to the data change status information corresponding to the target data source and the target data change policy set bound to the target data source, wherein the target data change policy set includes at least a data change policy, the data change status information is determined based on the change status of the data that has undergone data changes in the target data source, and the target change data is the data that has undergone data changes in the target data source and the data change status information satisfies the data change policy.

[0029] In the present invention, it is necessary to continuously monitor the changes in the target data source, which can be achieved through technical means such as database triggers, polling queries or real-time data streams. When the data in the target data source is detected to have changed, the system will record the detailed information of these changes, including the time of the change, the type of change (such as addition, modification, deletion) and the specific content of the change. It should be noted that in the present invention, for the judgment process of data changes in the target data source, it is necessary to first analyze the comparison results of the current and previous data changes in the target data source. Only when it is determined that the data change reaches a certain degree, it is determined that the data change has occurred in the target data source. That is, the present invention analyzes the degree of data change, so that the visualization screen will not be frequently refreshed due to some minor data changes every time the data is refreshed, thereby ensuring that the visualization screen can ensure the stability of the visualization screen while ensuring the real-time refresh of the displayed content.

[0030] Furthermore, the system will organize the monitored change information (such as the data table information and change type information corresponding to the changed data, etc.) into change status information (such as the number of data tables with changed data, the change type of the changed data is the newly added data type, etc.). These change status information records the changes of each data item in detail, and also corresponds to a strategy in the data change strategy set. When the change status information of the relevant data meets a corresponding strategy, the strategy is triggered, and then these relevant data are used as target change data for subsequent dynamic cache construction.

[0031] In the present invention, a set of target data change policy sets needs to be preset. This set of policy sets defines the conditions or rules that need to be met when data changes, for example, change type filtering: only focus on newly added or modified data, and ignore deleted data; data range restriction: only consider data changes within a specific time period or a specific business scope; change frequency control: limit the update frequency of data changes to avoid too frequent data updates that lead to unstable large-screen display. In the present invention, the data change policy set is bound to the corresponding data source (or data set). When it is determined that the data in the data source has changed and its change status meets the policy of the data change policy set, the policy can be triggered to determine the target change data.

[0032] When acquiring target change data, the present invention further processes and formats the data to ensure that they meet the requirements of large-screen display. This may include operations such as data aggregation, conversion, and sorting. Finally, the processed target change data will be pushed to the visualization large screen for users to view and analyze in real time.

[0033] Step 103: caching the target change data to refresh the data display content in the visualization large screen.

[0034] In the visualization screen system, data is constantly changing, and these changes need to be reflected in the visualization screen in real time or at a scheduled time. When the system receives target change data, these data usually contain updates or additions to certain key information. These target change data may come from multiple data sources, such as databases, API interfaces, user input, etc.

[0035] Furthermore, after receiving these target change data, pre-processing work such as data analysis and format conversion can be performed to ensure the consistency and accuracy of the data. Then, the received data is verified to ensure that it conforms to the expected format and range to avoid errors and anomalies in subsequent processing.

[0036] Furthermore, the verified target change data is integrated into the existing cache. Before refreshing the large screen, a snapshot set can be generated. If a problem occurs during the refresh process, it can be quickly restored to the previous state to avoid data loss or display abnormality. In the present invention, according to the data mapping relationship, the data in the dynamic cache is rendered to the corresponding position on the visualization large screen, involving multiple operations such as chart update, text replacement, color change, etc.

[0037] The data dynamic caching method provided by the present invention, when determining that the data display content in the visualization large screen has a data change in the corresponding target data source within the current cycle, obtains the change data of the target data source and the data change status satisfies the data change policy corresponding to the target data change policy set according to the change status information of the target data source and the target data change policy set, and then constructs dynamic cache data for refreshing the data display content in the visualization large screen according to the change data, thereby improving the efficiency and flexibility of refreshing the data of the visualization large screen.

[0038] Based on the above embodiment, the target data change strategy set also includes a time change strategy, and the time change strategy includes a fixed time change strategy and a fixed period change strategy, wherein the fixed time change strategy is used to trigger the corresponding policy operation when the preset time is reached, and the fixed period change strategy is used to trigger the corresponding policy operation at the beginning of each preset period.

[0039] Based on the above embodiment, the target data change strategy set includes at least two of the data change strategies and one of the fixed time change strategy or the fixed period change strategy.

[0040] In the present invention, the target data change strategy set is a comprehensive framework that aims to manage and execute change operations for different data objects. In this framework, the time change strategy is triggered based on specific conditions of time, which can be further divided into two categories: fixed time change strategy and fixed period change strategy, each of which has its own application scenario and trigger mechanism.

[0041] Specifically, the fixed time change strategy is a data change mechanism based on an absolute time point. The strategy pre-sets one or more specific time points. When the system time reaches these preset time points, the strategy operation associated with the time point will be automatically triggered. This strategy is particularly suitable for scenarios where data processing or updating is required at a specific time, for example, setting it to be automatically executed at 12:00 midnight every day. The advantage of the fixed time change strategy is its accuracy and predictability, which can ensure that data change operations are executed accurately and without error at the expected time.

[0042] Different from the fixed time change strategy, the fixed period change strategy is based on the time cycle cycle. The strategy first sets an initial time point and a time cycle (such as daily, weekly, monthly, etc.), and then triggers the strategy operation every preset cycle starting from this initial time point. This strategy is suitable for data change tasks that need to be performed regularly and repeatedly, for example, synchronizing the latest customer data from the external system at 3 pm every day. The fixed period change strategy ensures data consistency and system maintenance efficiency by regularly performing data change operations.

[0043] The target data change strategy set is a collection of data change strategies and time change strategies, which includes at least 2 data change strategies (these strategies can be triggered based on data content, data status or other non-time conditions, including 1 comparison strategy for judging data change status and 1 data strategy for judging target change data), and at least one time change strategy (which can be a fixed time change strategy or a fixed period change strategy), so that the visualization large screen system can flexibly combine and apply different strategies according to complex and changing business needs to achieve efficient and accurate data management.

[0044] Based on the above embodiment, the step of determining the target data source corresponding to the data display content in the visualization screen includes: According to the data table information or field information, obtain the target data table or target field corresponding to the data display content in the database; The target data table or the target field is used as the target data source.

[0045] In data visualization or data processing strategies, data is usually stored in relational databases (such as MySQL, PostgreSQL) or non-relational databases (such as MongoDB, Cassandra), and these data are distributed in different data tables and fields.

[0046] A data table is the basic unit for storing data in a database. It usually contains multiple rows and columns of data. Each row represents a record and each column represents a field. Based on the data table information, you can locate the specific table that contains the required data. A field is a column in a data table that is used to store a specific type of data (such as integers, strings, dates, etc.). Based on the field information, you can further pinpoint a specific field in the data table that contains the required data.

[0047] In the present invention, it is necessary to parse the data table information or field information in the configuration file corresponding to the data display content. This information may be provided in the form of a string, such as "equipment alarm table" and "storage capacity table-capacity field". The parsed information is used to perform a query in the database. For data table information, the entire table can be directly queried; for field information, the table name and field name need to be specified to query the data of a specific field. According to the query results, the target data table or target field is determined.

[0048] After the target data table or target field is determined, it can be regarded as the target data source corresponding to the data display content in the visualization screen. In the present invention, in the process of configuring the data source, it is usually involved to specify the database connection information (such as database type, address, port, user name, password, etc.), the detailed information of the target data table or field (such as table name, field name, data type, etc.). After the configuration is completed, data can be extracted from the target data source. For example, suppose there is a data display content of a visualization screen, which is used to display the alarm information and storage capacity of the equipment. Among them, the data source corresponding to the alarm information is the "equipment alarm table"; the data source corresponding to the storage capacity is the "storage capacity table-capacity field". The system parses this information, queries the "capacity" field of the "storage capacity table" in the database, and uses this field as the target data source. By directly locating the target data table or field, the time of blind searching in the database is reduced, and the efficiency of data processing is improved.

[0049] Based on the above embodiment, the steps of determining whether the target data source has changed are as follows: Get binary log information in real time; If it is determined according to the binary log information that the target data source currently has data changes, each data currently having data changes is converted into a corresponding target message body; Based on the change time information of the message body, each of the target message bodies is added to the message cache queue in sequence; Based on a preset data model format, format conversion is performed on the data corresponding to each target message body in the message cache queue to obtain pending change data of the current data change; If a comparison result between the pending change data of the current data change and the pending change data before the data change corresponding to the pending change data of the current data change in the target data source meets a preset data change condition, it is determined that the target data source has a data change; When it is determined that the target data source has a data change, obtaining the target change data in the current target data source according to the data change status information corresponding to the target data source and the target data change policy set bound to the target data source, includes: Determine, from the target data change policy set, a target data change policy corresponding to the pending change data currently undergoing data change, according to a policy configuration relationship between the pending change data currently undergoing data change and the data change policy in the target data change policy set; Acquire the data change status information corresponding to the pending change data that is currently undergoing data change, wherein the data change status information at least includes data change quantity information, data change type information, and data change duration of the pending change data that is currently undergoing data change; If the data change quantity information, the data change type information and the data change duration of the pending change data currently undergoing data change meet the target data change strategy, the pending change data currently undergoing data change is determined as the target change data.

[0050] Figure 2 The schematic diagram of the data dynamic cache architecture for the large-screen visualization provided by the present invention can be referred to Figure 2 As shown, in the present invention, based on the data refresh requirements of the visualization large screen, a corresponding data dynamic cache architecture is constructed, including a configuration center, a data receiver, a policy library manager, a policy set manager, a policy listener, a data set manager and a large screen docking service, wherein: The configuration center is used to store relevant configuration information, such as policy configuration, policy set configuration, and data source information. The configuration of policies can meet a variety of business scenarios. The data receiver is connected to the data source and is responsible for receiving the data change information pushed by the data source. Then, according to the subscription status of the policy, it pushes the changed data information to the policy library manager. The policy library manager is used to maintain the information of each policy and receive the change information of policy subscription; The policy set manager is used to maintain policy set information and receive change information of policy set subscriptions; The policy listener is used to receive data type policy change information and check whether the policy set meets the triggering conditions; The dataset manager is used to maintain dataset information, receive change information of policy sets, and refresh data cache.

[0051] The large-screen docking service is connected to the front-end large-screen UI component to provide data set query and push services.

[0052] Figure 3 The internal logic diagram of the data receiver provided by the present invention can be referred to Figure 3 As shown in the figure, the data receiver includes three core components, namely binary log listener, message cache queue and message processor, among which: The listener is used to listen to the binary log (binlog), which records all changes in the database (such as data insertion, update and deletion operations), and these records are stored in binary format, which is essential for tasks such as data recovery, replication and auditing. After analyzing the binary log information, confirming that the data display content of the task configuration corresponds to the relevant source table data, the listener converts the changed data row into the target message body and places it in the message cache queue.

[0053] On the basis of the above embodiment, if it is determined according to the binary log information that the target data source is currently experiencing data changes, converting each data currently experiencing data changes into a corresponding target message body includes: Determine the data change type information, source table information, changed data information and change time information of each data that has changed; The target message body is constructed according to the data change type information, the belonging source table information, the change data information and the change time information.

[0054] In the present invention, a complete message body format is as follows: Data change type information: ADD, UPDATE, DELETE; Source table information: the source table information to which the message belongs; Change data information: original information of the changed data row; Change time information: the time when the change occurred.

[0055] Furthermore, the message bodies read in the listener are sorted according to the change time, and placed in the data queue for processing. Then, the original data model corresponding to the message body is converted into a standard data model (i.e., the preset data model format) through the message processor. And according to the subscription mechanism of the data table, the relevant information is pushed to the policy set manager.

[0056] In the present invention, the strategies in the data change strategy set are divided into two types: time change strategy and data change strategy. The time change strategy includes two modes: fixed period and fixed time. The fixed period means that after the current strategy is triggered, the next strategy is triggered after an interval of n; the fixed time means that the strategy is triggered at a specific time point.

[0057] The data change strategy is a model constructed by the present invention based on the data change status of the data source. Among them, the data change strategy consists of three parts: the data source, the change type, and the comparison method. Among them, the data source can select a data table or a specific field in the database as the data source. For example, the device alarm table, the storage capacity table - capacity field, etc. can all be used as the data source of the strategy. The change type is the same as the message change type sent by the database in the message processor, and is divided into four types: all (i.e., covering all change types), new, modified, and deleted.

[0058] The comparison method is essentially a judgment logic, similar to the nature of ">" in "a>b" and "contains" in "a contain b". The present invention judges whether the strategy is triggered through the comparison method and the Boolean value returned by the comparison value, that is, judges whether the degree of data change of the data source has reached the standard of data change.

[0059] Specifically, the comparison method adopted by the present invention changes dynamically according to the selected data source and change type, and compares the change situation between the corresponding change data of the display content of the same data before and after the change. For example, if a field of the numerical field type and a change type of the modified type are selected, the comparison method can select the change rate ≥ x%; if a field of the string type is selected, then contains / equals / does not equal x can be selected; if a data table is selected and the change of the deleted type, the data volume change rate > x% or < x% can be selected. When the comparison result meets the above comparison method, the strategy can be triggered.

[0060] In the present invention, for the time change strategy, after it is created, the corresponding internal thread is directly started by the strategy listener for monitoring.

[0061] Furthermore, considering that a data change strategy may not meet the actual judgment needs in actual business, the concept of a strategy group, that is, a strategy set, is designed. A strategy group is a collection of a batch of strategies. When any strategy in the strategy group is triggered, the strategy can be triggered. The flexible combination of the time change strategy and the data change strategy in the present invention can effectively solve the data refresh needs in multiple scenarios.

[0062] Specifically, in the present invention, there is a corresponding configuration relationship between the pending change data and the data change policies in the target data change policy set, and this configuration relationship determines which policies will be applied to which data. Furthermore, for the current pending change data, its data change status information is obtained, and the data change status information at least includes: data change quantity information (such as how many records have been changed), data change type information (such as addition, modification, deletion, etc.) and data change duration (such as the time period in which the change occurred). Then compare the obtained data change status information with the target data change policy to check whether the data change quantity, type and duration meet the conditions set in the corresponding policy. If the change status information of the pending change data meets the conditions of the target data change policy, then these data are determined as target change data, and at the same time, these data will be further processed or refreshed according to the requirements of the policy.

[0063] For example, a policy expects to refresh data when the number of alarms in the data display content changes to more than 10, and refresh at least every 5 minutes. In this case, you need to create a data policy a, bind the alarm table, all types of changes, and the number of alarms ≥ 10, and create a periodic policy b for 5 minutes. Finally, add data policy a and data policy b as a whole to the policy set.

[0064] In the present invention, after the data change strategy is created, the subscription mechanism will be enabled, and the data table and change type in the data source will be used to initiate a subscription to the data receiver. When the corresponding data changes, the message is pushed to the policy set manager, and the policy listener can first perform data change judgment according to the comparison method in the policy set. If the comparison method returns a true result, it means that the policy is triggered. When the corresponding change status information meets this policy, the data set will trigger the corresponding interface and refresh the cached data.

[0065] The data dynamic caching method for a large-scale visualization screen provided by the present invention adds a data change strategy on the basis of fixed-period refresh. The essence of the strategy is the change of the data source table, and the strategy can be triggered when the degree of change reaches a certain set threshold. At the same time, multiple data change strategies and time change strategies are combined together to form a strategy set, and then the strategy set is bound to the data set, and finally, on the basis of realizing dynamic and flexible refresh of the large-screen data set, resource waste is reduced.

[0066] On the basis of the above embodiment, before determining the target data change policy corresponding to the pending change data currently undergoing data change from the target data change policy set according to the policy configuration relationship between the pending change data currently undergoing data change and the data change policy in the target data change policy set, the method further includes: Obtain multiple preset data change strategy sets consisting of different data change strategies; Based on the effective time of each of the preset data change policy sets, construct a time axis corresponding to all the preset data change policy sets; The target data change strategy set is determined from the plurality of preset data change strategy sets according to the position of the current corresponding time period on the time axis.

[0067] In the present invention, multiple data change policy sets are predefined and stored, each of which is composed of different data change policies, which involve data screening conditions, change operations, trigger conditions, etc. Further, a timeline is constructed according to the effective time of each preset data change policy set. The timeline is a linear representation for showing the effectiveness of each policy set in different time periods. Each policy set has one or more effective intervals on the timeline, which indicate the time periods in which the policy set is effective.

[0068] When a data change operation needs to be performed, first determine the current corresponding time period. Then, find out which preset data change policy sets are valid within this time period on the timeline. Finally, select one from these valid policy sets as the target data change policy set to guide the current data change operation. The present invention switches the policy group according to time to achieve adaptation to more scenarios. After the data set is refreshed, it is actively pushed to the large screen to reduce invalid interface calls on the large screen.

[0069] On the basis of the above embodiment, the target change data is cached to refresh the data display content in the visualization screen, including: After acquiring the target change data, generating target data refresh request information; Determine whether the target data refresh request information is the data refresh request information generated for the first time in the current cycle; if it is the data refresh request information generated for the first time in the current cycle, perform data caching processing on the target change data to obtain dynamic cache data; Refreshing the data display content in the visualization large screen based on the dynamic cache data; If it is not the data refresh request information generated for the first time in the current cycle, the target data refresh request information is deleted.

[0070] In the present invention, the data set is positioned consistently with the data set of the visualization screen, one or more data blocks of the visualization screen are bound to the north, and the corresponding interface processes the source data to form a new data model (i.e., the changed data in the preset data model format) in the south. On this basis, cache information is established inside the data set, and the data actively queried by the visualization screen can be directly obtained by querying this cache data. At the same time, a data set can be bound to multiple policy sets, and the effective time of the policy set can be set according to the timeline. When any policy set in an effective state is triggered, the data set will trigger the interface and refresh the cached data. Therefore, in the present invention, if there is no effective policy group within a certain timeline, it means that the data set will not be refreshed during this period of time.

[0071] Furthermore, if a single data set has triggered a data set refresh operation at the same time, if a refresh request appears again during the current refresh period, this request will be discarded. Specifically, when the target changed data (i.e., the data that needs to be updated to the cache) is detected, a target data refresh request message is first generated. This request message contains all the necessary information required for the refresh, such as the identifier of the data set, detailed information on the changed data, the time when the request was generated, etc. After the target data refresh request message is generated, check whether this request is the first data refresh request message generated in the current cycle. This can be achieved by comparing the time when the request was generated with the start time of the current cycle. If the request is generated for the first time after the start of the current cycle, then it is the first request; otherwise, it is not the first request.

[0072] If the result of the judgment is that the data refresh request information is generated for the first time in the current cycle, the target change data is cached, which includes extracting the required information from the target change data and storing it in the cache for subsequent quick access. In the process of caching data, a series of data processing can be performed, such as data verification, format conversion, etc., to ensure the accuracy and consistency of the cached data.

[0073] If the result is not the first data refresh request information generated in the current cycle, the request information is deleted. This is because only one data refresh operation needs to be processed at the same time. If the same request information is received again later, it will be regarded as redundant and discarded by the system. In this way, excessive refresh operations can be avoided from having a negative impact on performance and the timeliness and accuracy of cached data can be ensured.

[0074] Finally, when the dataset cache data is refreshed, the push mechanism will be triggered, and the latest dataset information will be actively pushed to the visualization screen through the docking service. At this time, the visualization screen will trigger the data refresh of the corresponding component.

[0075] Based on the above embodiment, the method further includes: When it is determined that the visualization large screen is in the initialization startup stage, an initial service data query is performed according to the data display content to obtain initial query data; Based on the initial query data, a data precipitation operation is performed to obtain initial cache data; According to the initial cache data, the data display content in the visualization large screen in the initialization startup phase is refreshed.

[0076] In the present invention, if this is the first time that the visualization large screen page is displayed, it is necessary to actively call the service query data. Since there is no cached data in the data set at this time, the corresponding data set will trigger a data precipitation, fill the cached data, and the large screen data will start to display.

[0077] Furthermore, after the initialization startup phase, the visualization screen will no longer actively initiate service requests except for manually refreshing the page. The backend service will then actively push data set change information, thereby effectively reducing invalid data calls.

[0078] In the present invention, the policy manager reads the configuration information, initializes the relevant policies, and sends data change subscription information to the data receiver. When all policies are initialized, the policy set manager initializes the policy group and binds the policy group to the snapshot set according to the configuration rules. After the data receiver is started, it starts to receive change information from the database and pushes the change information to the policy set manager according to the subscription configuration. After the service runs stably, the policy and policy set can be dynamically adjusted according to the running situation, so as to control the refresh frequency and refresh strategy of the snapshot set. In this process, the service can run normally and no coding operation is required.

[0079] The data dynamic caching method of the visualization large screen provided by the present invention is to add a data strategy on the basis of the concept of the visualization large screen periodic strategy. The strategy is based on the change of the underlying data of the database, and determines the triggering time of the strategy by receiving data messages and comparing them with the rules configured by itself. Through the strategy group, multiple strategy combinations and multiple strategy groups are bound to the data set with the time axis. When the strategy is triggered, the data refresh of the snapshot set is triggered, and finally the latest data is fed back to the visualization large screen through the data push mechanism inside the snapshot set.

[0080] The data dynamic caching system provided by the present invention is described below. The data dynamic caching system described below and the data dynamic caching method described above can be referenced to each other.

[0081] Figure 4 The structural diagram of the data dynamic caching system provided by the present invention is as follows: Figure 4As shown, the present invention provides a data dynamic caching system, including a data source determination module 401, a strategy execution module 402 and a cache control module 403, wherein the data source determination module 401 is used to determine the data display content in the visualization large screen in the current corresponding target data source; the strategy execution module 402 is used to, when it is determined that the target data source has a data change, obtain the target change data in the current target data source according to the data change status information corresponding to the target data source and the target data change strategy set bound to the target data source, wherein the target data change strategy set at least includes a data change strategy, the data change status information is determined based on the change status of the data in which the data change has occurred in the target data source, and the target change data is the data in which the data change has occurred in the target data source and the data change status information satisfies the data change strategy; the cache control module 403 is used to cache the target change data and refresh the data display content in the visualization large screen.

[0082] The data dynamic caching system provided by the present invention, when determining that the data display content in the visualization large screen has undergone data changes in the corresponding target data source within the current cycle, obtains change data in the target data source and the data change status satisfies the data change policy corresponding to the target data change policy set according to the change status information of the target data source and the target data change policy set, and then constructs dynamic cache data for refreshing the data display content in the visualization large screen according to these change data, thereby improving the efficiency and flexibility of refreshing the data of the visualization large screen.

[0083] The system provided by the present invention is used to execute the above-mentioned method embodiments. Please refer to the above-mentioned embodiments for the specific processes and detailed contents, which will not be repeated here.

[0084] Figure 5 A schematic diagram of the structure of an electronic device provided by the present invention, such as Figure 5As shown, the electronic device may include: a processor (Processor) 501, a communication interface (Communications Interface) 502, a memory (Memory) 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 complete mutual communication through the communication bus 504. The processor 501 may call the logic instructions in the memory 803 to execute the data dynamic caching method, which includes: determining the data display content in the visualization large screen in the current corresponding target data source; in the case of determining that the target data source has data change, according to the data change state information corresponding to the target data source and the target data change strategy set bound to the target data source, obtaining the target change data in the current target data source, wherein the target data change strategy set at least includes a data change strategy, the data change state information is determined based on the change of the data in the target data source, and the target change data is the data in the target data source where the data change occurs and the data change state information satisfies the data change strategy; the target change data is subjected to data caching processing for refreshing the data display content in the visualization large screen.

[0085] In addition, the logic instructions in the above-mentioned memory 503 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0086] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the data dynamic caching method provided by the above methods, the method including: determining the data display content in the visualization large screen in the current corresponding target data source; when it is determined that the target data source has a data change, obtaining the target change data in the current target data source according to the data change status information corresponding to the target data source and the target data change strategy set bound to the target data source, wherein the target data change strategy set at least includes a data change strategy, the data change status information is determined based on the change status of the data in which the data change occurs in the target data source, and the target change data is the data in which the data change occurs in the target data source and the data change status information satisfies the data change strategy; the target change data is subjected to data caching processing for refreshing the data display content in the visualization large screen.

[0087] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the data dynamic caching method provided by the above-mentioned embodiments, the method comprising: determining the data display content in the visualization large screen in the current corresponding target data source; when it is determined that the target data source has a data change, obtaining the target change data in the current target data source according to the data change status information corresponding to the target data source and the target data change policy set bound to the target data source, wherein the target data change policy set at least includes a data change policy, the data change status information is determined based on the change status of the data in which the data change occurs in the target data source, and the target change data is the data in which the data change occurs in the target data source and the data change status information satisfies the data change policy; the target change data is subjected to data caching processing for refreshing the data display content in the visualization large screen.

[0088] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0089] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data dynamic caching method, characterized in that: include: Determine the target data source corresponding to the data displayed on the visualization screen; In the case where it is determined that data change occurs in the target data source, the target change data in the current target data source is obtained according to the data change status information corresponding to the target data source and the target data change policy set bound to the target data source, wherein the target data change policy set includes at least a data change policy, the data change status information is determined based on the change status of the data that has undergone data change in the target data source, and the target change data is the data that has undergone data change in the target data source and the data change status information satisfies the data change policy; The target change data is subjected to data caching processing to refresh the data display content in the visualization large screen.

2. The data dynamic caching method according to claim 1, characterized in that: The target data change strategy set also includes a time change strategy, and the time change strategy includes a fixed time change strategy and a fixed period change strategy, wherein the fixed time change strategy is used to trigger a corresponding strategy operation when a preset time is reached, and the fixed period change strategy is used to trigger a corresponding strategy operation at the beginning of each preset period; the target data change strategy set includes at least 2 of the data change strategies, and 1 of the fixed time change strategy or the fixed period change strategy.

3. The data dynamic caching method according to any one of claims 1 to 2, characterized in that: Determining the target data source corresponding to the data display content in the visualization screen includes: According to the data table information or field information, obtain the target data table or target field corresponding to the data display content in the database; The target data table or the target field is used as the target data source.

4. The data dynamic caching method according to claim 3, characterized in that: The steps for determining that data changes occur in the target data source are as follows: Get binary log information in real time; If it is determined according to the binary log information that the target data source currently has data changes, each data currently having data changes is converted into a corresponding target message body; Based on the change time information of the message body, each of the target message bodies is added to the message cache queue in sequence; Based on a preset data model format, format conversion is performed on the data corresponding to each target message body in the message cache queue to obtain pending change data of the current data change; If a comparison result between the pending change data of the current data change and the pending change data before the data change corresponding to the pending change data of the current data change in the target data source meets a preset data change condition, it is determined that the target data source has a data change; When it is determined that the target data source has a data change, obtaining the target change data in the current target data source according to the data change status information corresponding to the target data source and the target data change policy set bound to the target data source, includes: Determine, from the target data change policy set, a target data change policy corresponding to the pending change data currently undergoing data change, according to a policy configuration relationship between the pending change data currently undergoing data change and the data change policy in the target data change policy set; Acquire the data change status information corresponding to the pending change data that is currently undergoing data change, wherein the data change status information at least includes data change quantity information, data change type information, and data change duration of the pending change data that is currently undergoing data change; If the data change quantity information, the data change type information and the data change duration of the pending change data currently undergoing data change meet the target data change strategy, the pending change data currently undergoing data change is determined as the target change data.

5. The data dynamic caching method according to claim 4, characterized in that: Before determining the target data change policy corresponding to the pending change data currently undergoing data change from the target data change policy set according to the policy configuration relationship between the pending change data currently undergoing data change and the data change policy in the target data change policy set, the method further includes: Obtain multiple preset data change strategy sets consisting of different data change strategies; Based on the effective time of each of the preset data change policy sets, construct a time axis corresponding to all the preset data change policy sets; The target data change strategy set is determined from the plurality of preset data change strategy sets according to the position of the current corresponding time period on the time axis.

6. The data dynamic caching method according to claim 2, characterized in that: The data caching process of the target change data is used to refresh the data display content in the visualization large screen, including: After acquiring the target change data, generating target data refresh request information; Determine whether the target data refresh request information is the data refresh request information generated for the first time in the current cycle; if it is the data refresh request information generated for the first time in the current cycle, perform data caching processing on the target change data to obtain dynamic cache data; Refreshing the data display content in the visualization large screen based on the dynamic cache data; If it is not the data refresh request information generated for the first time in the current cycle, the target data refresh request information is deleted.

7. The data dynamic caching method according to claim 1, characterized in that: The method further comprises: When it is determined that the visualization large screen is in the initialization startup stage, an initial service data query is performed according to the data display content to obtain initial query data; Based on the initial query data, a data precipitation operation is performed to obtain initial cache data; According to the initial cache data, the data display content in the visualization large screen in the initialization startup phase is refreshed.

8. A data dynamic caching system, characterized in that: include: A data source determination module is used to determine the target data source corresponding to the data display content in the visualization screen; a policy execution module, for obtaining target changed data in the current target data source according to data change status information corresponding to the target data source and a target data change policy set bound to the target data source, when it is determined that data change occurs in the target data source, wherein the target data change policy set includes at least a data change policy, the data change status information is determined based on a change condition of data in which data change occurs in the target data source, and the target changed data is data in which data change occurs in the target data source and the data change status information satisfies the data change policy; The cache control module is used to cache the target change data and refresh the data display content in the visualization large screen.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the data dynamic caching method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data dynamic caching method according to any one of claims 1 to 7 is implemented.

Images