Front-end and rear-end collaborative dynamic downsampling data display method and system

Abstract

The invention relates to the technical field of data processing, and discloses a front-end and rear-end collaborative dynamic downsampling data display method and system, and the method comprises the steps: collecting original sensing data through Internet of Things equipment; performing multi-layer downsampling processing on the original sensing data by using a back-end system, and constructing a multi-resolution data structure; screening the multi-resolution data structure based on a recommended sampling interval by using a back-end system to obtain first resolution data, a second extreme value and a second abnormal value; performing chart rendering by using the front-end system to obtain a first data display result; and when it is detected that the user executes an interaction operation on the first data display result, receiving optimization precision data returned by the back end and covering the first data display result. Through the cooperation of the front-end system and the rear-end system, the problems of difficulty in balancing performance and precision, data asynchronization caused by front-end and rear-end strategy separation, and easy loss of key abnormal points or extreme values in the traditional scheme are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a front-end and back-end cooperative dynamic down-sampling data display method and system. BACKGROUND

[0002] In the Internet of Things application scenario, sensor devices as the core nodes of data collection, usually capture various environmental data at a frequency of seconds or even higher, covering temperature, liquid level, pressure and other key indicators. These data are uploaded to the monitoring platform in real time through the network, and finally used for visual chart display and deep trend analysis, providing an important basis for users to master the device running state and predict potential risks. However, such high-frequency collection characteristics make the data naturally large in quantity and high in density. When full-quantity point information needs to be displayed on the front end, the chart component often faces severe performance challenges. Too many data points will cause the rendering process to be slow, and even cause the interface to freeze, seriously affecting the real-time performance of the monitoring system and user experience. The core contradiction of this performance bottleneck is that the high-density characteristics of the original data and the limited rendering capability of the front end, and the immediate demand of users for data visualization, cannot be reconciled. How to ensure data display efficiency while retaining as much key information as possible has become a problem to be solved in the data visualization link of the Internet of Things real-time monitoring system.

[0003] Currently, three main solutions have been formed to solve the above problems, but all have obvious limitations. The first solution relies on the backend to perform fixed-granularity aggregation processing during data storage, such as storing the average value of raw data every minute, and then directly using the aggregation result for front-end display. This method can reduce the amount of data, but the fixed time granularity cannot be flexibly adapted to different time spans or scaling levels of user operations. When the user needs to view the details of a certain subdivided time period, the aggregation granularity may be too coarse to lose key information, resulting in deviation in trend analysis. The second solution transfers the downsampling task to the front-end, which uses LTTB, VW and other simplified algorithms for local processing after obtaining the raw data. However, processing a large amount of raw data on the front-end will significantly consume the browser's computing resources and memory, not only increasing the page loading delay, but also causing uneven user experience due to device performance differences. The third solution relies on the chart library's own sampling mechanism to reduce the pressure of DOM element rendering by internally selecting some data points for display, but this independent processing within the library lacks global coordination and is difficult to form a linkage with the overall data processing strategy of the front-end and back-end. More importantly, the three existing solutions generally have the problem of fragmented processing strategies between the front-end and back-end. In the multi-chart linkage scenario, different charts may use different sampling logic, leading to asynchronous data display, which is likely to mislead users visually. At the same time, these solutions are difficult to optimize performance while retaining key abnormal points or extreme values in the data, which are often the core basis for determining whether a device is running normally in a monitoring system, and their loss will directly affect the accuracy and reliability of monitoring.

[0004] In summary, the traditional solutions have the defects of difficult balance between performance and accuracy, asynchronous data caused by fragmented front-end and back-end strategies, and easy loss of key abnormal points or extreme values. SUMMARY

[0005] Therefore, the present application provides a front-end and back-end cooperative dynamic downsampling data display method and system to solve the problems of difficult balance between performance and accuracy, asynchronous data caused by fragmented front-end and back-end strategies, and easy loss of key abnormal points or extreme values in the traditional solutions.

[0006] In a first aspect, the present application provides a front-end and back-end collaborative dynamic down-sampling data display method, which comprises: collecting original sensing data by using Internet of Things devices, uploading the original sensing data to a back-end system through a gateway; performing multi-layer down-sampling processing on the original sensing data by using the back-end system, constructing a multi-resolution data structure, analyzing the multi-resolution data structure to obtain a first extreme value and a first abnormal value; collecting visual area information of a chart by using a front-end system, and sending the visual area information to a back-end interface; calculating a recommended sampling interval based on the visual area information by using the back-end system, filtering the multi-resolution data structure based on the recommended sampling interval to obtain first resolution data, wherein the first resolution data is a single set of down-sampling data adapted to the recommended sampling interval; performing down-sampling processing on the first resolution data by using the back-end system to obtain second resolution data; filtering the first extreme value and the first abnormal value based on the visual area information by using the back-end system to obtain a second extreme value and a second abnormal value; integrating key data in the down-sampling processing and the filtering processing by using the back-end system to obtain metadata information, wherein the key data includes data granularity data, sampling algorithm data, sampling error estimation data, and a list of retained key points data; sending the metadata information, the second resolution data, the second extreme value, and the second abnormal value to the front-end system by using the back-end system, performing chart rendering by using the front-end system to obtain a first data display result; when detecting that a user performs an interactive operation on the first data display result, initiating a fine-grained data request for an interactive area by using the front-end system, receiving optimized precision data returned by the back-end, and covering the first data display result to obtain a second display result.

[0007] The front-end and back-end cooperative dynamic down-sampling data display method provided by the embodiment firstly collects original sensing data through the Internet of Things device and uploads the original sensing data to the back-end system through the gateway, thereby providing a source basis for the entire data processing flow and ensuring complete acquisition and transmission of the original data required for monitoring. Secondly, the back-end system performs multi-layer down-sampling processing on the original sensing data and constructs a multi-resolution data structure, and analyzes to obtain a first extreme value and a first abnormal value, thereby performing hierarchical processing on the massive high-frequency data in advance, providing multi-granularity data support for subsequent dynamic sampling, preliminarily retaining key data features, and avoiding omission of key information in subsequent processing. Subsequently, the front-end system collects chart visible region information and sends the chart visible region information to the back-end interface, thereby realizing accurate transmission of front-end view requirements to the back-end, enabling the back-end to perform targeted data processing based on the actual display scenario, and breaking the traditional front-end and back-end processing split state. Thereafter, the back-end system calculates a recommended sampling interval based on the visible region information and selects the first resolution data that is adapted, so that the back-end can quickly locate the data granularity that matches the current view from the pre-constructed multi-resolution data, avoiding problems of data being too dense or too sparse, and preliminarily realizing adaptation of data and view. Further, the back-end system performs down-sampling processing on the first resolution data to obtain second resolution data, further refines the data according to the specific bearing capacity of the front-end view, ensures that the amount of data finally transmitted to the front-end matches the front-end rendering capacity, and effectively reduces the front-end rendering pressure. Then, the back-end system selects a second extreme value and a second abnormal value based on the visible region information, so that the key abnormal points and extreme values under the current view can be retained in the down-sampling process, the problem of easy loss of key information in the traditional technology is solved, and the accuracy of data display is improved. In addition, the back-end system integrates key data in the down-sampling and selection process to obtain metadata information, thereby providing the front-end with a detailed description of data processing, enabling the front-end to more accurately render charts, allowing users to understand the reliability of the data, and enhancing the transparency of data display. Subsequently, the back-end sends the metadata, the second resolution data, and the second extreme value and abnormal value to the front-end and completes rendering to obtain a first data display result, thereby realizing efficient and accurate presentation of the data after cooperative down-sampling processing on the front-end, ensuring rendering performance, and taking into account key information of the data. Finally, the front-end initiates a fine-grained data request and receives optimized precision data to obtain a second display result when the user interacts, so that the system can dynamically respond to user operations and supplement higher-precision data when the user needs to view details, thereby realizing dynamic balance of performance optimization and detail display, and significantly improving the user interaction experience. The present invention solves the problems of difficulty in balancing performance and precision, data asynchronization caused by split front-end and back-end strategies, and easy loss of key abnormal points or extreme values in the traditional scheme.

[0008] In an optional implementation, the original sensor data includes at least one of temperature data, liquid level data, and pressure data, and the backend system stores the original sensor data in at least one of a time series database or a big data platform.

[0009] In an optional implementation, the granularity versions of the multi-layer down-sampling processing include at least two of original data, 10-second granularity data, 1-minute granularity data, and 1-hour granularity data, and the selected down-sampling algorithms for different granularities include at least one of an average value algorithm, an LTTB algorithm, and a MinMax algorithm.

[0010] In an optional implementation, the multi-resolution data structure is organized according to devices, points, and granularities.

[0011] In an optional implementation, the visible area information includes point number information, time range information, current visible area pixel width information, chart maximum tolerable point number information, user zoom level information, user behavior context information, and precision priority information.

[0012] In an optional implementation, the analysis of the multi-resolution data structure to obtain first extreme values includes identifying the multi-resolution data structure using a Z-score algorithm, and determining a first abnormal value when Z-score > 3.

[0013] In an optional implementation, in the metadata information, the data granularity data is a granularity corresponding to second resolution data, the sampling algorithm data is a down-sampling algorithm used to obtain the second resolution data, and the retained key point list data includes related information of second extreme values and second abnormal values.

[0014] In an optional implementation, the interaction operation includes at least one of a zoom operation and a hovering operation.

[0015] In an optional implementation, the method further includes, when the interface of the front-end system includes a multi-chart linkage chart, the front-end system sends a unified time range and a maximum displayable point number to the backend system, the backend system adopts a consistent sampling granularity for multiple charts, and returns all chart data to the front-end system after packaging.

[0016] In a second aspect, the present application provides a front-end and back-end cooperative dynamic down-sampling data display system, which comprises: a data acquisition module configured to collect original sensing data by using an Internet of Things device and upload the original sensing data to a back-end system through a gateway; a multi-layer down-sampling module configured to perform multi-layer down-sampling processing on the original sensing data by using the back-end system, construct a multi-resolution data structure, and analyze the multi-resolution data structure to obtain a first extreme value and a first abnormal value; an acquisition module configured to acquire visual area information of a chart by using a front-end system and send the visual area information to a back-end interface; a calculation module configured to calculate a recommended sampling interval based on the visual area information by using the back-end system, filter the multi-resolution data structure based on the recommended sampling interval, and obtain first resolution data, the first resolution data being a single set of down-sampling data adapted to the recommended sampling interval; a down-sampling module configured to perform down-sampling processing on the first resolution data by using the back-end system to obtain second resolution data; a filtering module configured to filter the first extreme value and the first abnormal value based on the visual area information by using the back-end system to obtain a second extreme value and a second abnormal value; an integration module configured to integrate key data in the down-sampling processing and the filtering processing by using the back-end system to obtain metadata information, the key data including data granularity data, sampling algorithm data, sampling error estimation data, and a list of retained key points; a rendering module configured to send the metadata information, the second resolution data, the second extreme value, and the second abnormal value to the front-end system by using the back-end system, perform chart rendering by using the front-end system, and obtain a first data display result; and an overlay module configured to initiate a fine-grained data request for an interactive area by using the front-end system when detecting that a user performs an interactive operation on the first data display result, receive optimized precision data returned by the back-end system, and overlay the first data display result to obtain a second display result. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0018] Figure 1 is a flowchart of a front-end and back-end cooperative dynamic down-sampling data display method according to an embodiment of the present application;

[0019] Figure 2 is a complete processing flowchart of Internet of Things data from acquisition to front-end display according to an embodiment of the present application;

[0020] Figure 3is a structural block diagram of a front-end and back-end cooperative dynamic down-sampling data display system according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In the Internet of Things application scenario, sensor devices, as the core nodes of data collection, usually continuously capture various types of environmental data at a frequency of seconds or even higher, covering key indicators such as temperature, liquid level, and pressure. These data are uploaded to the monitoring platform in real time through the network and ultimately used for visual chart display and deep trend analysis, providing an important basis for users to master the device running state and predict potential risks. However, this high-frequency collection characteristic naturally makes the data large in quantity and high in density. When full-quantity point information needs to be displayed on the front end, the chart component often faces severe performance challenges. Too many data points can cause a slow rendering process and even cause interface lag, seriously affecting the real-time performance and user experience of the monitoring system. The core contradiction of this performance bottleneck is that the high-density characteristics of the original data and the limited rendering capability of the front end, as well as the immediate demand of users for data visualization, are in conflict. How to ensure data display efficiency while retaining as much key information as possible has become a problem that needs to be solved in the data visualization link of the Internet of Things real-time monitoring system.

[0022] Currently, three main solutions have been formed to solve the above problems, but all have obvious limitations. The first solution relies on the backend to perform fixed-granularity aggregation processing during data storage, such as storing the average value of raw data every minute, and then directly using the aggregated results for front-end display. This method can reduce data volume, but the fixed time granularity cannot be flexibly adapted to different time spans or scaling levels of user operations. When users need to view details of a subdivided time period, key information may be lost due to coarse aggregation granularity, leading to deviation in trend analysis. The second solution transfers the downsampling task to the front-end, which uses LTTB, VW and other simplified algorithms for local processing after obtaining raw data. However, processing a large amount of raw data on the front-end will significantly consume the browser's computing resources and memory, increasing the page loading delay and possibly causing uneven user experience due to device performance differences. The third solution relies on the chart library's own sampling mechanism to reduce DOM element rendering pressure by displaying only a portion of the data points. However, this independent processing within the library lacks global coordination and is difficult to form a linkage with the overall data processing strategy of the front-end and back-end. More importantly, the three existing solutions generally have the problem of fragmented processing strategies between the front-end and back-end. In multi-chart linkage scenarios, different charts may use different sampling logic, leading to asynchronous data display and easily causing visual confusion for users. At the same time, these solutions are difficult to optimize performance while retaining key abnormal points or extreme values in the data. These special data points are often the core basis for determining whether a device is running normally, and their loss will directly affect the accuracy and reliability of the monitoring system.

[0023] In summary, traditional solutions have the defects of difficulty in balancing performance and accuracy, asynchronous data display due to fragmented front-end and back-end strategies, and easy loss of key abnormal points or extreme values.

[0024] The front-end and back-end cooperative dynamic down-sampling data display method provided by the embodiment firstly collects original sensing data through an Internet of Things device and uploads the original sensing data to a back-end system through a gateway, thereby providing a source basis for the entire data processing flow and ensuring complete acquisition and transmission of original data required for monitoring. Secondly, the back-end system performs multi-layer down-sampling processing on the original sensing data and constructs a multi-resolution data structure, and analyzes to obtain a first extreme value and a first abnormal value, thereby performing hierarchical processing on massive high-frequency data in advance, providing multi-granularity data support for subsequent dynamic sampling, preliminarily retaining key data features, and avoiding omission of key information in subsequent processing. Subsequently, the front-end system collects chart visible region information and sends the chart visible region information to a back-end interface, thereby realizing accurate transmission of front-end view requirements to the back-end, enabling the back-end to perform targeted data processing based on an actual display scenario, and breaking the status of traditional front-end and back-end processing fragmentation. Thereafter, the back-end system calculates a recommended sampling interval based on the visible region information and selects first-resolution data that is adapted, thereby enabling the back-end to quickly locate a data granularity that matches the current view from the pre-constructed multi-resolution data, avoiding problems of data being too dense or too sparse, and preliminarily realizing adaptation of data and views. Further, the back-end system performs down-sampling processing on the first-resolution data to obtain second-resolution data, further simplifies the data according to a specific bearing capacity of the front-end view, ensures that the amount of data finally transmitted to the front-end matches the front-end rendering capacity, and effectively reduces front-end rendering pressure. Then, the back-end system selects a second extreme value and a second abnormal value based on the visible region information, thereby enabling the back-end to retain key abnormal points and extreme values under the current view in a targeted manner in the down-sampling process, solving the problem of easy loss of key information in traditional technologies, and improving the accuracy of data display. In addition, the back-end system integrates key data in the down-sampling and selection processes to obtain metadata information, thereby providing the front-end with a detailed description of data processing, enabling the front-end to more accurately render charts, and enabling users to understand the reliability of data, thereby enhancing the transparency of data display. Subsequently, the back-end transmits the metadata, the second-resolution data, and the second extreme value and abnormal value to the front-end and completes rendering to obtain a first data display result, thereby realizing efficient and accurate presentation of data after cooperative down-sampling processing on the front-end, ensuring rendering performance, and taking into account key information of the data. Finally, the front-end initiates a fine-grained data request and receives optimized precision data to obtain a second display result when a user interacts, thereby enabling the system to dynamically respond to user operations and supplement higher-precision data when the user needs to view details, realizing dynamic balance of performance optimization and detail display, and significantly improving user interaction experience. The present invention solves the problems of difficulty in balancing performance and precision, out-of-sync data caused by fragmentation of front-end and back-end strategies, and easy loss of key abnormal points or extreme values in traditional solutions.

[0025] A front-end and back-end cooperative dynamic down-sampling data display method is provided in the embodiment.Figure 1 is a flow chart of a front-end and back-end collaborative dynamic down-sampling data display method according to an embodiment of the present application. As shown in Figure 1 , the flow includes the following steps:

[0026] Step S1, collecting original sensing data by using Internet of Things devices, and uploading the original sensing data to a back-end system through a gateway.

[0027] In some optional manners of some embodiments, the original sensing data includes at least one of temperature data, liquid level data and pressure data, and the back-end system stores the original sensing data in at least one of a time series database or a big data platform.

[0028] Specifically, original sensing data is collected by using Internet of Things devices, which covers at least one of temperature data, liquid level data, pressure data and other common environmental and equipment operation indicators in industrial scenarios. After collection, the original sensing data is stably uploaded to the back-end system by means of the protocol conversion and data forwarding functions of the gateway. In order to ensure efficient storage, query and subsequent analysis of data, the back-end system stores the original sensing data in at least one of a time series database such as InfluxDB, which is suitable for fast writing and time range query of time series data, and a big data platform such as Doris and TiDB, which can cope with distributed storage and complex analysis requirements of massive data, thereby building a basic data source for data processing and providing original materials for subsequent multi-layer down-sampling operations.

[0029] Step S2, performing multi-layer down-sampling processing on the original sensing data by using the back-end system, constructing a multi-resolution data structure, and analyzing the multi-resolution data structure to obtain a first extreme value and a first abnormal value.

[0030] In some optional manners of some embodiments, the granularity versions of the multi-layer down-sampling processing include at least two of original data, 10-second granularity data, 1-minute granularity data and 1-hour granularity data, and the selected down-sampling algorithms for different granularities include at least one of an average value algorithm, an LTTB algorithm and a MinMax algorithm.

[0031] Specifically, the backend system performs multi-layer down-sampling processing on the original sensor data to build a multi-resolution data structure. The multi-layer down-sampling processing sets multiple granularity versions, covering at least two of the original data, 10-second granularity data, 1-minute granularity data, and 1-hour granularity data, to adapt to the needs of different data display and analysis scenarios for data density. For each granularity version, flexible selection of down-sampling algorithms is used to select at least one from the average value algorithm, the LTTB algorithm, and the MinMax algorithm. For example, when the data needs to retain the overall trend, the average value algorithm can be used; when the extreme value distribution of the data is concerned, the MinMax algorithm is more suitable; when the key features are accurately retained while reducing the amount of data, the LTTB algorithm has obvious advantages. After the multi-resolution data structure is built, the system analyzes it and identifies the first extreme value in the data, such as the highest and lowest values of temperature, and the first abnormal value, such as the abnormal point deviating from the normal data pattern, to provide key information support for subsequent data display and interaction.

[0032] In an optional manner of some embodiments, the analysis of the multi-resolution data structure described above to obtain the first extreme value includes: using a Z-score algorithm to identify the multi-resolution data structure; and determining that the first abnormal value is obtained when Z-score > 3.

[0033] Specifically, in the process of analyzing the multi-resolution data structure to obtain the first extreme value and the first abnormal value, for the identification of the first abnormal value, the Z-score algorithm is used to calculate and judge the data in the multi-resolution data structure. The Z-score algorithm identifies the abnormality by measuring the deviation of a data point from other data in the data set. When the calculated Z-score is greater than 3, it means that the data point deviates greatly from other data and exceeds the normal range. At this time, the data point is determined as the first abnormal value. Such an identification method can accurately filter out data that deviates significantly from the normal mode from the multi-resolution data, provide a basis for retaining key abnormal information in the subsequent data display process, and ensure that abnormal data will not be missed in the down-sampling processing, thereby improving the accuracy and reliability of data display.

[0034] In an optional manner of some embodiments, the multi-resolution data structure is organized according to the structure of devices, points, and granularity.

[0035] Specifically, when the backend system constructs the multi-resolution data structure, it organizes the data according to the hierarchical logic of device, point, and granularity. In the Internet of Things application scenario, the system first distinguishes different Internet of Things devices, then for each device, it stores the corresponding down-sampled data according to the set multi-layer down-sampling granularity for each point, such as raw data, 10-second granularity, 1-minute granularity, etc. Subsequently, whether the front end requests data according to the visible area requirement or carries out extreme value and abnormal value screening, the target data can be quickly located based on the clear device-point-granularity hierarchy, improving the data retrieval and processing efficiency, so that the multi-resolution data can play a more accurate and efficient role in the front-end and back-end collaborative process, and ensure that the data display retains key information while considering performance.

[0036] Step S3, collecting the visible area information of the chart by the front-end system, and sending the visible area information to the back-end interface.

[0037] In some optional manners of some embodiments, the visible area information includes point number information, time range information, current visible area pixel width information, chart maximum tolerable point number information, user zoom level information, user behavior context information, and precision priority information.

[0038] Specifically, when the front-end system displays the chart, it actively collects various information related to the visible area of the chart. These information specifically includes point number information, which is used to clearly identify the monitoring point to which the data to be displayed belongs; time range information, which determines the time interval of the data to be presented by the chart; current visible area pixel width information, which reflects the actual display space size occupied by the chart in the front-end interface; chart maximum tolerable point number information, which represents the upper limit of the number of data points that the front-end chart component can efficiently render; user zoom level information, which reflects the view scale after the user performs zoom operation on the chart; user behavior context information, which records the relevant operation background of the user interacting with the chart; and precision priority information, which indicates the user's preference for maintaining trends or retaining extreme values during data display. The front-end system integrates these collected visible area information and sends it to the back-end interface, providing accurate basis for subsequent targeted down-sampling processing by the back-end system, ensuring that the data returned by the back-end can match the display requirements of the front-end.

[0039] Step S4, using the back-end system to calculate the recommended sampling interval based on the above visible area information, and filtering the above multi-resolution data structure based on the above recommended sampling interval to obtain first-resolution data, which is a single set of down-sampled data adapted to the above recommended sampling interval.

[0040] Specifically, after receiving the viewable area information sent by the front end, the backend system first analyzes the time range span and the number of displayable points in the information, and calculates a recommended sampling interval. Subsequently, the backend system filters out the most matched resolution level from the constructed multi-resolution data structure according to the recommended sampling interval, and thus obtains first resolution data that is a single set of down-sampled data adapted to the recommended sampling interval. Such filtering method can avoid the over-dense or over-sparse situation of data when displayed on the front end, ensure that the data volume matches the carrying capacity of the front-end chart, lay a foundation for further down-sampling processing, and at the same time, under the premise of ensuring display performance, retain the key features of the data as much as possible, so that the chart finally presented on the front end is clear and can reflect the core trend of the data.

[0041] Step S5, using the backend system to perform down-sampling processing on the first resolution data to obtain second resolution data.

[0042] Specifically, the backend system further performs down-sampling processing on the first resolution data filtered out, and finally obtains second resolution data. This process will select appropriate lightweight down-sampling algorithms such as LTTB algorithm or inflection point preservation algorithm according to the viewable area information and precision priority sent by the front end. Through such processing, the data can not only meet the displayable point number requirement of the front-end chart, but also more accurately adapt to the display requirements of the current view, avoiding the rendering pressure of the front end caused by excessive data volume, and at the same time, retaining key trends and features as much as possible in the process of simplifying data, providing a reliable data foundation for the subsequent efficient rendering and accurate display of the front end.

[0043] Step S6, using the backend system to filter the first extreme value and the first abnormal value based on the viewable area information to obtain a second extreme value and a second abnormal value.

[0044] Specifically, the backend system will filter the first extreme value and the first abnormal value obtained by analysis according to the viewable area information sent by the front end, especially the precision priority in the information, to obtain a second extreme value and a second abnormal value. In the filtering process, attention is focused on the extreme values and abnormal points that are within the current viewable time range and meet the user's precision requirements, such as when the user's priority is to retain extreme values, the highest value and the lowest value in the region will be highlighted; when the priority is to focus on abnormalities, abnormal data deviating from the normal range will be highlighted. Such filtering method can ensure that the second extreme value and the second abnormal value finally transmitted to the front end are closely related to the content displayed in the current view, avoiding the redundant transmission of irrelevant data, and ensuring that key feature data is not missed when displayed on the front end, improving the pertinence and accuracy of data display.

[0045] Step S7, the backend system integrates the key data in the downsampling processing and the screening processing to obtain metadata information, the key data including data granularity data, sampling algorithm data, sampling error estimation data and reserved key point list data.

[0046] In an optional manner of some embodiments, in the metadata information, the data granularity data is the granularity corresponding to the second resolution data, the sampling algorithm data is the downsampling algorithm used to obtain the second resolution data, and the reserved key point list data includes the related information of the second extreme value and the second abnormal value.

[0047] Specifically, the key data in the downsampling processing and the screening processing is integrated to form the metadata information. The key data specifically includes data granularity data, sampling algorithm data, sampling error estimation data and reserved key point list data. The data granularity data corresponds to the granularity of the second resolution data, which enables the front end to clearly know the refinement degree of the current data; the sampling algorithm data refers to the downsampling algorithm used to obtain the second resolution data, which facilitates the front end to understand the data processing method; and the reserved key point list data contains the related information of the second extreme value and the second abnormal value, which ensures that the front end can accurately identify and display these key data points. By integrating these information, the metadata can provide necessary auxiliary instructions for the chart rendering of the front end.

[0048] Step S8, the backend system sends the metadata information, the second resolution data, the second extreme value and the second abnormal value to the front end system, and the front end system performs chart rendering to obtain the first data display result.

[0049] In an optional manner of some embodiments, when the interface of the front end system includes a multi-chart linkage chart, the front end system sends a unified time range and a maximum displayable point number to the backend system, the backend system adopts a consistent sampling granularity for multiple charts, and returns all chart data to the front end system after packaging.

[0050] Specifically, the backend system sends the integrated metadata information, the processed second resolution data, and the screened second extreme value and second abnormal value to the front-end system. After receiving the data, the front-end system renders a chart according to the data granularity sampling algorithm in the metadata information, and finally presents a first data display result. When the interface of the front-end system contains multiple linked charts, the front-end system sends the unified time range and the maximum number of displayable points to the backend system. The backend system adopts a consistent sampling granularity for multiple charts to ensure that the data of different charts are consistent in display accuracy, and then returns the data of all charts to the front-end system, which not only ensures the synchronization of data display in multiple linked charts, but also avoids visual mismatch caused by inconsistent sampling granularity, and improves the accuracy and reliability of collaborative display of multiple charts.

[0051] Step S9, when detecting that the user performs an interactive operation on the first data display result, the front-end system initiates a fine-grained data request for the interactive area, receives the optimized accuracy data returned by the backend, and covers the first data display result to obtain a second display result.

[0052] In some optional manners of some embodiments, the interactive operation includes at least one of a zoom operation and a hovering operation.

[0053] Specifically, when the system detects that the user performs an interactive operation on the first data display result, the front-end system initiates a fine-grained data request for the interactive area. The interactive operation here includes at least one of a zoom operation and a hovering operation, such as zooming in on a certain area of the chart to view more detailed data, or hovering the mouse over a specific position of the chart to understand the specific value. After the front-end system initiates the request, it receives the optimized accuracy data returned by the backend. These data have higher accuracy than the previous second resolution data, and can present more detailed information of the interactive area. Then, the front-end system covers the first data display result with these optimized accuracy data to form a second display result. Such a processing manner can supplement detailed information in time when the user needs to understand the data of a specific area in depth, which not only guarantees the efficiency of initial display, but also meets the user's demand for detailed viewing, and improves the overall interactive experience.

[0054] Figure 2The figure is a complete processing flow chart of Internet of Things data from collection to front-end display according to an embodiment of the present application. The flow chart presents the complete processing flow of Internet of Things data from collection to front-end display. First, real-time data collection is carried out by an IoT device, and the collected data is then used to build a back-end sampling layer; after completion of the building, a hierarchical down-sampling cache link is entered; then, a sampling decision module plays a role, in the process, a front-end chart state perception request interacts with the sampling decision module to provide a basis for sampling decision in terms of front-end view dimensions; then, a data return + meta information stage is entered to return the processed data and related meta information; finally, front-end rendering and compensation down-sampling operations are performed to realize the visual presentation of the data, and each link is closely linked to ensure efficient data processing and accurate display.

[0055] The front-end and back-end cooperative dynamic down-sampling data display method provided by the embodiment firstly collects original sensing data through the Internet of Things device and uploads the original sensing data to the back-end system through the gateway, thereby providing a source basis for the entire data processing flow and ensuring complete acquisition and transmission of the original data required for monitoring. Secondly, the back-end system performs multi-layer down-sampling processing on the original sensing data and constructs a multi-resolution data structure, and analyzes to obtain a first extreme value and a first abnormal value, thereby performing hierarchical processing on the massive high-frequency data in advance, providing multi-granularity data support for subsequent dynamic sampling, preliminarily retaining key data features, and avoiding omission of key information in subsequent processing. Subsequently, the front-end system collects chart visible region information and sends the chart visible region information to the back-end interface, thereby realizing accurate transmission of front-end view requirements to the back-end, enabling the back-end to perform targeted data processing based on the actual display scenario, and breaking the traditional front-end and back-end processing split state. Thereafter, the back-end system calculates a recommended sampling interval based on the visible region information and selects the first resolution data that is adapted, so that the back-end can quickly locate the data granularity that matches the current view from the pre-constructed multi-resolution data, avoiding problems of data being too dense or too sparse, and preliminarily realizing adaptation of data and view. Further, the back-end system performs down-sampling processing on the first resolution data to obtain second resolution data, further refines the data according to the specific bearing capacity of the front-end view, ensures that the amount of data finally transmitted to the front-end matches the front-end rendering capacity, and effectively reduces the front-end rendering pressure. Then, the back-end system selects a second extreme value and a second abnormal value based on the visible region information, so that the key abnormal points and extreme values under the current view can be retained in the down-sampling process, the problem of easy loss of key information in the traditional technology is solved, and the accuracy of data display is improved. In addition, the back-end system integrates key data in the down-sampling and selection process to obtain metadata information, thereby providing the front-end with a detailed description of data processing, enabling the front-end to more accurately render charts, allowing users to understand the reliability of the data, and enhancing the transparency of data display. Subsequently, the back-end sends the metadata, the second resolution data, and the second extreme value and abnormal value to the front-end and completes rendering to obtain a first data display result, thereby realizing efficient and accurate presentation of the data after cooperative down-sampling processing on the front-end, ensuring rendering performance, and taking into account key information of the data. Finally, the front-end initiates a fine-grained data request and receives optimized precision data to obtain a second display result when the user interacts, so that the system can dynamically respond to user operations and supplement higher-precision data when the user needs to view details, thereby realizing dynamic balance of performance optimization and detail display, and significantly improving the user interaction experience. The present invention solves the problems of difficulty in balancing performance and precision, data asynchronization caused by split front-end and back-end strategies, and easy loss of key abnormal points or extreme values in the traditional scheme.

[0056] The above is an embodiment of the front-end and back-end collaborative dynamic down-sampling data display system provided in the present application. The following describes other embodiments of the front-end and back-end collaborative dynamic down-sampling data display provided in the present application. See the following for details.

[0057] In the present embodiment, a front-end and back-end collaborative dynamic down-sampling data display system is also provided, which is used to implement the above-described embodiments and preferred embodiments, and has been described above. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and contemplated.

[0058] The present embodiment provides a front-end and back-end collaborative dynamic down-sampling data display system, as shown in Figure 3 , comprising:

[0059] A data acquisition module is configured to collect original sensing data using an Internet of Things device and upload the original sensing data to a back-end system through a gateway.

[0060] A multi-layer down-sampling module is configured to perform multi-layer down-sampling processing on the original sensing data using the back-end system, construct a multi-resolution data structure, and analyze the multi-resolution data structure to obtain a first extreme value and a first abnormal value.

[0061] A collection module is configured to collect visual area information of a chart using a front-end system and send the visual area information to a back-end interface.

[0062] A calculation module is configured to calculate a recommended sampling interval based on the visual area information using the back-end system, filter the multi-resolution data structure based on the recommended sampling interval, and obtain first resolution data, which is a single set of down-sampling data adapted to the recommended sampling interval.

[0063] A down-sampling module is configured to perform down-sampling processing on the first resolution data using the back-end system to obtain second resolution data.

[0064] A filtering module is configured to filter the first extreme value and the first abnormal value based on the visual area information using the back-end system to obtain a second extreme value and a second abnormal value.

[0065] An integration module is configured to integrate key data in the down-sampling processing and the filtering processing using the back-end system to obtain metadata information, wherein the key data includes data granularity data, sampling algorithm data, sampling error estimation data, and a list of retained key points.

[0066] The rendering module is configured to send the metadata information, the second resolution data, the second extreme value and the second abnormal value to the front-end system by using a back-end system, perform chart rendering by using the front-end system, and obtain a first data display result.

[0067] The covering module is configured to, when detecting that a user performs an interaction operation on the first data display result, initiate a fine-grained data request on the interaction area by using the front-end system, receive optimized precision data returned by the back-end system, and cover the first data display result to obtain a second display result.

[0068] The front-end and back-end cooperative dynamic down-sampling data display method provided by the embodiment first, the original sensing data is collected by the Internet of Things device and uploaded to the back-end system through the gateway, providing a source basis for the entire data processing flow, ensuring the complete acquisition and transmission of the required original data for monitoring. Secondly, the original sensing data is processed by the back-end system through multi-layer down-sampling and the construction of a multi-resolution data structure, and the first extreme value and the first abnormal value are analyzed to provide multi-granularity data support for subsequent dynamic sampling, and to preliminarily retain key data features, avoiding the omission of key information in subsequent processing. Then, the chart visible area information is collected by the front-end system and sent to the back-end interface, realizing the accurate transmission of the front-end view requirements to the back-end, enabling the back-end to perform targeted data processing based on the actual display scenario, breaking the traditional front-end and back-end processing split. Thereafter, the back-end system calculates the recommended sampling interval based on the visible area information and selects the first resolution data that fits, enabling the back-end to quickly locate the data granularity that matches the current view from the pre-constructed multi-resolution data, avoiding the problems of data being too dense or too sparse, and preliminarily realizing the adaptation of data and view. Further, the first resolution data is processed by the back-end system to obtain second resolution data, and the data is further simplified according to the specific bearing capacity of the front-end view to ensure that the amount of data transmitted to the front-end matches the rendering capacity of the front-end, effectively reducing the rendering pressure of the front-end. Then, the back-end system filters the first extreme value and the first abnormal value based on the visible area information to obtain the second extreme value and the second abnormal value, which can retain key abnormal points and extreme values under the current view during the down-sampling process, solving the problem of key information loss in traditional technologies and improving the accuracy of data display. In addition, the back-end system integrates the key data in the down-sampling and filtering process to obtain metadata information, providing the front-end with detailed data processing instructions, enabling the front-end to more accurately render charts, while allowing users to understand the reliability of the data, enhancing the transparency of data display. Thereafter, the back-end sends the metadata, second resolution data, and second extreme value and abnormal value to the front-end and completes rendering to obtain the first data display result, realizing the efficient and accurate presentation of data after cooperative down-sampling on the front-end, ensuring rendering performance while taking into account key data information. Finally, the front-end initiates a fine-grained data request when the user interacts and receives optimized precision data to obtain the second display result, enabling the system to dynamically respond to user operations and supplement higher precision data when the user needs to view details, achieving a dynamic balance between performance optimization and detail display, and significantly improving the user interaction experience. The present invention solves the problems of traditional schemes, such as the difficulty in balancing performance and accuracy, the data out of sync caused by the split of front-end and back-end strategies, and the easy loss of key abnormal points or extreme values.

[0069] Further function descriptions of the above-mentioned modules and units are the same as those of the corresponding embodiments described above, and will not be repeated here.

Claims

1. A front-end and back-end collaborative dynamic down-sampling data presentation method, characterized in that, The method comprises: Collecting original sensing data by using an Internet of Things device, and uploading the original sensing data to a backend system through a gateway; Performing multi-layer down-sampling processing on the original sensing data by using the backend system, constructing a multi-resolution data structure, analyzing the multi-resolution data structure, and obtaining first extreme values and first abnormal values; Collecting visual area information of a chart by using a front-end system, and sending the visual area information to a backend interface; Calculating a recommended sampling interval based on the visual area information by using the backend system, filtering the multi-resolution data structure based on the recommended sampling interval, and obtaining first resolution data, which is a single set of down-sampled data adapted to the recommended sampling interval; Performing down-sampling processing on the first resolution data by using the backend system, and obtaining second resolution data; Filtering the first extreme values and the first abnormal values based on the visual area information by using the backend system, and obtaining second extreme values and second abnormal values; Integrating key data in the down-sampling processing and the filtering processing by using the backend system, and obtaining metadata information, wherein the key data includes data granularity data, sampling algorithm data, sampling error estimation data, and a retained key point list data; Sending the metadata information, the second resolution data, the second extreme values, and the second abnormal values to the front-end system by using the backend system, performing chart rendering by using the front-end system, and obtaining a first data display result; When detecting that a user performs an interactive operation on the first data display result, initiating a fine-grained data request for an interactive area by using the front-end system, receiving optimized precision data returned by the backend, and covering the first data display result to obtain a second display result.

2. The method of claim 1, wherein, The original sensing data includes at least one of temperature data, liquid level data, and pressure data, and the backend system stores the original sensing data in at least one of a time series database or a big data platform.

3. The method of claim 2, wherein, The granularity versions of the multi-layer down-sampling processing include at least two of original data, 10-second granularity data, 1-minute granularity data, and 1-hour granularity data, and the selected down-sampling algorithms for different granularities include at least one of an average value algorithm, an LTTB algorithm, and a MinMax algorithm.

4. The method of claim 3, wherein, The multi-resolution data structure is organized according to the structures of devices, points, and granularities.

5. The method of claim 4, wherein, The visual area information includes point number information, time range information, current visual area pixel width information, chart maximum tolerable point number information, user zoom level information, user behavior context information, and precision priority information.

6. The method of claim 5, wherein, The analysis of the multi-resolution data structure to obtain the first extreme values includes: Identifying the multi-resolution data structure by using a Z-score algorithm; When Z-score>3, determining the first abnormal values.

7. The method of claim 6, wherein, In the metadata information, the data granularity data is a granularity corresponding to the second resolution data, the sampling algorithm data is a down-sampling algorithm used to obtain the second resolution data, and the retained key point list data includes related information of the second extreme values and the second abnormal values.

8. The method of claim 7, wherein, The interactive operation includes at least one of a zoom operation and a hovering operation.

9. The method of claim 8, wherein, The method further comprises: when the interface of the front-end system contains a multi-chart linkage chart, the front-end system sends a unified time range and a maximum displayable point number to a back-end system, the back-end system adopts a consistent sampling granularity for multiple charts, and returns all chart data to the front-end system after packaging.

10. A front-end and back-end collaborative dynamic down-sampling data presentation system, characterized by, The system comprises: A data acquisition module configured to collect original sensing data using an Internet of Things device and upload the original sensing data to a back-end system through a gateway; A multi-layer down-sampling module configured to perform multi-layer down-sampling processing on the original sensing data using the back-end system, construct a multi-resolution data structure, and analyze the multi-resolution data structure to obtain first extreme values and first abnormal values; A collection module configured to collect visual area information of a chart using a front-end system and send the visual area information to a back-end interface; A calculation module configured to calculate a recommended sampling interval based on the visual area information using the back-end system, filter the multi-resolution data structure based on the recommended sampling interval, and obtain first resolution data, which is a single set of down-sampled data adapted to the recommended sampling interval; A down-sampling module configured to perform down-sampling processing on the first resolution data using the back-end system to obtain second resolution data; A filtering module configured to filter the first extreme values and the first abnormal values based on the visual area information using the back-end system to obtain second extreme values and second abnormal values; An integration module configured to integrate key data in the down-sampling processing and the filtering processing using the back-end system to obtain metadata information, the key data including data granularity data, sampling algorithm data, sampling error estimation data, and a list of retained key points; A rendering module configured to send the metadata information, the second resolution data, the second extreme values, and the second abnormal values to the front-end system using the back-end system, perform chart rendering using the front-end system, and obtain a first data display result; A coverage module configured to initiate a fine-grained data request for an interaction area using the front-end system when detecting that a user performs an interaction operation on the first data display result, receive optimized precision data returned by the back-end system, and cover the first data display result to obtain a second display result.

Images