A method of improving global query performance in edge networks

Abstract

Methods and systems are described for data management, particularly for processing global queries. Each global query includes user-defined query constraint values, such as slack or query response time limits. A query receiving node maintains a copy of previously updated data from all of its child nodes. The query receiving node first searches its local data store for the requested query data to minimize child node queries. If any portion of the requested data in the local data store fails to satisfy the query constraint values, the child node from which the data came is responsible for recursively performing the global query.

Description

[0001] This application claims priority to U.S. Patent Application No. 17 / 115,696, filed December 8, 2021, entitled “A method to improve global query performance in an edge network,” the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to electronic circuits, and more particularly to active isolators for radio frequency (RF) circuits. Background Technology

[0003] Data (represented as a collection of information in singular form) is an essential component of any networked system, including enterprise distributed systems, where data is stored in a widely distributed geographical environment and spans tiers of data centers arranged as edge networks.

[0004] For example, in advanced industrial automation scenarios, industrial complexes may include numerous resource-constrained Internet of Things (IoT) devices that can record a large amount of data metrics. Typically, these metrics are stored locally to conserve network bandwidth and other costs. However, ad-hoc global query mechanisms can be used for remote monitoring and management without incurring significant bandwidth or computational overhead. For instance, if a device malfunction is suspected, an operator can query specific relevant metrics for the device in the last minute, hour, or day.

[0005] As another example, smart cities typically use distributed sensors to collect data for areas such as pollution monitoring, traffic and traffic control, and healthcare. These sensors generate a wealth of valuable data and sensor information, but not all of this information needs to be processed through real-time streaming. Instead, data is often uploaded in batches. In such a network, some queries can be ad hoc, responding to specific events. For example, an operator might query the number of pedestrians and bicycles in a specific area affected by a car accident.

[0006] As another example, utility companies have been using smart meters to aggregate usage information from customers. While smart meters periodically send data to a centralized location for coarse-grained analysis, on-demand querying enables fine-grained analysis, which in turn allows for faster resource management.

[0007] Data management in enterprise distributed system scenarios, as described above, has always been a challenge. Specifically, the challenges become apparent when such applications migrate to the cloud, edge, and Internet of Things (IoT) domains, where data is stored in widely distributed geographical environments and spans data center tiers deployed as edge networks. For example, network links may have limited bandwidth, high latency variations, and may experience intermittent failures. In such networks, data can be updated at any node within the data center tier. When read and write operations are performed locally (i.e., on the local node), they do not need to be immediately replicated across the entire network. Therefore, a common approach is to use a high-performance local data repository to provide fast local read and write operations and periodically propagate updated data upwards using optimal or on-demand data synchronization strategies. This is known as eventual consistency.

[0008] In the eventually consistent scheme described above, two types of queries can be submitted: local queries and global queries. When a query is submitted to the top node of the eventually consistent network system, it may be difficult to guarantee the time accuracy of the query response, also known as freshness and completeness, because the required data may be distributed across one or more end nodes of the network system.

[0009] Therefore, while enterprise distributed systems can benefit from efficient and accurate global query mechanisms, achieving strong consistency over large geographical areas may be difficult in practice. To address this challenge, data-intensive edge computing applications have been built to meet the requirements of relaxed consistency guarantees (such as eventual consistency).

[0010] A common approach is to maintain a local copy of the eventually synchronized data on each node. This allows any query to be responded to by searching the locally stored synchronized data. While this achieves fast response times, the data can become stale because updates are not immediately propagated to all copies, but rather periodically or on demand. Similarly, edge computing applications often rely on distributed tables or key-value stores rather than classic relational databases. In distributed environments, the cost of supporting join tables and transactions can be prohibitively high, especially with large datasets. While relational and transactional databases in geographically distributed environments are an active area of ​​research, many current high-performance distributed databases are based on tables or key-value stores.

[0011] Alternatively, a thorough search of the entire system network could be performed to locate the latest data needed for a query response. However, this approach is both time-consuming and resource-intensive. Another alternative is stream processing, where queries are broken down into a graph of operators distributed across the entire system network. Data can be processed and aggregated as it streams from edge (or end) nodes to top nodes. However, this approach increases implementation complexity because it requires deploying, coordinating, and executing operators across different nodes. Furthermore, distributed stream processing across geographically distributed network systems can be difficult to execute due to issues such as unreliable links, frequent reconfigurations, and high latency. Therefore, stream processing incurs high setup and ongoing costs, making it more suitable for handling small, recurring, or sequential queries that are known in advance, rather than ad-hoc queries requiring data exploration.

[0012] In conclusion, there is a need for an improved data management system that can process local and global queries efficiently and promptly while ensuring data freshness. Summary of the Invention

[0013] In one aspect, this disclosure provides a hybrid approach for efficient on-demand global queries that satisfy user-defined freshness constraints by leveraging the underlying hierarchical structure of edge networks.

[0014] On the other hand, this disclosure provides an eventually consistent tabular data management system for edge computing applications, which allows users to intelligently control the trade-off between one or more query constraints and one or more quantified attributes of the query response. As a non-limiting example, a user can specify an exact freshness constraint for each individual query, or alternatively, a deadline for the query response can be specified as a query constraint. Queries can then be executed on a subset of the network, using local copies in intermediate network nodes as a cache to avoid querying edge nodes. In the absence of failures, the result set can include all data that satisfies the specified query constraints (as a cost of query response attributes, such as latency or resource usage).

[0015] On the other hand, the actual data attributes associated with the query constraints can be provided so that users can identify the precise data attribute values. For example, if the query constraints are in the form of freshness values, the actual freshness of the data can be provided in the form of timestamps in the query response so that users can identify the exact time when the data was generated.

[0016] On the other hand, this disclosure provides a data management system that allows for partial responses to queries in the event of, for example, intermittent link errors or other types of network failures. For instance, the user can be notified how much data is missing from the partial response.

[0017] In another aspect, this disclosure provides a method for data management in a hierarchical edge network, the method comprising: receiving a query for query data at a query receiving node having a plurality of child nodes, the query including query constraint values; searching for data in a local data storage device of the query receiving node based on the query constraint values; updating the search results with data found in the local data storage device that satisfies the query constraint values; recursively executing the query on one or more of the plurality of child nodes to locate data that satisfies the query constraint values ​​when at least a portion of the data that satisfies the query constraint values ​​is unavailable in the local data storage device of the node receiving the query; updating the search results based on the data that satisfies the query constraint values ​​received from the one or more child nodes; and reporting the search results as query data in response to the query.

[0018] In another aspect, this disclosure provides a computing system for data management, the computing system including one or more processing devices; and a memory storing instructions responsive to execution by the one or more processing devices, causing the computing system to: receive a query for query data at a query receiving node having a plurality of child nodes, the query including query constraint values; search for data in a local data storage device of the node receiving the query based on the query constraint values; update the query data with data found in the local data storage device that satisfies the query constraint values; recursively execute the query at one or more of the plurality of child nodes when at least a portion of the data that satisfies the query constraint values ​​is unavailable in the local data storage device of the node receiving the query; update the query data with data received from the one or more child nodes that satisfies the query constraint values; and report the query data as a response to the query.

[0019] In any of the above aspects, the query constraint value can be a slackness. L The freshness threshold is defined as T. q - L ,in, T q It is the time when the query was received.

[0020] In either of the above aspects, the data satisfying the query constraint value can be received in batches from one or more child nodes.

[0021] In any of the foregoing aspects, the update may also include an actual freshness value associated with the data received from one or more child nodes.

[0022] In any of the above aspects, the actual freshness value can be the minimum between the latest update time of the query receiving node and the actual freshness value returned by one or more child nodes.

[0023] Any of the above aspects may further include comparing the actual freshness value of the data from each of the plurality of child nodes with a freshness threshold to determine one or more child nodes for which a query needs to be performed.

[0024] When a link error causes one of the child nodes to become unreachable, any of the above aspects may further include: updating the search results with the data from the unreachable child node from the local data storage device.

[0025] When a link error causes a child node among multiple child nodes to become unreachable, any of the above aspects may further include: updating the search results with partial data that satisfies the query constraint value; and updating the query data to include the unreachable child nodes.

[0026] When a link error causes a child node among multiple child nodes to become unreachable, any of the above aspects may further include: estimating the number of missing data; updating the query data to include the missing data estimate.

[0027] In any of the foregoing aspects, the estimation may include calculating the number of missing data, specifically as follows: .

[0028] In any of the foregoing aspects, the estimation may include using time series forecasting to calculate the number of missing data.

[0029] Any of the above aspects may further include receiving updated data from each of the plurality of child nodes; marking the received updated data with a status indicator having a first status value; and maintaining a copy of the data received from each of the plurality of child nodes in the local data storage device.

[0030] Any of the above aspects may further include updating the data marked with a first state value to the parent node; and changing the state indicator of the data to a second state value.

[0031] In either of the above aspects, the updated data can be received periodically from each of the multiple child nodes.

[0032] In either of the above aspects, the updated data can be received from each of the multiple child nodes upon request.

[0033] In any of the above aspects, the query constraint value can be a query response time limit.

[0034] In either of the above aspects, the reporting step can be completed before the query response time limit.

[0035] When a child node fails to respond before the query response time limit, any of the above aspects may further include: updating the query data with data from the unreachable child node from the local data storage device.

[0036] Any of the above aspects may also include modifying the query response limits at each level by one or more child nodes to take into account the latency and processing time of one or more child nodes.

[0037] In either of the above aspects, the one or more child nodes can execute queries in parallel.

[0038] In any of the above aspects, the query may be an aggregate query for determining data information, and updating the query data further includes: determining the data information based on the data that satisfies the query constraint value; and updating the query data using the data information.

[0039] In any of the above aspects, the aggregate query may include a conditional clause, and the determination may further include: applying the conditional clause to the data that satisfies the constraint value to determine intermediate data; and determining the data information based on the intermediate data.

[0040] In any of the above aspects, the data information may include maximum (MAX), minimum (MIN), average (AVG), total number of data (COUNT), and sum (SUM).

[0041] In any of the foregoing, the conditional clause may include WHERE, GROUP BY, ORDER, LIMIT, and DISTINCT.

[0042] At least some of the above aspects can advantageously provide an active isolator for broadband performance, which has improved linearity and is capable of operating with lower power consumption and less loss.

[0043] According to another aspect of this disclosure, a computing device is provided, comprising a processor, a memory, and a communication subsystem. The memory tangibly stores executable instructions for execution by the processor. In response to execution by the processor, the executable instructions cause the computing device to perform the methods described above and herein.

[0044] According to another aspect of this disclosure, a non-transitory machine-readable medium is provided, on which executable instructions for execution by a processor of a computing device are tangibly stored. In response to execution by the processor, the executable instructions cause the computing device to perform the methods described above and herein.

[0045] Other aspects and features of this disclosure will become apparent to those skilled in the art after reviewing the following detailed description of embodiments. Attached Figure Description

[0046] Figure 1 An exemplary edge network with three levels according to this disclosure is shown;

[0047] Figure 2 A block diagram of an exemplary simplified computing system is shown, which can be used to implement Figure 1 One or more network nodes are shown;

[0048] Figure 3 An exemplary schematic block diagram of a network module is shown, which can... Figure 1 Implemented at either the top node 110 or the core node 120;

[0049] Figure 4 The hypothetical edge network and corresponding event timeline according to this disclosure are shown;

[0050] Figure 5 It shows that it can be made by Figure 3 A flowchart illustrating an exemplary method by which the query processing module 308 executes to process a global query;

[0051] Figure 6 A timeline illustrating the relaxation concept of this disclosure is shown;

[0052] Figure 7 Exemplary pseudocode for a global query processing algorithm is shown, which can be executed by network module 300 for use with relaxation. L The query constraint value.

[0053] Similar reference numerals can be used to denote similar components in different accompanying drawings. Detailed Implementation

[0054] This disclosure is made with reference to the accompanying drawings, in which embodiments are illustrated. However, many different embodiments may be used, and therefore the description should not be construed as limiting it to the embodiments set forth herein. Rather, these embodiments are provided to make this application thorough and complete. Where possible, the same reference numerals are used in the drawings and the following description to refer to the same elements, and prime number notation is used in alternative embodiments to indicate the same elements, operations, or steps. The separate blocks or separations of functional elements of the illustrated systems and devices do not necessarily require physical separation of these functions, as communication between these elements can occur without any such physical separation via message passing, function calls, shared memory spaces, etc. Therefore, although functions are shown separately herein for ease of explanation, these functions do not need to be implemented in physically or logically separated platforms. Different devices may have different designs such that while some devices implement some functions in fixed-function hardware, others may implement those functions in a programmable processor with code available from a machine-readable medium. Finally, elements referred to in the singular may be in the plural form, and vice versa.

[0055] As described herein, the term "edge network" refers to a hierarchical network where each node can be a data center where a portion of an edge computing application may be deployed. In some embodiments, the top or top node of the edge network is a server, such as a cloud server with high-performance computing and storage resources, which may be well-suited for scalability. As the network hierarchy descends, the resources of the data centers acting as intermediate nodes may become increasingly constrained, while being closer to the end users and data-generating devices (such as sensors) that serve as data sources. At the very edge of the network are edge nodes, which may be small data centers, typically consisting of a limited number of machines with limited computing power.

[0056] As described in this article, the term "edge computing application" refers to an application deployed on an edge network to divide computing and storage tasks among nodes in the network.

[0057] In this disclosure, data centers along the path from the top node (i.e., the cloud data center) to the edge node are referred to as core nodes. Users and data-generating devices (such as sensors) are not considered part of the network.

[0058] As described herein, a “module” can refer to a combination of hardware processing circuitry and machine-readable instructions (software and / or firmware) that can be executed on the hardware processing circuitry. Hardware processing circuitry can include any or some combinations of microprocessors, the core of multi-core microprocessors, microcontrollers, programmable integrated circuits, programmable gate arrays, digital signal processors, or other hardware processing circuitry.

[0059] Figure 1 A simplified example of an edge network 100 with three levels (represented by three rows of rectangles below the top node 110) is shown, which can be used to implement the example described herein. It should be understood that although three levels are shown, the edge network suitable for implementing the example described herein can have any number of levels. The disclosed edge network 100 includes a top node 110, core nodes 120A to D (collectively referred to as core nodes 120), edge nodes 130A to F (collectively referred to as edge nodes 130), and data generation devices 140A to 140F (collectively referred to as data generation devices 140).

[0060] The data generating device 140 can be any IoT device coupled to network 100, configured to generate data that can be used for monitoring and control. The data generating device 140 can have limited computing resources, such as a single sensor or embedded system built for a specific purpose. For example, in an Industrial IoT (IIoT) environment, the data generating device 140 can be a machine on a factory floor capable of generating large amounts of data, which is typically used for low-latency process control decisions on a production line. In other embodiments, such as smart cities, the data generating device 140 can be traffic sensors, lights, meters, etc., used to generate data to assist in city management operations.

[0061] The data generated by one or more data generation devices 140 can then be aggregated at one or more edge nodes. In some embodiments, edge node 130 is a device capable of routing network traffic. Such a device typically has higher computing power and resources compared to data generation device 140. For example, edge node 130 can be a base station, router or switch, gateway, or small data center. In some embodiments, edge node 130 can be configured to monitor and control machines in a factory or group of factories where more resource-intensive predictive maintenance models can be applied with less stringent latency requirements. In some embodiments, edge node 130 can provide fast read and write operations to high-performance local data storage devices.

[0062] The intermediate node located between the top node 110 and the edge node 130 is also referred to as the core node 120. The core node 120 can be a data center that aggregates data from a subset of the edge nodes 130. As shown in the diagram, data generated by data generation devices 140B and 140C is collected by edge nodes 130C and 130D and then aggregated by the core node 120B. As a non-limiting example, a core node can be a control server responsible for controlling and monitoring a factory in a region or a smart city within a state or province. As shown in the diagram, a core node can have multiple levels. For example, core node 120B is a child node (also called a descendant node) of core node 120A, and core node 120D is a child node of core node 120C. Although a maximum of two levels of core nodes are shown, it should be understood that any number of core node levels can be implemented within network 100.

[0063] Top node 110 can be a cloud server / data center as shown in the figure, or any other server-based system, serving as the top node of the network. Furthermore, while a single instance of a top node is shown, other embodiments may have two or more top nodes. Top node 110 can aggregate data generated by data generation devices 140 coupled to edge network 100 and provide an access point for human operators to monitor status and guide operational decisions. In some embodiments, top node 110 can be a cloud data center with high-performance computing and storage resources suitable for scalability. Top node 110 may include a backend management system capable of providing a web-based control panel or user interface (UI) displaying one or more operational characteristics of the data generation devices coupled to edge network 100 or the network itself, such as global operational status, inventory, and schedules. Due to the greater availability of resources in the cloud, top node 110 can also be used to train machine learning based on historical data.

[0064] Figure 2 A block diagram of an exemplary simplified computing system 200 is shown, which can be used to implement Figure 1 One or more network nodes are shown, such as top node 110, core nodes 120A, 120B, 120C, and 120D, and edge nodes 130A to D and 130E to F. Other computing systems suitable for implementing the embodiments described in this disclosure may be used, which may include components different from those discussed below. In some examples, the computing system may be implemented across multiple physical hardware units, such as in parallel computing, distributed computing, virtual servers, or cloud computing configurations. Although Figure 2 A single instance of each component is shown, but multiple instances of each component may exist in the computing system 200.

[0065] The computing system 200 may include one or more processing devices 202, such as a central processing unit (CPU), graphics processing unit (GPU), tensor processing unit (TPU), neural processing unit (NPU), microprocessor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), dedicated logic circuit, dedicated artificial intelligence processing unit, or a combination thereof.

[0066] The computing system 200 may also include one or more optional input / output (I / O) interfaces 204, which can be connected to one or more optional input devices 206 and / or optional output devices 208. In the example shown, the input devices 206 (e.g., keyboard, mouse, microphone, touchscreen, and / or keypad) and output devices 208 (e.g., monitor, speaker, and / or printer) are shown as optional and external to the computing system 200. In other examples, one or more of the input devices 206 and / or the output devices 208 may be components of the computing system 200. In other examples, there may be no input devices 206 and output devices 208, in which case the I / O interfaces 204 may not be needed.

[0067] The computing system 200 may include one or more network interfaces 210 for wired or wireless communication with the data generation device 140 or other nodes within the network 100. The network interface 210 may include wired links (e.g., Ethernet cables) and / or wireless links (e.g., one or more antennas) for intranet and / or extranet communication.

[0068] The computing system 200 may further include one or more local data storage units 212, which may include high-capacity storage units such as solid-state drives, hard disk drives, disk drives, and / or optical disk drives. In some embodiments, the local data storage unit 212 may store data aggregated from the data generation device 140 through edge nodes (i.e., 130A to D, 130E to F) and core nodes (i.e., 120A to B, 120C to D). The computing system 200 may include one or more memories 214, which may include volatile or non-volatile memories (e.g., flash memory, random access memory (RAM), and / or read-only memory (ROM)). The multiple non-transient memories 214 may store instructions executed by the multiple processing devices 202, such as instructions for performing the examples described in this disclosure. The multiple memories 214 may include other software instructions, such as software instructions for implementing an operating system and other applications / functions. In some examples, memory 214 may include software instructions for execution by processing device 202 to train and / or implement trained neural networks, as disclosed herein.

[0069] In other examples, one or more datasets and / or modules may be provided by external memory (e.g., an external drive that communicates with the computing system 200 via wired or wireless communication) or by transient or non-transient computer-readable media. Examples of non-transient computer-readable media include RAM, ROM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, CD-ROM, or other portable storage devices.

[0070] There may be one or more buses 216 for providing communication between components of the computing system 200, including multiple processing devices 202, multiple optional I / O interfaces 204, multiple network interfaces 210, multiple local data storage units 212, and / or multiple memories 214. Bus 216 may be any suitable bus architecture, such as a memory bus, peripheral bus, or video bus.

[0071] In some embodiments, network 100 can provide two types of queries: local queries and global queries. Both types of queries can access data written locally to each of nodes 110, 120, and 130, but differ in data attributes. Specifically, local queries typically provide fast read and write operations performed directly on the local data storage unit 212. Local queries can be executed by applications or supported by an edge-centric eventually consistent distributed database. Global queries can include on-demand read queries that provide guarantees of user-specified query constraints (i.e., freshness or response time). In some embodiments, in response to execution on nodes (i.e., 110, 120, 130), the query response is computed based on the most recent local and descendant data up to the user-specified constraint value. By maintaining the update time of data stored at core node 120, network 100 can avoid querying remote edge node 130, thereby achieving faster response times and saving network bandwidth.

[0072] Figure 3 A simplified schematic block diagram of network module 300 is shown, which can be implemented at either the top node 110 or the core node 120. In the illustrated embodiment, network module 300 includes a data storage device 302, a data transmission module 304, a data receiving module 306, and a query processing module 308.

[0073] As described herein, a “module” can refer to a combination of hardware processing circuitry and machine-readable instructions (software and / or firmware) that can be executed on the hardware processing circuitry. Hardware processing circuitry can include any combination of microprocessors, the core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a system on a chip (SoC) or other hardware processing circuitry.

[0074] Data storage device 302, also known as a persistent local data storage module, is configured to provide persistent storage for local data and / or replicated data. In some embodiments, data storage device 302 may be configured to store data and any metadata associated with the data, such as timestamps. A data storage device can be said to be persistent if it retains data even after a power outage. A persistent data storage device can also refer to a non-volatile and non-transient storage device.

[0075] Data storage device 302 is configured to store and maintain local copies of remotely generated or remotely located data, such as data generated by data generation devices 140A to F. This type of data storage device 302 can also be referred to as a materialized view or snapshot, which is a table segment or database subject containing query results, and whose content is periodically refreshed based on queries (whether for local or remote tables). Data storage device 302 may receive search requests 310 from query processing module 308 to locate query data requested by queries (local query 330 or global queries 332 and 334). Search request 310 may include any query constraint values ​​specified in the query. Data is searched in data storage device 302 for any data that satisfies the specified query constraint values. When data satisfying the query constraint values ​​is found in data storage device 302, the data is added to the search results. Alternatively, if no data satisfying the query constraint values ​​is found, the search results may remain empty or display any other suitable indicator to indicate that no data was found. The destination of the search results depends on the source of the query. For example, the search results of a search request 310 generated in response to a local query 330 or a global query 332 received in the current network module 300 can be returned to the query processing module 308. Alternatively, when a corresponding search request 310 is generated in response to a global query 334 received from a parent node, the search results can be sent to the data transmission module 304 via data retrieval 312.

[0076] In some embodiments, data storage device 302 can be implemented on Apache Cassandra using a standalone single-datacenter Cassandra cluster, which allows for horizontal scaling within the datacenter by adding nodes to the local cluster. Furthermore, Cassandra QUORUM reads and writes can be configured to provide strong consistency within data storage device 302. It should be understood that network 100 is independent of the underlying data storage device and can be adapted to other systems. Because data storage device 302 in each network module 300 is independent of other instances of network module 300, network 100 can achieve horizontal scaling within nodes. Data can be written from data receiving module 306 to data storage device 302 via data update 318, as detailed below. (As described herein, "updating" data can include modifying data, replacing data, resetting data, and any other method of recording or obtaining an updated version of data of interest; "updating" search results, discussed below, can also include any method of recording or obtaining an updated version.)

[0077] Data transmission module 304 is configured to transmit data to other nodes in network 100. In some embodiments, data transmission module 304 is configured to transmit search results as uplink data transmission (also referred to as upload or push) 314 from data storage device 302 to a parent node in response to a global query 334. In some embodiments, data transmission module 304 is a background process configured to execute continuously to handle periodic (regular or intermittent) service requests and is often referred to as a "push daemon". When a global query 334 is received from the parent node, query processing module 308 sends a search request 310 to data storage device 302 to locate the query data requested by the global query. Regardless of whether the query data is found, the search results are transmitted to data transmission module 304 as data retrieval 312. The search results are then transmitted to the parent node, which then receives the global query 334 as uplink data transmission (also referred to as upload or push) 314, which is received by the parent node's data receiving module 306 as received data 316. In some embodiments, uplink data transmission 314 may include a query ID field indicating the global query 334 it is responding to. In some embodiments, to avoid link saturation or overloading of the parent node, excessively large uplink data transmissions 314 can be truncated into multiple uplink data transmissions 314 (also referred to as batches) before transmission (or push). In some embodiments, batches can be transmitted in chronological order (i.e., from oldest to newest, or vice versa) by sorting them by the timestamps of the data contained therein. The update cycle and / or batch size can be configured to control the trade-off between data freshness and resource usage (i.e., link bandwidth, CPU, and data storage space). In some embodiments, when the data transmission module 304 receives confirmation of successful data transmission from the parent node, it can notify the data storage device 302 to modify the status indicator associated with the transmitted data. For example, "dirty" data, after being pushed to the parent node, can be marked as "clean" data to indicate that it has been synchronized with the parent node.

[0078] In some embodiments, the data pushed from a child node to a parent node also includes one or more metadata fields associated with the data itself. For example, the metadata may include a timestamp indicating the update time (i.e., the time the data was received), thus describing the freshness of the pushed data. In some embodiments, the update time is defined recursively. As a non-limiting example, for network module 300 on a non-edge node (i.e., top node 110 or core node 120), the update time may be set to the minimum of the latest update times of all its child nodes. For edge node 130, if the push includes all “dirty” data, the update time may be set to the current timestamp, or if the data is transmitted or pushed in batches, the update time may be set to the update timestamp of the latest row data pushed up to the parent node. In some embodiments, the timestamp containing the update time may be used by the parent node to ensure freshness. In some embodiments, when the query data to be pushed to the parent node does not contain “dirty” data, the child node responsible for transmission may transmit empty uplink data transmission 314 to its parent node with the update time as described above. This avoids spurious subqueries in the query mechanism.

[0079] The data receiving module 306 is configured to receive data transmitted from one or more child nodes. In some embodiments, one or more uplink data transmissions 314 from the data transmission modules 304 of one or more child nodes are received by the data receiving module 306 as received data 316. The received data 316 may include multiple (i.e., ...) data transmissions from one or more child nodes. Figure 3 The three shown are upstream data transmission 314. The data receiving module 306 can be configured to run continuously in the background, commonly referred to as the "receive daemon".

[0080] Upon receiving data 316, the data receiving module 306 forwards the received data 316 to the node's local data storage device 302 as a data update 318 to be stored. In some embodiments, the received data 316 is stored along with a status indicator to indicate whether the data has been synchronized within the network 100. For example, newly received data 316 may be stored with the status indicator "dirty" until it is transmitted upwards or pushed to its parent node by the data transmission module 304, where the status indicator may be changed to "clean". The received data 316 may also be stored in the data storage device 306 along with corresponding metadata fields, such as a timestamp indicating the latest update time of the received data. In some embodiments, the timestamp may be used as a filtering condition to satisfy query constraint values ​​(such as freshness). When the received data 316 includes an empty stream with an update time, the update time is forwarded to the data storage device 302 via data update 318 to update the timestamp of the expected data by matching the query ID field and the query data initiated by the global query 334 request. In some embodiments, the received data 316 may be stored in the data storage device 302, with the corresponding child node ID indicating which child node the data was received from. This allows the current network module 300 to track and monitor child nodes that may need to be queried for data synchronization, as well as the source of any particular received data 316.

[0081] In some embodiments, the data receiving module 316 may receive a partial result set from a set of child nodes. The received partial results may be stored in the data storage device 302 and merged with any locally found data in the search results, which are then sent as data retrieval 312 to the data transmission module 304 and pushed up to its parent node.

[0082] The query processing module 308 is configured to receive and process queries including local queries 330 from local applications, global queries 332 received at the current network module 300, and global queries 334 received from the parent node.

[0083] When query processing module 308 receives local query 330, it sends search request 310 to search data in data storage device 302 for data that satisfies the query constraint values ​​specified in local query 330. If found, the data can be appended to the search result. Alternatively, the search result can be left empty (null) or assigned any other value to indicate that no data was found. The search result is returned to query processing module 308, which in turn responds to the local application with the search result and uses the local application to generate local query 330. Thus, data storage device 302, serving as a materialized view of data in network 100, allows network module 300 to locally respond to local query 330.

[0084] When the global query 332 is received locally in the network module 300, the query processing module 308 sends a search request 310 to search for data in the data storage device 302 that satisfies the query constraint values ​​specified in the global query 332. If found, the data is appended to the search result; otherwise, the search result may remain empty (or null) or have any other value indicating that no data was found, and the search result is returned to the query processing module 308. If the queried data is found in the data storage device 302, the query processing module 308 responds to the query with the search result. If no data satisfying the query constraint values ​​is found in the data storage device 302 (i.e., the search result is empty), the query processing module 308 generates one or more subqueries 320 to be sent to one or more child nodes. The subqueries 320 generated by the query processing module 308 are received by the child nodes as a global query 334. Each subquery 320 (or the global query 334 of a child node) may contain a query ID, which may be incorporated into the uplink data transmission 314 from the child node, such that the uplink data transmission 314 can be matched with the corresponding subquery 320. In some embodiments, the query processing module 308 may determine one or more child nodes that need to be queried in order to obtain the query data requested by the subquery 320 and generate the subquery 320 accordingly.

[0085] Therefore, when a node in network 100 receives a global query with query constraint values ​​such as freshness, the node will execute the query on its local storage device (a materialized view of network data), which can provide the query data sought by the query, thereby avoiding the time-consuming process of the query-end node. As a non-limiting example, in accordance with... Figure 4 In the hypothetical Level 3 edge network 400 of this disclosure, the network can be used to monitor data generating devices, such as robotic arms (not shown) in a factory, and timelines displaying relationships between time parameters. In the illustrated embodiment, operational events are monitored. K For example, power consumption. Figure 4 As shown, for time T =8 at the top node A The received request has L Assuming a global query with a relaxation degree of 2, the network must retrieve data in... T All data updated before =6 (i.e., data) K 1 、K 2 、K 3 ).because K 2 It has been synchronized to the node. B Therefore, the network does not need to send data to the nodes. DQuery data. Similarly, for K 1 Since it has been synchronized to the node C Therefore, no node query is required. F .data K 3 It might be "dirty" data that hasn't been synchronized yet, so it's necessary to query the node. E As can be seen from the above, the network according to this disclosure may only require access to nodes. B , C and E To obtain the requested data, instead of accessing all nodes. D , E , F .

[0086] In some embodiments, nodes in network 100 may periodically (at regular or irregular time intervals) generate subqueries 320 for data synchronization purposes via query processing module 308. This operation, which requests data that has not yet been synchronized (i.e., data entries marked as “dirty”), can be accomplished by specifying a slack value based on the node’s update time, thereby ensuring eventual consistency of network 100.

[0087] When a global query 334 (or subquery 320) is received from the parent node, whether as part of data synchronization or a user-generated global query, the query processing module 308 sends a search request 310 to search the data storage device 302 for data that satisfies the query constraint values ​​specified in the global query 334. If found, the data is appended to the search result; otherwise, the search result may remain empty (or null) or any other value to indicate that no data was found. The search results are forwarded to the data transmission module 304, which sends the search results back to the parent node from which it received the global query 334, either as an uplink data transmission 314, either entirely or in batches.

[0088] Figure 5 A flowchart is shown of an exemplary method 500 for processing queries that can be executed by the query processing module 308.

[0089] At 502, the global query 332 or a global query 334 from a parent node is received at the global query processor 308 of network module 300. The receiving network module 300 can be either the top node 110 or the core node 120. Global queries 332 and 334 include user-specified query constraint values, one or more metadata attributes that specify query data or query search parameters.

[0090] In some embodiments, the query constraint value can be a time precision parameter of the queried data, also known as relaxation. L RelaxationL Specify the time requested by the user t Previously generated data will be included in the query data. Relaxation effectively relaxes or loosens the freshness requirements of the query data.

[0091] Figure 6 A timeline is shown to better illustrate the relaxation concept of this disclosure. As shown in the figure, the query time... T q The nodes of network 100 receive values ​​including relaxation values. L The time required for the query. The purpose of Network 100 is to process queries and provide information contained in the definition... The response is a set of updated data (i.e., through insertions, deletions, and updates) that occurred before a freshness threshold. While slack may limit data freshness, the query results may be fresher than the limit. Therefore, the query data can also include the actual freshness time. T f This makes in T f All previously updated data is included in the query data. T f Subsequently generated data may be included in the results, but this is not guaranteed. In some embodiments, T f The exact value depends on which data has already been synchronized within the network (i.e., copied on the parent node).

[0092] Because query execution and propagation between nodes takes time, there is an inherent time delay in providing a response. Therefore, any query data generated in response to a query may be slightly outdated, even with slack. L The same applies to =0. In some cases, the actual freshness of the data... T f It may be earlier than the query time. T q (Right now T f > T q Therefore, the timeout value can be defined as the response time. T a The difference between the actual freshness time and the actual freshness time, i.e. .like Figure 6 As shown, in some embodiments, network 100 can guarantee the following relationships between various time parameters:

[0093]

[0094] For further explanation of how to operate the relaxation parameter, please refer to [link / reference]. Figure 4As shown in the diagram, network 400 includes nodes that serve as cloud data centers. A ,node B and C It is a core node, a node D , E and F It is an edge node. It monitors events. K 1 to K 6 From coupling to edge nodes D to F The data is created by a data generation device (not shown), and K 1 and K 2 Events have been copied to the parent node. C and B In the exemplary network shown, at the node A of T q =8 received a global query to retrieve all data, its slackness L =2. This provides a freshness threshold of 6, meaning that network 400 must guarantee events. K 1 , K 2 , K 3 The update timestamps of the data, 2, 3, and 4, are all before the freshness threshold and are included in the query response. Therefore, at least the node must be queried. B , C and E Because they all contain data that should be included in the query response. Due to the nodes... F arrive C The last propagation time was recent enough that the event... K 1 The required data has been synchronized and stored locally on the node. C In the middle, there is no need to query the node again. F If the global query receives the same time T q =8 has a looser slack. L =4.5, then only query the node. B and C To obtain K 1 and K 2 That might be enough, and there's no need to query nodes anymore. E This is because its update time of 4 exceeds the freshness threshold of 3.5. This allows for faster response times, but at the cost of more outdated data.

[0095] In some other embodiments, the query constraint value can also be a query response time limit for completing the query search. This query response time limit effectively acts as a latency constraint. In these embodiments, even if not all the freshest data is included, the response to the query must be provided within the specified response time limit. The actual freshness time for each data update can be considered. T f Provided to users to indicate the freshness of the data. In some embodiments, a child node that fails to respond in a timely manner to meet the query response time limit can be considered a failed link, as detailed below. In these embodiments, a child node receiving a query from its parent node can reduce the query response time limit to account for latency between the parent and child nodes and increase processing margin. In addition to executing the query, another query is executed on the local data storage device of each child node responsible for executing the query. For each child node that does not receive its response within the query response time limit, the result of the corresponding local query for such child node is used to update the query data response.

[0096] It should be understood that there can be more than one query constraint value, and any other suitable query constraint value can be specified.

[0097] Back Figure 5 In step 504, the query receiving node searches its local data storage device 302 to determine if data satisfying the query constraint value can be found locally, or if a recursive query of its child nodes is necessary. If the required query data is found in the data storage device 302, a query response can be generated faster compared to a recursive search through network nodes. The query constraint value is the relaxation level. L In some embodiments, the decision depends on each child node. c The latest update time received is represented as If one or more Compared to freshness threshold Closer (i.e.) If the local storage data from one or more child nodes is new enough to generate a response to the query, then the need to query its child nodes is eliminated.

[0098] In step 506, when one or more child nodes c Data update time Earlier than the freshness threshold (Right now When ), then the child node c Pushing data to the parent node takes too long, so queries can be passed to child nodes instead. cExecute the following steps. Steps 402 through 404 can be executed recursively on each queried child node (or as part of a recursive operation) to return the result to its parent node.

[0099] In some embodiments, queries can be executed in parallel by nodes. In these embodiments, queries can be asynchronously scheduled to child nodes and then executed locally.

[0100] In step 508, when a response is received from the child node responsible for executing the query, the response is appended to the query data as a response to the query. In some embodiments, such as when the query constraint value is slackness... L In this case, the query receiving node can update the query data to include the actual freshness time of its data. T f The actual freshness is recursively defined, similar to the latest update time. In some embodiments, it can be defined as the latest update time of the current node plus the freshness returned by the responses of each child node. T f The minimum value between. Therefore, in some embodiments, the actual freshness value T f This depends on the push update cycle of the child nodes and the depth of the hierarchical network. For example, if a remote edge node needs to be queried to obtain the necessary data, the data propagation time may be longer. In addition, if the network hierarchy increases, the query may need to be executed multiple times down the network hierarchy, which increases processing time and thus affects the freshness of the data returned to the query receiving node.

[0101] In 510, the query data can be presented to the user, for example, through output device 208, as a response to the query.

[0102] Figure 7 Exemplary pseudocode is shown for a global query processing algorithm that can be executed by network module 300, where the query constraint value is the relaxation level. L The algorithm accepts queries at the specified time. T q Received queries q and relaxation L and the current node n As input, it returns the actual freshness time. T f The result (i.e., the query response data) R In lines 3 through 6, loop through the current node. n Each child node in all child nodes c To determine the source from the child node c Was the last update earlier than the freshness threshold? If so, then the child nodes...c Add to the list of visited child nodes and query q Asynchronous send to child nodes c The query is executed on the local data storage device to retrieve data from child nodes whose data freshness is updated to the freshness threshold of row 7. The query response is then... R Use in line 8 R loc Incremental update. In line 9, freshness time. T f Set to the latest update time of the current node (i.e. The freshness returned by the response of each child node found in rows 10 to 12. T f The minimum value between. Query response. R and actual freshness T f Return it as output on line 13.

[0103] In networks such as geographically distributed edge networks, failures can be common. In particular, because networks are large and intermittent link failures are not uncommon, the ability to execute queries in the event of a loss of network connectivity becomes crucial. In some embodiments, query timeouts are allowed when one or more nodes requiring data are deemed unreachable. When network 100 generates search results for a query in the event of a link failure at an unreachable node, it may include information about the unreachable node. When a link to a child node being queried fails or times out, the freshness of the data for a particular query may not be guaranteed. In some embodiments, in the event of a network failure, network 100 provides a complete but less fresh answer that includes stale results with missing child nodes. Specifically, the query response is complete but does not meet the freshness guarantee (i.e., slackness) requested by the user. Instead, the query response may include data with a more lenient freshness value, which depends on the last update time of the child node connected by the failed link. In other words, the query response can guarantee actual freshness. T f It is complete, but the actual freshness may be lower than the freshness threshold: As a non-limiting example, a user can request data with a 5-minute slack (i.e., data generated in the past 5 minutes), and network 100 can return a complete query response with data up to 15 minutes ago. Alternatively, in the event of a link failure, network 100 can provide a partial but up-to-date query response. The query response can satisfy a user-specified freshness guarantee. However, it can only use data from nodes that respond promptly, and not any new data from subtrees that cannot be queried due to link failures.

[0104] In some embodiments, for each query response, network 100 provides analytics information that may include, for example, the number of nodes queried, the number of data included in the query response data, and an estimate of the amount of data not included due to query constraints (i.e., slackness) or link failures.

[0105] In some embodiments, each network module 300 may determine the number of child nodes it needs to query for a given query (i.e., Figure 7 (Lines 3 to 6 in the table), and the total amount of data received from its child nodes and the total amount of data obtained from its local data storage device.

[0106] In some embodiments, each network module 300 tracks the rate of row updates (and insertions) received from each child node in order to estimate the number of new or updated data rows. In some embodiments, in addition to recording the last update time, each network module 300 also records the last update time received from the child node. K +1 timestamp of the update, and the number of new rows reported on the child nodes. Therefore, the data from each child node can be represented according to equation (1). c The update rate is estimated from the node c The end K Average update rate per update:

[0107]

[0108] in, It comes from the child node. c The latest push update timestamp It comes from the child node. c The earliest push update timestamp, It is a report from the child nodes during each corresponding push update. c The number of new data, of which i =The range is between 0 and k Between -1 and 1. Therefore, in time... Above, child nodes c The estimated number of new or updated ones is The query data response may include the sum of estimates for all child nodes. In some embodiments, where more accurate estimates may be required, more sophisticated time series forecasting methods may be used. One such time series forecasting method is disclosed in "Time Series Forecasting Using Artificial Wavelet Neural Networks and Multiresolution Analysis: Applications to Wind Speed ​​Data" by B. Doucoure, K. Agbossou, and A. Cardenas, excerpted from Renewable Energy, Vol. 92, pp. 202–211, 2016, the disclosure of which is incorporated herein by reference.

[0109] In some embodiments, different types of global queries (referred to as aggregate queries) exist, which can be used to determine one or more pieces of information about the data that satisfy query constraints or the query data (i.e., minimum / maximum, number of data entries, average, etc.). In some embodiments, aggregate queries in network 100 may adopt the format of Cassandra Query Language (CQL) queries, where query constraints are specified by additional conditions, such as data freshness (slackness) or result latency (query response time limit). In these embodiments, global queries may also adopt features provided by CQL, such as aggregate functions, including maximum data value (MAX), minimum data value (MIN), average data value (AVG), total data value (COUNT), total number of data satisfying query constraints, and sum of data (SUM), as well as aggregate queries with conditional clauses, including WHERE to specify a particular data row to query; GROUP BY to compress selected data rows that share the same set of column values ​​into one row; ORDER to select the order in which the results are returned; LIMIT to limit the number of rows returned by the query; and DISTINCT to remove duplicates and return distinct data values.

[0110] For aggregation queries, query data can be updated based on the query type, rather than using the actual data. For example, data values ​​can be added to a SUM aggregation query. Therefore, the content and format of the query data can vary depending on the type of aggregation query. For example, for aggregation queries on network module 300, such as MIN, MAX, SUM, COUNT, a single value is provided in the query data response from the child nodes. For aggregation queries that include AVG (average), the query data response can include two values: the average and the number of elements in the set required to perform the aggregation.

[0111] Clauses are used as conditions applied to related data before aggregation. Aggregation and clauses can be applied to data by parent nodes, child nodes, or a combination of both. For example, for an aggregation query with a GROUP BY clause, the related data is first grouped into intermediate data based on the GROUP BY condition, and the child nodes determine the aggregation for each group of intermediate data. The results are then pushed up to the parent node, which merges the results of each group with the results from other child nodes. For an aggregation query with a WHERE clause, the clause is applied locally on the child node to the data that meets the WHERE condition to obtain intermediate data, which is then sent to the parent node for aggregation. For DISTINCT, ORDER, and LIMIT clauses, aggregation can be performed at the last level of the aggregation (i.e., the query receiving node), rather than at intermediate nodes.

[0112] Although this disclosure describes methods and processes using actions in a certain order, one or more actions of the methods and processes may be omitted or modified as appropriate. One or more actions may be performed in an order other than that described, as appropriate.

[0113] Although this disclosure describes at least part of the methodological aspects, those skilled in the art will understand that this disclosure also relates to various components, whether hardware components, software, or any combination thereof, for performing at least some aspects and features of the methods. Accordingly, the technical solutions of this disclosure can be embodied in the form of a software product. Suitable software products can be stored in pre-recorded storage devices or other similar non-volatile or non-transitory computer-readable media, including DVDs, CD-ROMs, USB flash drives, removable hard drives, or other storage media. The software product includes instructions tangibly stored thereon, which enable processing devices (e.g., personal computers, servers, or network devices) to perform examples of the methods disclosed herein.

[0114] This disclosure may be implemented in other specific forms without departing from the subject matter of the claims. The exemplary embodiments described are merely illustrative in all respects and not restrictive. Features selected from one or more of the foregoing embodiments may be combined to create alternative embodiments not explicitly described, and features suitable for such combinations will be understood within the scope of this disclosure. Therefore, the scope of this disclosure is defined by the appended claims rather than by the foregoing description. The scope of the claims should not be limited to the embodiments set forth in the examples, but should be given the broadest interpretation consistent with the entire description.

[0115] All values ​​and sub-ranges within the scope of the disclosure are also disclosed. Furthermore, although the systems, devices, and processes disclosed and shown herein may include a specific number of elements / components, modifications may be made to say the systems, devices, and components to include more or fewer of such elements / components. For example, although any element / component disclosed may be referred to as a single number, embodiments disclosed herein may be modified to include multiple such elements / components. The subject matter described herein is intended to cover and encompass all appropriate technical changes.

Claims

1. A method for data management in a hierarchical edge network, the method comprising: A query for query data is received at a query receiving node having multiple child nodes, the query including query constraint values, the multiple child nodes including edge nodes and core nodes, the edge nodes being operable to aggregate data from one or more data generating devices, and the core nodes being operable to aggregate data from a subset of edge nodes, wherein the query constraint value is a relaxation L to define a freshness threshold as Tq-L, where Tq is the time when the query was received; Based on the query constraint value, search for data in the local data storage device of the query receiving node; The search results are updated using the data that satisfies the query constraint values ​​found in the local data storage device; In response to at least a portion of the data satisfying the query constraint value being unavailable in the local data storage device of the query receiving node, The query is recursively executed on one or more of the multiple child nodes to locate data that satisfies the query constraint value; The search results are updated based on data received from the one or more child nodes that satisfy the query constraint values; as well as In response to the query, the search results are reported as the query data.

2. The method of claim 1, wherein data satisfying the query constraint value is received in batches from the one or more child nodes.

3. The method of any one of claims 1 or 2, wherein the update further includes an actual freshness value associated with the data received from the one or more child nodes.

4. The method of claim 3, wherein the actual freshness value is the minimum value between the latest update time of the query receiving node and the actual freshness value returned by the one or more child nodes.

5. The method according to claim 3, further comprising: The actual freshness value of the data from each of the plurality of child nodes is compared with a freshness threshold to determine one or more child nodes from which the query needs to be executed.

6. The method according to claim 1 or 2, further comprising: when a link error causes one of the plurality of child nodes to become unreachable; The search results are updated using data from unreachable child nodes of the local data storage device.

7. The method according to claim 1 or 2, further comprising: when a link error causes one of the plurality of child nodes to become unreachable; Update the search results using partial data that satisfies the query constraint values; as well as Update the query data to include unreachable child nodes.

8. The method according to claim 1 or 2, further comprising: when a link error causes one of the plurality of child nodes to become unreachable; Estimate the number of missing data; as well as Update the query data to include the missing data estimate.

9. The method of claim 8, wherein the estimation comprises calculating the number of the missing data as: .

10. The method of claim 8, wherein the estimation includes calculating the number of missing data using time series forecasting.

11. The method according to claim 1 or 2, further comprising: Receive updated data from each of the plurality of child nodes; The received updated data is marked using a status indicator with a first status value; as well as A copy of the data received from each of the plurality of child nodes is maintained in the local data storage device.

12. The method of claim 11, further comprising: Update the parent node with the data marked with the first state value; as well as Change the status indicator of the data marked with the first status value to the second status value.

13. The method of claim 11, wherein the updated data is periodically received from each of the plurality of child nodes.

14. The method of claim 11, wherein the updated data is received from each of the plurality of child nodes upon request.

15. The method according to claim 1 or 2, wherein the query constraint value is a query response time limit.

16. The method of claim 15, wherein the reporting step is completed before the query response time limit.

17. The method of claim 15, further comprising: when a child node fails to respond before the query response time limit, the method includes: The query data is updated using data from the unreachable child nodes of the local data storage device.

18. The method of claim 15, further comprising: The query response time limit at each level is modified by the one or more child nodes to take into account the latency and processing time of the one or more child nodes.

19. The method of claim 1 or 2, wherein the one or more child nodes execute the query in parallel.

20. The method according to claim 1 or 2, wherein the query is an aggregate query for determining data information, and the method further comprises: The data information is determined based on the data that satisfies the query constraint values; as well as The query data is updated using the data information.

21. The method of claim 20, wherein the aggregate query includes a conditional clause, and the determination further includes: The conditional clause is applied to the data that satisfies the constraint value to determine intermediate data; as well as The data information is determined from the intermediate data.

22. The method according to claim 21, wherein the data information includes maximum value (MAX), minimum value (MIN), average value (AVG), total number of data (COUNT), and sum (SUM).

23. The method of claim 21, wherein the conditional clause includes WHERE, GROUP BY, ORDER, LIMIT, and DISTINCT.

24. A computing system for data management, comprising: One or more processing devices; as well as Memory, storing instructions that, in response to execution by the one or more processor devices, cause the computing system to perform the method according to any one of claims 1 to 23.

25. A non-transitory machine-readable medium having tangibly stored executable instructions for execution by one or more processing devices of a computing system, wherein the executable instructions, in response to execution by the plurality of processing devices, cause the computing system to perform the method according to any one of claims 1 to 23.

Images