A data collection and edge-cloud collaborative processing system for an internet of things scene

Abstract

This invention discloses a data acquisition and edge-cloud collaborative processing system for IoT scenarios, comprising: a data acquisition module for dividing acquisition windows and establishing temporal relationships; an edge processing module for generating edge processing results and differential data; a cloud collaboration module for generating cloud processing results and constructing processing dependencies; a backpropagation determination module for locking acquisition windows using an improved APPNP model; a data recovery module for backpropagating and restoring continuously acquired data; a local processing module for reprocessing incomplete cloud processing locations and resuming transmission; and a result output module for summarizing results and generating edge-cloud collaborative processing results. This invention relates to the field of IoT data processing technology, achieving accurate IoT data backpropagation and improving the efficiency and integrity of edge-cloud collaborative processing.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) data processing technology, and in particular to a data acquisition and edge-cloud collaborative processing system for IoT scenarios. Background Technology

[0002] With the increasing number of IoT terminals, sensing devices continuously generate time-series operational data. Existing IoT data processing systems mostly employ a collaborative architecture of edge computing and cloud computing. Edge nodes perform data filtering, compression, feature extraction, or local analysis, and then upload the processing results to the cloud. The cloud then performs cross-device data aggregation, correlation analysis, and business processing. In some solutions, when cloud processing results are abnormal or insufficient, the original data or differential data is retrieved from the edge nodes to supplement the data content required for cloud computing.

[0003] Existing edge-cloud collaborative processing methods often determine the data backhaul range based on devices, tasks, or fixed data windows during the data replenishment stage, lacking fine-grained dependency tracking between cloud processing locations, edge processing results, and the original acquisition window. When a cloud processing location lacks necessary data, it easily expands the original data backhaul range, and data that already meets processing requirements may participate in repeated transmission or recalculation. For cloud processing tasks with multiple data sources, the required replenishment depth varies among different data sources, and uniformly expanding the replenishment range increases edge-cloud communication volume and cloud-based redundant processing. Current technologies also lack a mechanism to determine replenishment boundaries separately for different data dependency branches and to only extend the data processing of the corresponding branch when local recovery is insufficient.

[0004] Therefore, how to provide a data acquisition and edge-cloud collaborative processing system for IoT scenarios is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a data acquisition and edge-cloud collaborative processing system for IoT scenarios. This invention adopts an improved APPNP model for backpropagation along processing dependencies. By locking propagation branches, directional data replenishment, and local reprocessing, it achieves edge-cloud collaborative data recovery, which has the advantages of reducing data backhaul, reducing redundant calculations, and maintaining the stability of continuous data processing.

[0006] A data acquisition and edge-cloud collaborative processing system for IoT scenarios according to an embodiment of the present invention includes: The data acquisition module is used to acquire continuously collected data, divide the acquisition windows, and connect adjacent acquisition windows to form a time sequence relationship; The edge processing module is used to process the edge processing results formed by the acquisition window, retain the differential data formed by the edge processing results not covering the acquisition content, and associate the edge processing results with the acquisition window; The cloud collaboration module is used to process edge processing results to form cloud processing results, lock the unfinished cloud processing locations, connect the cloud processing locations, edge processing results and acquisition windows, and connect the timing relationships to form processing dependencies. The back-determination module is used to input the cloud processing location and processing dependency into the improved APPNP model, and to form a propagation branch by back propagation along the processing dependency. When the propagation branch first reaches the acquisition window of the retained differential data, the acquisition window is locked, the locked propagation branch is terminated, and the unlocked propagation branch is retained to continue propagation. The data recovery module is used to gather the data recovery range formed by the locked acquisition window, combine the corresponding differential data and edge processing results, and recover the continuously acquired data within the locked acquisition window. The local processing module is used to reprocess the cloud processing positions associated with continuously collected data within the locked acquisition window, update the cloud processing results, and unlock the corresponding acquisition window and propagation branch for cloud processing positions that have not yet been completed, so as to continue reverse propagation. The results output module is used to summarize the edge processing results and the cloud processing results, and output the edge-cloud collaborative processing results.

[0007] Optionally, the data acquisition module forms a time-series relationship, specifically including: Arrange the continuously collected data in the order of collection, and divide the continuously collected data into segments to form a collection window; Lock the end and beginning data of adjacent acquisition windows, and use the end data as the boundary to push the beginning data back to the previous acquisition window, maintaining the original acquisition order of the beginning data; The first-end data that remains in continuous acquisition sequence with the previous acquisition window after being turned back is snapped to the previous acquisition window, and the first-end data that is not snapped back is sent back to the next acquisition window. Reconnect the previous acquisition window with the next acquisition window to establish a timing relationship.

[0008] Optionally, the edge processing module retains the differential data formed by the edge processing results not covering the collected content, specifically including: The edge processing results are generated by processing the acquisition window, and the edge processing results are mapped to the acquisition data positions in the acquisition window; Lock the data collection locations covered by the edge processing results of two adjacent data collection locations, and extract the data collection content between the two data collection locations that is not covered by the edge processing results; The captured content is shrunk from both ends toward the adjacent data positions, while maintaining the original acquisition order before shrinking. The captured content after shrinking at both ends is then associated with the corresponding edge processing results. The remaining collected content that is not associated with the edge processing results continues to shrink to the adjacent collected data location until all uncovered collected content is associated, and the associated collected content is formed into differential data.

[0009] Optionally, the cloud collaboration module establishes processing dependencies, specifically including: Lock the unfinished cloud processing location and connect the edge processing results that participated in the cloud processing location to the corresponding cloud processing location; Connect each edge processing result to the corresponding acquisition window, and connect adjacent acquisition windows along the temporal sequence; Edge processing results connected to the same acquisition window retain their independent connections to the cloud processing location, allowing each independent connection to converge in the same acquisition window; Preserve the timing relationships between each independent connection and acquisition window to form processing dependencies.

[0010] Optionally, the backpropagation branch formed in the backpropagation determination module along the processing dependency relationship specifically includes: Starting from the location of the incomplete cloud processing, the propagation proceeds backward along the processing dependencies to the corresponding edge processing result. The edge processing results are then passed back to the corresponding acquisition window, so that the back transmission paths that reach the same acquisition window converge at the acquisition window, and each back transmission path is retained. The sequence continues to propagate backward from the preceding acquisition window, following the temporal relationship between acquisition windows. Each reverse propagation path is identified as a propagation branch.

[0011] Optionally, the backfilling determination module locks the acquisition window when the propagation branch first reaches the acquisition window for retaining differential data, specifically including: The propagation branches sequentially check the arrival of the acquisition windows. Propagation branches that have not reached the acquisition windows for retaining differential data continue to propagate along the processing dependencies. When the propagation branch first reaches the acquisition window for retaining differential data, the current acquisition window and the corresponding propagation branch are locked, and the locking of the propagation branch is terminated. Propagation branches that do not reach the acquisition window for retaining differential data are retained as unlocked propagation branches.

[0012] Optionally, the backfilling determination module retains the unlocked propagation branches for continued propagation, specifically including: After completing one round of back propagation, the locked propagation branches are removed from the next round of propagation, and only the unlocked propagation branches are retained; The current collection window reached by the unlocked propagation branch is used as the starting point for the next round of propagation, and the propagation continues in reverse along the processing dependency; When different unlocked propagation branches arrive at the same acquisition window where differential data is not retained, each unlocked propagation branch is retained separately and the unlocked propagation branches are not merged. For the first acquisition window in the arrival time sequence, the unlocked propagation branch is terminated and removed, and the reverse propagation is repeated until there are no more unlocked propagation branches.

[0013] Optionally, the data recovery module recovers continuously acquired data within the locked acquisition window, specifically including: The data acquisition windows are collected and arranged according to their temporal sequence to form a data recovery range. Extract the differential data and edge processing results corresponding to each locked acquisition window within the data recovery range; Locate the edge processing result within the data acquisition window, and fill the data acquisition locations not covered by the edge processing result with differential data. Restore continuously collected data within the locked collection window according to the data collection location.

[0014] Optionally, updating the cloud processing results in the local processing module specifically includes: Locate the cloud processing location associated with continuously collected data within the locked acquisition window along the processing dependency relationship, and freeze the cloud processing locations other than those associated with continuously collected data within the locked acquisition window; The continuously collected data within the locked collection window is sent back to the cloud processing location associated with the continuously collected data within the locked collection window, the cloud processing location associated with the continuously collected data within the locked collection window is reprocessed, and the cloud processing results are updated. Release the lock on the corresponding acquisition window and corresponding propagation branch for cloud processing locations that have not yet been completed; The data continues to propagate backward from the unlocked acquisition window along the corresponding propagation branch.

[0015] Optionally, the result output module outputs the edge-cloud collaborative processing results, specifically including: The cloud processing results are mapped to the cloud processing location associated with the continuously collected data within the locked collection window, and the corresponding locked collection window is determined. Replace the edge processing results connected to the corresponding cloud processing location with the cloud processing results, and retain the edge processing results that are not replaced; Connect the cloud processing results and the edge processing results along the temporal relationship of the corresponding acquisition window; It aggregates cloud processing results and edge processing results, and outputs edge-cloud collaborative processing results.

[0016] The beneficial effects of this invention are: (1) The present invention takes the unfinished cloud processing location as the starting point of reverse propagation, traces the processing dependency one branch at a time, locks the acquisition window and terminates the corresponding propagation when the propagation branch first reaches the acquisition window for retaining differential data, and retains only the unlocked propagation branch to continue propagation, so that different data sources obtain the data replenishment boundary corresponding to the processing requirements, avoids expanding the replenishment range as a whole according to the device, task or fixed window, and reduces the range of irrelevant acquisition data participating in the edge cloud transmission.

[0017] (2) The present invention keeps the propagation branches that reach the same acquisition window independent, does not perform propagation branch merging, and uses differential data and edge processing results to restore the corresponding continuous acquisition data after locking the acquisition window, so that different dependent branches in the same cloud processing task can complete data recovery according to their respective missing degree, avoiding the data loss of a single branch from affecting other branches that have met the processing conditions, and enhancing the pertinence of the multi-source IoT data replenishment process.

[0018] (3) After data recovery, this invention only reprocesses the cloud processing positions associated with the continuously collected data within the locked acquisition window and freezes the other cloud processing positions. If the reprocessing is not yet completed, only the corresponding acquisition window and propagation branch are unlocked and reverse propagation continues, so that the data recovery range expands branch by branch with the actual processing results, avoiding overall cloud back-calculation and repeated recovery, and maintaining the continuous connection between edge processing results and cloud processing results. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a module structure diagram of a data acquisition and edge-cloud collaborative processing system for IoT scenarios proposed in this invention; Figure 2 This invention presents a diagram illustrating the handling of dependencies, backpropagation, and propagation state matrix. Figure 3 This is a diagram showing the changes in the propagation round window state and propagation branch proposed in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0021] refer to Figures 1-3 A data acquisition and edge-cloud collaborative processing system for IoT scenarios includes: The data acquisition module is used to acquire continuously collected data, divide the acquisition windows, and connect adjacent acquisition windows to form a time sequence relationship; The edge processing module is used to process the edge processing results formed by the acquisition window, retain the differential data formed by the edge processing results not covering the acquisition content, and associate the edge processing results with the acquisition window; The cloud collaboration module is used to process edge processing results to form cloud processing results, lock the unfinished cloud processing locations, connect the cloud processing locations, edge processing results and acquisition windows, and connect the timing relationships to form processing dependencies. The back-determination module is used to input the cloud processing location and processing dependency into the improved APPNP model, and to form a propagation branch by back propagation along the processing dependency. When the propagation branch first reaches the acquisition window of the retained differential data, the acquisition window is locked, the locked propagation branch is terminated, and the unlocked propagation branch is retained to continue propagation. The APPNP model is a graph neural network model based on a personalized PageRank mechanism for adjacency propagation. The improved APPNP model uses the incomplete cloud processing location as the propagation starting point. The propagation based on adjacency in the APPNP model is adjusted to propagate backward along the processing dependency, from the incomplete cloud processing location to the corresponding edge processing result, the acquisition window, and the preceding acquisition window. When different backward propagation paths reach the same acquisition window, they are retained separately and not merged. The retained backward propagation paths are formed into propagation branches. The data recovery module is used to gather the data recovery range formed by the locked acquisition window, combine the corresponding differential data and edge processing results, and recover the continuously acquired data within the locked acquisition window. The local processing module is used to reprocess the cloud processing positions associated with continuously collected data within the locked acquisition window, update the cloud processing results, and unlock the corresponding acquisition window and propagation branch for cloud processing positions that have not yet been completed, so as to continue reverse propagation. The results output module is used to summarize the edge processing results and the cloud processing results, and output the edge-cloud collaborative processing results.

[0022] In this embodiment, the formation of a time sequence relationship in the data acquisition module specifically includes: Arrange the continuously collected data in the order of collection, and divide the continuously collected data into segments to form a collection window; Lock the end and beginning data of adjacent acquisition windows, and use the end data as the boundary to push the beginning data back to the previous acquisition window, maintaining the original acquisition order of the beginning data; The end-of-line data is taken from the end position of the previous acquisition window, and the beginning-of-line data is taken from the beginning position of the next acquisition window. Both are derived from continuously acquired data arranged in the acquisition order. When turning back, the position of the end-of-line data is used as the boundary. The beginning-of-line data is moved to the previous acquisition window item by item according to the original acquisition order. The beginning-of-line data after being moved in is checked to see if it maintains the continuous acquisition order with the acquisition data in the previous acquisition window, without changing the original acquisition order between the beginning-of-line data. The first-end data that remains in continuous acquisition sequence with the previous acquisition window after being turned back is snapped to the previous acquisition window, and the first-end data that is not snapped back is sent back to the next acquisition window. When the data is engaged, the first data collected in the continuous acquisition sequence will be retained in the previous acquisition window; if the acquisition sequence is interrupted, the engagement will stop, and the first data collected in the unengaged end will be restored to the next acquisition window, maintaining the original acquisition sequence before the reversal. Reconnect the previous acquisition window with the next acquisition window to establish a timing relationship.

[0023] In this embodiment, the edge processing module retains the differential data formed by the edge processing results not covering the collected content, specifically including: The edge processing results are generated by processing the acquisition window, and the edge processing results are mapped to the acquisition data positions in the acquisition window; Lock the data collection locations covered by the edge processing results of two adjacent data collection locations, and extract the data collection content between the two data collection locations that is not covered by the edge processing results; The location of the collected data covered by the edge processing result is the position of the collected data processed when the corresponding edge processing result is formed in the acquisition window; between two adjacent locations of collected data covered by edge processing results, the collected data not covered by the edge processing result is determined as the collected content not covered by the edge processing result, and the original acquisition order of the corresponding collected data is maintained. The captured content is shrunk from both ends toward the adjacent data positions, while maintaining the original acquisition order before shrinking. The captured content after shrinking at both ends is then associated with the corresponding edge processing results. During contraction, starting from both ends of the captured content, the data is moved item by item to the corresponding adjacent data position in the order of proximity to the adjacent data position, and the captured content after the two ends are moved is associated with the corresponding edge processing results. The remaining collected content that is not associated with the edge processing results continues to shrink to the adjacent collected data location until all uncovered collected content is associated, and the associated collected content is formed into differential data.

[0024] When generating differential data, the corresponding collected content is arranged according to its position in the collection window after association, maintaining the original collection order of the corresponding collected content.

[0025] In this embodiment, the cloud collaboration module establishes processing dependencies, specifically including: Lock the unfinished cloud processing location and connect the edge processing results that participated in the cloud processing location to the corresponding cloud processing location; The processing location where the corresponding cloud processing result is not output after the edge processing result is processed in the cloud is defined as the incomplete cloud processing location; Connect each edge processing result to the corresponding acquisition window, and connect adjacent acquisition windows along the temporal sequence; Edge processing results connected to the same acquisition window retain their independent connections to the cloud processing location, allowing each independent connection to converge in the same acquisition window; Independent connections are connections that are separately maintained between edge processing results and their corresponding cloud processing locations. If several edge processing results are connected to the same acquisition window, the connections will converge at the acquisition window, but will still maintain their respective connections to their corresponding cloud processing locations and will not be merged into a single connection. Preserve the timing relationships between each independent connection and acquisition window to form processing dependencies.

[0026] The processing dependencies consist of cloud processing location, edge processing results, connections between acquisition windows, and temporal relationships between acquisition windows.

[0027] In this embodiment, the backpropagation of the processing dependency in the back-propagation determination module forms a propagation branch, specifically including: Starting from the location of the incomplete cloud processing, the propagation proceeds backward along the processing dependencies to the corresponding edge processing result. The edge processing results are then passed back to the corresponding acquisition window, so that the back transmission paths that reach the same acquisition window converge at the acquisition window, and each back transmission path is retained. The sequence continues to propagate backward from the preceding acquisition window, following the temporal relationship between acquisition windows. The preceding acquisition window is the acquisition window that is located before and adjacent to the arrival acquisition window in the timing relationship; when different reverse propagation paths arrive at the same acquisition window, each reverse propagation path is retained separately and not merged into the same reverse propagation path; Each reverse propagation path is identified as a propagation branch.

[0028] The propagation branch is a separately retained reverse transmission path, which starts from the incomplete cloud processing position, reaches the acquisition window through the corresponding edge processing result, and continues to be transmitted to the previous acquisition window along the temporal relationship.

[0029] In this embodiment, the backfilling determination module locks the acquisition window when the propagation branch first reaches the acquisition window for retaining differential data. Specifically, this includes: The propagation branches sequentially check the arrival of the acquisition windows. Propagation branches that have not reached the acquisition windows for retaining differential data continue to propagate along the processing dependencies. The acquisition window that retains differential data is the acquisition window that the edge processing module has already formed for acquiring differential data; during verification, the acquisition windows that arrive are checked one by one in the reverse propagation order to see if they retain differential data; When the propagation branch first reaches the acquisition window for retaining differential data, the current acquisition window and the corresponding propagation branch are locked, and the locking of the propagation branch is terminated. The first arrival refers to the first arrival of the acquisition window for retaining differential data during the reverse propagation process from the propagation origin of the same propagation branch; the termination of locking the propagation branch means stopping the corresponding propagation branch from continuing to propagate to the previous acquisition window. Propagation branches that do not reach the acquisition window for retaining differential data are retained as unlocked propagation branches.

[0030] In this embodiment, the unlocked propagation branches are retained in the backfilling determination module to continue propagation, specifically including: After completing one round of back propagation, the locked propagation branches are removed from the next round of propagation, and only the unlocked propagation branches are retained; One round of backpropagation is the process by which each unlocked propagation branch completes a forward acquisition window transfer from the current propagation starting point along the processing dependency relationship; The current collection window reached by the unlocked propagation branch is used as the starting point for the next round of propagation, and the propagation continues in reverse along the processing dependency; When different unlocked propagation branches arrive at the same acquisition window where differential data is not retained, each unlocked propagation branch is retained separately and the unlocked propagation branches are not merged. For the first acquisition window in the arrival time sequence, the unlocked propagation branch is terminated and removed, and the reverse propagation is repeated until there are no more unlocked propagation branches.

[0031] In the timing relationship, the first acquisition window is the acquisition window that is at the beginning of the acquisition order and has no preceding acquisition window.

[0032] In this embodiment, the data recovery module recovers continuously acquired data within the locked acquisition window, specifically including: The data acquisition windows are collected and arranged according to their temporal sequence to form a data recovery range. The data recovery range consists of locked acquisition windows, which are arranged according to their sequential positions in the time sequence. Extract the differential data and edge processing results corresponding to each locked acquisition window within the data recovery range; When generating edge processing results, the collected content in the data collection locations covered by the edge processing results is preserved; Locate the edge processing result within the data acquisition window, and fill the data acquisition locations not covered by the edge processing result with differential data. During filling, the collected content in the differential data is placed in the same collection data position according to the collection data position corresponding to the differential data, and the collection data position covered by the edge processing result remains unchanged. Restore continuously collected data within the locked collection window according to the data collection location.

[0033] During recovery, the collected content retained in the edge processing results and the entered differential data are arranged according to the position of the collected data in the locked acquisition window.

[0034] In this embodiment, updating the cloud processing results in the local processing module specifically includes: Locate the cloud processing location associated with continuously collected data within the locked acquisition window along the processing dependency relationship, and freeze the cloud processing locations other than those associated with continuously collected data within the locked acquisition window; The cloud processing location associated with continuously collected data within the locked acquisition window is determined by locating the cloud processing location from the locked acquisition window along the processing dependency relationship through the corresponding edge processing result; when frozen, reprocessing of cloud processing locations other than those associated with continuously collected data within the locked acquisition window is stopped, and the corresponding cloud processing result remains unchanged; The continuously collected data within the locked collection window is sent back to the cloud processing location associated with the continuously collected data within the locked collection window, the cloud processing location associated with the continuously collected data within the locked collection window is reprocessed, and the cloud processing results are updated. During the feedback process, the continuously collected data within the restored locked acquisition window is passed along the processing dependency to the cloud processing location associated with the continuously collected data within the locked acquisition window. The corresponding cloud processing location is then reprocessed, and the cloud processing result generated by the reprocessing is used to update the corresponding cloud processing result. Release the lock on the corresponding acquisition window and corresponding propagation branch for cloud processing locations that have not yet been completed; The cloud processing location that is still incomplete is the cloud processing location that has not yet output the corresponding cloud processing result after reprocessing; when unlocking, the lock on the corresponding acquisition window and the corresponding propagation branch is canceled; The data continues to propagate backward from the unlocked acquisition window along the corresponding propagation branch.

[0035] In this embodiment, the result output module outputs the edge-cloud collaborative processing results, specifically including: The cloud processing results are mapped to the cloud processing location associated with the continuously collected data within the locked collection window, and the corresponding locked collection window is determined. When determining the corresponding locked acquisition window, the cloud processing result is mapped to the cloud processing position that forms the cloud processing result, and the locked acquisition window is located from the cloud processing position through the corresponding edge processing result along the processing dependency relationship; Replace the edge processing results connected to the corresponding cloud processing location with the cloud processing results, and retain the edge processing results that are not replaced; During replacement, the edge processing results connected to the corresponding cloud processing location are removed, and the removed edge processing results are replaced with cloud processing results. The edge processing results that are not replaced remain unchanged. Connect the cloud processing results and the edge processing results along the temporal relationship of the corresponding acquisition window; It aggregates cloud processing results and edge processing results, and outputs edge-cloud collaborative processing results.

[0036] The edge-cloud collaborative processing result consists of the cloud processing result connected according to the temporal relationship and the edge processing result that has not been replaced.

[0037] Example 1: To verify the feasibility of this invention in practice, it was applied to an industrial IoT device status acquisition and edge-cloud collaborative processing scenario, and a test environment corresponding to the actual edge nodes and cloud processing links was established. 120 IoT acquisition terminals were set up in the scenario, continuously generating device status data. Four processing nodes were set up on the edge side, and several processing locations were set up in the cloud. Continuously acquired data was written in the acquisition order, with each acquisition window forming a window of 2000 consecutive acquisition data entries. After adjacent acquisition windows were formed, the end acquisition data of the previous acquisition window and the beginning acquisition data of the next acquisition window were read. Using the location of the end acquisition data as the boundary, the beginning acquisition data was folded back to the previous acquisition window in its original acquisition order. The beginning acquisition data that maintained the continuous acquisition order after folding back was appended to the previous acquisition window; the beginning acquisition data that did not maintain the continuous acquisition order was sent back to the next acquisition window, and then the adjacent acquisition windows were reconnected to form a temporal relationship. Through this process, the division of acquisition windows did not change the original sequential order of the continuously acquired data.

[0038] Edge processing nodes process continuously acquired data according to the acquisition window and form edge processing results. To cover situations where the edge processing results do not completely correspond to the acquired content, some acquired content not covered by the edge processing results is retained during the test run. After the edge processing results are formed, they are mapped to the acquired data positions in the acquisition window. The acquired data positions covered by two adjacent edge processing results are locked, and the uncovered acquired content between them is truncated. The truncated acquired content is shrunk from both ends towards adjacent acquired data positions, maintaining the original acquisition order, and the moved acquired content at both ends is associated with the corresponding edge processing results. The remaining acquired content that has not been associated continues to shrunk to both sides until all uncovered acquired content is associated. The associated acquired content is arranged according to the acquired data position and forms differential data. The edge processing results retain the acquired content in the covered acquired data positions, providing corresponding data for the recovery of subsequent continuously acquired data.

[0039] In the cloud-based collaborative processing phase, edge processing results are connected to their corresponding cloud processing locations, and then each edge processing result is connected to its corresponding acquisition window. The established temporal relationships between acquisition windows are maintained. If several edge processing results are connected to the same acquisition window, the independent connections between each edge processing result and the cloud processing location are preserved, converging only at the acquisition window. The connections between cloud processing locations, edge processing results, and acquisition windows, along with the temporal relationships between acquisition windows, collectively form the processing dependency. During the test run, some cloud processing locations were prevented from outputting cloud processing results to create incomplete cloud processing locations. In 10 test groups, the number of incomplete cloud processing locations was controlled between 6 and 11.

[0040] The backpropagation module uses the incomplete cloud processing locations and processing dependencies to improve the propagation of the APPNP model. The APPNP model uses a personalized PageRank mechanism for adjacency propagation. In this embodiment, the propagation starting point is limited to the incomplete cloud processing location, and the propagation based on adjacency is adjusted to backpropagation along processing dependencies. Propagation passes from the incomplete cloud processing location to the corresponding edge processing result, then to the corresponding acquisition window. Upon reaching the acquisition window, it continues propagation along the temporal sequence to the preceding acquisition window. Different backpropagation paths are retained separately when they reach the same acquisition window; different backpropagation paths are not merged, and each backpropagation path forms a propagation branch. The propagation branch checks the acquired windows according to the backpropagation order. When it first reaches the acquisition window that retains the differential data, it locks the current acquisition window and the corresponding propagation branch, and stops the corresponding propagation branch from propagating to the preceding acquisition window. Propagation branches that have not reached the acquisition window that retains the differential data are retained as unlocked propagation branches and enter the next round of propagation.

[0041] After each round of backpropagation, locked propagation branches are removed from subsequent propagation, while unlocked propagation branches use the currently reached acquisition window as the starting point for the next round of propagation. If different unlocked propagation branches reach the same acquisition window that does not retain differential data, each propagation branch is still retained. Unlocked propagation branches terminate propagation and are removed after reaching the first acquisition window in the time sequence. The propagation process continues until no more unlocked propagation branches remain. Using this method, the backfilling range is no longer determined by all historical acquisition windows, but rather by the locked acquisition windows that each propagation branch first hits. During testing, as the amount of continuously acquired data increased, the number of locked acquisition windows increased from 11 to 21, still significantly less than the total number of acquisition windows.

[0042] During the data recovery phase, the locked acquisition windows are arranged according to their temporal sequence to form a data recovery range. Differential data and edge processing results corresponding to each locked acquisition window are extracted. The acquisition data positions covered by the edge processing results remain unchanged, and the differential data is filled into the positions not covered by the edge processing results according to their original acquisition data positions. The acquisition content retained in the edge processing results and the filled differential data are arranged according to the acquisition data positions within the locked acquisition windows, restoring the continuous acquisition data within the locked acquisition windows. The restored continuous acquisition data is then sent back to the associated cloud processing positions along the processing dependencies. Except for the cloud processing positions associated with the continuous acquisition data within the locked acquisition windows, other cloud processing positions stop reprocessing and retain their existing cloud processing results, performing only partial reprocessing on the associated positions. Positions that still do not output cloud processing results after reprocessing are unlocked, and the corresponding acquisition window and propagation branch are released from locking. Backward propagation continues from the unlocked acquisition window along the corresponding propagation branch.

[0043] In the output phase, the updated cloud processing results are mapped to the cloud processing locations that formed the results, and then the corresponding edge processing results are used to locate the locked acquisition window along the processing dependencies. Edge processing results connected to the corresponding cloud processing locations are replaced by the cloud processing results, while unreplaced edge processing results remain unchanged. The cloud processing results and unreplaced edge processing results are connected according to the sequential position of the corresponding acquisition windows in the time sequence, and aggregated to form the edge-cloud collaborative processing results. The control method uses conventional full-data recovery processing, i.e., after detecting incomplete cloud processing locations, a full retrieval and cloud reprocessing are performed on the continuously acquired data of the current test group, without first-hit locking of propagation branches, continuation of unlocked propagation branches, or freezing of unrelated cloud processing locations. For each test group, the amount of continuously acquired data, the number of incomplete cloud processing locations, the number of locked acquisition windows, the amount of recovered data, the time consumed by recovery and reprocessing, and the completeness rate of continuously acquired data recovery are recorded.

[0044] Table 1: Comparison of edge cloud interpolation and local reprocessing results under different acquisition loads In Table 1, the continuous data acquisition volume of the 10 test groups increased from 84,000 to 125,000 records, and the number of locations where cloud processing was not completed increased from 6 to 11. Under the same input conditions, the amount of data acquired using the conventional full-data acquisition method was basically the same as the continuous data acquisition volume of the current test group. This invention only performs acquisition on the locked acquisition window that is hit for the first time in the propagation branch, and the amount of acquired data is controlled between 22,000 and 42,000 records. Calculated based on 10 sets of data, the average amount of acquired data acquired by this invention is reduced by about 69.6%. The reduction rate of each group remained between 66.4% and 73.8%, indicating that after the scale of continuous data acquisition increases, the first-hit lock can still limit the range of data acquisition, and there is no situation where the acquisition range expands synchronously with all acquired data.

[0045] Regarding the time consumed for data recovery and reprocessing, the conventional full data recovery method increased from 15.8s to 23.3s, while this invention increased from 7.1s to 10.1s. Based on 10 sets of data, the average time was reduced by approximately 55.7%. This invention only reprocesses cloud processing locations associated with continuously acquired data within the locked acquisition window; other cloud processing locations retain their current cloud processing results, reducing redundant processing unrelated to the current recovery range. As the number of incomplete cloud processing locations increases, the processing time of this invention maintains a relatively gradual growth trend, and the local processing and freezing mechanisms can still control the reprocessing range under high data loads.

[0046] The completeness rate of continuously acquired data recovery ranged from 99.3% to 99.8% across 10 test groups, with an average of approximately 99.58%. The completeness rate did not decrease synchronously when the amount of supplemented data decreased significantly. Differential data retained the acquired content not covered by edge processing results and its corresponding acquisition data positions. During data recovery, this data was rearranged with the acquired content retained in the edge processing results according to the acquisition data positions, enabling the locked acquisition window to be restored to continuously acquired data. The data in the table shows that, under the conditions of this implementation, the first hit lock of the propagation branch, differential data supplementation, and local cloud reprocessing can maintain a high data recovery completeness while narrowing the supplementation range.

[0047] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A data acquisition and edge-cloud collaborative processing system for Internet of Things (IoT) scenarios, characterized in that, include: The data acquisition module is used to acquire continuously collected data, divide the acquisition windows, and connect adjacent acquisition windows to form a time sequence relationship; The edge processing module is used to process the edge processing results formed by the acquisition window, retain the differential data formed by the edge processing results not covering the acquisition content, and associate the edge processing results with the acquisition window; The cloud collaboration module is used to process edge processing results to form cloud processing results, lock the unfinished cloud processing locations, connect the cloud processing locations, edge processing results and acquisition windows, and connect the timing relationships to form processing dependencies. The back-determination module is used to input the cloud processing location and processing dependency into the improved APPNP model, and to form a propagation branch by back propagation along the processing dependency. When the propagation branch first reaches the acquisition window of the retained differential data, the acquisition window is locked, the locked propagation branch is terminated, and the unlocked propagation branch is retained to continue propagation. The data recovery module is used to gather the data recovery range formed by the locked acquisition window, combine the corresponding differential data and edge processing results, and recover the continuously acquired data within the locked acquisition window. The local processing module is used to reprocess the cloud processing positions associated with continuously collected data within the locked acquisition window, update the cloud processing results, and unlock the corresponding acquisition window and propagation branch for cloud processing positions that have not yet been completed, so as to continue reverse propagation. The results output module is used to summarize the edge processing results and the cloud processing results, and output the edge-cloud collaborative processing results.

2. The data acquisition and edge-cloud collaborative processing system for IoT scenarios according to claim 1, characterized in that, The data acquisition module forms a time-series relationship, specifically including: Arrange the continuously collected data in the order of collection, and divide the continuously collected data into segments to form a collection window; Lock the end and beginning data of adjacent acquisition windows, and use the end data as the boundary to push the beginning data back to the previous acquisition window, maintaining the original acquisition order of the beginning data; The first-end data that remains in continuous acquisition sequence with the previous acquisition window after being turned back is snapped to the previous acquisition window, and the first-end data that is not snapped back is sent back to the next acquisition window. Reconnect the previous acquisition window with the next acquisition window to establish a timing relationship.

3. The data acquisition and edge-cloud collaborative processing system for IoT scenarios according to claim 1, characterized in that, The edge processing module retains differential data formed by the edge processing results not covering the collected content, specifically including: The edge processing results are generated by processing the acquisition window, and the edge processing results are mapped to the acquisition data positions in the acquisition window; Lock the data collection locations covered by the edge processing results of two adjacent data collection locations, and extract the data collection content between the two data collection locations that is not covered by the edge processing results; The captured content is shrunk from both ends toward the adjacent data positions, while maintaining the original acquisition order before shrinking. The captured content after shrinking at both ends is then associated with the corresponding edge processing results. The remaining collected content that is not associated with the edge processing results continues to shrink to the adjacent collected data location until all uncovered collected content is associated, and the associated collected content is formed into differential data.

4. The data acquisition and edge-cloud collaborative processing system for IoT scenarios according to claim 1, characterized in that, The cloud-based collaboration module establishes processing dependencies, specifically including: Lock the unfinished cloud processing location and connect the edge processing results that participated in the cloud processing location to the corresponding cloud processing location; Connect each edge processing result to the corresponding acquisition window, and connect adjacent acquisition windows along the temporal sequence; Edge processing results connected to the same acquisition window retain their independent connections to the cloud processing location, allowing each independent connection to converge in the same acquisition window; Preserve the timing relationships between each independent connection and acquisition window to form processing dependencies.

5. A data acquisition and edge-cloud collaborative processing system for IoT scenarios according to claim 1, characterized in that, The backpropagation branch formed in the backpropagation determination module along the processing dependency relationship specifically includes: Starting from the location of the incomplete cloud processing, the propagation proceeds backward along the processing dependencies to the corresponding edge processing result. The edge processing results are then passed back to the corresponding acquisition window, so that the back transmission paths that reach the same acquisition window converge at the acquisition window, and each back transmission path is retained. The sequence continues to propagate backward from the preceding acquisition window, following the temporal relationship between acquisition windows. Each reverse propagation path is identified as a propagation branch.

6. The data acquisition and edge-cloud collaborative processing system for IoT scenarios according to claim 1, characterized in that, The backfilling determination module locks the acquisition window when the propagation branch first reaches the acquisition window for retaining differential data, specifically including: The propagation branches sequentially check the arrival of the acquisition windows. Propagation branches that have not reached the acquisition windows for retaining differential data continue to propagate along the processing dependencies. When the propagation branch first reaches the acquisition window for retaining differential data, the current acquisition window and the corresponding propagation branch are locked, and the locking of the propagation branch is terminated. Propagation branches that do not reach the acquisition window for retaining differential data are retained as unlocked propagation branches.

7. The data acquisition and edge-cloud collaborative processing system for IoT scenarios according to claim 1, characterized in that, The backfilling determination module retains the unlocked propagation branches for continued propagation, specifically including: After completing one round of back propagation, the locked propagation branches are removed from the next round of propagation, and only the unlocked propagation branches are retained; The current collection window reached by the unlocked propagation branch is used as the starting point for the next round of propagation, and the propagation continues in reverse along the processing dependency; When different unlocked propagation branches arrive at the same acquisition window where differential data is not retained, each unlocked propagation branch is retained separately and the unlocked propagation branches are not merged. For the first acquisition window in the arrival time sequence, the unlocked propagation branch is terminated and removed, and the reverse propagation is repeated until there are no more unlocked propagation branches.

8. A data acquisition and edge-cloud collaborative processing system for IoT scenarios according to claim 1, characterized in that, The data recovery module recovers continuously acquired data within the locked acquisition window, specifically including: The data acquisition windows are collected and arranged according to their temporal sequence to form a data recovery range. Extract the differential data and edge processing results corresponding to each locked acquisition window within the data recovery range; Locate the edge processing result within the data acquisition window, and fill the data acquisition locations not covered by the edge processing result with differential data. Restore continuously collected data within the locked collection window according to the data collection location.

9. A data acquisition and edge-cloud collaborative processing system for IoT scenarios according to claim 1, characterized in that, The local processing module updates the cloud processing results, specifically including: Locate the cloud processing location associated with continuously collected data within the locked acquisition window along the processing dependency relationship, and freeze the cloud processing locations other than those associated with continuously collected data within the locked acquisition window; The continuously collected data within the locked collection window is sent back to the cloud processing location associated with the continuously collected data within the locked collection window, the cloud processing location associated with the continuously collected data within the locked collection window is reprocessed, and the cloud processing results are updated. Release the lock on the corresponding acquisition window and corresponding propagation branch for cloud processing locations that have not yet been completed; The data continues to propagate backward from the unlocked acquisition window along the corresponding propagation branch.

10. A data acquisition and edge-cloud collaborative processing system for IoT scenarios according to claim 1, characterized in that, The result output module outputs the edge-cloud collaborative processing results, specifically including: The cloud processing results are mapped to the cloud processing location associated with the continuously collected data within the locked collection window, and the corresponding locked collection window is determined. Replace the edge processing results connected to the corresponding cloud processing location with the cloud processing results, and retain the edge processing results that are not replaced; Connect the cloud processing results and the edge processing results along the temporal relationship of the corresponding acquisition window; It aggregates cloud processing results and edge processing results, and outputs edge-cloud collaborative processing results.

Images