Information big data management system based on edge computing

Abstract

The present application relates to big data processing and edge computing technical field, specifically to information big data management system based on edge computing, including mutation detection module, skip list management module and decision execution module, the mutation detection module carries out "noise-silence" dual-state detection to big data stream in edge node, only intercepts mutation micro-slice at state conversion and calculates 128 bit SimHash mutation fingerprint; the skip list management module weaves the "mutation fingerprint-physical block" bidirectional skip list with reversible fuse mark on local PCM, realizes the index of mutation fingerprint to physical block address; the decision execution module takes the skip list root handle as the entrance, drives the "write once" type edge inference machine to generate the immutable decision frame, writes back the data source terminal through the low delay channel and updates the fuse state, so as to complete the closed loop processing of cold mutation data.

Description

Technical Field

[0001] This invention relates to the field of big data processing and edge computing technology, and in particular to an information big data management system based on edge computing. Background Technology

[0002] Big data and edge computing technologies are widely used in scenarios such as the Industrial Internet, smart cities, and financial risk control. Business systems continuously generate high-frequency data streams such as log streams, sensor streams, and transaction streams. Among these, the data that truly carries critical incremental information are often small in quantity but have a significant impact, such as sudden abnormal fluctuations during long-term stable operation, rare attack characteristic fragments, or short-term abnormal signals before equipment failure. To improve real-time performance and reduce the pressure on central cloud, the industry has begun to push some analytical capabilities down to edge nodes. At the edge, raw big data streams are preprocessed, filtered, and decisions are made and distributed, with the aim of quickly capturing critical anomalies and generating executable control instructions while minimizing bandwidth and storage resource consumption.

[0003] However, in existing technologies, edge big data management generally adopts the approach of uploading the entire raw big data stream at fixed intervals or fixed volumes, or coarsely extracting data fragments using simple threshold triggering. Most solutions lack the ability to accurately distinguish between "quiet states" and "noisy states" based on statistical characteristics. This results in cold mutation data either being submerged in massive amounts of normal data or the extracted fragments being lengthy and containing a lot of redundant information, placing a heavy burden on edge storage and computing. In terms of data organization and indexing, common practices often rely on sequential files, linear linked lists, or general tree index structures. Insertion and query performance are difficult to balance in high-concurrency scenarios. Access to non-volatile storage such as edge PCM often lacks lightweight structured management and fine-grained access control mechanisms that combine mutation fingerprints. At the decision execution level, existing solutions typically require multiple reads and writes of temporary intermediate results, lacking a "write-once, immutable" decision frame management method. They also lack circuit breaker control and automatic reset strategies that are tied to the state of a single mutation data point. This makes it difficult to complete a secure closed loop from mutation detection, data access, inference generation to decision write-back within millisecond-level latency, which can easily lead to problems such as repeated decisions, difficulty in tracing results, or continuous increase in the load on edge nodes. Summary of the Invention

[0004] This invention provides an information big data management system based on edge computing to solve the problems of difficulty in timely and accurate identification of cold mutation data, low efficiency of edge data storage and retrieval, and lack of security and controllability in decision-making closed loop.

[0005] An information big data management system based on edge computing includes a mutation detection module, a skip list management module, and a decision execution module, among which; The mutation detection module is used to perform "noise-silence" bi-state detection on the original big data stream at the edge node, and outputs a mutation micro-slice containing only "potential mutation segments" and its mutation fingerprint, wherein the mutation fingerprint is a 128-bit SimHash value. Based on the mutation fingerprint, the skip list management module instantly weaves a bidirectional skip list of "mutation fingerprint-physical block" in the local PCM storage area, and implants a reversible fuse mark in the skip list node to obtain the root handle of the skip list with the fuse mark. The decision execution module is used to start the "write-once" edge inference engine with the root handle of the skip list as the key, directly generate and latch executable decision frames, and write the decision frames back to the data source terminal to complete the millisecond-level closed loop of cold mutation data on the edge side.

[0006] Optionally, the mutation detection module includes: The original big data stream is continuously extracted based on a sliding window of preset duration to obtain a series of window data segments arranged in chronological order; By calculating the statistical entropy of each of the window data segments, a statistical entropy sequence synchronized with the timeline of the original large data stream is generated; Based on the statistical entropy sequence, each statistical entropy value is compared with a predefined silent threshold and a noise threshold to determine the state of the corresponding window data segment: if the statistical entropy value is consistently lower than the silent threshold, it is marked as "silent state"; if the statistical entropy value exceeds the noise threshold, it is marked as "noisy state". By monitoring the state marking results, the transition point from "silent state" to "noise state" in the original big data stream is identified, and this transition point is precisely defined as the starting boundary of a potential mutation segment; Based on the starting boundary, a fixed-length continuous data segment is extracted from the original big data stream, and this data segment is encapsulated into a complete mutation microslice; By performing a 128-bit SimHash algorithm on the mutant microslice, a unique mutant fingerprint is generated, thus completing the pairing and output of the mutant microslice and its mutant fingerprint.

[0007] Optionally, the silence threshold and the noise threshold are dynamic thresholds obtained based on statistical analysis of the long-term historical statistical entropy sequence of the original big data stream.

[0008] Optionally, the condition for determining that the value is continuously lower than the threshold is: the statistical entropy value of N consecutive window data segments is lower than the silent threshold, where N≥2.

[0009] Optionally, the skip list management module includes: The mutation fingerprint is received from the mutation detection module, and the physical block address associated with the mutation fingerprint and stored in the local PCM storage area is obtained to form an initial mutation fingerprint-physical block mapping relationship. Based on the mutation fingerprint-physical block mapping relationship, a new skip list node is created. This skip list node stores at least the mutation fingerprint, the corresponding physical block address pointer, and forward and backward pointers for constructing a doubly linked list. Based on the numerical value of the mutation fingerprint, the new skip list node is inserted into the ordered structure of the existing "mutation fingerprint-physical block" bidirectional skip list, and the forward and backward pointers between adjacent nodes are updated to maintain the order and traversability of the bidirectional skip list. In the newly inserted skip list node, a state field is initialized as a reversible circuit breaker flag, and the initial state of the reversible circuit breaker flag is set to "on". After completing node insertion and reversible circuit breaker implantation, update and return a reference to the highest-level head node of the bidirectional skip list of the "mutation fingerprint-physical block" as the root handle of the skip list with circuit breaker obtained in this operation.

[0010] Optionally, the mutation fingerprint-physical block mapping relationship is obtained by querying a hash table maintained in the local PCM storage area, which uses the mutation fingerprint as the key and the corresponding physical block address as the value.

[0011] Optionally, the new skip list node is inserted into the ordered structure of the existing "mutation fingerprint-physical block" bidirectional skip list. Specifically, starting from the highest level of the bidirectional skip list, the insertion position is searched layer by layer by comparing the numerical values ​​of the mutation fingerprint, and the forward and backward pointers of the new skip list node and its adjacent nodes are updated after a suitable position is found at each level.

[0012] Optionally, the decision execution module includes: Receive the root handle of the skip list with the fuse flag from the skip list management module, and use the root handle to access the corresponding "mutation fingerprint-physical block" bidirectional skip list; Based on the root handle of the skip list with the fuse mark, locate the target skip list node in the bidirectional skip list of "mutation fingerprint-physical block" and check the reversible fuse mark status in the target skip list node; if the status is "on", extract the physical block address stored in the node and read the corresponding physical block data from the PCM storage area according to the physical block address. Using the read physical block data as input, the "write-once" edge inference engine is started to execute a preset inference algorithm and generate an executable decision frame with immutable content and a locked logical address. The generated executable decision frame is transmitted back to the data source terminal that produced the original big data stream; After confirming that the executable decision frame has been successfully written back to the data source terminal, the reversible circuit breaker flag status of the corresponding target skip table node in the bidirectional skip table of the "mutation fingerprint-physical block" is updated to complete the millisecond-level closed-loop processing of this cold mutation data on the edge side.

[0013] Optionally, accessing the corresponding "mutation fingerprint-physical block" bidirectional skip list using the skip list root handle specifically includes: parsing the skip list root handle with the fuse flag, obtaining the reference it contains to the highest-level head node of the bidirectional skip list, and verifying the validity of the reference.

[0014] Optionally, checking the reversible fuse flag status in the target skip list node specifically involves: when the reversible fuse flag status is detected as "fuse-out", pausing the reading operation of physical block data and waiting until the status is restored to "conducting" or a preset timeout mechanism is triggered.

[0015] The beneficial effects of this invention are: 1. This invention performs "noise-silent" bi-state detection on the original big data stream at the edge node based on statistical entropy using a mutation detection module. It extracts mutation micro-slices of a fixed time length only in the neighborhood of the transition point from "silent state" to "noise state" and calculates a 128-bit SimHash mutation fingerprint for each mutation micro-slice. This achieves fine screening and high compression of cold mutation data, significantly reduces the storage and computing load on the edge side, and improves the detection sensitivity and positioning accuracy of low-frequency abnormal events.

[0016] 2. This invention uses a skip list management module to instantly weave a bidirectional skip list of "mutation fingerprint-physical block" in the local PCM storage area. It uses a hash table to establish a mapping between mutation fingerprints and physical block addresses, and constructs a skip list node with a hierarchical array, forward and backward pointers, timestamps, and reversible circuit breaker flags for each mutation fingerprint. Then, it generates multi-level forward links through a probabilistic algorithm and outputs the root handle of the skip list with circuit breaker flags. This enables fast retrieval and insertion of cold mutation data with O(logN) complexity, while supporting fine-grained dynamic control of access permissions for individual physical blocks, significantly improving the controllability and maintainability of cold mutation data management.

[0017] 3. This invention, through a decision execution module using the root handle of a skip list with a circuit breaker flag as the entry point, locates the target skip list node with a reversible circuit breaker flag marked "on" in the bidirectional skip list of "mutation fingerprint-physical block". It then loads the PCM physical block data into a "write-once" edge inference engine using direct memory access. A lightweight machine learning model embedded in read-only storage is invoked to generate an executable decision frame with immutable content and a logically locked address. This frame is then appended with a digital signature and timestamp and written back to the data source terminal via a dedicated low-latency communication channel. Finally, the reversible circuit breaker flag of the corresponding node is switched to "frozen" and an automatic reset mechanism is used. This achieves one-time secure decision-making and millisecond-level closed-loop processing for the same cold mutation data, ensuring the uniqueness, security, and traceability of edge-side decisions. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the system flow according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the skip list management module according to an embodiment of the present invention. Detailed Implementation

[0020] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0021] It should be noted that the use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0022] Generally, terms can be understood at least partly from their use in context. For example, depending at least partly on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or a combination of features, structures, or characteristics in a plural sense. Additionally, the term "based on" can be understood not necessarily to convey an exclusive set of factors, but rather, alternatively, depending at least partly on the context, to allow for the presence of other factors that are not necessarily explicitly described.

[0023] like Figures 1-2 As shown, the information big data management system based on edge computing includes a mutation detection module, a skip list management module, and a decision execution module, wherein; The mutation detection module is used to perform "noise-silence" bi-state detection on the raw big data stream at the edge node, and outputs a mutation micro-slice containing only "potential mutation segments" and its mutation fingerprint. The mutation fingerprint is a 128-bit SimHash value. The specific steps are as follows: 1. The original large data stream is continuously truncated using a sliding window of preset duration, resulting in a series of window data segments arranged in chronological order, specifically: In the edge node's processor, a preset duration parameter is pre-configured for each data type of the raw big data stream, and the number of data points contained in each sliding window is calculated based on the sampling rate of the raw big data stream. Upon receiving the raw big data stream, the edge node first caches the data in a circular buffer according to its timestamp order. Then, with the preset duration as the window length, it sequentially reads the corresponding number of data points starting from the beginning of the buffer, encapsulating this segment into the first window data segment. Subsequently, the starting position of the window is shifted backward by the number of data points corresponding to one window length, ensuring no overlap with the previous window data segment, and the same number of data points are read again from the new starting position to form the second window data segment. The edge node repeats this truncation process until the remaining data length in the buffer at the current moment is insufficient to form a complete window data segment. This constructs a series of continuous, non-overlapping, chronologically ordered window data segments on the timeline, and each window data segment is appended with its starting timestamp and its starting position index in the raw big data stream.

[0024] 2: By calculating the statistical entropy of each window data segment, a statistical entropy sequence synchronized with the timeline of the original large data stream is generated, specifically as follows: For each obtained window data segment, the edge node calculates the corresponding statistical entropy value according to the definition of the Shannon entropy formula. To do this, firstly, within the current window data segment, according to the data encoding method of the original big data stream, the data points are mapped to several discrete symbols or feature values, and the frequency of occurrence of each symbol or feature value in the window data segment is counted to obtain the frequency of occurrence of each symbol or feature value.

[0025] Subsequently, the frequency of each symbol or feature is divided by the total number of data points within the window segment to obtain the probability of its occurrence within the window. Based on the definition of Shannon entropy, each edge node is used as input to calculate its contribution to the statistical entropy, and all contributions are summed to obtain the statistical entropy value for that window segment.

[0026] To ensure the stability and repeatability of the calculation process, edge nodes use a fixed logarithmic base when performing the above calculations, and set a lower limit for extremely small probability values ​​to avoid numerical underflow.

[0027] Finally, the edge nodes bind the statistical entropy value of each window to the corresponding window start timestamp according to the order of the window data segments on the timeline, generate a statistical entropy sequence synchronized with the timeline of the original big data stream, and store it in the local cache for subsequent steps.

[0028] 3. Based on the statistical entropy sequence, each statistical entropy value is compared with a predefined silent threshold and a noise threshold to determine the state of the corresponding window data segment: if the statistical entropy value is consistently below the silent threshold, it is marked as "silent state"; if the statistical entropy value exceeds the noise threshold, it is marked as "noisy state". Specifically: During the system initialization phase, edge nodes extract a representative sample data from the long-term historical data of the original big data stream, calculate the long-term historical statistical entropy sequence, and perform statistical analysis on the long-term historical statistical entropy sequence to obtain the statistical characteristic interval reflecting the normal silent conditions and normal noise conditions.

[0029] Based on this statistical characteristic interval, the edge nodes select a silence threshold and a noise threshold to distinguish between silent and noisy states, and store these two thresholds as dynamic thresholds in a configuration table. During system operation, the edge nodes can periodically update the long-term historical statistical entropy sequence in a sliding manner, thereby re-evaluating and adjusting the silence threshold and noise threshold in a timely manner to keep the thresholds adaptive to the current data distribution.

[0030] In the actual judgment process, for each statistical entropy value in the statistical entropy sequence, the edge node first compares it with the currently effective silent threshold and noise threshold respectively.

[0031] To determine the silent state, the edge node maintains a silent counter. Whenever a new statistical entropy value is lower than the silent threshold, the silent counter is incremented by one; when the statistical entropy value is not lower than the silent threshold, the silent counter is cleared to zero.

[0032] When the value of the silence counter reaches the preset number of consecutive windows N and N≥2, the edge node will mark the current window data segment and the N-1 window data segments before it as "silent state".

[0033] For the determination of noise state, when the statistical entropy value exceeds the noise threshold, the edge node immediately marks the current window data segment as "noise state" and records the time index and sequence number of the window.

[0034] For statistical entropy values ​​between the silent threshold and the noise threshold, edge nodes remain in an undefined state until the statistical entropy value of a subsequent window satisfies the criteria for either a "silent state" or a "noisy state".

[0035] Through the aforementioned continuous comparison and counting mechanism, edge nodes establish a stable "quiet state" or "noise state" state label sequence for each window data segment without changing the statistical entropy sequence itself.

[0036] 4. By monitoring the state marking results, identify the transition point in the original big data stream from "silent state" to "noisy state," and precisely define this transition point as the starting boundary of a potential mutation segment, specifically: After completing the state marking, the edge node traverses the state marking sequence according to the time order of the window data segments. For the k-th window data segment, the edge node reads its state marking and the state marking of the (k-1)-th window data segment. When it detects that the state marking of the (k-1)-th window data segment is "silent" and the state marking of the k-th window data segment is "noise", it is determined that a state transition from "silent" to "noise" has occurred between the (k-1)-th and k-th window data segments.

[0037] The edge node records the transformation event as a potential mutation event and uses the start timestamp of the k-th window data segment and its starting position index in the original big data stream as the starting boundary of the potential mutation segment.

[0038] To avoid repeatedly generating multiple potential mutation segments when multiple consecutive window data segments are marked as "noise state", after the edge node identifies a transition point from "silent state" to "noise state", when it continues to scan backward, it only generates a new potential mutation segment starting boundary when the first transition from "silent state" to "noise state" occurs again.

[0039] By using this state-marked sequence-based monitoring method, edge nodes can accurately locate the first occurrence of each cold mutation event and obtain a clear starting boundary for subsequent mutation microslices.

[0040] 5: Based on the initial boundary, extract a fixed-length continuous data segment from the original large data stream and encapsulate this data segment into a complete mutation microslice, specifically: Once the starting boundary of a potential mutation segment is determined, the edge node locates the specific data point corresponding to that starting boundary in the original data buffer based on the recorded starting timestamp and starting position index. For each data type of the original big data stream, the edge node pre-sets a fixed time length parameter according to the sampling rate of that data type and the observation requirements for mutation behavior characteristics, and converts this fixed time length into the corresponding number of data points.

[0041] Subsequently, the edge node reads a number of consecutive data points corresponding to a fixed time length, starting from the data point corresponding to the starting boundary, and treats this continuous data as the original data content of a complete mutant microslice.

[0042] When encapsulating mutant microslices, edge nodes not only store the aforementioned continuous data points themselves, but also additionally record the start timestamp, end timestamp, start position index, and data type identifier of the mutant microslice, forming a structured mutant microslice data unit.

[0043] If, during actual operation, it is found that the data length from the starting boundary to the end of the current buffer is insufficient to constitute a complete fixed time length, the edge node can temporarily suspend encapsulation until subsequent data arrives and meets the fixed time length requirement, thereby ensuring the consistency of each mutation microslice in terms of time length and data volume.

[0044] Finally, the edge nodes cache the encapsulated mutation microslices in their local queue for processing, providing input for subsequent mutation fingerprint calculations.

[0045] 6: By performing a 128-bit SimHash algorithm on the mutant microslice, a unique mutant fingerprint is generated, ultimately completing the pairing and output of the mutant microslice and its mutant fingerprint. Specifically: After the edge node retrieves a mutation microslice from the queue to be processed, it first maps the continuous data in the mutation microslice into a series of feature terms for fingerprint calculation according to the data organization method of the original big data stream.

[0046] For numerical data, edge nodes can be converted from continuous numerical sequences into several discrete feature terms through fixed-length segmentation, differential mode encoding, or symbolic mode encoding. For character-based or structured data, edge nodes can be segmented according to field boundaries or semantic boundaries to obtain a stable set of feature items.

[0047] Subsequently, the edge nodes assign a weight value corresponding to the importance of each feature item according to the preset weight allocation rules, and input each feature item into the pre-selected hash function to obtain a hash vector component with a length of 128 bits.

[0048] In the above process, the set of features extracted from a single mutant microslice is denoted as ,in This indicates the number of features contained in the current mutant microslice, and each feature... Corresponding to a weight value For each feature term Through a pre-selected hash function Calculate its 128-bit hash vector: ; in, Representing characteristic terms In the The binary hash result of the bits.

[0049] For each feature item's hash vector component, the edge nodes perform a weighted summation according to its weight value, summing the weighted results of all feature items bit by bit to form a 128-dimensional weighted summation vector. To do this, first, each bit of the binary hash result... Mapped to symbolic value Its definition is: ; Then, for each bit dimension Calculate the weighted sum of all feature terms along this dimension: ; in, The weighted accumulation vector is represented at the th... The value in the dimension.

[0050] Next, the edge node performs a sign function check on each dimension of the weighted accumulation vector: if the accumulation result in that dimension is greater than zero, the corresponding bit is set to 1; if the accumulation result in that dimension is less than or equal to zero, the corresponding bit is set to 0. That is, the... Bit mutation fingerprint bit Defined as: ; Through the bitwise processing of the above sign function, the edge node obtains a binary sequence of length 128 bits: ; This binary sequence is the mutation fingerprint that uniquely corresponds to the current mutation microslice. For the overall mutation fingerprint, For mutation fingerprints in the first The bit value at each bit.

[0051] After completing the mutation fingerprint calculation, the edge node associates and binds the mutation fingerprint with the corresponding mutation microslice to form a pair of records consisting of the mutation microslice and the mutation fingerprint. This pairing record is then used as the output of the mutation detection module and passed to the skip list management module through the internal interface, thereby completing the pairing and output of the mutation microslice and its mutation fingerprint.

[0052] Based on mutation fingerprints, the skip list management module in real-time weaves a bidirectional skip list of "mutation fingerprint-physical block" in the local PCM storage area, and implants reversible fuse-breaking flags in the skip list nodes to obtain the root handle of the skip list with fuse-breaking flags. The specific steps are as follows: 1. Receive the mutation fingerprint from the mutation detection module and obtain the physical block address associated with the mutation fingerprint, stored in the local PCM storage area, to form the initial mutation fingerprint-physical block mapping relationship, specifically: After receiving the mutation microslices and their mutation fingerprints output by the mutation detection module through the internal interface, the skip list management module first initiates a query operation on the hash table maintained in the local PCM storage area for the mutation fingerprint.

[0053] The hash table uses a key-value storage structure, where the key is the 128-bit SimHash value corresponding to the mutation fingerprint, which is uniformly encoded and used as the hash key, and the value is the physical block address that corresponds one-to-one with the mutation fingerprint.

[0054] The skip list management module performs a hash function calculation on the mutation fingerprint, locates the corresponding bucket position in the hash table, and then traverses the stored key-value entries in that bucket, comparing the encoded value of the mutation fingerprint with the key value in the entry in turn. When a key value that is exactly the same is found, the physical block address recorded in that entry is read and cached.

[0055] If the hash table does not contain an entry corresponding to the mutation fingerprint, the skip table management module allocates a new physical block in the PCM storage area according to a predetermined strategy, writes the mutation microslice content to the address of the physical block, and inserts a record in the hash table with the mutation fingerprint as the key and the address of the newly allocated physical block as the value.

[0056] Through the above queries and, when necessary, creation operations, the skip list management module forms a unique mutation fingerprint-physical block mapping relationship in its internal memory, corresponding to the current mutation fingerprint, providing basic data for the creation of subsequent skip list nodes.

[0057] 2. Based on the mutation fingerprint-physical block mapping relationship, create a new skip list node. This skip list node should at least store the mutation fingerprint, the corresponding physical block address pointer, and forward and backward pointers for constructing a doubly linked list, specifically: The skip list management module allocates structured storage space in memory for new skip list nodes, and sets mutation fingerprint field, physical block address pointer field, hierarchical array field, forward and backward pointer set field, and timestamp field in the structure.

[0058] The mutation fingerprint field is used to store the obtained mutation fingerprint values ​​so that they can be located in an ordered manner during comparison and traversal.

[0059] The physical block address pointer field is used to store the address of the corresponding physical block in the local PCM storage area, so that subsequent modules can directly access the mutation microslice data through this pointer.

[0060] The level array field is used to store the forward link pointers of the skip list node at each level. For each level, one element is allocated in the level array, which stores the forward pointer address pointing to the subsequent node in the same level.

[0061] The forward and backward pointer set fields are used to construct the doubly linked list structure on the base layer. At each level, a forward pointer and a backward pointer are maintained to point to the predecessor node and successor node in the linked list of this level.

[0062] When a node is created, the timestamp field is read by the skip list management module from the current system time and written into the node structure. This is used to record the time when the mutation fingerprint-physical block mapping relationship was created. Subsequently, when the node is accessed, the skip list management module can update the timestamp field to the most recent access time, thereby supporting a time-based maintenance strategy.

[0063] With the initialization of the above fields, a skip list node containing mutation fingerprint, physical block address pointer, multi-level forward link structure and time information is fully constructed, ready to be inserted into the "mutation fingerprint-physical block" bidirectional skip list.

[0064] 3. Based on the numerical value of the mutation fingerprint, insert the new skip list node into the existing ordered structure of the "mutation fingerprint-physical block" bidirectional skip list, and update the forward and backward pointers between adjacent nodes to maintain the order and traversability of the bidirectional skip list. Specifically: Before inserting a new skip list node, the skip list management module first generates a random number of layers for the node based on a probability algorithm. This number of layers does not exceed the preset maximum level limit.

[0065] For each generated layer, the module allocates a corresponding forward pointer slot in the node's hierarchy array to build a multi-layer forward link structure.

[0066] Subsequently, the skip list management module starts from the head node of the current highest level of the bidirectional skip list "mutation fingerprint-physical block", compares the new mutation fingerprint with the mutation fingerprints of the nodes being traversed one by one, and moves backward along the forward pointer at that level until it finds the insertion interval boundary where the mutation fingerprint value at that level is not greater than the current node and is greater than the previous node.

[0067] Upon reaching this position, the module sets the forward pointer of the new skip list node to point to the original successor node at this level, and sets the backward pointer of the new skip list node to point to the original predecessor node. At the same time, it updates the forward pointer of the predecessor node to point to the new skip list node, and updates the backward pointer of the successor node to point to the new skip list node.

[0068] After completing the insertion at the current level, the skip list management module switches down to the next level and performs the same insertion operation on the forward pointer slot of the corresponding level in the new node level array as on the current level linked list. By going down level by level, the new skip list node completes the access of forward and backward pointers on all participating levels.

[0069] The forward and backward pointers at the base level are used to maintain the continuous and ordered traversal capability of the entire "mutation fingerprint-physical block" bidirectional skip list at the lowest level, ensuring that all nodes can be accessed sequentially through the base level.

[0070] Through the above-described layer-by-layer insertion and pointer update process, the skip list management module inserts new skip list nodes into the ordered structure of the bidirectional skip list without disrupting the original order, while maintaining the traversability of the entire skip list at all levels.

[0071] 4: In the newly inserted skip list node, initialize a status field as a reversible circuit breaker flag, and set the initial state of this reversible circuit breaker flag to "on", specifically: The skip list management module reserves a reversible circuit breaker flag field inside the new skip list node structure. This field is implemented as a binary flag that occupies a single bit or a single byte.

[0072] After the node insertion operation is completed, the module immediately sets the value of the flag to the encoded value that indicates "on", so as to make it clear that the node is currently allowed to be accessed by the decision execution module.

[0073] The reversible circuit breaker flag has two states: "on" and "off". In the "on" state, any traversal operation initiated based on the skip list root handle can continue to access the mutation microslice data in the local PCM storage area along the physical block address pointer when it locates the node. When subsequent feedback from the decision execution module or control instructions from the upper-level policy engine necessitate pausing access to physical blocks corresponding to certain mutation fingerprints, the skip list management module can set the reversible circuit breaker flag status in these nodes to "circuit breaker". When a node marked as "circuit breaker" is detected during traversal, the physical block access process of that node is skipped directly, retaining only the structural link relationship, thereby achieving fine-grained control over the access permissions of individual nodes.

[0074] Since the circuit breaker flag is reversible, the skip list management module can switch the flag status from "fuse-out" to "on" again after receiving the instruction to restore access, thus restoring the access capability of the physical block corresponding to the node.

[0075] 5: After completing node insertion and reversible circuit breaker flag implantation, update and return the reference to the highest-level head node of the bidirectional skip list of this "mutation fingerprint-physical block", as the root handle of the skip list with circuit breaker flag obtained in this operation, specifically: After completing the multi-level insertion of new nodes and the initialization of reversible circuit breaker flags in nodes, the skip list management module updates the metadata of the overall structure of the current "mutation fingerprint-physical block" bidirectional skip list.

[0076] The module internally maintains a metadata record about the skip list, which includes at least the maximum level of the current skip list, the total number of nodes, and a reference to the head node of the highest level.

[0077] When a new skip list node is successfully inserted, the module first increments the total number of nodes by one, and then determines whether the maximum level field needs to be updated based on the relationship between the new node level and the original maximum level.

[0078] When the number of new node levels exceeds the original maximum level, the module creates a new head node above the highest level and sets the new head node as the new highest level head node, and uses the original head node as the head node of its next level, thereby adjusting the hierarchical structure of the entire skip list.

[0079] Subsequently, the skip list management module packages the reference pointing to the current highest-level head node together with the updated maximum level and total number of nodes to form a skip list root handle with a circuit breaker flag. This skip list root handle is returned to the decision execution module as the output of this module.

[0080] When the decision execution module executes the "write-once" edge inference engine, it can use the root handle of the skip list as the only entry point. By using the highest-level head node reference and metadata information contained in the root handle, it can efficiently traverse and access the entire "mutation fingerprint-physical block" bidirectional skip list, thereby completing the rapid location of the physical block indexed by the mutation fingerprint and the reading of decision data.

[0081] The decision execution module uses the skip list root handle as a key to start a "write-once" edge inference engine, directly generate and latch executable decision frames, and write the decision frames back to the data source terminal, completing the millisecond-level closed loop of cold mutation data at the edge. The specific steps are as follows: 1. Receive the root handle of the skip list with a circuit breaker flag from the skip list management module, and use the root handle to access the corresponding "mutation fingerprint-physical block" bidirectional skip list, specifically: The decision execution module receives the root handle of the skip list with a circuit breaker flag from the skip list management module through an internal function call interface. The root handle of the skip list is a structured data object, which contains at least metadata information such as a reference to the head node of the highest level of the "mutation fingerprint-physical block" bidirectional skip list, the maximum level of the current skip list, and the total number of nodes.

[0082] The decision execution module first performs an integrity check on the root handle, including checking the version number field of the root handle, checking whether the maximum level recorded in the root handle is within the preset upper limit, and checking whether the total number of nodes is a non-zero value.

[0083] Subsequently, the decision execution module parses the highest-level head node reference from the root handle, performs an access test on the memory address pointed to by the reference, reads the basic identification fields of the node at that address, such as the skip list head node mark, the level identifier field, the null pointer mark field, etc., and determines whether the reference is valid based on the preset set of valid values.

[0084] When the identifier field of the head node is found to be consistent with the preset specification, the decision execution module registers the reference as the entry point of the bidirectional skip list of "mutation fingerprint-physical block" in the current session. Subsequently, when locating the target skip list node, the highest-level head node corresponding to the reference is used as the starting position for multi-level traversal.

[0085] If the reference validation fails, the decision execution module marks the root handle as invalid, reports the error status to the upper-level control logic, and stops the current decision execution process to avoid accessing the corrupted skip list structure.

[0086] 2: Based on the root handle of the skip list with the fuse flag, locate the target skip list node in the "mutation fingerprint-physical block" bidirectional skip list, and check the reversible fuse flag status in the target skip list node; if the status is "on", extract the physical block address stored in the node, and read the corresponding physical block data from the PCM storage area according to the physical block address, specifically: After the root handle is successfully resolved, the decision execution module, based on the maximum level recorded in the root handle, starts from the head node of the highest level and moves forward along the forward pointer at each level according to a predetermined traversal strategy.

[0087] During the traversal process, the module can determine the specific target skip list node to be located based on the sorting strategy of the mutation fingerprints to be processed and the scheduling result of the current processing queue. For example, it can select the mutation fingerprint corresponding to the head of the queue, select a specific mutation fingerprint according to the priority table, or scan sequentially.

[0088] During the localization process, the decision execution module compares the value of the mutation fingerprint stored in the current node with the value of the target mutation fingerprint at each level. When it finds that the mutation fingerprint value of the current node is not less than the target mutation fingerprint value, it switches to the next level and continues to compare with the current node as the new starting point until the target skip list node with the mutation fingerprint equal to the target mutation fingerprint is located at the baseline level.

[0089] After locating the target skip list node, the decision execution module reads the reversible circuit breaker flag field in that node to determine if the current state is "on". If the state is "frozen", the decision execution module immediately suspends physical block access operations on that node, adds the node to a waiting list, and starts a status polling mechanism to reread the reversible circuit breaker flag status of the node at fixed time intervals.

[0090] When the marked state is found to be restored to "conducting" within the preset timeout period, the decision execution module retrieves the node from the waiting list and continues subsequent processing; if the timeout period is still "circuit broken", the module marks the node as an unavailable node and reports to the upper layer control logic that the current cold mutation data cannot complete the closed-loop processing.

[0091] When the reversible fuse flag is in the "on" state, the decision execution module reads the physical block address pointer field from the target skip list node, writes the physical block address as a parameter into the input configuration register of the "write-once" edge inference engine, and initiates a data transfer operation from the PCM storage area to the edge inference engine input buffer through the edge node's direct memory access controller. This direct memory access operation reads the physical block data in the PCM storage area sequentially according to the physical block address and physical block length at the hardware level, avoiding the central processing unit's involvement in data transfer byte by byte, thus improving read efficiency.

[0092] After the data transfer is completed, the decision execution module sets a valid input buffer flag on the edge inference engine side to indicate that the corresponding physical block data has been fully loaded.

[0093] 3: Using the read physical block data as input, the "write-once" edge inference engine is started to execute a preset inference algorithm, generating an executable decision frame with immutable content and a locked logical address, specifically: Once the input buffer valid flag of the "write-once" edge inference engine is set, the decision execution module sends a start command to the edge inference engine. This command contains a preset inference algorithm identifier, an input data length identifier, and a logical address identifier for the output area.

[0094] The default inference algorithm is a lightweight machine learning model. This model has been compiled into an execution unit adapted to the edge inference machine instruction set during the system deployment phase and is embedded in the read-only memory area inside the edge inference machine. The model parameters and structure cannot be modified during runtime.

[0095] After receiving the start command, the edge inference engine loads the weight parameters and network structure definition of the lightweight machine learning model from the read-only storage area, reads physical block data from the input buffer layer by layer according to the preset data flow path, and inputs it into each computing unit in sequence to complete the processing processes such as feature extraction, pattern recognition, risk assessment or control command generation. Finally, it generates a structured decision content in the internal output buffer.

[0096] After confirming the completion of the inference process, the decision execution module reads the decision content from the edge inference engine output buffer and allocates a logically address-locked storage area in the non-volatile storage region of the edge node for the executable decision frame. This storage area is allocated with write permissions restricted to a one-time write mode; after a successful write, any overwrite or erase operations are prohibited.

[0097] The decision execution module writes the decision content in the output buffer into the non-volatile storage area locked by the logical address in one go, and generates a timestamp field to record the generation time of the executable decision frame.

[0098] Subsequently, the decision execution module calls the encryption module of the edge node to perform encryption operations on the executable decision frame content and timestamp field using the edge node's private key, thereby obtaining the corresponding digital signature.

[0099] Ultimately, the executable decision frame is solidified in a non-volatile storage area locked by a logical address in the form of "decision content + timestamp + digital signature", forming an executable decision frame object with immutable content and integrity verification capability.

[0100] 4: Transmit the generated executable decision frame back to the data source terminal that produced the original big data stream, specifically: After obtaining the executable decision frame object, the decision execution module reads the complete content of the decision frame, as well as the digital signature and timestamp fields, from the non-volatile storage area locked by the logical address, and constructs a decision delivery message based on the address information of the data source terminal.

[0101] The message contains an identification field, a source identification field, a destination identification field, an executable decision frame payload field, and a check field.

[0102] The decision execution module selects a pre-configured dedicated low-latency communication channel, encapsulates the message into link layer frames, network layer data packets, and transport layer data segments according to the layering requirements of the communication protocol stack, and sends it to the network access point where the data source terminal is located through the corresponding physical interface.

[0103] During transmission, the decision execution module enables the transmission sequence number and retransmission timer to perform a limited number of retransmissions in the event of momentary link jitter or packet loss.

[0104] When the data source terminal receives the message, it verifies the integrity and authenticity of the executable decision frame content, digital signature and timestamp contained therein, and returns a receipt confirmation to the edge node after the verification is successful.

[0105] The receipt confirmation includes the original transmission sequence number and confirmation flag fields. After receiving the receipt confirmation, the decision execution module compares the confirmation with the locally maintained transmission sequence number to confirm that the executable decision frame has been reliably received by the target data source terminal and is ready for execution.

[0106] If no valid confirmation is received within the preset waiting time interval, the decision execution module can resend the executable decision frame according to the preset retransmission strategy until the maximum number of retransmissions is reached. If no confirmation is received at this time, the transmission will be marked as a failure and an exception will be reported to the upper-layer control logic.

[0107] 5. After confirming that the executable decision frame has been successfully written back to the data source terminal, update the reversible circuit breaker flag status of the corresponding target skip table node in the "mutation fingerprint-physical block" bidirectional skip table, completing the millisecond-level closed-loop processing of this cold mutation data at the edge side, specifically: Once the decision execution module confirms that the executable decision frame has been successfully written back to the data source terminal based on the received confirmation receipt, it retrieves the target skip list node identifier corresponding to this decision execution from the local record, and locates the target skip list node again through the aforementioned root handle and the highest level head node reference.

[0108] After successful location, the decision execution module modifies the reversible circuit breaker flag field in the node, updating its status from "conducting" to "circuit breaker". At the same time, it reads the current system time and writes it into the status change timestamp field of the node to record the specific time when the node enters the "circuit breaker" state.

[0109] At this point, for subsequent traversal operations initiated based on the same skip list root handle, when the target skip list node is accessed again and the reversible circuit breaker flag status check is performed, it will be found that the node is in a "circuit breaker" state, thereby automatically pausing the reading of its physical block data and preventing the repeated execution of reasoning and decision-making processes on the same cold mutation data.

[0110] After the status update, the decision execution module records the identifier of the target skip list node and the corresponding executable decision frame logical address into a closed-loop management table. This closed-loop management table is used to track the entire execution status of each cold mutation data from detection, skip list weaving to decision frame generation and writing back to the data source terminal.

[0111] After the upper-level control logic or data source terminal in the system successfully executes the executable decision frame instruction and generates the execution result, it can send the execution result back to the edge node through the preset feedback interface. After receiving the execution result and confirming that the execution was successful, the decision execution module finds the corresponding target skip list node. Under the premise of satisfying the preset security policy, it resets the reversible circuit breaker flag status of the node from "circuit breaker" to "on", so that access capability can be restored when additional analysis of the same physical block is needed in the future.

[0112] If no execution result feedback is received within the preset automatic reset time window, the decision execution module will automatically trigger the reset operation of the reversible circuit breaker mark of the node when the time window is reached, according to the automatic reset strategy.

[0113] Through the aforementioned state update and automatic reset mechanisms, the decision execution module achieves complete closed-loop control of cold mutation data from detection, decision-making to feedback, and ensures that the response and processing of mutation data are completed at the edge with a millisecond-level latency.

[0114] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0115] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An information big data management system based on edge computing, characterized in that, It includes a mutation detection module, a skip list management module, and a decision execution module, among which; The mutation detection module is used to perform "noise-silence" bi-state detection on the original big data stream at the edge node, and outputs a mutation micro-slice containing only "potential mutation segments" and its mutation fingerprint, wherein the mutation fingerprint is a 128-bit SimHash value. Based on the mutation fingerprint, the skip list management module instantly weaves a bidirectional skip list of "mutation fingerprint-physical block" in the local PCM storage area, and implants a reversible fuse mark in the skip list node to obtain the root handle of the skip list with the fuse mark. The decision execution module is used to start the "write-once" edge inference engine with the root handle of the skip list as the key, directly generate and latch executable decision frames, and write the decision frames back to the data source terminal to complete the millisecond-level closed loop of cold mutation data on the edge side.

2. The information big data management system based on edge computing according to claim 1, characterized in that, The mutation detection module includes: The original big data stream is continuously extracted based on a sliding window of preset duration to obtain a series of window data segments arranged in chronological order; By calculating the statistical entropy of each of the window data segments, a statistical entropy sequence synchronized with the timeline of the original large data stream is generated; Based on the statistical entropy sequence, each statistical entropy value is compared with a predefined silent threshold and a noise threshold to determine the state of the corresponding window data segment: if the statistical entropy value is consistently lower than the silent threshold, it is marked as "silent state"; if the statistical entropy value exceeds the noise threshold, it is marked as "noisy state". By monitoring the state marking results, the transition point from "silent state" to "noise state" in the original big data stream is identified, and this transition point is precisely defined as the starting boundary of a potential mutation segment; Based on the starting boundary, a fixed-length continuous data segment is extracted from the original big data stream, and this data segment is encapsulated into a complete mutation microslice; By performing a 128-bit SimHash algorithm on the mutant microslice, a unique mutant fingerprint is generated, thus completing the pairing and output of the mutant microslice and its mutant fingerprint.

3. The information big data management system based on edge computing according to claim 2, characterized in that, The silence threshold and the noise threshold are dynamic thresholds obtained by statistical analysis of the long-term historical statistical entropy sequence of the original big data stream.

4. The information big data management system based on edge computing according to claim 2, characterized in that, The condition for determining that the value is continuously lower than the threshold is: the statistical entropy value of N consecutive window data segments is lower than the silent threshold, where N≥2.

5. The information big data management system based on edge computing according to claim 2, characterized in that, The skip list management module includes: The mutation fingerprint is received from the mutation detection module, and the physical block address associated with the mutation fingerprint and stored in the local PCM storage area is obtained to form an initial mutation fingerprint-physical block mapping relationship. Based on the mutation fingerprint-physical block mapping relationship, a new skip list node is created. This skip list node stores at least the mutation fingerprint, the corresponding physical block address pointer, and forward and backward pointers for constructing a doubly linked list. Based on the numerical value of the mutation fingerprint, the new skip list node is inserted into the ordered structure of the existing "mutation fingerprint-physical block" bidirectional skip list, and the forward and backward pointers between adjacent nodes are updated to maintain the order and traversability of the bidirectional skip list. In the newly inserted skip list node, a status field is initialized as a reversible circuit breaker flag, and the initial state of the reversible circuit breaker flag is set to "on". After completing node insertion and reversible circuit breaker flag implantation, update and return a reference to the highest-level head node of the bidirectional skip list of the "mutation fingerprint-physical block" as the root handle of the skip list with circuit breaker flag obtained in this operation.

6. The information big data management system based on edge computing according to claim 5, characterized in that, The mutation fingerprint-physical block mapping relationship is obtained by querying a hash table maintained in the local PCM storage area, which uses the mutation fingerprint as the key and the corresponding physical block address as the value.

7. The information big data management system based on edge computing according to claim 5, characterized in that, Inserting the new skip list node into the ordered structure of the existing "mutation fingerprint-physical block" bidirectional skip list specifically includes: starting from the highest level of the bidirectional skip list, searching for the insertion position layer by layer by comparing the numerical values ​​of the mutation fingerprint, and updating the forward and backward pointers of the new skip list node and its adjacent nodes after finding a suitable position at each level.

8. The information big data management system based on edge computing according to claim 5, characterized in that, The decision execution module includes: Receive the root handle of the skip list with the fuse flag from the skip list management module, and use the root handle to access the corresponding "mutation fingerprint-physical block" bidirectional skip list; Based on the root handle of the skip list with the fuse mark, locate the target skip list node in the bidirectional skip list of "mutation fingerprint-physical block" and check the reversible fuse mark status in the target skip list node; if the status is "on", extract the physical block address stored in the node and read the corresponding physical block data from the PCM storage area according to the physical block address. Using the read physical block data as input, the "write-once" edge inference engine is started to execute a preset inference algorithm and generate an executable decision frame with immutable content and a locked logical address. The generated executable decision frame is transmitted back to the data source terminal that produced the original big data stream; After confirming that the executable decision frame has been successfully written back to the data source terminal, the reversible circuit breaker flag status of the corresponding target skip table node in the "mutation fingerprint-physical block" bidirectional skip table is updated to complete the millisecond-level closed-loop processing of this cold mutation data at the edge side.

9. The information big data management system based on edge computing according to claim 8, characterized in that, The step of using the root handle of the skip list to access the corresponding "mutation fingerprint-physical block" bidirectional skip list specifically includes: parsing the root handle of the skip list with the circuit breaker flag, obtaining the reference it contains to the highest-level head node of the bidirectional skip list, and verifying the validity of the reference.

10. The information big data management system based on edge computing according to claim 8, characterized in that, The process of checking the reversible fuse flag status in the target skip list node specifically involves: when the reversible fuse flag status is detected as "fuse-out", pausing the reading operation of physical block data and waiting until the status is restored to "conducting" or a preset timeout mechanism is triggered.

Images