Data processing method and system based on AI chip

Abstract

This invention provides a data processing method and system based on an AI chip, relating to the field of electronic digital data processing technology. The method includes: acquiring the current data unit in a target data stream; calculating its data similarity C with previous data units; and calculating the entropy decay index D between adjacent computational layers in the layer-by-layer processing of a neural network. {N} The entropy decay index D {N} Used to quantify the degree of decrease in information uncertainty; the data similarity C and entropy decay index D {N} Perform a joint judgment; when C is higher than the similarity threshold and D... {N} When the data falls below the decay threshold, processing of subsequent computational layers is interrupted, and the feature data of the current layer is discarded as redundant data. This invention achieves dynamic pruning of the computational path by constructing a dual verification criterion of input similarity and entropy change in the model's internal processing. This reduces redundant computation and improves the processing efficiency and energy efficiency of AI chips without sacrificing risk identification accuracy.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, specifically to a data processing method and system based on an AI chip. Background Technology

[0002] With the booming development of smart cities and ubiquitous sensing networks, video surveillance systems have become indispensable nerve endings in social security, industrial production, and urban management. Behind this, artificial intelligence (AI) technology, with deep neural networks at its core, is endowing massive amounts of video data with unprecedented depth and breadth, enabling it to "see" and "understand." AI chips, as the computing power foundation of all this, are driving an industrial revolution from "seeing clearly" to "understanding."

[0003] However, beneath this feast of computing power lies a tremendous challenge. Video data streams possess inherent spatiotemporal continuity, meaning they contain a massive amount of redundant information. Surveillance cameras often capture long periods of static scenes or periodic, minute changes—for example, an empty street at midnight, the shadows of trees swaying gently in the wind, or a silent corridor in an office building. These "silent moments," which constitute the vast majority of the total data volume, form a surging "digital torrent."

[0004] Existing processing paradigms mostly employ a brute-force computing strategy that treats everything the same. Regardless of whether the scene is calm or turbulent, AI chips must perform a complete and complex neural network forward propagation on every frame of data. This "computationally incompatible" processing method applies high-intensity deep learning to undifferentiated temporal data, resulting in astonishing energy waste, high storage costs, and unnecessary hardware load. On the other hand, while traditional simple methods such as motion detection can filter out some static images, they cannot handle pseudo-changes such as gradual changes in light and shadow movement, nor can they understand the deep semantics of the scene, easily confusing genuine, subtle signs of risk with meaningless background noise.

[0005] Therefore, there is a need in this field for a dynamic, adaptive computing mechanism that can intelligently "brake" or "accelerate" based on the information evolution of the data content itself, thereby promoting the optimization of system energy efficiency and response speed while ensuring that key information is not missed.

[0006] The information disclosed in the background section above is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The purpose of this invention is to provide a data processing method and system based on AI chips to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a data processing method based on an AI chip, the specific steps of which include: acquiring a target data stream containing multiple data units;

[0009] For the current data unit, calculate its data similarity C with the previous data units in the target data stream;

[0010] The current data unit is processed by forward propagation layer by layer through multiple computational layers of the neural network model;

[0011] After forward propagation to the Nth computational layer, the information entropy H of the set of current-level feature data generated by the Nth computational layer in response to the input data is calculated. {N} And obtain the information entropy H of a set of previous layer feature data generated by the (N-1)th computation layer and used as input to the Nth computation layer. {N-1} ;

[0012] Based on H {N} With H {N-1} Calculate the entropy decay exponent D {N} Entropy decay exponent D {N} Used to quantify the degree of reduction in information uncertainty during the processing from layer N-1 to layer N;

[0013] Regarding the entropy decay exponent D {N} When combined with data similarity C, the entropy decay index D is used for joint judgment. {N} If the data is below the preset decay threshold and the data similarity C is above the preset similarity threshold, the processing of subsequent calculation layers is interrupted, the current level feature data is removed as redundant data, and the final processing result is marked as a regular safety label. Otherwise, the current level feature data is retained, and the final processing result is marked as a change risk label.

[0014] A data processing system based on an AI chip, the system being used to execute the aforementioned data processing method based on an AI chip, including:

[0015] The data stream acquisition module is configured to acquire a target data stream containing multiple data units;

[0016] The data similarity calculation module is configured to calculate the data similarity C between the current data unit and previous data units in the target data stream;

[0017] The neural network processing module is configured to perform layer-by-layer forward propagation processing on the current data unit through multiple computational layers within it.

[0018] The dynamic monitoring and entropy calculation module, connected to the neural network processing module, is configured to perform real-time calculation of the information entropy H of the current level feature data generated by the Nth computational layer during the forward propagation process. {N} Obtain the information entropy H of the feature data from the previous layer generated by the (N-1)th computation layer. {N-1} And calculate the entropy decay exponent D{N};

[0019] The intelligent decision-making and control module, connected to the data similarity calculation module and the dynamic monitoring and entropy calculation module respectively, is configured to monitor the entropy decay index D. {N} It performs a joint judgment with the data similarity C, and generates processing control instructions based on the judgment results;

[0020] The data management module receives and processes control commands and is configured to perform one of the following operations:

[0021] When the control instruction is interrupted, the processing of subsequent computation layers is interrupted, the feature data of the current layer is removed as redundant data, and the final processing result is marked as a regular security label.

[0022] When the processing control instruction is to continue processing, the current level feature data is retained, the processing of subsequent calculation layers continues, and the final processing result is marked as a change risk label.

[0023] Compared with the prior art, the beneficial effects of the present invention are: the present invention constructs a dual verification criterion of input data similarity and entropy change in the internal processing of the neural network, and simultaneously identifies redundant data units that do not require deep processing in real time, and further dynamically prunes the subsequent neural network calculation path, thereby saving ineffective computing resources and improving the overall processing throughput and energy efficiency of the AI ​​chip.

[0024] This invention combines data similarity, which characterizes the external correlation of data, with the entropy decay index, which quantifies the stability of feature extraction within the model. This avoids the risk of misjudging data by skipping processing simply because of superficial similarity. It further ensures that effective deep feature extraction can still be performed on highly disguised and deep-level abnormal data, achieving an effective balance between computational efficiency and risk identification accuracy, and enhancing the reliability and robustness of the system.

[0025] This invention adds a hardware cache query mechanism based on spatiotemporal hash values ​​before neural network processing, and uses the combination of locality-sensitive hashing and temporal context to accurately identify highly repetitive data sequences. This further enables the system to achieve hardware-level fast hit and result reuse for the vast majority of stable and unchanging background data, and completely skips the entire neural network processing flow. As a result, when processing extremely redundant data streams, the computation latency is reduced to the nanosecond level, and the power consumption of AI chips is minimized.

[0026] This invention integrates three mechanisms: hardware cache pre-filtering, task importance-based forced processing, and dynamic pruning of the model's internal state. It also constructs a multi-layered, progressive processing architecture that progresses from "hardware-level coarse screening" to "task-level fine-tuning" and then to "model-level fine-tuning." This further enables the allocation of computing resources to be dynamically adjusted in a refined and multi-dimensional manner based on the novelty of the data, the importance of the content, and the effectiveness of the model processing. This results in a highly adaptive, hierarchically optimized, and intelligent data processing system that balances extreme efficiency with absolute reliability. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the overall application process of the present invention;

[0028] Figure 2 This is a schematic diagram illustrating the execution logic of the data processing method steps of the AI ​​chip of the present invention;

[0029] Figure 3 This is a schematic diagram of the execution logic of the "state significance coefficient S" and "spatiotemporal hash value" of the present invention;

[0030] Figure 4 This is a schematic diagram of the system flow of the present invention. Detailed Implementation

[0031] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0032] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0033] Example 1:

[0034] Please see Figures 1 to 4 The present invention provides a technical solution:

[0035] The data processing method based on AI chips includes the following steps: acquiring a target data stream containing multiple data units;

[0036] For the current data unit, calculate its data similarity C with the previous data units in the target data stream;

[0037] The steps for calculating data similarity C are specifically configured as follows:

[0038] Obtain the first pixel matrix of the current data unit and the second pixel matrix of the previous data unit;

[0039] Based on the first pixel matrix and the second pixel matrix, the mean μ and standard deviation σ of each are calculated respectively, and the covariance σxy of the first pixel matrix and the second pixel matrix is ​​calculated, thereby constructing a statistical feature set containing brightness, contrast and structural information;

[0040] Then, the statistical feature set is numerically stabilized by the stabilization constant injection method. The mean μ, standard deviation σ and covariance σxy are combined with the preset stabilization constants w1, w2 and w3 respectively to generate luminance component, contrast component and structure component with numerical robustness.

[0041] Finally, a multiplicative fusion operation is performed on the luminance component, contrast component, and structure component after stability processing. The calculation results of the three components are multiplied one by one to obtain a scalar output, which is then used as the data similarity C to characterize the degree of consistency between the contents of the two data units.

[0042] The current data unit is processed by forward propagation layer by layer through multiple computational layers of the neural network model;

[0043] The steps for building a neural network model are as follows: setting up a network topology containing multiple computational layers, initializing the learnable parameters in the network topology, using a preset training dataset related to the target processing task, iteratively training the network topology, terminating training when the performance index of the neural network model reaches the preset convergence criterion, and deploying the trained neural network model containing fixed learnable parameters into the AI ​​chip for subsequent layer-by-layer forward propagation processing.

[0044] After forward propagation to the Nth computational layer (N is an integer greater than 1), the information entropy H of the set of current-level feature data generated by the Nth computational layer in response to the input data is calculated. {N} And obtain the information entropy H of a set of previous layer feature data generated by the (N-1)th computation layer and used as input to the Nth computation layer. {N-1} ;

[0045] Entropy decay exponent D {N} The method for obtaining this information is as follows: calculate the absolute decay of information entropy from layer N-1 to layer N. The absolute decay of information entropy is equal to the information entropy H of the feature data from the previous layer. {N-1} Information entropy H of the current level feature data {N} difference;

[0046] The absolute decay of information entropy is normalized by dividing the absolute decay by the information entropy H of the previous level feature data, which serves as the baseline. {N} To obtain a dimensionless relative decay rate;

[0047] The relative decay rate obtained after normalization is assigned the entropy decay exponent D. {N} .

[0048] Based on H {N} With H {N-1} Calculate the entropy decay exponent D {N} Entropy decay exponent D {N} Used to quantify the degree of reduction in information uncertainty during the processing from layer N-1 to layer N;

[0049] Regarding the entropy decay exponent D {N} When combined with data similarity C, the entropy decay index D is used for joint judgment. {N} If the data is below the preset decay threshold and the data similarity C is above the preset similarity threshold, the processing of subsequent calculation layers is interrupted, the current level feature data is removed as redundant data, and the final processing result is marked as a regular safety label. Otherwise, the current level feature data is retained, and the final processing result is marked as a change risk label.

[0050] Further explanation: Before making a joint judgment, obtain the state significance coefficient S, which is used to characterize the importance level of the target data stream processing task;

[0051] A preset significance threshold is set. When the state significance coefficient S is higher than the significance threshold, the entropy decay exponent D is ignored. {N} The result of the data similarity C judgment is used to ensure that the processing of subsequent calculation layers is not interrupted.

[0052] If the state significance coefficient S is not lower than the significance threshold, the processing interruption decision is determined to be false, and the processing of subsequent calculation layers continues.

[0053] If the state significance coefficient S is lower than the significance threshold, then it is further determined whether the data similarity C is higher than the similarity threshold and whether the entropy decay index H... {N} If the threshold is lower than the decay threshold, and both conditions are met simultaneously, the interruption decision is determined to be true.

[0054] Further explanation: Before making a joint judgment, the methods for obtaining the state significance coefficient S include:

[0055] A pre-defined list of content to be monitored is used to establish a mapping relationship between content identifiers and basic salience scores; content identifiers include at least one or more of the following: specific personnel, smoke, abnormally opened access control or unauthorized devices;

[0056] Perform content parsing processing on the target data stream to identify and extract one or more current content objects contained in the data stream; perform a matching query between the current content object and the list of content to be followed to determine whether the current content object matches any content identifier in the list;

[0057] Based on the results of the matching query, determine the state significance coefficient S. The output score of the state significance coefficient S is a standardized interval of [0,1].

[0058] If the query is successful, the basic significance score corresponding to the content identifier of the successful query is obtained as the state significance coefficient S.

[0059] If the query does not find a match, the preset default significance score is used as the state significance coefficient S.

[0060] Further explanation: Before performing layer-by-layer forward propagation processing on the current data unit, the method also includes:

[0061] Calculate the spatiotemporal hash value of the current data unit. The spatiotemporal hash value is generated based on the spatial characteristics of the current data unit and the temporal context of the previous data unit in the target data stream.

[0062] The query step in the hardware cache includes: for the spatiotemporal hash value, if the query hits, directly obtain a set of cached data pre-stored in the current hardware cache, remove the current level feature data as redundant data, mark the final processing result as a regular security label, and skip the layer-by-layer forward propagation processing and subsequent joint judgment steps; if the query misses, proceed to the layer-by-layer forward propagation processing and subsequent joint judgment steps.

[0063] The specific steps for generating a spacetime hash value include:

[0064] Calculate the spatial hash value of the current data unit using the locality-sensitive hash function;

[0065] The spatial hash value is combined with the stored historical hash value corresponding to the previous data unit by a preset bitwise operation to generate a spatiotemporal hash value.

[0066] The query steps specifically include: calculating the Hamming distance between the spatiotemporal hash value of the current data unit and each spatiotemporal hash value stored in the hardware cache; when any Hamming distance is lower than a preset distance threshold, it is determined that the query has been hit, and the pre-stored calculation result corresponding to the spatiotemporal hash value with the smallest Hamming distance is obtained; when all Hamming distances are not lower than the preset distance threshold, it is determined that the query has not been hit, and the process proceeds to the layer-by-layer forward propagation process and subsequent joint judgment steps.

[0067] The following are specific implementation instructions for the above content: Please refer to [link / reference]. Figures 1 to 4 The present invention provides a specific implementation scheme for a data processing method based on an AI chip.

[0068] The application scenario in this embodiment is set as follows: In an office, a surveillance camera at a fixed angle is continuously pointed at a wall with a unique texture. The system's preset thresholds are as follows:

[0069] Hash Hamming distance threshold: 3; Data similarity threshold C thresh : 0.95; Entropy decay exponent threshold D thresh : 0.05; State significance threshold S thresh 0.7;

[0070] Data similarity threshold C thresh With entropy decay threshold D thresh The joint calibration method is as follows:

[0071] The data similarity threshold is denoted as Cthresh: it represents a preset threshold used to determine whether the current data unit is highly consistent with the previous data unit in terms of content. In this embodiment, a preferred value is 0.95. The entropy decay index threshold is denoted as Dthresh: it represents a preset threshold used to determine whether the information entropy of the neural network has stabilized and no longer significantly decreased when processing data. In this embodiment, a preferred value is 0.05. C thresh With D thresh Together, they constitute the core of the dynamic computational pruning mechanism of this invention, and there is a close mutual constraint between them: higher C thresh and lower D thresh Combining data makes the system tend to classify more data as redundant, thereby maximizing computational efficiency. However, this may lead to the risk of missing highly deceptive data (i.e., data that appears highly similar on the surface but contains subtle anomalies that require deep networks to detect). Conversely, lower C... thresh and higher D thresh Combining these elements makes the system extremely sensitive, with most data undergoing full-depth processing. While this provides high security, it severely diminishes the beneficial effects of this invention in terms of efficiency optimization.

[0072] Therefore, the optimal values ​​for these two parameters were obtained through joint calibration experiments of the following systems:

[0073] This embodiment prepares a 100-hour public video surveillance dataset, which covers long periods of static background (redundant data) and various types of unexpected events, such as pedestrian intrusion, abnormal vehicle parking, and abandoned objects (non-redundant / risk data). To simulate high-level camouflage risk, image processing techniques are used to artificially inject different levels of minute perturbations into some static video clips, such as: subtle color changes at the pixel level, flickering of individual pixels, and slow movement of extremely small background objects. In offline data analysis, a data similarity threshold C is set. threshThe search range is [0.90, 0.99], with a step size of 0.01; the entropy decay exponent threshold D is set. thresh The search range is [0.01, 0.10], with a step size of 0.01. For each parameter pair (C thresh D thresh The entire dataset is used as input to evaluate the system's decision-making behavior.

[0074] The following two key indicators are statistically analyzed:

[0075] False-Negative Rate: The proportion of risky data frames containing artificially injected minor disturbances that are incorrectly identified as redundant by the system and prematurely interrupted.

[0076] Redundant-Computation-Rate: The proportion of all real static background redundant data frames that were processed to full depth without triggering an interrupt mechanism.

[0077] Ultimately, a parameter pair was selected that could control the false negative rate below 0.1% while minimizing redundant computation. Experimental results show that when C thresh =0.95 and D thresh When the value is 0.05, the system achieves the best balance between the two indicators mentioned above.

[0078] The method for calibrating the hash Hamming distance threshold is as follows: Let the hash Hamming distance threshold be denoted as H. thresh The hash Hamming distance threshold is used to determine whether the perceptual hash values ​​of two data units are sufficiently close during coarse-grained pre-screening; in this embodiment, a preferred value is 3. The hash Hamming distance threshold is used in computationally expensive C... thresh Previously, nearly identical data units were quickly filtered out. The choice of its value also involved a trade-off:

[0079] Lower thresholds (such as 0 or 1) are too strict, potentially passing data units that should be skipped to subsequent processing due to single pixel noise in data transmission, thus reducing the efficiency of pre-screening. Higher thresholds (such as greater than 5) are too lenient, potentially filtering out data units with subtle but meaningful changes, increasing the risk of missed detections.

[0080] The calibration method is as follows: 100,000 consecutive data frames were randomly selected from the above dataset and divided into two groups: "redundant frame pairs" and "variable frame pairs". The perceptual hash Hamming distance of each pair of data frames was calculated, and its probability distribution histogram was plotted. Analysis revealed that the Hamming distances of "redundant frame pairs" were mostly distributed in the [0,2] interval, while the Hamming distances of "variable frame pairs" were mainly distributed above 5. thresh=3 is the optimal split point between these two distributions. It can filter out redundant frames with an accuracy of over 99.5%, while ensuring that the probability of changing frames being incorrectly filtered is less than 0.01%.

[0081] The method for calibrating the state significance threshold is as follows: Let the state significance threshold be denoted as St. hresh The risk score, representing whether the final output risk score of a data unit after complete processing is sufficient to be labeled as a "change risk label," is a threshold valued at 0.7 in this embodiment. The state significance threshold is denoted as St. hresh The sensitivity of the system's final alarm is determined by this threshold. Its calibration is a typical classification threshold selection problem: a low state saliency threshold will make the system overly sensitive, easily reporting harmless normal fluctuations as risks, resulting in a large number of false alarms. A high state saliency threshold will make the system insensitive, potentially missing some real threats with unclear risk characteristics, leading to false negatives. The calibration method is as follows: using a fully trained neural network model, all data in the validation set are processed to obtain a risk score for each data unit. Based on the true labels of the validation set (i.e., whether it is a real risk event), the receiver operating characteristic curve (ROC-Curve) of the system is plotted. The ROC-Curve shows the relationship between the system's true positive rate and false positive rate under different thresholds. thresh =0.7 was selected as the preferred value because it corresponds to the "inflection point" on the ROC curve that is closest to the top left corner (i.e., perfect classification). At the "inflection point", the system can obtain the highest possible true positive rate while maintaining a low false positive rate, thus achieving the best balance between alarm accuracy and coverage.

[0082] The event description for Scenario 1 is set as follows: Between time t0 and t1, there is no activity in the office, the lighting is stable, and there is only slight noise from the sensor itself.

[0083] Execution steps: The system acquires each frame of the previous data unit at time t0 and the current data unit at time t1, and calculates the spatiotemporal hash value;

[0084] The system already stores the spatiotemporal hash value at time t0. Obtain the 64-bit binary string: H t0 =10110010...11001011.

[0085] The system uses the locality-sensitive hash function to calculate the spatial hash value of the data unit at time t1. Since the image remains almost unchanged, its spatial hash value is close to that at time t0.

[0086] Through bitwise operations (with H) t0(Perform an XOR operation) to generate the spatiotemporal hash value H at time t1. t1 =10110010...11001010 (only the last digit changes due to noise);

[0087] System calculation H t1 H stored in hardware cache t0 The Hamming distance between them. Hamming distance = 1. The Hamming distance (1) is lower than the preset Hamming distance threshold (3), so it is determined as "query hit". The system immediately interrupts all subsequent calculation processes. Directly retrieve the value of H from the cache. t0 The associated pre-stored results (e.g., a low-dimensional feature vector representing "no change in scene").

[0088] The current result is marked as a "regular security label" and output. Through a hash caching mechanism, the system successfully identifies the extremely high redundancy between consecutive frames, skipping the costly neural network processing and saving the computing power and power consumption of the AI ​​chip.

[0089] The event description for Scenario 2 is as follows: At time t2 (previous data unit), indoor lighting is normal. At time t3 (current data unit), due to external clouds blocking the sun, the overall indoor lighting intensity decreases uniformly, causing the walls to appear darker in the camera's view.

[0090] The execution steps are as follows: Hash cache miss: Due to changes in illumination, a large number of pixel values ​​change. The Hamming distance between the newly generated spatiotemporal hash value and the hash value in the cache is much greater than 3. The query misses, the process continues, and the system extracts the 3x3 pixel matrix at times t2 and t3 as samples.

[0091] Previous data unit (time t2, normal illumination): Second pixel matrix (y): [[130,135,132],[128,132,138],[134,129,131]];

[0092] Current data unit (time t3, when the light dims): First pixel matrix (x): [[110,115,112],[108,112,118],[114,109,111]];

[0093] x represents the "current data unit": that is, the data acquired at the current time (e.g., t3) that needs to be analyzed. In this embodiment, it corresponds to the first pixel matrix, which is the pixel data after the illumination has darkened.

[0094] y represents "previous data unit": that is, the data from the previous time step (e.g., t2) that serves as a comparison benchmark. In this embodiment, it corresponds to the second pixel matrix, i.e., the pixel data under normal lighting conditions.

[0095] Example of data similarity C calculation process: Obtain the first pixel matrix and the second pixel matrix, calculate the mean μ and standard deviation σ of each character, and calculate the covariance σ between the two. xy Construct a statistical feature set;

[0096] μ x =112.11; μ y =133.11; Standard deviation: σ x =2.96; σ y =2.96; Covariance: σ xy =8.765;

[0097] The statistical feature set is processed using a stabilization constant injection method to generate three components: brightness, contrast, and structure. Specifically:

[0098] When calculating the components, the stabilization constants w1, w2, and w3 are explicitly set as small positive numbers to prevent the denominator from being zero.

[0099] The value of the first stable constant w1 is determined based on the dynamic range of the pixel data. Specifically, Where L is the dynamic range of the pixel data, and K1 is a preset first coefficient. For an 8-bit image, L = 255. K1: In a preferred embodiment, K1 = 0.01. In a specific embodiment, the pixel data is 8-bit grayscale data with a dynamic range of 255. In this case, the first stable constant w1 is 6.5025.

[0100] Second stable constant The value of is determined based on the dynamic range of the pixel data; specifically, Where L is the dynamic range of the pixel data. This is a preset second coefficient. In a preferred embodiment, K2 = 0.03.

[0101] In a specific embodiment, when the pixel data is 8-bit grayscale data, its dynamic range L is 255. At this time, the second stable constant... The value is .

[0102] The value of the third stability constant w3 is the same as that of the second stability constant. Related, specifically .

[0103] In a specific embodiment, when the pixel data is 8-bit grayscale data, based on the second stable constant... The value of is 58.5225. At this point, the third stable constant is... The value is 58.5225 / 2 = 29.26125.

[0104] Luminance component (l): Luminance component: 0.9856;

[0105] Contrast component (c): Contrast component: 1.0;

[0106] Structural components (s): Structural component: 1.0;

[0107] A multiplicative fusion operation is performed, multiplying the three component results to obtain the final data similarity C. Data similarity C = l × c × s = 0.9856 × 1.0 × 1.0 = 0.9856; the final calculated data similarity C is 0.9856. This is a very high similarity value, conveying the following accurate information to the monitoring system: the brightness has indeed changed: the brightness component l is 0.9856, less than 1, which accurately captures the fact that the overall scene has darkened (the mean decreased from 133.11 to 112.11).

[0108] Contrast and structure remain unchanged: both the contrast component c and the structure component s are 1.0, indicating that despite the change in brightness, the internal contrast (light and dark contrast) and spatial structure (texture direction) of the wall texture are completely consistent with the previous moment.

[0109] Conclusion: Non-critical event. In summary, the algorithm can "understand" that this is merely a change in lighting, rather than someone moving or an object being placed in front of a wall. Therefore, the monitoring system can use this high similarity value of 0.9856 to determine that the current frame change is a "non-threatening" normal environmental fluctuation, thus avoiding triggering unnecessary motion detection alarms.

[0110] Neural network processing and entropy calculation: The current data unit is fed into a pre-trained neural network model (specifically the Res-Net model) deployed on an AI chip for processing.

[0111] When processing the (N-1)th layer, the output feature data of the current layer represents the depth perception of the wall texture. Its information entropy H is calculated. {N-1} =4.25 bits. Bits are the basic unit of information entropy, used to quantify the complexity and uncertainty of the internal state of the feature data output by the (N-1)th layer of a neural network model, encoded using an average of 4.25 binary bits;

[0112] The propagation continues forward along the current layer's feature data to layer N. Since changes in illumination are low-level information and have been largely filtered out in deeper networks, the network still consistently identifies this as "wall texture," with only a slight decrease in information uncertainty. The calculated information entropy H of layer N is... {N} =4.18 bits.

[0113] Calculate the entropy decay exponent D {N} =(H {N-1} -H {N} ) / H {N-1} =(4.25-4.18) / 4.25=0.0165.

[0114] Judgment 1: Data similarity C(0.9856) > C thresh (0.95), the result is true, which indicates that despite pixel-level changes, the macroscopic structure and content of the scene remain highly consistent.

[0115] Judgment 2: Entropy decay exponent D {N} (0.0165) < D thresh (0.05), the judgment result is true, indicating that the information uncertainty of the neural network decreases slowly during the deep feature extraction process, and no mutation information that needs further analysis is found.

[0116] If both conditions are met, the specific processing control instructions are as follows: the system interrupts the processing of subsequent computation layers of the neural network, removes the feature data generated by the Nth layer as redundant data, and marks the final processing result as a "regular safety label".

[0117] Setting the event description for Scenario 3: Event description: At time t4, there is no one in front of the wall. At time t5, a person wearing specific work clothes (preset as "content of interest") enters the frame.

[0118] Execution steps:

[0119] Hash cache miss: The scenario has changed drastically, and hash lookups will inevitably miss.

[0120] Obtain the state significance coefficient S: While performing similarity and entropy calculations, the system performs content parsing in parallel.

[0121] The content analysis module identifies a "human-shaped" target in the image and further matches it with its "specific work clothes" feature;

[0122] This example is based on a pre-defined "list of content of interest," which pre-configures basic salience scores for different content identifiers. When the system parses specific content from the target data stream (such as monitoring footage), it will directly query this table to obtain its corresponding status salience coefficient S. The value of S ranges from [0,1]. The higher the value, the higher the importance level and the higher the processing priority. The list of content of interest is shown in Table 1 below:

[0123]

[0124] When a task begins: the system first parses the content of the input image or data stream.

[0125] Query and Assignment: If the parsing result is "Specific personnel detected in the core computer room area", the system queries Table 1 and matches rule number 3, then the S-value for this task is assigned a value of 0.95. If the parsing result is "Normal screen, no special events", the system queries Table 1 and does not match any rules numbered 1-4, then the default rule number 5 is applied, and the S-value for this task is assigned a value of 0.1. The current feature matches the "Specific Personnel" entry in the "List of Contents of Interest", and the basic significance score for the specific personnel is 0.95. Therefore, the system determines the significance coefficient S of the current state to be 0.95.

[0126] Priority handling for joint judgment: When the system enters the joint judgment module, it first checks the S value.

[0127] Judgment: Significance coefficient S(0.95) > S thresh (0.7), the judgment result is true, indicating that the change in the current scene is not a normal environmental fluctuation or general activity, but has triggered a "high-value attention event" (i.e., the appearance of specific personnel) in the predefined strategy. Decision and output: The judgment is true, triggering the highest priority instruction. The system ignores the data similarity C (which will be very low at this time) and entropy decay index D that will be calculated later. {N} The judgment result (which will be very high at this point) is then determined. The entire processing flow of the neural network is forcibly maintained without interruption, instructing the AI ​​chip to complete deep analysis of all computational layers to obtain the most accurate and complete feature information about the specific individual. The final deep analysis result is then labeled as a "change risk tag" and reported.

[0128] The core technical feature of this embodiment is:

[0129] 1. A dynamic processing interruption criterion combining "input similarity" and "internal processing entropy change" is constructed. This scheme abandons the single, static processing mode of traditional methods. It innovatively establishes a dual-verification decision mechanism: not only does it perform an "external" coarse screening at the data input layer using data similarity (C) to determine whether the data is superficially redundant; more importantly, it delves into the neural network, using the entropy decay index (D{N}) to quantify the stability of the model's "internal" processing when extracting features from the data. Processing interruption is only triggered when both conditions are met: "external data is highly similar" and "internal processing tends to stabilize (information entropy no longer decreases significantly)". This design avoids resource waste caused by processing redundant data and prevents risky missed judgments caused by skipping data that is "superficially similar but contains anomalies".

[0130] 2. A method for quantifying the internal processing state of a neural network based on inter-layer information entropy decay is proposed. To achieve accurate insight into the internal processing state of the model, this scheme proposes to use information entropy (H) as an indicator to measure the uncertainty of the feature data extracted by the neural network at a specific layer. Furthermore, the entropy decay index D between adjacent computational layers (layer N-1 and layer N) is calculated. {N} This study quantifies for the first time whether the model is still effectively learning and abstracting new information. A low entropy decay index accurately characterizes the fact that the model has not discovered new and valuable features during the processing phase, and that the processing has entered a stable or saturated state.

[0131] 3. An adaptive computing resource pruning mechanism based on dual verification criteria was implemented. The ultimate goal of this solution is to optimize the computing power allocation of AI chips. Based on the aforementioned dual verification criteria, this solution sets an "early-exit" dynamic pruning point in the forward propagation path of the neural network. Once the interruption condition is met, the system will immediately terminate subsequent deeper-level calculations and actively delete the generated intermediate-level feature data, directly outputting the "normal safety" label. This adaptive, per-data-unit-based dynamic adjustment of the computing path promotes the precise allocation of computing resources to data units that truly have information increment and potential risks, thereby improving the overall energy efficiency and processing throughput of AI chips when processing continuous data streams without sacrificing security.

[0132] Example 2, please refer to Figure 4 AI chip-based data processing systems include:

[0133] The data stream acquisition module is configured to acquire a target data stream containing multiple data units;

[0134] The data similarity calculation module is configured to calculate the data similarity C between the current data unit and previous data units in the target data stream;

[0135] The neural network processing module is configured to perform layer-by-layer forward propagation processing on the current data unit through multiple computational layers within it.

[0136] The dynamic monitoring and entropy calculation module, connected to the neural network processing module, is configured to perform real-time calculation of the information entropy H of the current level feature data generated by the Nth computational layer during the forward propagation process. {N} Obtain the information entropy H of the feature data from the previous layer generated by the (N-1)th computation layer. {N-1} And calculate the entropy decay exponent D. {N} ;

[0137] The intelligent decision-making and control module, connected to the data similarity calculation module and the dynamic monitoring and entropy calculation module respectively, is configured to monitor the entropy decay index D. {N} It performs a joint judgment with the data similarity C, and generates processing control instructions based on the judgment results;

[0138] The data management module receives and processes control commands and is configured to perform one of the following operations:

[0139] When the control instruction is interrupted, the processing of subsequent calculation layers is interrupted, the feature data of the current layer is removed as redundant data, and the final processing result is marked as a regular security label.

[0140] When the processing control instruction is to continue processing, the current level feature data is retained, the processing of subsequent calculation layers continues, and the final processing result is marked as a change risk label.

[0141] It should be noted that all calculation formulas in this application employ regression analysis, including but not limited to machine learning algorithms, to deeply analyze the collected parameters and identify their natural trends and interrelationships. Specialized software, such as Python's Scikit-learn library or the R language, is used to automatically generate mathematical models that match the data. Then, cross-validation and other methods are used to objectively evaluate the model performance, and continuous feedback and optimization are combined to ensure that the created formulas truly reflect the inherent laws of the data, thereby guaranteeing their effectiveness and accuracy. In all calculation formulas in this application, the parameters in each formula undergo dimensionless processing within a consistent range to ensure that different physical quantities are compared on the same scale; dimensionless processing techniques include, but are not limited to, min-max-normalization and Z-score standardization.

[0142] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A data processing method based on an AI chip, characterized in that, The specific steps include: acquiring a target data stream containing multiple data units; For the current data unit, calculate its data similarity C with the previous data units in the target data stream; The current data unit is processed by forward propagation layer by layer through multiple computational layers of the neural network model; Before performing layer-by-layer forward propagation processing on the current data unit, the method also includes: Calculate the spatiotemporal hash value of the current data unit. The spatiotemporal hash value is generated based on the spatial characteristics of the current data unit and the temporal context of the previous data unit in the target data stream. The query step in the hardware cache includes: for the spatiotemporal hash value, if the query hits, directly obtain a set of cached data pre-stored in the current hardware cache, remove the current level feature data as redundant data, mark the final processing result as a regular security label, and skip the layer-by-layer forward propagation processing and subsequent joint judgment steps; if the query misses, proceed to the layer-by-layer forward propagation processing and subsequent joint judgment steps. After forward propagation to the Nth computational layer, the information entropy H of the set of current-level feature data generated by the Nth computational layer based on the input data is calculated. {N} And obtain the information entropy H of a set of previous layer feature data generated by the (N-1)th computation layer and used as input to the Nth computation layer. {N-1} ; Based on H {N} With H {N-1} Calculate the entropy decay exponent D {N} Entropy decay exponent D {N} Used to quantify the degree of reduction in information uncertainty during the processing from layer N-1 to layer N; Regarding the entropy decay exponent D {N} When combined with data similarity C, the entropy decay index D is used for joint judgment. {N} If the data is below the preset decay threshold and the data similarity C is above the preset similarity threshold, the processing of subsequent calculation layers is interrupted, the current level feature data is removed as redundant data, and the final processing result is marked as a regular safety label. Otherwise, the current level feature data is retained, and the final processing result is marked as a change risk label.

2. The data processing method based on an AI chip according to claim 1, characterized in that, The steps for calculating data similarity C are specifically configured as follows: Obtain the first pixel matrix of the current data unit and the second pixel matrix of the previous data unit; Based on the first pixel matrix and the second pixel matrix, the mean μ and standard deviation σ of each are calculated respectively, and the covariance σxy of the first pixel matrix and the second pixel matrix is ​​calculated, thereby constructing a statistical feature set containing brightness, contrast and structural information; Then, the statistical feature set is numerically stabilized by the stabilization constant injection method. The mean μ, standard deviation σ and covariance σxy are combined with the preset stabilization constants w1, w2 and w3 respectively to generate luminance component, contrast component and structure component with numerical robustness. Finally, a multiplicative fusion operation is performed on the luminance component, contrast component, and structure component after stability processing. The calculation results of the three components are multiplied one by one to obtain a scalar output, which is then used as the data similarity C to characterize the degree of consistency between the contents of the two data units.

3. The data processing method based on an AI chip according to claim 1, characterized in that, The steps for building a neural network model are as follows: setting up a network topology containing multiple computational layers, initializing the learnable parameters in the network topology, using a preset training dataset related to the target processing task, iteratively training the network topology, terminating training when the performance index of the neural network model reaches the preset convergence criterion, and deploying the trained neural network model containing fixed learnable parameters into the AI ​​chip for subsequent layer-by-layer forward propagation processing.

4. The data processing method based on an AI chip according to claim 1, characterized in that, Entropy decay exponent D {N} The method for obtaining this information is as follows: calculate the absolute decay of information entropy from layer N-1 to layer N. The absolute decay of information entropy is equal to the information entropy H of the feature data from the previous layer. {N-1} Information entropy H of the current level feature data {N} difference; The absolute decay of information entropy is normalized by dividing the absolute decay by the information entropy H of the previous level feature data, which serves as the baseline. {N} To obtain a dimensionless relative decay rate; The relative decay rate obtained after normalization is assigned the entropy decay exponent D. {N} .

5. The data processing method based on an AI chip according to claim 1, characterized in that, Before making a joint judgment, obtain the state significance coefficient S, which is used to characterize the importance level of the target data stream processing task; A preset significance threshold is set. When the state significance coefficient S is higher than the significance threshold, the entropy decay exponent D is ignored. {N} The result of the data similarity C judgment is used to ensure that the processing of subsequent calculation layers is not interrupted. If the state significance coefficient S is not lower than the significance threshold, the processing interruption decision is determined to be false, and the processing of subsequent calculation layers continues. If the state significance coefficient S is lower than the significance threshold, then it is further determined whether the data similarity C is higher than the similarity threshold and whether the entropy decay index H... {N} If the threshold is lower than the decay threshold, and both conditions are met simultaneously, the interruption decision is determined to be true.

6. The data processing method based on an AI chip according to claim 5, characterized in that, Before making a joint judgment, the methods for obtaining the state significance coefficient S include: A pre-defined list of content to be monitored is used to establish a mapping relationship between content identifiers and basic salience scores; content identifiers include at least one or more of the following: specific personnel, smoke, abnormally opened access control or unauthorized devices; Perform content parsing processing on the target data stream to identify and extract one or more current content objects contained in the data stream; perform a matching query between the current content object and the list of content to be followed to determine whether the current content object matches any content identifier in the list; Based on the results of the matching query, determine the state significance coefficient S. The output score of the state significance coefficient S is a standardized interval of [0,1]. If the query is successful, the basic significance score corresponding to the content identifier of the successful query is obtained as the state significance coefficient S. If the query does not find a match, the preset default significance score is used as the state significance coefficient S.

7. The data processing method based on an AI chip according to claim 1, characterized in that, The specific steps for generating a spacetime hash value include: Calculate the spatial hash value of the current data unit using the locality-sensitive hash function; A spatiotemporal hash value is generated by performing a preset bitwise operation on the spatial hash value and the stored historical hash value corresponding to the previous data unit.

8. The data processing method based on an AI chip according to claim 1, characterized in that: The query steps specifically include: calculating the Hamming distance between the spatiotemporal hash value of the current data unit and each spatiotemporal hash value stored in the hardware cache; when any Hamming distance is lower than a preset distance threshold, it is determined that the query has been hit, and the pre-stored calculation result corresponding to the spatiotemporal hash value with the smallest Hamming distance is obtained; when all Hamming distances are not lower than the preset distance threshold, it is determined that the query has not been hit, and the process proceeds to the layer-by-layer forward propagation process and subsequent joint judgment steps.

9. A data processing system based on an AI chip, characterized in that: The system is used to execute the data processing method based on an AI chip as described in any one of claims 1-8, including: The data stream acquisition module is configured to acquire a target data stream containing multiple data units; The data similarity calculation module is configured to calculate the data similarity C between the current data unit and previous data units in the target data stream; The neural network processing module is configured to perform layer-by-layer forward propagation processing on the current data unit through multiple computational layers within it. The dynamic monitoring and entropy calculation module, connected to the neural network processing module, is configured to calculate the information entropy H of the current-level feature data generated by the Nth computational layer in real time during the forward propagation process. {N} Obtain the information entropy H of the feature data from the previous layer generated by the (N-1)th computation layer. {N-1} And calculate the entropy decay exponent D{N}; The intelligent decision-making and control module, connected to the data similarity calculation module and the dynamic monitoring and entropy calculation module respectively, is configured to monitor the entropy decay index D. {N} It performs a joint judgment with the data similarity C, and generates processing control instructions based on the judgment results; The data management module receives and processes control commands and is configured to perform one of the following operations: When the control instruction is interrupted, the processing of subsequent computation layers is interrupted, the feature data of the current layer is removed as redundant data, and the final processing result is marked as a regular security label. When the processing control instruction is to continue processing, the current level feature data is retained, the processing of subsequent calculation layers continues, and the final processing result is marked as a change risk label.

Images