Water conservancy internet of things concurrent processing method and device

Abstract

The application provides a water conservancy Internet of Things concurrent processing method and device, relates to the field of data processing, and solves the technical problem that the existing technology cannot consider data priority and load fluctuation, resulting in delayed key data processing or decreased system performance. The method comprises the following steps: obtaining a water conservancy Internet of Things device data stream; performing three-dimensional value scoring on the data stream based on a heat factor, a business weight factor and a timeliness factor to determine a first three-dimensional score; dynamically optimizing the weight parameters of the heat factor, the business weight factor and the timeliness factor through a self-learning weight adjustment mechanism based on the first three-dimensional score to obtain a second three-dimensional score; performing hierarchical processing on the data stream according to the second three-dimensional value score and system load conditions to obtain a data stream hierarchical processing result; and executing a corresponding data processing strategy based on the data stream hierarchical processing result, and gradually retreating to full-service in a step-by-step manner when a predicted fallback condition is met. The application is used in the water conservancy Internet of Things concurrent processing process.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to a method and apparatus for concurrent processing of water conservancy Internet of Things (IoT) devices. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) in water conservancy, systems such as hydrological monitoring, rainfall early warning, and river water level monitoring need to process large amounts of real-time data streams. Furthermore, critical business data differs significantly from ordinary monitoring data in terms of access frequency, business importance, and validity period. Existing technologies typically employ single-dimensional caching strategies or fixed-threshold degradation methods, relying solely on access frequency or static load thresholds for data eviction and degradation. This fails to adequately consider business value and timeliness, potentially leading to the erroneous eviction of critical data. Simultaneously, the system is prone to service delays or crashes during load fluctuations or sudden peaks. Therefore, there is an urgent need for a concurrent processing method for the IoT in water conservancy that comprehensively considers data importance, improving system processing efficiency while ensuring the priority of critical business data to achieve overall system stability. Summary of the Invention

[0003] This application provides a method and apparatus for concurrent processing of water conservancy Internet of Things, which solves the technical problem that the existing technology is unable to balance data priority and load fluctuation, resulting in untimely processing of key data or degraded system performance.

[0004] To achieve the above objectives, this application adopts the following technical solution: Firstly, a concurrent processing method for water conservancy IoT is provided, comprising: acquiring data streams from water conservancy IoT devices; performing a three-dimensional value score on the data streams based on a popularity factor, a business weight factor, and a timeliness factor to determine a first three-dimensional score; dynamically optimizing the weight parameters of the popularity factor, business weight factor, and timeliness factor through a self-learning weight adjustment mechanism based on the first three-dimensional score to obtain a second three-dimensional score; performing hierarchical processing on the data streams according to the second three-dimensional value score and the system load to obtain hierarchical processing results for the data streams; executing corresponding data processing strategies based on the hierarchical processing results for the data streams, and gradually rolling back to full service when the predicted rollback conditions are met.

[0005] In conjunction with the first aspect mentioned above, one possible implementation involves scoring the data stream in three dimensions based on a popularity factor, a business weight factor, and a timeliness factor. This includes: calculating the popularity factor for each data stream, where the popularity factor is the ratio of the number of accesses within a preset time window to the length of the time window; determining the business weight factor based on the device metadata corresponding to each data stream, and assigning a higher weight value to critical business devices than to ordinary devices; calculating the timeliness factor based on the generation time and validity period of each data stream, and weighting and correcting the value of data nearing expiration using a hyperbolic tangent function; and weighting and combining the popularity factor, business weight factor, and timeliness factor according to the weight parameters to obtain the first three-dimensional value score for each data stream.

[0006] In conjunction with the first aspect mentioned above, in one possible implementation, the weight parameters of the popularity factor satisfy the following formula:

[0007]

[0008] in, These are the initial popularity weight parameters. As a dynamic decay factor, Based on the basic heat compensation value, For scene sensitivity coefficient, Generates the time interval up to the present for the data stream.

[0009] In conjunction with the first aspect mentioned above, one possible implementation involves dynamically optimizing the weight parameters of the popularity factor, business weight factor, and timeliness factor through a self-learning weight adjustment mechanism. This includes: obtaining service quality indicators during system operation as feedback signals, including response latency, throughput, and system stability; using a reinforcement learning algorithm based on the feedback signals to iteratively update the weight parameters of the popularity factor, business weight factor, and timeliness factor; and recalculating the three-dimensional value score of the data stream using the updated weight parameters to obtain a second three-dimensional value score.

[0010] In conjunction with the first aspect mentioned above, one possible implementation involves tiered processing of the data stream, including: constructing a system load state equation based on Lyapunov stability theory and determining the optimal solution for degradation depth; based on the optimal solution for degradation depth, performing full caching of the data stream when the system load is below a first threshold; based on the optimal solution for degradation depth, discarding data streams with second-dimensional value scores in a preset low range when the system load reaches a second threshold; based on the optimal solution for degradation depth, transferring non-critical business data streams to a local cache when the system load reaches a third threshold, maintaining high-speed processing only for critical business data streams; based on the optimal solution for degradation depth, performing direct database reads and asynchronous cache writes for critical business data streams when the system load reaches a fourth threshold, and no longer retaining data streams with low second-dimensional value scores.

[0011] In conjunction with the first aspect mentioned above, in one possible implementation, the system load equation satisfies the following formula:

[0012] in, Under load conditions The system state matrix, To reduce the control quantity, To control the input matrix, As a distractor, This is the interference matrix.

[0013] In conjunction with the first aspect mentioned above, in one possible implementation, the optimal solution for degradation depth is... Satisfy the following formula:

[0014] For the target load state, This represents the current load status. These are the weighting coefficients. This is due to a loss in service quality.

[0015] In conjunction with the first aspect mentioned above, in one possible implementation, a corresponding data processing strategy is executed based on the data stream hierarchical processing results, and the system gradually rolls back to full service when the prediction rollback conditions are met. This includes: predicting the system load within a preset time window based on the Kalman prediction algorithm; and when the prediction results meet the condition that the load is lower than a preset threshold and the confidence level is greater than a predetermined value, gradually reducing the hierarchical level until the system is restored to full cache service.

[0016] In conjunction with the first aspect mentioned above, in one possible implementation, the prediction result satisfies the following formula:

[0017] in, The system state matrix, To predict load status, Here is the Kalman gain matrix. For the observed values, This is the observation matrix.

[0018] Secondly, a concurrent processing device for water conservancy IoT is provided, comprising: a communication unit and a processing unit; the communication unit is used to acquire data streams from water conservancy IoT devices; the processing unit is used to perform a three-dimensional value score on the data stream based on a popularity factor, a business weight factor, and a timeliness factor to determine a first three-dimensional score; based on the first three-dimensional score, dynamically optimize the weight parameters of the popularity factor, the business weight factor, and the timeliness factor through a self-learning weight adjustment mechanism to obtain a second three-dimensional score; according to the second three-dimensional value score and the system load, perform hierarchical processing on the data stream to obtain a hierarchical processing result; execute the corresponding data processing strategy based on the hierarchical processing result of the data stream, and gradually roll back to full service when the predicted rollback conditions are met.

[0019] Thirdly, this application provides an electronic device, including: a processor and a storage medium; the storage medium includes instructions, and the processor is configured to execute the instructions to implement the methods described in the first aspect and any possible implementation thereof. This electronic device may be an electronic device or a chip within an electronic device.

[0020] Fourthly, this application provides a concurrent processing system for water conservancy IoT, including: sensors and electronic devices; wherein, the sensors are used to acquire data streams from water conservancy IoT devices; the electronic devices are used to perform a three-dimensional value score on the data stream based on a popularity factor, a business weight factor, and a timeliness factor to determine a first three-dimensional score; based on the first three-dimensional score, the weight parameters of the popularity factor, the business weight factor, and the timeliness factor are dynamically optimized through a self-learning weight adjustment mechanism to obtain a second three-dimensional score; the data stream is processed in a hierarchical manner according to the second three-dimensional value score and the system load to obtain a hierarchical processing result; the corresponding data processing strategy is executed based on the hierarchical processing result, and the system gradually rolls back to full service when the predicted rollback conditions are met.

[0021] Fifthly, this application provides a computer-readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0022] In a sixth aspect, this application provides a computer program product containing instructions that, when run on an electronic device, cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0023] This application provides a method and apparatus for concurrent processing of water conservancy Internet of Things (IoT) data. It performs a three-dimensional value score on the data stream's heat factor, business weight factor, and timeliness factor, and dynamically optimizes the score weights using a self-learning weight adjustment mechanism to prioritize the processing of critical business data. Furthermore, it implements tiered processing based on the three-dimensional value score and system load, and gradually rolls back to full service when predictive rollback conditions are met. This effectively controls system load fluctuations, improves data processing efficiency, ensures the priority of critical data, and enhances the overall stability and reliability of the system. It solves the technical problem of existing technologies struggling to balance data priority and load fluctuations, leading to untimely processing of critical data or degraded system performance.

[0024] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0025] Figure 1 A system architecture diagram of a water conservancy Internet of Things concurrent processing system provided in this application embodiment; Figure 2 A flowchart illustrating a concurrent processing method for water conservancy Internet of Things (IoT) provided in this application embodiment; Figure 3 A flowchart illustrating another concurrent processing method for water conservancy IoT provided in this application embodiment; Figure 4 A flowchart illustrating another concurrent processing method for water conservancy IoT provided in this application embodiment; Figure 5 A schematic diagram of the structure of a water conservancy Internet of Things concurrent processing device provided in an embodiment of this application; Figure 6This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0027] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0028] The concurrent processing method for water conservancy Internet of Things provided in this application embodiment can be applied to, for example... Figure 1 In the water conservancy Internet of Things concurrent processing system 100 shown, such as Figure 1 As shown, the system includes: a sensor 101 and an electronic device 102.

[0029] Among them, sensor 101 is used to acquire data streams from water conservancy IoT devices.

[0030] Electronic device 102 is used to perform three-dimensional value scoring on data streams based on popularity factors, business weight factors, and timeliness factors to determine a first three-dimensional score; based on the first three-dimensional score, the weight parameters of popularity factors, business weight factors, and timeliness factors are dynamically optimized through a self-learning weight adjustment mechanism to obtain a second three-dimensional score; according to the second three-dimensional value score and system load, the data streams are processed in a hierarchical manner to obtain data stream hierarchical processing results; based on the data stream hierarchical processing results, corresponding data processing strategies are executed, and when the predicted rollback conditions are met, the data streams are rolled back to full service level by level.

[0031] To address the technical problem in existing technologies where it is difficult to balance data priority and load fluctuations, leading to untimely processing of critical data or degraded system performance, this application provides a concurrent processing method for water conservancy IoT devices. The method includes: acquiring data streams from water conservancy IoT devices; performing a three-dimensional value score on the data stream based on a popularity factor, a business weight factor, and a timeliness factor to determine a first three-dimensional score; dynamically optimizing the weight parameters of the popularity factor, business weight factor, and timeliness factor through a self-learning weight adjustment mechanism based on the first three-dimensional score to obtain a second three-dimensional score; performing hierarchical processing on the data stream according to the second three-dimensional value score and system load conditions to obtain a hierarchical processing result; executing corresponding data processing strategies based on the hierarchical processing result, and gradually rolling back to full service when the predicted rollback conditions are met.

[0032] Figure 2 This application provides a flowchart illustrating a concurrent processing method for water conservancy IoT, as shown in the embodiments below. Figure 2 As shown, the method includes: S201. Obtain data streams from water conservancy IoT devices.

[0033] Among them, the data flow of water conservancy Internet of Things equipment refers to the real-time monitoring data sequence collected by water conservancy monitoring equipment (such as water level sensors, flow meters, etc.).

[0034] In one possible implementation, the data acquisition device connects to various water conservancy monitoring devices via an Internet of Things (IoT) communication protocol, periodically receives real-time data uploaded by the devices, and stores it in a data buffer pool.

[0035] It should be noted that data stream acquisition must ensure real-time performance and integrity to avoid data loss or delay.

[0036] S202. Based on the popularity factor, business weight factor and timeliness factor, perform a three-dimensional value score on the data stream to determine the first three-dimensional score.

[0037] Among them, the popularity factor refers to the frequency of data stream access or invocation; the business weight factor refers to the importance of data stream to core business; and the timeliness factor refers to the freshness of data.

[0038] In one possible implementation, a popularity factor is calculated for each data stream, which is the ratio of the number of accesses within a preset time window to the length of the time window. A business weight factor is determined based on the device metadata corresponding to each data stream, with critical business devices assigned a higher weight value than ordinary devices. A timeliness factor is calculated based on the generation time and validity period of each data stream, and the value of data nearing expiration is weighted and adjusted using a hyperbolic tangent function. Finally, the popularity factor, business weight factor, and timeliness factor are weighted together to obtain the first three-dimensional score.

[0039] It should be noted that the initial weight parameters can be set based on historical business data and will be dynamically optimized in the future.

[0040] S203. Based on the first three-dimensional score, the weight parameters of the popularity factor, business weight factor and timeliness factor are dynamically optimized through a self-learning weight adjustment mechanism to obtain the second three-dimensional score.

[0041] In one possible implementation, service quality indicators during system operation are obtained as feedback signals. These service quality indicators include response latency, throughput, and system stability. Based on the feedback signals, a reinforcement learning algorithm is used to iteratively update the weight parameters of the popularity factor, the business weight factor, and the timeliness factor. The three-dimensional value score of the data stream is recalculated using the updated weight parameters to obtain a second three-dimensional value score.

[0042] It should be noted that the self-learning process is a continuous iterative update to adapt to the dynamic changes in business needs and load status in the water conservancy IoT scenario, thereby avoiding the scoring distortion problem caused by fixed weights.

[0043] As an example, in this embodiment of the application, the iterative update of the weight parameters of the popularity factor, the business weight factor, and the timeliness factor is achieved through a reinforcement learning algorithm. The reduction of system latency is used as a reward signal. The strategy is iterated on different weight combinations and finally converges to the optimal weight parameters to obtain the second three-dimensional score.

[0044] As an example, in an embodiment of this application, the second three-dimensional score Satisfy the following formula:

[0045] in, For the popularity factor score, Score the business weight factor. For the timeliness factor score, For the weight parameters of the heat factor, The weighting parameters of the business weighting factor. This is the weighting parameter for the timeliness factor.

[0046] S204. Based on the second and third-dimensional value scores and system load, the data stream is classified and processed to obtain the data stream classification processing results.

[0047] Among them, system load refers to the operating status indicators of the data processing system, such as CPU utilization, memory usage, and processing latency.

[0048] In one possible implementation, a system load state equation is constructed based on Lyapunov stability theory, and an optimal solution for the degradation depth is determined. Based on the optimal solution for the degradation depth, when the system load is below the first threshold, full caching is performed on the data stream. Based on the optimal solution for the degradation depth, when the system load reaches the second threshold, data streams with second-dimensional value scores in a preset low range are discarded. Based on the optimal solution for the degradation processing depth, when the system load reaches the third threshold, non-critical business data streams are transferred to the local cache, and only critical business data streams are kept under high-speed processing. Based on the optimal solution for the degradation depth, when the system load reaches the fourth threshold, direct database reads and asynchronous cache writes are performed on critical business data streams, and data streams with low second-dimensional value scores are no longer retained.

[0049] It should be noted that the grading criteria can be dynamically adjusted according to the system's maximum load capacity.

[0050] S205. Execute the corresponding data processing strategy based on the data stream hierarchical processing results, and roll back to the full service level by level when the prediction rollback conditions are met.

[0051] Among them, the predicted rollback condition refers to the judgment condition that the system predicts the load will drop below the threshold or the urgency of business needs will increase.

[0052] In one possible implementation, the system load within a preset time window is predicted based on the Kalman prediction algorithm; when the prediction result satisfies the condition that the load is lower than a preset threshold and the confidence level is greater than a predetermined value, the grading level is gradually reduced until the system is restored to full cache service.

[0053] As an example, in an embodiment of this application, the prediction result satisfies the following formula:

[0054] in, The system state matrix, To predict load status, Here is the Kalman gain matrix. For the observed values, This is the observation matrix.

[0055] As an example, in this embodiment of the application, the Kalman predictor predicts the system load within a three-minute time window. If the prediction result shows that the CPU utilization rate will be less than 70% and the prediction confidence is greater than 95%, the rollback control device triggers a step-by-step rollback.

[0056] Optionally, the step-by-step rollback satisfies the following rule: maintain at least a minimum observation time after each rollback. During this observation period, rollback is prohibited to avoid frequent switching of cache states that could cause system instability.

[0057] This application embodiment aggregates data streams from water conservancy IoT devices in real time and quantifies the value of the data streams using three-dimensional factors: popularity, business weight, and timeliness. A self-learning weight adjustment mechanism dynamically optimizes the factor parameters, allowing the three-dimensional value assessment to be continuously corrected based on business feedback, better aligning with actual business needs and improving the accuracy of data value mining. Based on this, the optimized second-dimensional score is combined with system load conditions to perform tiered processing of the data streams: high-value data receives priority resource support, while low-value data is delayed or eliminated under high load conditions, thus preventing low-value data from consuming core resources and causing system overload. When the system load decreases and the predicted rollback conditions are met, full service is restored step by step. This releases more resources while smoothly transitioning to avoid load fluctuations and sudden service changes caused by frequent switching, achieving efficient utilization and stable operation of system resources. It solves the problem of balancing data priority and load fluctuations in existing technologies, avoiding untimely processing of critical data or system performance degradation, and ensuring the simultaneous guarantee of efficiency, stability, and business priority in data services within the water conservancy IoT scenario.

[0058] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 3 As shown, the above S202 can be specifically implemented through the following S301 to S304, which are explained in detail below: S301. Calculate the heat factor for each data stream. The heat factor is the ratio of the number of accesses within a preset time window to the length of the time window.

[0059] In one possible implementation, for each data stream, the number of times it is accessed within a preset time window is counted, and this number of accesses is divided by the length of the time window to obtain the popularity factor of the data stream. The preset time window can be fixed at the minute or hour level, or it can be dynamically adjusted according to system load and business scenarios.

[0060] It should be noted that the time window is not a static parameter. To adapt to sudden changes in load in water conservancy IoT scenarios, the scoring device can dynamically shorten or extend the window length according to the system load, so that the heat factor can accurately reflect the access characteristics of the data flow under different pressure scenarios.

[0061] As an example, in an embodiment of this application, the heat factor satisfies the following formula:

[0062] in, The number of visits within a time window. This represents the length of the time window.

[0063] As an example, in an embodiment of this application, the heat factor satisfies the following formula:

[0064]

[0065] in, These are the initial popularity weight parameters. As a dynamic decay factor, Based on the basic heat compensation value, For scene sensitivity coefficient, Generates the time interval up to the present for the data stream.

[0066] S302. Determine the business weight factor based on the device metadata corresponding to each data stream, and assign a higher weight value to critical business devices than to ordinary devices.

[0067] Among them, the business weight factor refers to the weight assigned based on the business importance of the device to which the data stream belongs, used to highlight the priority of different businesses in data processing. It is directly related to the core nature of water conservancy business, for example, flood control scheduling is higher than daily monitoring.

[0068] In one possible implementation, the device metadata corresponding to the data stream is first read, including information such as device type, deployment location, and service level. This data stream is then assigned a corresponding service weight factor based on a pre-defined service priority mapping table. Data streams from critical service devices will be given higher weights.

[0069] Optionally, the business weight factor can adopt a multi-level weight model, such as three levels (high, medium, low) or five levels (critical, important, ordinary, low, non-critical), or it can adopt a continuous numerical weight to facilitate fine-tuning. In this embodiment, the flood control equipment will be automatically weighted to 200% of the benchmark value, but this embodiment does not limit this.

[0070] It should be noted that the business weighting factors are not fixed values, but can be updated as business needs change. For example, during the flood season, the weight of rainfall monitoring equipment data can be temporarily increased; during the drought monitoring season, the weight of water level evaporation monitoring equipment may be increased.

[0071] S303. Calculate the timeliness factor based on the generation time and validity period of each data stream, and use the hyperbolic tangent function to weight and correct the value of data that is about to expire.

[0072] The timeliness factor refers to an indicator that adjusts the value of data based on the time remaining until the data's expiration date. It reflects the immediate value of the data for current decision-making, with data nearing its expiration date requiring priority processing.

[0073] In one possible implementation, the scoring device calculates the remaining validity period based on the data stream's generation time and expiration date. The ratio of the remaining time to the total validity period is then used as input and subjected to a nonlinear transformation using a hyperbolic tangent function to obtain a timeliness factor. This function exhibits linear variation near zero and gradually saturates near one, thus amplifying the value of data nearing expiration.

[0074] As an example, in the embodiments of this application, the timeliness factor Satisfy the following formula:

[0075] in, It is the ratio of remaining time to total validity period. This is the magnification factor.

[0076] S304. The heat factor, business weight factor and timeliness factor are weighted and combined according to the weight parameters to obtain the first three-dimensional value score of each data stream.

[0077] Among them, the first three-dimensional score refers to the overall value quantification result obtained by combining the popularity factor, business weight factor and timeliness factor, which is the direct basis for subsequent classification processing.

[0078] In one possible implementation, the three-dimensional factors are weighted and summed with their corresponding weight parameters to obtain the first three-dimensional score for each data stream. The weight parameters can be initialized empirically, for example, with the business weight factor having the largest proportion, and then dynamically optimized in subsequent steps through a self-learning mechanism.

[0079] Optionally, the first three-dimensional score satisfies the following formula:

[0080] in, For the popularity factor score, Score the business weight factor. For the timeliness factor score, For the weight parameters of the heat factor, The weighting parameters of the business weighting factor. This is the weighting parameter for the timeliness factor.

[0081] This application's embodiments identify data stream access activity by calculating a heat factor, enabling timely capture of sudden data demands; highlight the importance of core water conservancy businesses through a business weight factor, ensuring that critical business data is prioritized for processing when resources are scarce; and emphasize the immediate value of data nearing expiration through a timeliness factor, preventing critical information from becoming invalid due to delayed processing. Finally, the three types of factors are weighted and integrated to form a unified three-dimensional value scoring system, providing a quantitative basis for subsequent graded processing. This not only achieves a multi-dimensional and accurate characterization of data value but also ensures that resource allocation takes into account activity, business priority, and timeliness, thereby effectively improving the efficiency and stability of water conservancy IoT data processing.

[0082] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 4 As shown, the above S204 can be specifically implemented through the following S401 to S405, which are explained in detail below: S401. Construct the system load state equation based on Lyapunov stability theory and determine the optimal solution for degradation depth.

[0083] In one possible implementation, a load state equation is established based on Lyapunov stability theory. By comparing the energy functions under different degradation depths, the optimal degradation depth solution under the current system constraints is obtained. This determines whether the dynamic system is stable under different states, ensuring that the system does not oscillate or diverge during multi-stage degradation processing.

[0084] As an example, in an embodiment of this application, the system load equation satisfies the following formula:

[0085] in, Under load conditions The system state matrix, To reduce the control quantity, To control the input matrix, As a distractor, This is the interference matrix.

[0086] As an example, in an embodiment of this application, the optimal solution for degradation depth is... Satisfy the following formula:

[0087] For the target load state, This represents the current load status. These are the weighting coefficients. This is due to a loss in service quality.

[0088] It should be noted that the optimal solution for degradation depth obtained through this theory can avoid performance instability caused by excessive or insufficient degradation under load fluctuation conditions.

[0089] S402. Based on the optimal solution of degradation depth, when the system load is lower than the first threshold, the data stream is fully cached.

[0090] In one possible implementation, after detecting that the system load is below a first threshold, the caching device directly performs full caching on all data streams, writing the data completely to the cache to speed up read operations. The first threshold refers to the criterion for determining when the system is in a low-load state, typically measured by CPU utilization or usage rate.

[0091] It should be noted that full caching is suitable for scenarios with sufficient system resources, and can provide the best user experience while ensuring business performance.

[0092] As an example, in an embodiment of this application, when the system load threshold is less than 70%, full caching is performed on the data stream.

[0093] S403. Based on the optimal solution of the degradation depth, when the system load reaches the second threshold, the data stream with the second three-dimensional value score in the preset low range is eliminated.

[0094] In one possible implementation, after detecting that the load has reached a second threshold, the cache management device invokes the three-dimensional scoring module to sort all data streams by their second and third-dimensional value scores, and eliminates data streams that are below a preset low-value range to free up cache space. This process only eliminates low-value data streams, while high-value and critical business data streams remain unaffected, thus ensuring that the system can maintain the normal operation of core businesses even when the load increases.

[0095] As an example, in this embodiment of the application, when the system load threshold is greater than or equal to 70% and less than 85%, the last 20% of the data in the second three-dimensional value score is eliminated.

[0096] S404. Based on the optimal solution for degradation processing depth, when the system load reaches the third threshold, non-critical business data streams are transferred to the local cache, and only critical business data streams are processed at high speed.

[0097] In one possible implementation, after detecting that the load has reached a third threshold, the cache scheduling device transfers non-critical business data streams from the cache to the local disk cache, retaining only critical business data streams for processing in the cache. Critical business data streams can be determined by a business weight factor; data streams with weight values ​​exceeding a preset threshold are considered critical.

[0098] It should be noted that although local caching is slower than high-speed caching, it can maintain the integrity of data storage and ensure that non-critical data still has available copies.

[0099] As an example, in this embodiment of the application, when the system load threshold is greater than or equal to 85% and less than 95%, non-critical business data streams are transferred to local cache, and only critical business data streams are processed at high speed.

[0100] S405. Based on the optimal solution of degradation depth, when the system load reaches the fourth threshold, the critical business data stream is directly read from the database and asynchronously written to the cache, and the data stream with low value scores in the second and third dimensions is no longer retained.

[0101] In one possible implementation, the data processing device adopts an extreme protection strategy after detecting that the load has reached the fourth threshold: for critical business data streams, it reads directly from the database and uses asynchronous writing to ensure the reliability of the results; for data streams with low value scores, it no longer caches or stores them to avoid wasting resources.

[0102] As an example, in an embodiment of this application, the asynchronous write cache satisfies the following formula:

[0103] in, The length of the cache queue. For data write rate, For cache processing speed, , These are the scheduling parameters.

[0104] As an example, in this embodiment of the application, the strategy ensures that under extreme system pressure, limited resources serve only the most critical data streams. When the system load threshold exceeds 95%, direct database reads and asynchronous writes to the cache are performed on critical business data streams, while data streams with low second and third-dimensional value scores are no longer retained. The system only retains reservoir safety monitoring data and discards the cache of ordinary rainfall monitoring data.

[0105] This application's embodiments construct the system load state equation by introducing Lyapunov stability theory and determine the optimal solution for degradation depth, enabling the system to maintain global asymptotic stability during hierarchical processing. When the load is below a low threshold, full caching is performed to fully utilize system redundancy resources and ensure high-speed service for all data streams. When the load enters a medium range, low-value data streams are discarded, releasing cache space and alleviating system pressure. When the load further increases, non-critical business data streams are transferred to local cache, providing high-speed processing only for critical business data streams to ensure priority operation of core businesses. When the system load reaches the extreme threshold, a direct database read and asynchronous cache write approach is used to protect critical business data, while low-value data is no longer retained, ensuring that limited resources are concentrated on serving the most important data streams. This achieves dynamic hierarchical control and stable evolution of the system under different load levels, avoiding resource waste and preventing system overload, ultimately ensuring the efficiency, stability, and critical business continuity of the water conservancy IoT under extreme concurrency conditions.

[0106] The above primarily describes the solutions of the embodiments of this application from the perspective of device implementation. It is understood that each device, such as a water conservancy IoT concurrent processing device, includes at least one of the hardware structures and software modules corresponding to the execution of each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0107] This application embodiment can divide the water conservancy Internet of Things concurrent processing device into functional units according to the above method example. For example, each function can be divided into different functional units, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or software functional units. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0108] When using integrated units, Figure 5 A possible structural schematic diagram of the water conservancy Internet of Things concurrent processing device (referred to as water conservancy Internet of Things concurrent processing device 50) involved in the above embodiments is shown. The water conservancy Internet of Things concurrent processing device 50 includes a processing unit 501 and a communication unit 502, and may also include a storage unit 503. Figure 5The structural diagram shown can be used to illustrate the structure of the concurrent processing device for water conservancy Internet of Things involved in the above embodiments.

[0109] when Figure 5 The schematic diagram shown illustrates the structure of the concurrent processing device for water conservancy IoT involved in the above embodiments. The processing unit 501 is used to control and manage the operation of the concurrent processing device for water conservancy IoT, the communication unit 502 is used for the concurrent processing device for water conservancy IoT to communicate with other devices, and the storage unit 503 is used to store the program code and data of the concurrent processing device for water conservancy IoT.

[0110] For example, communication unit 502 is used to acquire data streams from water conservancy IoT devices; processing unit 501 is used to perform three-dimensional value scoring on the data stream based on popularity factors, business weight factors, and timeliness factors to determine a first three-dimensional score; based on the first three-dimensional score, the weight parameters of popularity factors, business weight factors, and timeliness factors are dynamically optimized through a self-learning weight adjustment mechanism to obtain a second three-dimensional score; according to the second three-dimensional value score and system load, the data stream is processed in a hierarchical manner to obtain a hierarchical data stream processing result; based on the hierarchical data stream processing result, the corresponding data processing strategy is executed, and when the predicted rollback conditions are met, the data stream is rolled back to full service level by level.

[0111] In one possible implementation, the processing unit 501 is further configured to perform a three-dimensional value score on the data stream based on a popularity factor, a business weight factor, and a timeliness factor, including: calculating the popularity factor for each data stream, where the popularity factor is the ratio of the number of accesses within a preset time window to the length of the time window; determining the business weight factor based on the device metadata corresponding to each data stream, and assigning a higher weight value to critical business devices than to ordinary devices; calculating the timeliness factor based on the generation time and validity period of each data stream, and weighting and correcting the value of data nearing expiration using a hyperbolic tangent function; and weighting and combining the popularity factor, business weight factor, and timeliness factor according to the weight parameters to obtain a first three-dimensional value score for each data stream.

[0112] In one possible implementation, the weight parameters of the popularity factor satisfy the following formula:

[0113]

[0114] in, These are the initial popularity weight parameters. As a dynamic decay factor, Based on the basic heat compensation value, For scene sensitivity coefficient, Generates the time interval up to the present for the data stream.

[0115] In one possible implementation, the processing unit 501 is further configured to dynamically optimize the weight parameters of the popularity factor, business weight factor, and timeliness factor through a self-learning weight adjustment mechanism, including: obtaining service quality indicators during system operation as feedback signals, the service quality indicators including response latency, throughput, and system stability; using a reinforcement learning algorithm based on the feedback signals to iteratively update the weight parameters of the popularity factor, business weight factor, and timeliness factor; and recalculating the three-dimensional value score of the data stream from the updated weight parameters to obtain a second three-dimensional value score.

[0116] In one possible implementation, the processing unit 501 is further configured to perform hierarchical processing on the data stream, including: constructing a system load state equation based on Lyapunov stability theory and determining the optimal solution for the degradation depth; based on the optimal solution for the degradation depth, performing full caching processing on the data stream when the system load is below a first threshold; based on the optimal solution for the degradation depth, eliminating data streams with second-dimensional value scores in a preset low range when the system load reaches a second threshold; based on the optimal solution for the degradation processing depth, transferring non-critical business data streams to a local cache when the system load reaches a third threshold, maintaining high-speed processing only for critical business data streams; based on the optimal solution for the degradation depth, performing direct database reads and asynchronous cache writes on critical business data streams when the system load reaches a fourth threshold, and no longer retaining data streams with low second-dimensional value scores.

[0117] In one possible implementation, the system load equation satisfies the following formula:

[0118] in, Under load conditions The system state matrix, To reduce the control quantity, To control the input matrix, As a distractor, This is the interference matrix.

[0119] In one possible implementation, the optimal solution for degradation depth is... Satisfy the following formula:

[0120] For the target load state, This represents the current load status. These are the weighting coefficients. This is due to a loss in service quality.

[0121] In one possible implementation, the processing unit 501 is further configured to execute the corresponding data processing strategy based on the data stream hierarchical processing result, and to gradually roll back to full service when the prediction rollback condition is met, including: predicting the system load within a future preset time window based on the Kalman prediction algorithm; when the prediction result satisfies that the load is lower than a preset threshold and the confidence level is greater than a predetermined value, gradually reducing the hierarchical level until restoring to full cache service.

[0122] In one possible implementation, the prediction result satisfies the following formula:

[0123] in, The system state matrix, To predict load status, Here is the Kalman gain matrix. For the observed values, This is the observation matrix.

[0124] The processing unit 501 can be a processor or a controller, and the communication unit 502 can be a communication interface, transceiver, transceiver circuit, transceiver device, etc. The term "communication interface" is a general term and may include one or more interfaces. The storage unit 503 can be a memory. When the water conservancy IoT concurrent processing device 50 is a chip, the processing unit 501 can be a processor or a controller, and the communication unit 502 can be an input interface and / or an output interface, pins, or circuits, etc. The storage unit 503 can be a storage unit within the chip (e.g., a register, cache, etc.) or a storage unit located outside the chip (e.g., read-only memory (ROM), random access memory (RAM, etc.)).

[0125] The communication unit can also be called a transceiver unit. The antenna and control circuit with transceiver functions in the water conservancy IoT concurrent processing device 50 can be considered as the communication unit 502 of the water conservancy IoT concurrent processing device 50, and the processor with processing functions can be considered as the processing unit 501 of the water conservancy IoT concurrent processing device 50. Optionally, the device in the communication unit 502 used to implement the receiving function can be considered as the communication unit. The communication unit is used to execute the receiving steps in the embodiments of this application, and the communication unit can be a receiver, a receiver circuit, etc. The device in the communication unit 502 used to implement the transmitting function can be considered as the transmitting unit. The transmitting unit is used to execute the transmitting steps in the embodiments of this application, and the transmitting unit can be a transmitter, a transmitter, a transmitting circuit, etc.

[0126] Figure 5If the integrated units in the process are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. Storage media for storing computer software products include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0127] Figure 5 The units in the process can also be called modules; for example, a processing unit can be called a processing module.

[0128] This application also provides a hardware structure diagram of an electronic device (denoted as electronic device 60), see [link to diagram]. Figure 6 The electronic device 60 includes a processor 601, and optionally, a memory 602 connected to the processor 601.

[0129] In the first possible implementation, see Figure 6 The electronic device 60 also includes a transceiver 603. The processor 601, memory 602, and transceiver 603 are connected via a bus. The transceiver 603 is used to communicate with other devices or communication networks. Optionally, the transceiver 603 may include a transmitter and a receiver. The device in the transceiver 603 that implements the receiving function can be considered as a receiver, which is used to perform the receiving steps in the embodiments of this application. The device in the transceiver 603 that implements the transmitting function can be considered as a transmitter, which is used to perform the transmitting steps in the embodiments of this application.

[0130] Based on the first possible implementation method Figure 6 The structural diagram shown can be used to illustrate the structure of the electronic device involved in the above embodiments.

[0131] in, Figure 6 This can also be illustrated by a system chip in an electronic device. In this case, the actions performed by the aforementioned electronic device can be implemented by this system chip; the specific actions performed can be found above and will not be repeated here.

[0132] In implementation, each step of the method provided in this embodiment can be completed by integrated logic circuits in the processor or by instructions in software form. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0133] The processor in this application may include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., which are various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor may be a standalone semiconductor chip or integrated with other circuits into a single semiconductor chip. For example, it may be integrated with other circuits (such as encoding / decoding circuits, hardware acceleration circuits, or various bus and interface circuits) to form a System-on-a-Chip (SoC), or it may be integrated as a built-in processor within an ASIC. The ASIC with the integrated processor may be packaged separately or together with other circuits. In addition to the cores for executing software instructions to perform calculations or processing, the processor may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

[0134] The memory in the embodiments of this application may include at least one of the following types: read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; or electrically erasable programmable-only memory (EEPROM). In some scenarios, the memory may also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto.

[0135] This application also provides a computer-readable storage medium including instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0136] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0137] This application also provides a chip including a processor and an interface circuit. The interface circuit is coupled to the processor. The processor is used to run computer programs or instructions to implement the above-described method. The interface circuit is used to communicate with other modules outside the chip.

[0138] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0139] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0140] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A concurrent processing method for water conservancy Internet of Things (IoT), characterized in that, include: Acquire data streams from water conservancy IoT devices; The data stream is scored using a three-dimensional value assessment based on popularity factors, business weight factors, and timeliness factors to determine the first three-dimensional score. The process of performing a three-dimensional value score on the data stream based on popularity factors, business weight factors, and timeliness factors to determine the first three-dimensional score includes: Calculate the popularity factor for each data stream, where the popularity factor is the ratio of the number of accesses within a preset time window to the length of the time window; The business weight factor is determined based on the device metadata corresponding to each data stream, and key business devices are assigned a higher weight value than ordinary devices. A timeliness factor is calculated based on the generation time and validity period of each data stream, and the value of data nearing expiration is weighted and corrected using a hyperbolic tangent function. The popularity factor, the business weight factor, and the timeliness factor are weighted and combined according to the weight parameters to obtain the first three-dimensional value score of each data stream; Based on the first three-dimensional score, the weight parameters of the popularity factor, the business weight factor, and the timeliness factor are dynamically optimized through a self-learning weight adjustment mechanism to obtain the second three-dimensional score. Based on the second three-dimensional value score and system load, the data stream is classified and processed to obtain the data stream classification processing result; The hierarchical processing of the data stream includes: The system load state equation is constructed based on Lyapunov stability theory, and the optimal solution for degradation depth is determined. Based on the optimal solution for the degradation depth, when the system load is below the first threshold, the data stream is fully cached. Based on the optimal solution for the degradation depth, when the system load reaches the second threshold, the data stream with the second three-dimensional value score in the preset low range is eliminated. Based on the optimal solution for the degradation processing depth, when the system load reaches the third threshold, non-critical business data streams are transferred to the local cache, while only critical business data streams are processed at high speed. Based on the optimal solution for the degradation depth, when the system load reaches the fourth threshold, direct database read and asynchronous cache write are performed on the critical business data streams, and data streams with low second and third-dimensional value scores are no longer retained. Based on the data stream hierarchical processing results, the corresponding data processing strategy is executed, and the service is rolled back to full service level by level when the predicted rollback conditions are met.

2. The method according to claim 1, characterized in that, The weighting parameters of the heat factor satisfy the following formula: in, These are the initial popularity weight parameters. As a dynamic decay factor, Based on the basic heat compensation value, For scene sensitivity coefficient, Generates the time interval up to the present for the data stream.

3. The method according to claim 1, characterized in that, The step of dynamically optimizing the weight parameters of the popularity factor, the business weight factor, and the timeliness factor through a self-learning weight adjustment mechanism includes: The service quality indicators during system operation are obtained as feedback signals, including response latency, throughput, and system stability. Based on the feedback signal, a reinforcement learning algorithm is used to iteratively update the heat factor weight parameter, business weight factor, and timeliness weight factor. The three-dimensional value score of the data stream is recalculated using the updated weight parameters to obtain a second three-dimensional value score.

4. The method according to claim 1, characterized in that, The system load equation satisfies the following formula: in, Under load conditions The system state matrix, To reduce the control quantity, To control the input matrix, As a distractor, This is the interference matrix.

5. The method according to claim 1, characterized in that, The optimal solution for the downgrade depth Satisfy the following formula: in, For the target load state, This represents the current load status. These are the weighting coefficients. This is due to a loss in service quality.

6. The method according to claim 1, characterized in that, The step of executing the corresponding data processing strategy based on the data stream hierarchical processing results, and gradually rolling back to the full service when the predicted rollback conditions are met, includes: The system load within a preset time window is predicted based on the Kalman prediction algorithm. When the prediction result meets the conditions that the load is lower than the preset threshold and the confidence level is greater than the predetermined value, the classification level is gradually reduced until the full cache service is restored.

7. The method according to claim 6, characterized in that, The prediction result satisfies the following formula: in, The system state matrix, To predict load status, Here is the Kalman gain matrix. For the observed values, This is the observation matrix.

8. A concurrent processing device for water conservancy Internet of Things, characterized in that, The device includes: a communication unit and a processing unit; The communication unit is used to acquire data streams from water conservancy IoT devices; The processing unit is configured to: perform a three-dimensional value score on the data stream based on a popularity factor, a business weight factor, and a timeliness factor to determine a first three-dimensional score; based on the first three-dimensional score, dynamically optimize the weight parameters of the popularity factor, the business weight factor, and the timeliness factor through a self-learning weight adjustment mechanism to obtain a second three-dimensional score; perform hierarchical processing on the data stream according to the second three-dimensional value score and system load to obtain a hierarchical processing result; execute the corresponding data processing strategy based on the hierarchical processing result, and gradually roll back to full service when the predicted rollback conditions are met; the three-dimensional value score on the data stream based on the popularity factor, the business weight factor, and the timeliness factor to determine the first three-dimensional score includes: calculating the popularity factor of each data stream, where the popularity factor is the ratio of the number of accesses within a preset time window to the length of the time window; determining the business weight factor based on the device metadata corresponding to each data stream, and assigning a higher weight value to critical business devices than to ordinary devices; calculating the timeliness factor based on the generation time and validity period of each data stream, and applying a hyperbolic tangent function to adjacent data streams... The value of expired data is weighted and corrected; the popularity factor, the business weight factor, and the timeliness factor are weighted and combined according to the weight parameters to obtain the first three-dimensional value score of each data stream; the data stream is graded and processed, including: constructing a system load state equation based on Lyapunov stability theory and determining the optimal solution for degradation depth; based on the optimal solution for degradation depth, when the system load is below a first threshold, performing full caching processing on the data stream; based on the optimal solution for degradation depth, when the system load reaches a second threshold, eliminating data streams with a second three-dimensional value score in a preset low range; based on the optimal solution for degradation depth, when the system load reaches a third threshold, transferring non-critical business data streams to local cache, and maintaining high-speed processing only for critical business data streams; based on the optimal solution for degradation depth, when the system load reaches a fourth threshold, performing direct database read and asynchronous cache write for critical business data streams, and no longer retaining data streams with low second three-dimensional value scores; executing the corresponding data processing strategy based on the data stream grading and processing results, and gradually rolling back to full service when the predicted rollback conditions are met.

Images