Resource self-adaptive real-time visualization method and system for multi-device data stream

Abstract

The invention discloses a resource adaptive real-time visualization method and system for multi-device data streams, and a computer, and belongs to the technical field of real-time streaming data processing and resource dynamic allocation. The universal design of the method can cover all large-scale parallel streaming data scenes, double service isolation and annular memory double partition are adopted, the access stability problem of high-concurrency streaming data is fundamentally solved, and the architecture is universal and stable; real-time resource perception and dynamic scheduling are carried out, the resource priority of a core task is ensured, the performance reduction of a whole link is avoided, and the resource adaptation is efficient; the ring-shaped shared memory double partitions realize communication between millisecond-level services, the real-time requirement of streaming data is adapted, and the communication is low in delay; the method is not bound with specific industries (medical treatment / industry / Internet of Things), can adapt to different scales of terminal devices and data types through configuration, and is high in scene expansibility.

Description

Technical Field

[0001] This invention relates to the fields of computer, real-time streaming data processing and dynamic resource allocation, and particularly to a resource adaptive real-time visualization method and system for multi-device data streams. Background Technology

[0002] With the widespread application of IoT technology in fields such as medical monitoring and sports research, there is a need for real-time, continuous streaming data monitoring and visualization of physiological indicators (such as pulse rate and blood oxygen saturation) of large numbers of individuals (more than 50 people). Data streams in these scenarios are characterized by high concurrency, unbounded accessibility, and low latency requirements. However, existing technical solutions suffer from two major drawbacks, making them difficult to adapt to: Architectural stability bottleneck: Existing solutions mostly adopt a single software service architecture, simultaneously handling streaming data reception, parsing, storage, and high-load graphics rendering tasks. Under the impact of high-concurrency data streams from dozens of devices, CPU, memory, and GPU resources compete fiercely for resources across different tasks. This not only leads to data processing delays and visualization rendering stutters, but also makes the system prone to memory overflow due to the accumulation of streaming data, ultimately causing the entire system to crash and failing to meet the high reliability requirements of medical monitoring.

[0003] The passive nature of resource allocation is a significant drawback: even when some solutions attempt to split into modules, they lack collaborative resource allocation mechanisms between and within services. The system cannot perceive its own operational status or changes in streaming data load, and cannot dynamically adjust task priorities when faced with data surges or fluctuations in host computing resources. This leads to all tasks slowing down, crippling core functions: core processes such as data reception and anomaly alerts compete for resources with non-core tasks like historical data archiving and UI rendering, ultimately affecting the timeliness of monitoring.

[0004] Therefore, there is an urgent need in this field for a generalized integrated solution that combines architectural decoupling with deep collaboration in dynamic resource allocation. This solution can ensure data access stability through service isolation and achieve dynamic adaptation of task priorities through real-time resource awareness, thereby meeting the high reliability and low latency requirements of large-scale parallel streaming data scenarios. Summary of the Invention

[0005] To achieve the above-mentioned objectives and other advantages of the present invention, a first objective of the present invention is to provide a resource adaptive real-time visualization method for multi-device data streams, comprising the following steps: Parallel streaming data from multiple terminal devices is received and preprocessed by an independently operating data access and processing service; The preprocessed streaming data is visualized in real time by a standalone data visualization service. Data communication between the data access and processing service and the data visualization service is achieved through an operating system-level ring shared memory, wherein the ring shared memory is at least divided into a physically isolated streaming data buffer and a control signal area. The data access and processing service divides its processing tasks into core tasks and non-core tasks according to a preset task layering strategy. The data access and processing service monitors the resource status of the host system in real time, and dynamically adjusts the resource allocation priority of the core tasks and the non-core tasks based on the resource status, while writing the resource status information into the control signal area. The data visualization service reads the resource status information from the control signal area and adaptively adjusts the resource consumption strategy of its visualization tasks accordingly.

[0006] Furthermore, the step of implementing data communication between the data access and processing service and the data visualization service through an operating system-level ring shared memory includes: The data access and processing service writes the preprocessed regularized streaming data into the streaming data buffer. The regularized streaming data includes at least a device identifier, a unified timestamp, and a data payload. The data visualization service reads the regularized streaming data from the streaming data buffer and visualizes it. When the streaming data buffer is full, new data is written by overwriting the oldest unread data to prioritize the transmission of new data.

[0007] Furthermore, the control signal area is further divided into logically isolated monitoring data sub-areas and status identifier sub-areas; The real-time monitoring of the host system's resource status includes: periodically collecting system resource indicators by the monitoring thread in the data access and processing service, and writing the system resource indicators into the monitoring data sub-area; The dynamic adjustment of resource allocation priority based on the resource status includes: the scheduling thread in the data access and processing service reads the system resource indicators in the monitoring data sub-area, and determines whether the current system is in an overloaded or idle state according to the preset resource threshold, and then executes the corresponding dynamic scheduling strategy.

[0008] Furthermore, the dynamic scheduling strategy includes: When the system is determined to be in an overload state, an overload degradation strategy is executed: the priority of the thread corresponding to the core task is increased and / or bound to a specific CPU core, while the resource allocation of the non-core task is reduced or suspended. When it is determined that the current system is in an idle state for a preset duration, an idle recovery strategy is executed: the resource allocation of the non-core tasks is gradually restored; If an overload state is triggered again during the idle recovery process, the system will immediately revert to the overload degradation strategy.

[0009] Furthermore, reducing or suspending resource allocation for the non-core tasks includes at least one of the following: extending the execution interval of historical data archiving tasks in the data access and processing service; simplifying the rendering logic of non-critical data in the data visualization service or reducing its rendering frame rate.

[0010] Furthermore, the scheduling thread in the data access and processing service also writes a status identifier representing the system's overload or idle state into the status identifier sub-area; The data visualization service reads the status identifier from the status identifier sub-area and adjusts its visualization tasks accordingly, including: when the status identifier is overloaded, rendering only key data and triggering alarms; when the status identifier is idle, restoring complete visualization rendering.

[0011] Furthermore, the core tasks include at least: data reception, data verification, and time synchronization; the non-core tasks include at least: historical data archiving and storage.

[0012] A second objective of this invention is to provide a resource-adaptive real-time visualization system for multi-device data streams, the system being used to implement the aforementioned method, comprising: The data access and processing service module is configured to run as an independent process, used to receive and preprocess parallel streaming data from multiple terminal devices, and to execute core tasks and non-core tasks according to the task layering strategy. The data visualization service module is configured to run as an independent process and is used for real-time visualization of preprocessed streaming data. A ring-shaped shared memory communication module, implemented based on operating system-level shared memory, includes at least a physically isolated streaming data buffer and a control signal area, used to realize low-latency data and status communication between the data access and processing service module and the data visualization service module; The resource monitoring and adaptive scheduling module is integrated into the data access and processing service module. It is used to monitor system resources in real time, dynamically adjust task priorities based on resource status, and write resource status information into the control signal area. The data visualization service module is configured to adaptively adjust its visualization strategy based on the resource status information read from the control signal area.

[0013] Furthermore, in the ring shared memory communication module: The streaming data buffer is configured to store the regular streaming data written by the data access and processing service module in a FIFO manner, and to be read by the data visualization service module. Its capacity is determined based on the number of terminal devices and the preset data retention time. The control signal area is further divided into a logically isolated monitoring data sub-area and a status identifier sub-area by memory address offset. The monitoring data sub-area is used to store system resource monitoring data written by the resource monitoring and adaptive scheduling module, and the status identifier sub-area is used to store system status identifiers written by the resource monitoring and adaptive scheduling module.

[0014] Furthermore, the resource monitoring and adaptive scheduling module includes: The monitoring thread periodically collects at least one system resource indicator among CPU utilization, memory usage, and network I / O throughput at a preset low frequency, and securely writes the system resource indicator into the monitoring data sub-area through a read-write lock. The scheduling thread reads the system resource indicators of the monitoring data sub-area and compares them with preset overload thresholds and idle thresholds to determine the system load status. The scheduling thread executes a dynamic scheduling strategy based on the judgment result. The dynamic scheduling strategy includes: locking core task resources and downgrading non-core tasks in an overloaded state, and gradually restoring non-core tasks in an idle state. The scheduling thread also writes a status identifier representing the load status into the status identifier sub-area.

[0015] Furthermore, the data visualization service module is configured with a status monitoring unit, which periodically reads the status identifier from the status identifier sub-area and controls the working mode of the visualization rendering unit according to the status identifier: when overloaded, it switches to a simplified mode that only renders key data and triggers alarms; when idle, it switches to a standard mode that performs full data rendering.

[0016] Furthermore, the data access and processing service module establishes a connection with the terminal device or data gateway through a network protocol, and its preprocessing operations include at least data verification, invalid packet removal, and time-series synchronization of streaming data from multiple devices.

[0017] Compared with the prior art, the beneficial effects of the present invention are: The universal design of this invention can cover all large-scale parallel streaming data scenarios. It adopts dual-service isolation and ring-based memory dual partitioning to fundamentally solve the access stability problem of high-concurrency streaming data, resulting in a universal and stable architecture. Real-time resource awareness and dynamic scheduling ensure the resource priority of core tasks, avoid performance degradation across the entire link, and achieve efficient resource adaptation. Ring-based shared memory dual partitioning enables millisecond-level inter-service communication, adapting to the real-time requirements of streaming data with low communication latency. It is not bound to a specific industry (medical / industrial / IoT) and can be configured to adapt to different scales of terminal devices and data types, resulting in strong scenario scalability.

[0018] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 A method for resource-adaptive real-time visualization of multi-device data streams Figure 1 ; Figure 2 A schematic diagram illustrating the principle of resource adaptive real-time visualization for multi-device data streams; Figure 3 A method for resource-adaptive real-time visualization of multi-device data streams Figure 2 ; Figure 4 A flowchart illustrating the data communication process between the data access and processing service and the data visualization service via an operating system-level ring shared memory. Figure 5 This is a flowchart of the dynamic scheduling strategy. Figure 6 A block diagram of a resource-adaptive real-time visualization system for multi-device data streams; Figure 7 This is a schematic diagram of a computer device. Figure 8 This is a schematic diagram of a computer-readable storage medium. Detailed Implementation

[0020] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0021] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0022] The drawing numbers in this application are only used to distinguish the steps in the scheme and are not used to limit the execution order of the steps. The specific execution order is as described in the specification.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0024] This invention provides a method and system for highly stable access, resource adaptive processing, and real-time visualization of streaming data from large-scale terminal devices. It is particularly suitable for centralized monitoring of more than 50 Bluetooth physiological sensing devices in medical monitoring scenarios, and solves the integrated problem of large-scale access stability and dynamic system resource adaptation.

[0025] Example 1 A resource-adaptive real-time visualization method for multi-device data streams, such as... Figures 1-3 As shown, it includes the following steps: S1. Parallel streaming data from multiple terminal devices is received and preprocessed by an independently operating data access and processing service; S2. Real-time visualization of preprocessed streaming data is performed by an independently running data visualization service; This system adheres to the principles of service isolation and efficient memory-level communication, providing the infrastructure for general scenarios. The dual-service process isolation design includes: Data Access and Processing Service (DIaS): an independent process (without a user interface, supporting background / system service operation), responsible for parallel streaming data access, data verification, time-series synchronization, and preprocessing of large-scale terminal devices, dividing internal tasks into core / non-core categories to provide a basis for resource scheduling; Data Visualization Service (DVisS): An independent process (supporting desktop / front-end visualization), responsible for real-time display of streaming data, user interaction, and key information alerts; The two services are independent processes at the operating system level, with resources isolated from each other. Even if DVisS experiences a brief pause due to rendering load or user interaction, DIAS can still continuously receive and process streaming data without being affected, thus avoiding the spread of single point of failure at the architectural level.

[0026] S3. Data communication between the data access and processing service and the data visualization service is achieved through an operating system-level ring shared memory, wherein the ring shared memory is at least divided into a physically isolated streaming data buffer and a control signal area. In some embodiments, such as Figure 4 As shown, the steps for implementing data communication between the data access and processing service and the data visualization service through operating system-level ring shared memory include: S31. The data access and processing service writes the preprocessed regularized streaming data into the streaming data buffer. The regularized streaming data includes at least a device identifier, a unified timestamp, and a data payload. S32. The data visualization service reads the regularized streaming data from the streaming data buffer and visualizes it. S33. When the streaming data buffer is full, new data is written by overwriting the oldest unread data to prioritize the transmission of new data.

[0027] This system provides a ring-shaped shared memory with dual physical partitions. It employs an operating system-level ring-shaped shared memory to achieve low-latency communication between services, abstracted as two independent physical partitions. The core balances simplicity and functionality through physical isolation and logical sub-partitions. The streaming data buffer (physical partition 1) stores parallel streaming data preprocessed by DIAS, including device identifiers, unified timestamps, and data payloads. Its capacity is adapted to the requirement of "hundreds of devices × 10 minutes of data". FIFO transmission is achieved through write pointers (controlled by DIAS) and read pointers (controlled by DVisS). When the buffer is full, the oldest unread data is overwritten to ensure that new data is prioritized and to avoid data accumulation. This partition is only used for streaming data transmission and does not carry any control information.

[0028] The control signal area (physical partition 2, internally divided into 2 logical sub-partitions) has a lightweight capacity (KB level) and is physically isolated from the streaming data buffer. Internally, it is divided into two independent logical sub-partitions by memory address offsets, which do not interfere with each other: Monitoring data sub-partition (logical sub-partition 1): It is dedicated to storing system resource monitoring data (CPU utilization, memory usage, network I / O throughput), which is exclusively written by the DIAS monitoring thread and exclusively read by the scheduling thread, and thread safety is ensured by read-write locks; Status identifier sub-partition (logical sub-partition 2): It is dedicated to storing storage and coordination status identifiers (such as "overload / idle" status codes + timestamps), which are exclusively written by the DIAS scheduling thread and exclusively read by DVisS, and millisecond-level status synchronization is achieved with the help of an event triggering mechanism.

[0029] This system provides a general streaming data processing flow: DIAS establishes a connection with terminal devices / data gateways via network protocols (TCP / UDP) and receives parallel streaming data; after data verification to remove invalid packets and time synchronization to unify the timestamps of multiple devices, the regularized streaming data is written into the streaming data buffer; the write operation of DIAS and the read operation of DVisS are asynchronous and independent, ensuring that the data stream is not interrupted.

[0030] S4. The data access and processing service divides its processing tasks into core tasks and non-core tasks according to a preset task layering strategy. Preferably, the core tasks include at least: data reception, data verification, and time synchronization; the non-core tasks include at least: historical data archiving and storage.

[0031] This system achieves dual-service collaboration by integrating monitoring and scheduling threads within DIAS, combined with a control signal area. Specifically, the task layering is based on task priorities for streaming data scenarios, clearly defining the following layers: Core tasks: DIAS data reception, data verification, and time-series synchronization; DVisS key data visualization and anomaly signal alarms; Non-core tasks: DIAS historical data archiving and storage; DVisS non-critical data rendering and data smoothing.

[0032] The dual-service collaboration mechanism provided by this system involves the scheduling thread writing the overload / idle status into the control signal area. DIAS reads this status in real time and adapts accordingly: when overloaded, only key data is displayed, and when idle, the full visualization is restored, thus realizing resource collaboration between services.

[0033] Real-time resource monitoring specifically involves starting a low-priority monitoring thread in DIAS to collect system resource metrics (CPU utilization, memory usage, network I / O throughput) at a frequency of 1-2 times per second, storing them in the control signal area of ​​the circular shared memory, and ensuring thread safety through read-write locks for the scheduling thread to read.

[0034] S5. The data access and processing service monitors the resource status of the host system in real time, and dynamically adjusts the resource allocation priority of the core task and the non-core task based on the resource status, while writing the resource status information into the control signal area. In some embodiments, the control signal area is further divided into a logically isolated monitoring data sub-area and a status identifier sub-area; The real-time monitoring of the host system's resource status includes: periodically collecting system resource indicators by the monitoring thread in the data access and processing service, and writing the system resource indicators into the monitoring data sub-area; The dynamic adjustment of resource allocation priority based on the resource status includes: the scheduling thread in the data access and processing service reads the system resource indicators in the monitoring data sub-area, and determines whether the current system is in an overloaded or idle state according to the preset resource threshold, and then executes the corresponding dynamic scheduling strategy.

[0035] Preferably, the scheduling thread in the data access and processing service also writes a status identifier representing the system overload or idle state into the status identifier sub-area; The data visualization service reads the status identifier from the status identifier sub-area and adjusts its visualization tasks accordingly, including: when the status identifier is overloaded, rendering only key data and triggering an alarm; when the status identifier is idle, resuming full visualization rendering. Specifically, such as Figure 5 As shown, the dynamic scheduling strategy includes: S51. When it is determined that the current system is in an overload state, execute the overload degradation strategy: increase the priority of the thread corresponding to the core task and / or bind it to a specific CPU core, while reducing or suspending the resource allocation of the non-core task. Preferably, reducing or suspending resource allocation for the non-core tasks includes at least one of the following: extending the execution interval of historical data archiving tasks in the data access and processing service; simplifying the rendering logic of non-critical data in the data visualization service or reducing its rendering frame rate.

[0036] S52. When it is determined that the current system is in an idle state for a preset duration, execute the idle recovery strategy: gradually restore the resource allocation of the non-core tasks; S53. During the idle recovery process, if an overload state is triggered again, the system immediately reverts to the overload degradation strategy.

[0037] This system provides a dynamic scheduling strategy: preset configurable thresholds (overload: CPU > 80% / Memory > 75%; idle: CPU < 60% / Memory < 65% for 5 seconds), and scheduling threads are executed based on these metrics. Overload degradation: Lock core task resources (increase thread priority, bind CPU cores), and degrade non-core tasks (extend DIAS archiving interval, simplify DVisS rendering logic). Idle task recovery: Gradually restore non-core tasks according to the principle of starting with the least important tasks and then moving to the most important ones, to avoid resource fluctuations; Interruption mechanism: If an overload is triggered during the recovery process, the system will immediately revert to a degraded state.

[0038] S6. The data visualization service reads the resource status information from the control signal area and adaptively adjusts the resource consumption strategy of its visualization task accordingly.

[0039] The core objective of this invention is to overcome the architectural and resource deficiencies of existing technologies and provide a generalized dual-service decoupling architecture and resource adaptive scheduling method and system: architectural isolation is achieved by splitting data access processing services and data visualization / consumption services; efficient communication between services is achieved through ring-shared memory dual partitioning; and a real-time resource awareness-dynamic priority adjustment-dual-service collaboration mechanism is used to adapt to data flow and resource fluctuations, ultimately achieving highly stable reception, low-latency processing, and real-time visualization of large-scale parallel streaming data.

[0040] Example 2 A resource-adaptive real-time visualization system for multi-device data streams is disclosed. This system implements the aforementioned method. For a detailed description of the method, please refer to the corresponding description in the above method embodiments; it will not be repeated here. Figure 2 , Figure 3 , Figure 6 As shown, the system 100 includes: The data access and processing service module 110 is configured to run as an independent process, used to receive and preprocess parallel streaming data from multiple terminal devices, and to execute core tasks and non-core tasks according to the task layering strategy. Preferably, the data access and processing service module establishes a connection with the terminal device or data gateway through a network protocol, and its preprocessing operations include at least data verification, invalid packet removal, and time-series synchronization of streaming data from multiple devices.

[0041] This system achieves dual-service collaboration by integrating monitoring and scheduling threads within DIAS, combined with a control signal area. Tasks are hierarchically layered based on the priority of tasks in streaming data scenarios: Core tasks: DIAS data reception, data verification, and time-series synchronization; DVisS key data visualization and anomaly signal alarms; Non-core tasks: DIAS historical data archiving and storage; DVisS non-critical data rendering and data smoothing.

[0042] The data visualization service module 120 is configured to run as an independent process and is used to visualize preprocessed streaming data in real time. This system adheres to the principles of service isolation and efficient memory-level communication, providing the infrastructure for general scenarios. The dual-service process isolation design includes: Data Access and Processing Service (DIaS): an independent process (without a user interface, supporting background / system service operation), responsible for parallel streaming data access, data verification, time-series synchronization, and preprocessing of large-scale terminal devices, dividing internal tasks into core / non-core categories to provide a basis for resource scheduling; Data Visualization Service (DVisS): An independent process (supporting desktop / front-end visualization), responsible for real-time display of streaming data, user interaction, and key information alerts; The two services are independent processes at the operating system level, with resources isolated from each other. Even if DVisS experiences a brief pause due to rendering load or user interaction, DIAS can still continuously receive and process streaming data without being affected, thus avoiding the spread of single point of failure at the architectural level.

[0043] The ring-shaped shared memory communication module 130 is implemented based on operating system-level shared memory and includes at least a physically isolated streaming data buffer and a control signal area, used to realize low-latency data and status communication between the data access and processing service module and the data visualization service module. The data visualization service module is configured to adaptively adjust its visualization strategy based on the resource status information read from the control signal area.

[0044] This system provides a ring-shaped shared memory with dual physical partitions. It employs an operating system-level ring-shaped shared memory to achieve low-latency communication between services, abstracted as two independent physical partitions. The core balances simplicity and functionality through physical isolation and logical sub-partitions. The streaming data buffer (physical partition 1) stores parallel streaming data preprocessed by DIAS, including device identifiers, unified timestamps, and data payloads. Its capacity is adapted to the requirement of "hundreds of devices × 10 minutes of data". FIFO transmission is achieved through write pointers (controlled by DIAS) and read pointers (controlled by DVisS). When the buffer is full, the oldest unread data is overwritten to ensure that new data is prioritized and to avoid data accumulation. This partition is only used for streaming data transmission and does not carry any control information.

[0045] The control signal area (physical partition 2, internally divided into 2 logical sub-partitions) has a lightweight capacity (KB level) and is physically isolated from the streaming data buffer. Internally, it is divided into two independent logical sub-partitions by memory address offsets, which do not interfere with each other: Monitoring data sub-partition (logical sub-partition 1): It is dedicated to storing system resource monitoring data (CPU utilization, memory usage, network I / O throughput), which is exclusively written by the DIAS monitoring thread and exclusively read by the scheduling thread, and thread safety is ensured by read-write locks; Status identifier sub-partition (logical sub-partition 2): It is dedicated to storing storage and coordination status identifiers (such as "overload / idle" status codes + timestamps), which are exclusively written by the DIAS scheduling thread and exclusively read by DVisS, and millisecond-level status synchronization is achieved with the help of an event triggering mechanism.

[0046] This system provides a general streaming data processing flow: DIAS establishes a connection with terminal devices / data gateways via network protocols (TCP / UDP) and receives parallel streaming data; after data verification to remove invalid packets and time synchronization to unify the timestamps of multiple devices, the regularized streaming data is written into the streaming data buffer; the write operation of DIAS and the read operation of DVisS are asynchronous and independent, ensuring that the data stream is not interrupted.

[0047] Preferably, the data visualization service module is configured with a status monitoring unit, which periodically reads the status identifier from the status identifier sub-area and controls the working mode of the visualization rendering unit according to the status identifier: when overloaded, it switches to a simplified mode that only renders key data and triggers alarms; when idle, it switches to a standard mode that performs full data rendering.

[0048] Preferably, in the ring shared memory communication module: The streaming data buffer is configured to store the regular streaming data written by the data access and processing service module in a FIFO manner, and to be read by the data visualization service module. Its capacity is determined based on the number of terminal devices and the preset data retention time. The control signal area is further divided into a logically isolated monitoring data sub-area and a status identifier sub-area by memory address offset. The monitoring data sub-area is used to store system resource monitoring data written by the resource monitoring and adaptive scheduling module, and the status identifier sub-area is used to store system status identifiers written by the resource monitoring and adaptive scheduling module.

[0049] The resource monitoring and adaptive scheduling module 140 is integrated into the data access and processing service module. It is used to monitor system resources in real time, dynamically adjust task priorities based on resource status, and write resource status information into the control signal area. Preferably, the resource monitoring and adaptive scheduling module includes: The monitoring thread periodically collects at least one system resource indicator among CPU utilization, memory usage, and network I / O throughput at a preset low frequency, and securely writes the system resource indicator into the monitoring data sub-area through a read-write lock. The scheduling thread reads the system resource indicators of the monitoring data sub-area and compares them with preset overload thresholds and idle thresholds to determine the system load status. The scheduling thread executes a dynamic scheduling strategy based on the judgment result. The dynamic scheduling strategy includes: locking core task resources and downgrading non-core tasks in an overloaded state, and gradually restoring non-core tasks in an idle state. The scheduling thread also writes a status identifier representing the load status into the status identifier sub-area.

[0050] Specifically, the real-time resource monitoring provided by this system is as follows: a low-priority monitoring thread is started in DIAS to collect system resource indicators (CPU utilization, memory usage, network I / O throughput) at a frequency of 1-2 times / second, and store them in the control signal area of ​​the circular shared memory. Thread safety is ensured through read-write locks, and the data is available for the scheduling thread to read.

[0051] The dynamic scheduling strategy is as follows: Preset configurable thresholds (overload: CPU > 80% / Memory > 75%; idle: CPU < 60% / Memory < 65% for 5 seconds), and scheduling threads are executed based on these metrics. Overload degradation: Lock core task resources (increase thread priority, bind CPU cores), and degrade non-core tasks (extend DIAS archiving interval, simplify DVisS rendering logic). Idle task recovery: Gradually restore non-core tasks according to the principle of "light tasks first, heavy tasks later" to avoid resource fluctuations; Interruption mechanism: If an overload is triggered during the recovery process, the system will immediately revert to a degraded state.

[0052] The dual-service collaboration mechanism works as follows: the scheduling thread writes the overload / idle status to the control signal area, and DIAS reads the status in real time and adapts accordingly: when overloaded, only key data is displayed, and when idle, the full visualization is restored, thus realizing resource collaboration between services.

[0053] Example 3 A computer device 200, such as Figure 7 As shown, the system includes a memory 210, a processor 220, and a computer program 230 stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a resource-adaptive real-time visualization method for multi-device data streams. For a detailed description of the method, please refer to the corresponding description in the above method embodiments; it will not be repeated here.

[0054] Example 4 A computer-readable storage medium, such as Figure 8 As shown, a computer program is stored thereon, which, when executed by a processor, implements the steps of a resource adaptive real-time visualization method for multi-device data streams. For a detailed description of the method, please refer to the corresponding description in the above method embodiments, which will not be repeated here.

[0055] The number of devices and processing scale described herein are for the purpose of simplifying the description of the invention. Applications, modifications, and variations of the invention will be readily apparent to those skilled in the art.

[0056] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

[0057] The apparatus, computer device, and non-volatile computer storage medium and method provided in the embodiments of this specification are corresponding. Therefore, the apparatus, computer device, and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, computer device, and non-volatile computer storage medium will not be repeated here.

[0058] Those skilled in the art will also know that, besides implementing the controller in the form of purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller take the form of logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices included within it for implementing various functions can also be considered structures within that hardware component. Alternatively, the devices for implementing various functions can be considered as both software units implementing the method and structures within a hardware component.

[0059] The systems, apparatuses, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above apparatuses are described separately as various units based on their functions. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0060] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0061] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0064] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0065] This specification may be described in the general context of computer-executable instructions, such as program units, that are executed by a computer. Generally, program units include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification may also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program units may reside in local and remote computer storage media, including storage devices.

[0066] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0067] The above description is merely an embodiment of this specification and is not intended to limit the scope of one or more embodiments of this specification. Various modifications and variations can be made to one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of one or more embodiments of this specification.

Claims

1. A resource adaptive real-time visualization method for multi-device data streams, characterized in that, Includes the following steps: Parallel streaming data from multiple terminal devices is received and preprocessed by an independently operating data access and processing service; The preprocessed streaming data is visualized in real time by a standalone data visualization service. Data communication between the data access and processing service and the data visualization service is achieved through an operating system-level ring shared memory, wherein the ring shared memory is at least divided into a physically isolated streaming data buffer and a control signal area. The data access and processing service divides its processing tasks into core tasks and non-core tasks according to a preset task layering strategy. The data access and processing service monitors the resource status of the host system in real time, and dynamically adjusts the resource allocation priority of the core tasks and the non-core tasks based on the resource status, while writing the resource status information into the control signal area. The data visualization service reads the resource status information from the control signal area and adaptively adjusts the resource consumption strategy of its visualization tasks accordingly.

2. The resource adaptive real-time visualization method for multi-device data streams as described in claim 1, characterized in that, The steps for implementing data communication between the data access and processing service and the data visualization service through operating system-level ring shared memory include: The data access and processing service writes the preprocessed regularized streaming data into the streaming data buffer. The regularized streaming data includes at least a device identifier, a unified timestamp, and a data payload. The data visualization service reads the regularized streaming data from the streaming data buffer and visualizes it. When the streaming data buffer is full, new data is written by overwriting the oldest unread data to prioritize the transmission of new data.

3. The resource adaptive real-time visualization method for multi-device data streams as described in claim 1, characterized in that, The control signal area is further divided into a logically isolated monitoring data sub-area and a status identifier sub-area; The real-time monitoring of the host system's resource status includes: periodically collecting system resource indicators by the monitoring thread in the data access and processing service, and writing the system resource indicators into the monitoring data sub-area; The dynamic adjustment of resource allocation priority based on the resource status includes: the scheduling thread in the data access and processing service reads the system resource indicators in the monitoring data sub-area, and determines whether the current system is in an overloaded or idle state according to the preset resource threshold, and then executes the corresponding dynamic scheduling strategy.

4. The resource adaptive real-time visualization method for multi-device data streams as described in claim 3, characterized in that, The dynamic scheduling strategy includes: When the system is determined to be in an overload state, an overload degradation strategy is executed: the priority of the thread corresponding to the core task is increased and / or bound to a specific CPU core, while the resource allocation of the non-core task is reduced or suspended. When it is determined that the current system is in an idle state for a preset duration, an idle recovery strategy is executed: the resource allocation of the non-core tasks is gradually restored; If an overload state is triggered again during the idle recovery process, the system will immediately revert to the overload degradation strategy.

5. The resource adaptive real-time visualization method for multi-device data streams as described in claim 4, characterized in that, The reduction or suspension of resource allocation for non-core tasks includes at least one of the following: extending the execution interval of historical data archiving tasks in the data access and processing service; simplifying the rendering logic of non-critical data in the data visualization service or reducing its rendering frame rate.

6. The resource adaptive real-time visualization method for multi-device data streams as described in claim 3, characterized in that, The scheduling thread in the data access and processing service also writes a status flag representing the system overload or idle state into the status flag sub-area. The data visualization service reads the status identifier from the status identifier sub-area and adjusts its visualization tasks accordingly, including: when the status identifier is overloaded, rendering only key data and triggering alarms; when the status identifier is idle, restoring complete visualization rendering.

7. A resource adaptive real-time visualization method for multi-device data streams as described in any one of claims 1 to 6, characterized in that, The core tasks include at least: data reception, data verification, and time synchronization; the non-core tasks include at least: historical data archiving and storage.

8. A resource-adaptive real-time visualization system for multi-device data streams, characterized in that, The system is used to implement the method as described in any one of claims 1 to 7, comprising: The data access and processing service module is configured to run as an independent process, used to receive and preprocess parallel streaming data from multiple terminal devices, and to execute core tasks and non-core tasks according to the task layering strategy. The data visualization service module is configured to run as an independent process and is used for real-time visualization of preprocessed streaming data. A ring-shaped shared memory communication module, implemented based on operating system-level shared memory, includes at least a physically isolated streaming data buffer and a control signal area, used to realize low-latency data and status communication between the data access and processing service module and the data visualization service module; The resource monitoring and adaptive scheduling module is integrated into the data access and processing service module. It is used to monitor system resources in real time, dynamically adjust task priorities based on resource status, and write resource status information into the control signal area. The data visualization service module is configured to adaptively adjust its visualization strategy based on the resource status information read from the control signal area.

9. A resource-adaptive real-time visualization system for multi-device data streams as described in claim 8, characterized in that, In the ring-shaped shared memory communication module: The streaming data buffer is configured to store the regular streaming data written by the data access and processing service module in a FIFO manner, and to be read by the data visualization service module. Its capacity is determined based on the number of terminal devices and the preset data retention time. The control signal area is further divided into a logically isolated monitoring data sub-area and a status identifier sub-area by memory address offset. The monitoring data sub-area is used to store system resource monitoring data written by the resource monitoring and adaptive scheduling module, and the status identifier sub-area is used to store system status identifiers written by the resource monitoring and adaptive scheduling module.

10. A resource-adaptive real-time visualization system for multi-device data streams as described in claim 9, characterized in that, The resource monitoring and adaptive scheduling module includes: The monitoring thread periodically collects at least one system resource indicator among CPU utilization, memory usage, and network I / O throughput at a preset low frequency, and securely writes the system resource indicator into the monitoring data sub-area through a read-write lock. The scheduling thread reads the system resource indicators of the monitoring data sub-area and compares them with preset overload thresholds and idle thresholds to determine the system load status. The scheduling thread executes a dynamic scheduling strategy based on the judgment result. The dynamic scheduling strategy includes: locking core task resources and downgrading non-core tasks in an overloaded state, and gradually restoring non-core tasks in an idle state. The scheduling thread also writes a status identifier representing the load status into the status identifier sub-area.

11. A resource-adaptive real-time visualization system for multi-device data streams as described in claim 10, characterized in that, The data visualization service module is configured with a status monitoring unit, which periodically reads the status identifier from the status identifier sub-area and controls the working mode of the visualization rendering unit according to the status identifier: when overloaded, it switches to a simplified mode that only renders key data and triggers alarms; when idle, it switches to a standard mode that performs full data rendering.

12. A resource-adaptive real-time visualization system for multi-device data streams as described in claim 8, characterized in that, The data access and processing service module establishes a connection with the terminal device or data gateway through a network protocol. Its preprocessing operations include at least data verification, invalid packet removal, and time synchronization of streaming data from multiple devices.

Images