Data management systems and methods for cell and gene therapy
By utilizing protocol adapters and a multi-layered architecture to process heterogeneous data in a data management system for cell and gene therapy, the problem of data silos has been solved, enabling efficient and real-time data management and compliant storage, thus meeting the data compliance requirements for cell and gene therapy.
Patent Information
- Application Number
- CN202611317194.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-27
- Publication Date
- 2026-11-17
AI Technical Summary
The existing data management model lacks a unified management mechanism, resulting in low data management efficiency, insufficient real-time performance, severe data silos, and inconsistent formats that fail to meet compliance audit requirements in cell and gene therapy.
A data management system is provided, including a device access layer module, a data processing layer module, a data storage layer module, and an application service layer module. It realizes unified format conversion and standardization of heterogeneous device data through multiple protocol adapters, uses message queues for data distribution, and adopts relational databases, time-series databases, and cache databases for differentiated storage. It combines Kafka Streams and Apache Flink for stream processing and computation.
It enables unified access and standardization of multi-source heterogeneous data, improves data processing efficiency and real-time performance, reduces labor costs, meets data compliance requirements under regulatory frameworks such as GMP, and enhances the standardization and controllability of the production process.
Smart Images

Figure CN123120352A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of multi-device data collaborative management technology, specifically providing a data management system and method for cell and gene therapy. Background Technology
[0002] Cell and gene therapy (CGT) is a core direction of next-generation precision medicine. Its production process involves multiple complex steps such as cell culture, quality testing, and cell storage, requiring the coordinated work of various equipment such as cell culture equipment, quality testing equipment, and cell storage equipment.
[0003] Currently, the common data management model in CGT industrial scenarios involves each production device independently recording and storing its own generated data, resulting in a decentralized data storage structure. When data aggregation, analysis, or audit traceability is required, operators must log into each device's terminal or access its local database, manually export data files, and then manually organize, compare, and summarize them using spreadsheets and other tools. In this heterogeneous data environment, it is difficult to establish effective correlations between data generated by different devices. Therefore, the existing data management model suffers from technical problems such as low data management efficiency, insufficient real-time performance, severe data silos, and inconsistent formats that fail to meet compliance audit requirements. Summary of the Invention
[0004] This application aims to solve the aforementioned technical problems, namely, to address the issues of low data management efficiency, insufficient real-time performance, severe data silos, and inconsistent formats that fail to meet compliance audit requirements caused by the lack of a unified management mechanism in existing data management models.
[0005] In a first aspect, this application provides a data management system for cell and gene therapy, comprising a device access layer module, a data processing layer module, a data storage layer module, and an application service layer module, wherein,
[0006] The device access layer module is used to receive heterogeneous device data reported by various heterogeneous devices, convert the heterogeneous device data of each protocol into standardized business data in a unified format through multiple protocol adapters, and send the standardized business data to the first topic of the message queue.
[0007] The data processing layer module is used to pull the standardized business data from the first topic, perform data cleaning, protocol conversion, data enhancement, rule judgment and real-time calculation on the standardized business data in sequence, generate target business data and alarm data, send the target business data to the second topic of the message queue, and send the alarm data to the third topic of the message queue.
[0008] The data storage layer module includes a relational database, a time-series database, and a cache database. The data storage layer module is used to pull the target business data from the second topic and write the target business data into the time-series database; and to pull the alarm data from the third topic, write the alarm data into the relational database, and push it to the publish-subscribe channel of the cache database.
[0009] The application service layer module provides a data query interface and a real-time data push interface. The data query interface is used to obtain data from the data storage layer module, and the real-time data push interface is used to push real-time data to the client. The application service layer module is also used to perform permission verification on the data query interface and the real-time data push interface.
[0010] Optionally, the device access layer module includes an MQTT protocol adapter, an HTTP protocol adapter, and a TCP protocol adapter;
[0011] The MQTT protocol adapter is built on the Vert.x framework to create an MQTT server, which is used to receive topic messages published by devices via the MQTT protocol;
[0012] The HTTP protocol adapter is used to provide a RESTful endpoint to receive JSON data, XML data, or file data reported by the device via a POST request; the file data includes image, CSV, Excel, or PDF format files;
[0013] The TCP protocol adapter is built on the Netty framework to create a TCP server, which is used to receive binary messages reported by devices through a custom TCP protocol.
[0014] Optionally, the device access layer module further includes a device registration management submodule, which is used to pre-store the registration information of each device, including device identifier, protocol type and data parsing template;
[0015] After receiving the heterogeneous device data, the device access layer module extracts the device identifier from the heterogeneous device data, finds the corresponding registration information based on the device identifier, and performs protocol parsing and format conversion on the heterogeneous device data according to the protocol type and data parsing template in the registration information to generate the standardized business data.
[0016] Optionally, the relational database is used to store device metadata, user information, alarm records, and rule configuration data;
[0017] The time-series database is used to store time-series sampling data reported by the device, and the time-series sampling data includes timestamps, device identifiers, and indicator values collected by the device;
[0018] The cache database is used to cache the device's real-time status data, real-time alarm data, and rule configuration data.
[0019] Optionally, the data storage layer module is further configured to retrieve the target business data from the second topic and then store the target business data into the time-series database through a batch write interface; the batch write interface accumulates multiple target business data within a preset time window, and when the accumulated data reaches a preset batch threshold or the waiting time reaches a preset timeout threshold, the accumulated multiple target business data are written into the time-series database at once.
[0020] Optionally, the system further includes a front-end display layer module, which is used to display the device overview screen, device details page, data analysis page and alarm center page, and receive real-time data pushed by the application service layer module through web sockets;
[0021] The device overview screen displays map markers, key indicator cards, and real-time alarm scroll bars; the device details page displays real-time data curves, historical data queries, and device parameter configurations; the data analysis page displays multi-device comparisons, custom time queries, and report exports; and the alarm center page displays alarm lists and processing records.
[0022] Optionally, the front-end presentation layer module adopts a front-end and back-end separation architecture, is built using the Vue.js framework with ElementUI or Ant Design component library, uses ECharts or Highcharts chart library to display real-time data curves and historical trend charts, and integrates Leaflet or Baidu Map component to display the device's geographical location and operating status on the device overview screen.
[0023] Optionally, the data processing layer performs the following processing using a hybrid stream processing engine combining Kafka Streams and Apache Flink:
[0024] Data cleaning is used to remove outliers from standardized business data that exceed the preset normal range, add the current system timestamp to data with missing timestamps, and perform format validation on the data types of each field. Data that fails the format validation is forwarded to the dead letter topic of the message queue.
[0025] Protocol conversion processing is used to find the corresponding parsing script based on the device model and parse the reported binary data or non-standard field names into structured fields;
[0026] Data augmentation processing is used to retrieve the static information of the corresponding device from the relational database based on the device identifier in the data, and store the associated static information into the data stream. The static information is pre-cached in KafkaStreams' GlobalKTable.
[0027] The rule-based judgment process is used to match data according to preset threshold rules. Single-event threshold rules are executed by Kafka Streams, while multi-event time-series pattern matching is executed by Apache Flink CEP. When data triggers an alarm rule, alarm data is generated and sent to the third topic of the message queue. Business data that does not trigger an alarm rule continues to flow to the next stage.
[0028] Real-time computation processing is used to perform real-time calculations on business data using a sliding window averaging algorithm, filtering out instantaneous noise and glitches in the data to extract stable signal features and generate target business data.
[0029] Data splitting processing is used to send the target business data to a second topic of the message queue for consumption by the storage layer, and to send the alarm data to a third topic of the message queue for consumption by the real-time push channel.
[0030] Optionally, the application service layer module is built on the Spring Boot framework and is used to provide RESTful APIs for front-end calls to perform device management functions, data query functions, alarm management functions, real-time data push functions, and user authentication and permission management functions.
[0031] The device management functions include device registration, device editing, device deletion, and device list and details query;
[0032] The data query function includes aggregating and querying historical data from the time-series database based on device identifier and time range;
[0033] The alarm management functions include alarm history query, alarm confirmation, and alarm rule configuration;
[0034] The real-time data push function pushes real-time device data or alarm data to the client via the WebSocket protocol.
[0035] The user authentication and permission management function integrates the Spring Security and SaToken frameworks to implement role-based access control.
[0036] In a second aspect, this application provides a data management method for cell and gene therapy, comprising:
[0037] The device access layer receives heterogeneous device data reported by various heterogeneous devices, converts the heterogeneous device data of each protocol into standardized business data in a unified format through multiple protocol adapters, and sends the standardized business data to the first topic of the message queue.
[0038] The standardized business data is pulled from the first topic through the data processing layer. The standardized business data is then subjected to data cleaning, protocol conversion, data enhancement, rule judgment and real-time calculation processing in sequence to generate target business data and alarm data. The target business data is sent to the second topic of the message queue and the alarm data is sent to the third topic of the message queue.
[0039] The target business data is retrieved from the second topic through the data storage layer and written into the time series database; and the alarm data is retrieved from the third topic, written into the relational database and pushed to the publish-subscribe channel of the cache database.
[0040] The application service layer provides a data query interface and a real-time data push interface. Data can be queried from the relational database, the time-series database, or the cache database through the data query interface, and real-time data can be pushed to the client through the real-time data push interface.
[0041] By adopting the above technical solution, this application, through setting up a device access layer module containing multiple protocol adapters, can be compatible with and adapt to multiple communication protocols, uniformly converting heterogeneous device data from different sources and with different formats into standardized business data. This breaks down data format barriers between devices, solves the data silo problem, and achieves unified access and standardization of multi-source heterogeneous data. Furthermore, by setting up a data processing layer module, a data storage layer module, and an application service layer module, and utilizing message queues for data distribution between modules, it achieves automated and hierarchical data processing, constructing a fully automated management process from data access, cleaning, processing, storage to application. This replaces the inefficient and error-prone manual aggregation mode, significantly improving data processing efficiency and real-time performance. This approach reduces labor costs. By storing alarm data and target business data in relational databases and time-series databases respectively, and using a cache database to cache the latest status data, it achieves differentiated and optimized data storage. This ensures efficient writing and querying performance of business data, as well as reliable storage of structured data such as alarms. At the same time, caching improves the access speed of real-time data. By centrally managing and standardizing the dispersed data, a complete view of the equipment data lifecycle can be formed, providing a solid data foundation for data analysis, process traceability, and quality auditing. This helps meet the stringent compliance requirements for data integrity and traceability under GMP and other regulatory frameworks in the field of cell and gene therapy, and improves the standardization and controllability of the production process. Attached Figure Description
[0042] The preferred embodiments of this application are described below with reference to the accompanying drawings, in which:
[0043] Figure 1 This is a schematic diagram of the structure of a data management system for cell and gene therapy provided in an embodiment of this application. Figure 1 ;
[0044] Figure 2 This is a schematic diagram of the structure of a data management system for cell and gene therapy provided in an embodiment of this application. Figure 2 ;
[0045] Figure 3 This is a flowchart of the steps of a data management method for cell and gene therapy provided in the embodiments of this application. Detailed Implementation
[0046] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0047] To address the aforementioned technical problems, this application provides a data management system assembly method for cell and gene therapy. By setting up a device access layer module containing multiple protocol adapters, it can be compatible with and adapt to various communication protocols, uniformly converting heterogeneous device data from different sources and with different formats into standardized business data. This breaks down data format barriers between devices, solves the data silo problem, and achieves unified access and standardization of multi-source heterogeneous data. By setting up a data processing layer module, a data storage layer module, and an application service layer module, and using message queues for data distribution between modules, it achieves automated and hierarchical data processing, constructing a full-link automated management process from data access, cleaning, processing, storage to application. This replaces the inefficient and error-prone manual aggregation mode, significantly improving efficiency. This approach improves data processing efficiency and real-time performance while reducing labor costs. By storing alarm data and target business data in relational databases and time-series databases respectively, and using a cache database to cache the latest status data, it achieves differentiated and optimized data storage. This ensures efficient writing and querying performance of business data, as well as reliable storage of structured data such as alarms. At the same time, caching improves the access speed of real-time data. By centrally managing and standardizing dispersed data, a complete view of the equipment data lifecycle can be formed, providing a solid data foundation for data analysis, process traceability, and quality auditing. This helps meet the stringent compliance requirements for data integrity and traceability under GMP and other regulatory frameworks in the cell and gene therapy field, and improves the standardization and controllability of the production process.
[0048] Please see Figure 1 This application provides a data management system for cell and gene therapy, including a device access layer module 100, a data processing layer module 200, a data storage layer module 300, and an application service layer module 400.
[0049] The device access layer module 100 is used to receive heterogeneous device data reported by various heterogeneous devices, convert the heterogeneous device data of each protocol into standardized business data in a unified format through multiple protocol adapters, and send the standardized business data to the first topic of the message queue.
[0050] In this embodiment, the device access layer module 100 integrates multiple specific protocol adapters. For example, the device access layer module 100 includes an MQTT protocol adapter, an HTTP protocol adapter, and a TCP protocol adapter. After receiving heterogeneous device data, the device access layer module 100 determines the target protocol adapter from among the multiple adapters based on the protocol type corresponding to the heterogeneous device data. It then calls the target protocol adapter to parse and transform the heterogeneous device data, converting it into standardized business data in a unified format, and sends the standardized business data to the first topic of the message queue. The standardized business data is in JSON format and includes a device identifier (deviceId), a timestamp, and a fields field. The fields field is a key-value pair array used to store the specific business data reported by the device.
[0051] The design of the device access layer module 100 addresses the data heterogeneity issue at its source. By translating data from different devices and using different protocols into standardized business data within the system, it provides a unified and clean data foundation for all subsequent processing flows. In this way, the data management system shields the complexity of the underlying hardware and communication protocols, achieving unified access and standardization of multi-source heterogeneous data and breaking down data format barriers between devices.
[0052] Next, the data processing layer module 200 is used to pull standardized business data from the first topic, and sequentially perform data cleaning, protocol conversion, data enhancement, rule judgment and real-time calculation on the standardized business data to generate target business data and alarm data. The target business data is sent to the second topic of the message queue, and the alarm data is sent to the third topic of the message queue.
[0053] The data processing layer module 200 is designed to extract valuable information from massive amounts of raw data and make intelligent decisions and responses based on preset business rules. By splitting the processed data—sending target business data to the second topic of the message queue and alarm data to the third topic—the data management system achieves differentiated processing of data with different priorities. This ensures that critical alarm events are processed first, while target business data enters the storage process in an orderly manner, thereby achieving automation and intelligence in data processing.
[0054] The data storage layer module 300 includes a relational database, a time-series database, and a cache database. The data storage layer module is used to pull target business data from the second topic and write the target business data into the time-series database; and to pull alarm data from the third topic, write the alarm data into the relational database, and push it to the publish-subscribe channel of the cache database.
[0055] The data storage layer module 300 is designed to select the most suitable storage solution for data with different characteristics. By storing alarm data that requires transaction support and complex queries in a relational database, storing massive amounts of time-series data in a dedicated time-series database, and placing frequently accessed real-time status data in a high-speed cache, the system solves the problem that a single database cannot meet all needs, and achieves overall optimization of storage resources and query performance.
[0056] The application service layer module 400 provides a data query interface and a real-time data push interface. The data query interface is used to obtain data from the data storage layer module 300, and the real-time data push interface is used to push real-time data to the client. The application service layer module 400 is also used to perform permission verification on the data query interface and the real-time data push interface.
[0057] The application service layer module 400 is designed to encapsulate the complex underlying data storage and access logic, providing concise, secure, and standard API services. Upper-layer applications (such as the front-end presentation layer module 500) do not need to concern themselves with the specific database where the data is stored or how to perform efficient queries; they only need to call the interfaces provided by the application service layer. In this way, the system achieves complete decoupling between the front-end and back-end, ensuring not only the security and consistency of data access but also improving the development efficiency and flexibility of upper-layer applications. The data management system provided in this application, through its layered and decoupled architecture, achieves fully automated management of the entire data chain, from data access, processing, and storage to application.
[0058] In a specific example, the data management system uses Apache Kafka as the unified data bus for message queue selection. Kafka's core value lies in three aspects: First, its peak-shaving and valley-filling capabilities effectively buffer instantaneous peaks in device data reporting, preventing downstream processing systems from being overwhelmed by traffic surges; second, its partitioning model naturally guarantees the sequentiality of data from the same device (same key), meeting the stringent time-series processing requirements of IoT scenarios; and third, its support for independent subscriptions by multiple consumer groups, allowing the same data to be simultaneously supplied to different consumers such as the stream processing engine, data storage service, and real-time push channel, achieving write-once, consume-multiple times data.
[0059] In terms of theme planning, the data management system defines three core themes: the first theme receives standardized business data from the access layer as the input source for this layer; the second theme carries the target business data generated after a complete processing chain for the storage layer to consume; and the third theme specifically receives real-time alarm data for the push channel to achieve second-level notification.
[0060] In selecting the stream processing engine, the data management system adopts a layered hybrid architecture, flexibly choosing the most suitable technology components based on the complexity of the business scenario. For simple processing scenarios, lightweight processing logic is written using Kafka Streams. This engine is directly embedded in the application process as a client library, eliminating the need to maintain a separate cluster and enabling stateless or lightly stateful streaming operations with sub-millisecond latency. For complex processing scenarios, Apache Flink is integrated to perform heavy computations such as window aggregation and pattern matching. Flink provides exactly-once processing semantics, terabyte-level state management capabilities, and a robust event-time handling mechanism, easily handling out-of-order and delayed data. The two complement each other: Kafka Streams handles the rapid preprocessing and distribution of the entire dataset, while Flink performs deep analysis only on the subset of data requiring complex computations, thus achieving an optimal balance between development efficiency, resource costs, and processing latency.
[0061] Furthermore, in a preferred embodiment, to ensure broad compatibility with various devices commonly used in cell and gene therapy, the device access layer module 100 integrates several specific protocol adapters. Specifically, it includes an MQTT protocol adapter, an HTTP protocol adapter, and a TCP protocol adapter. The MQTT protocol adapter, built on the Vert.x framework, is an MQTT server used to receive topic messages published by devices via the MQTT protocol. The HTTP protocol adapter provides a RESTful endpoint to receive JSON, XML, or file data reported by devices via POST requests; file data includes image, CSV, Excel, or PDF format files. The TCP protocol adapter, built on the Netty framework, is a TCP server used to receive binary messages reported by devices via a custom TCP protocol.
[0062] This design enables the system to natively support the three most mainstream communication protocols in the current industrial IoT and internet fields, covering everything from sensors requiring low-power, low-bandwidth communication to analytical instruments that need to report data via web services, and traditional industrial equipment using proprietary binary protocols. By providing dedicated, high-performance adapters for each mainstream protocol, the system achieves efficient and reliable access to heterogeneous devices, solving the data access difficulties caused by protocol inconsistencies.
[0063] In another preferred embodiment, to achieve automated management and parsing of access devices and their data, the device access layer module 100 also includes a device registration management submodule. This submodule pre-stores registration information for all managed devices in the system. This information acts like an ID card for each device, detailing its unique device identifier, the protocol type used, and the data parsing template used to parse its reported data. When the device access layer module 100 receives a batch of heterogeneous device data, it first extracts the device identifier from the data and then uses this identifier to query the device registration management submodule to obtain the corresponding parsing rules. Based on these rules, the system can automatically perform protocol parsing and format conversion on the data. By introducing this device registration management submodule, the system decouples the data parsing logic from the code, transforming it into a configurable template. When a new type of device needs to be accessed, the administrator only needs to register the device in the system and configure its data parsing template, without modifying the code. This improves the system's flexibility and scalability, solving the maintenance difficulties and poor scalability problems caused by hard-coded parsing logic.
[0064] In one optional implementation, the three databases in the data storage layer module 300 each have clearly defined and complementary responsibilities. A relational database, such as MySQL, is specifically used to store highly structured data requiring transaction guarantees, such as device metadata (model, location, etc.), user information, complete alarm records, and alarm rule configuration data. A time-series database, such as InfluxDB, is specifically used to store large amounts of time-series sampled data reported by devices; its data model (timestamps, device identifiers, and device-collected metric values) is naturally well-suited for efficiently processing time-series data. A caching database, such as Redis, is used to cache data that requires fast access, including real-time device status data, the latest real-time alarm data, and rule configuration data cached to handle high-concurrency reads. Through this differentiated storage strategy, the system finds the optimal home for different types of data, thereby maximizing write and query efficiency while ensuring data reliability.
[0065] Furthermore, to address the challenges of high data acquisition frequency and large data volume in scenarios such as cell culture, the data storage layer module 300 employs a batch write optimization strategy when storing target business data into the time-series database. Specifically, it sets up a buffer in memory through a batch write interface. Within a preset time window (e.g., 5 seconds), this interface continuously accumulates the target business data pulled from the second topic. When the amount of data accumulated in the buffer reaches a preset batch threshold (e.g., 1000 records), or the waiting time reaches a preset timeout threshold (even if less than 1000 records), the interface writes all the data in the buffer to the time-series database in one go through a single database connection. This "consolidation" strategy reduces the number of network I / O operations and transaction overhead with the database, lowers the database write load, thereby improving data throughput and solving the database performance bottleneck problem that may be caused by high-frequency single-record data writing.
[0066] In a preferred embodiment, such as Figure 2 As shown, the data management system provided in this application also includes a front-end presentation layer module 500, serving as the human-computer interaction interface. This module is responsible for presenting the data processed and stored in the backend to the user in an intuitive and user-friendly manner. Its core functions are to display a device overview screen, device details page, data analysis page, and alarm center page. The device overview screen displays map annotations, key indicator cards, and real-time alarm scroll bars; the device details page displays real-time data curves, historical data queries, and device parameter configurations; the data analysis page displays multi-device comparisons, custom time queries, and report exports; and the alarm center page displays alarm lists and processing records. To achieve real-time data updates, the front-end presentation layer module 500 establishes a persistent connection with the application service layer module 400 via WebSocket, enabling it to receive real-time data actively pushed by the backend. The introduction of this module transforms the system's powerful backend data processing capabilities into user-perceptible and operable front-end functions, forming a complete closed loop from data collection to final value presentation, solving the problem that raw data is difficult for users to understand and use intuitively.
[0067] Building upon the aforementioned implementation methods, to enhance development efficiency and user experience, the front-end presentation layer module 500 can adopt a mature front-end / back-end separation architecture. The front-end can be built using the Vue.js framework, coupled with UI component libraries such as Element UI or Ant Design to quickly create a beautiful and consistent user interface. For data visualization, professional charting libraries such as ECharts or Highcharts are used to display real-time data curves and historical trend charts, leveraging their rich chart types and interactive features to vividly present the data. For geographic information display, map components such as Leaflet or Baidu Maps are integrated to intuitively display the device's geographical location and operating status on the device overview screen. By adopting these mature industry technologies, the development cycle can be significantly shortened, and the performance, stability, and scalability of the front-end application can be ensured, providing users with a superior interactive experience.
[0068] In a preferred embodiment, the data processing flow executed by the data processing layer module 200 includes several refined steps, specifically including data cleaning operation, protocol conversion operation, data augmentation operation, rule judgment operation, real-time calculation operation, and data diversion operation.
[0069] Data cleaning is performed by Kafka Streams. The cleaning process comprises three core tasks: First, format validation checks each data entry to ensure it contains required fields, the field types are correct, and the timestamp is within a reasonable range. Incorrectly formatted data is filtered and forwarded to a dead-letter queue for manual review. Second, deduplication uses Kafka Streams' state storage, based on device identifiers, data timestamps, and sequence numbers, to detect and remove duplicate reports, preventing contamination of subsequent statistical results. Finally, outlier filtering identifies data that clearly exceeds physical limits (e.g., a temperature sensor reporting -999℃ or a negative pressure value), classifying it as a sensor malfunction or transmission error, and transferring it to the dead-letter queue instead of further processing. The output of this stage is a clean, correctly formatted, standardized data stream.
[0070] Protocol conversion involves more than just format transformation. It also includes finding the corresponding parsing script based on the device model, resolving the reported binary data or non-standard field names into structured fields with clear business meaning that are consistent within the system. Specifically, this includes unifying the same physical quantity reported by different devices into the International System of Units (SI) (e.g., converting all pressure values to Pascals), mapping vendor-defined field names to the system's unified naming convention (e.g., mapping "temp", "temperature", and "T" to "temperature"), and formatting the raw timestamp to the system's unified time precision standard. This step is also implemented by Kafka Streams using a stateless mapping method, ensuring that all subsequent processing steps deal with semantically consistent data with unified units.
[0071] Data augmentation operations, performed during data stream processing, comprise two layers: The first layer involves retrieving static information about the device (such as its geographical location, department, model, installation date, maintenance personnel, and maintenance records) from a relational database in real time based on the device identifier within the data. This information is stored in a configuration topic and then broadcast to the local memory of all instances via Kafka Streams' GlobalKTable mechanism, achieving millisecond-level stream table association. The second layer involves dynamic context supplementation, including a summary of the device's recent operational status, such as the average value over the past five minutes, the last reported time, and the cumulative number of reports for the day. This information is maintained and queried using Kafka Streams' local state storage (based on RocksDB). Through this series of processes, the system not only ensures the accuracy and standardization of the data but also imbues the previously isolated measurement point data with rich business context, enhancing its value and usability.
[0072] Rule-based decision-making is a crucial decision point in the data processing layer, determining the direction of data flow and serving as the source of alarm data. It employs a hybrid processing architecture, processing rules based on their complexity. Simple rules are executed directly by Kafka Streams: for example, when the temperature exceeds a preset threshold of 85℃, or when the temperature change rate exceeds 10℃ per minute within one minute, the system immediately classifies it as an alarm event, marks the data, and sends it to the third topic. Complex rules are handled by the Flink CEP (Complex Event Processing) engine: for example, detecting multi-device time-series correlation patterns such as "within 3 seconds after device A's temperature exceeds 80℃, device B's pressure simultaneously falls below 0.5 MPa," or missing event patterns such as "a device has not sent a heartbeat for 5 consecutive reporting cycles." These complex logics involving multiple events, multiple devices, and time constraints are handled by Flink CEP's rule matching engine. All data triggering alarm rules is output to the third topic and enters the real-time push channel; regular business data that does not trigger any alarm rules continues to flow to the next stage.
[0073] Real-time computation operations perform window aggregation and statistical analysis on regular data streams that, after rule-based judgment, are not marked as alarms. This operation is handled by Apache Flink, fully leveraging its powerful stateful stream processing capabilities. Specific computation types include: sliding window aggregation, such as calculating the average and maximum temperature of each device or region every 10 seconds over the past minute for real-time trend monitoring; rolling window aggregation, such as summarizing the frequency and success rate of device reporting every 5 minutes for billing or service level assessment; and session window detection, such as identifying events where device reporting is interrupted for more than a certain duration for offline diagnostics. In terms of temporal semantics, all window operations adopt event time and incorporate a watermark mechanism to handle out-of-order data and delayed arrival issues, ensuring the time accuracy of the computation results. These aggregation results are output as target business data to a second topic for persistent storage in the device storage layer.
[0074] After the above five processing steps, the data is clearly divided into two output paths. The first path is alarm data, originating from all records that triggered business rules in the rule judgment stage. This data is sent to the third topic of the message queue. Real-time push channels subscribed to this topic (such as WebSocket services) will push alarm notifications to operations and maintenance personnel within milliseconds, ensuring that critical events can be responded to in a timely manner. The second path is target business data, containing regular business records that have undergone complete cleaning, enhancement, and aggregation calculations. This data is sent to the second topic of the message queue. Storage layer services subscribed to this topic (such as Elasticsearch for log retrieval, HDFS for historical archiving, and relational databases for structured storage) will persistently store the data for subsequent use in scenarios such as report generation, trend analysis, and AI model training. The two-path data splitting design ensures that alarm events are not affected by regular processes such as storage I / O, achieving real-time priority processing of high-priority data.
[0075] In another preferred embodiment, before performing data cleaning operations, the data processing layer module 200 further includes: pulling standardized business data from the first topic of the message queue. After performing real-time computing operations, the data processing layer module 200 further includes: periodically persisting the computing state of the real-time computing operations to a distributed file system through Apache Flink's Checkpoint mechanism to achieve state recovery in the event of task failure. The data stored in the dead-letter queue includes the original data content and a description of the failure reason, which can be queried and reprocessed by operations and maintenance personnel through the management interface.
[0076] In one optional implementation, the application service layer module 400 can be built on the Spring Boot framework, providing a complete set of RESTful APIs for front-end calls. These APIs cover the core business functions of the system, including: device management functions for the entire lifecycle management of devices (such as device registration, editing, deletion, and querying); data query functions for on-demand aggregation and querying of historical data from a time-series database; alarm management functions for querying, confirming, and configuring alarms; and real-time data push functions for proactively pushing real-time data to clients via the WebSocket protocol. Furthermore, to ensure system security, this module integrates mainstream security frameworks such as Spring Security and SaToken to implement role-based access control (RBAC), ensuring that users with different permissions can only access the data and functions they are authorized to access. In this way, the application service layer module 400 becomes a fully functional and secure back-end service center.
[0077] like Figure 3As shown, this application embodiment also provides a data management method for cell and gene therapy, which is a process-oriented manifestation of the functions of each module of the above-mentioned system. The method specifically includes the following steps: S11 to S14:
[0078] Step S11: Receive heterogeneous device data reported by various heterogeneous devices through the device access layer, convert the heterogeneous device data of each protocol into standardized business data in a unified format through multiple protocol adapters, and send the standardized business data to the first topic of the message queue.
[0079] Step S12: Pull standardized business data from the first topic through the data processing layer, and perform data cleaning, protocol conversion, data enhancement, rule judgment and real-time calculation on the standardized business data in sequence to generate target business data and alarm data. Send the target business data to the second topic of the message queue and send the alarm data to the third topic of the message queue.
[0080] Step S13: Pull target business data from the second topic through the data storage layer and write the target business data into the time series database; and pull alarm data from the third topic, write the alarm data into the relational database and push it to the publish-subscribe channel of the cache database.
[0081] Step S14: Provide data query interface and real-time data push interface through the application service layer. Query data from relational database, time-series database or cache database through the data query interface, and push real-time data to the client through the real-time data push interface.
[0082] In one specific embodiment, the data management system described in this application can be deployed in a server environment serving an automated cell culture platform (model H-ACCS-AC-PY). In this scenario, the device access layer module runs on a dedicated gateway server configured with load balancing, responsible for handling concurrent connections from various cell processing devices such as incubators, centrifuges, and microscopes. The data processing layer module is deployed as a stream processing framework cluster to meet the computational needs of complex data processing. The message queue adopts a high-throughput cluster deployment, with corresponding partitions and replicas configured for different topics to ensure the reliability and efficiency of data transmission.
[0083] In this embodiment, the deployment of the data storage layer module has also been optimized. The relational database adopts a master-slave replication mode to achieve read-write separation and improve query performance. The time-series database adopts a distributed cluster deployment to support the rapid writing and aggregation query of massive cell culture process data. The caching database adopts a sentinel mode deployment to ensure the high availability of the caching service. The application service layer module and the front-end presentation layer module are packaged into container images and deployed and managed through a container orchestration system, realizing elastic scaling and rapid iteration of services. Through this specific hardware and software configuration, the entire system provides stable, efficient, and reliable data collaborative management capabilities for the automated cell culture platform.
[0084] The following describes a minimal protection embodiment integrating multiple preferred technical features of this application. This embodiment aims to demonstrate the technical effects achievable when these technical features work together. In this embodiment, the data management system fully includes a device access layer module, a data processing layer module, a data storage layer module, an application service layer module, and a front-end presentation layer module. The device access layer module integrates adapters for MQTT, HTTP, and TCP protocols and has a built-in device registration management submodule. The data processing layer module executes a complete data processing flow, including cleaning, enhancement, and filtering. The data storage layer module adopts a hybrid storage architecture and implements a batch write strategy to a time-series database and a publish-subscribe mechanism for alarms. The application service layer module is built based on the Spring Boot framework and integrates a security framework. The front-end presentation layer module is built using technologies such as Vue.js, providing complete user interaction functionality.
[0085] The operation of this minimum protection scope embodiment will now be described using a specific workflow. Assume a cell culture chamber (device identifier "INC-001", protocol type MQTT) registered in the device registration management submodule reports data including temperature and CO2 concentration. The MQTT protocol adapter of the device access layer module receives this message, queries the device registration management submodule to obtain the parsing rules, successfully parses the data, encapsulates it into standardized business data, and then sends it to the first topic of the message queue.
[0086] The data processing layer module pulls the data from the first topic. First, it performs data cleaning to confirm the values are within normal ranges. Then, it performs data augmentation, adding static information "location:'Cleanroom A'" to the data record based on the "INC-001" registration information. Next, the rules engine determines that the data has not triggered any alarms. Therefore, this processed data is sent to the second topic of the message queue as target business data.
[0087] The consumer in the data storage layer module pulls the data from the second topic and places it into a batch write buffer, awaiting writing to the time-series database. Simultaneously, the latest status data is updated in the cache database with the key "INC-001". When the buffer is full or times out, the entire batch of data is written to the time-series database at once.
[0088] Meanwhile, a lab administrator is accessing the front-end presentation layer module via a browser. On the device details page he sees, a graph maintaining a WebSocket connection with the application service layer module is displaying a real-time data curve for "INC-001". When data in the cache database is updated, the application service layer module pushes the new data points to the front end via WebSocket, and the graph curve extends accordingly in real time. This seamless and complete process demonstrates how the embodiments of this application efficiently and in real-time complete the entire process from data acquisition to final presentation.
[0089] This embodiment achieves comprehensive technical effects by combining all the aforementioned technical features. First, by integrating multiple protocol adapters and device registration management, the system's data access capabilities and scalability are enhanced. Second, a refined data processing workflow ensures data quality and accuracy. Third, the combination of batch writing and caching mechanisms allows the system to maintain high performance and low latency even when processing massive amounts of data. Finally, the fully functional and modernized front-end and back-end applications improve user convenience. The synergistic effect of these technical features enables this system to be applied to complex application scenarios such as cell and gene therapy, which have stringent requirements for data real-time performance, accuracy, and compliance, achieving the goal of efficient, intelligent, and reliable collaborative data management.
[0090] The data management system and method described in this application have broad applicability in practical applications. One of its core application scenarios is in the fields of cell and gene therapy (CGT), biopharmaceuticals, and other life science research. In these scenarios, a large number of automated devices, such as cell culture incubators, bioreactors, centrifuges, flow cytometers, and liquid nitrogen tanks, require continuous 24 / 7 monitoring. This system can seamlessly integrate these heterogeneous devices, enabling centralized monitoring and recording of key parameters in the cell culture process (such as temperature, humidity, CO2 concentration, and pH).
[0091] For example, in a fully automated cell culture workstation, this system can collect real-time environmental parameters of the incubator, the rotation speed of the shaker, and cell morphology images captured by a timed microscopic imaging system. The data processing layer module can analyze this multimodal data in real time; for example, it can use a rule engine to determine whether the CO2 concentration deviates from the set range and trigger an alarm. After all the data is stored, it forms an electronic record of the entire life cycle of each batch of cells from resuscitation and culture to harvest, which is crucial for meeting compliance requirements of regulations such as GMP (Good Manufacturing Practice). Researchers can use the data analysis page of the front-end presentation layer module to compare cell growth rates under different culture conditions or trace back all environmental parameters when a problem occurred in a particular batch, thereby optimizing the production process and conducting root cause analysis.
[0092] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.
Claims
1. A data management system for cell and gene therapy, characterized in that, It includes a device access layer module, a data processing layer module, a data storage layer module, and an application service layer module, among which, The device access layer module is used to receive heterogeneous device data reported by various heterogeneous devices, convert the heterogeneous device data of each protocol into standardized business data in a unified format through multiple protocol adapters, and send the standardized business data to the first topic of the message queue. The data processing layer module is used to pull the standardized business data from the first topic, perform data cleaning, protocol conversion, data enhancement, rule judgment and real-time calculation on the standardized business data in sequence, generate target business data and alarm data, send the target business data to the second topic of the message queue, and send the alarm data to the third topic of the message queue. The data storage layer module includes a relational database, a time-series database, and a cache database. The data storage layer module is used to pull the target business data from the second topic and write the target business data into the time-series database; and to pull the alarm data from the third topic, write the alarm data into the relational database, and push it to the publish-subscribe channel of the cache database. The application service layer module provides a data query interface and a real-time data push interface. The data query interface is used to obtain data from the data storage layer module, and the real-time data push interface is used to push real-time data to the client. The application service layer module is also used to perform permission verification on the data query interface and the real-time data push interface.
2. The data management system for cell and gene therapy according to claim 1, characterized in that, The device access layer module includes an MQTT protocol adapter, an HTTP protocol adapter, and a TCP protocol adapter; The MQTT protocol adapter is built on the Vert.x framework to create an MQTT server, which is used to receive topic messages published by devices via the MQTT protocol; The HTTP protocol adapter is used to provide a RESTful endpoint to receive JSON data, XML data, or file data reported by the device via a POST request; the file data includes image, CSV, Excel, or PDF format files; The TCP protocol adapter is built on the Netty framework to create a TCP server, which is used to receive binary messages reported by devices through a custom TCP protocol.
3. The data management system for cell and gene therapy according to claim 1, characterized in that, The device access layer module also includes a device registration management submodule, which is used to pre-store the registration information of each device. The registration information includes device identifier, protocol type and data parsing template. After receiving the heterogeneous device data, the device access layer module extracts the device identifier from the heterogeneous device data, finds the corresponding registration information based on the device identifier, and performs protocol parsing and format conversion on the heterogeneous device data according to the protocol type and data parsing template in the registration information to generate the standardized business data.
4. The data management system for cell and gene therapy according to claim 1, characterized in that, The relational database is used to store device metadata, user information, alarm records, and rule configuration data; The time-series database is used to store time-series sampling data reported by the device, and the time-series sampling data includes timestamps, device identifiers, and indicator values collected by the device; The cache database is used to cache the device's real-time status data, real-time alarm data, and rule configuration data.
5. The data management system for cell and gene therapy according to claim 1, characterized in that, The data storage layer module is also used to pull the target business data from the second topic and then store the target business data into the time series database through a batch write interface. The batch write interface accumulates multiple target business data within a preset time window. When the accumulated data reaches a preset batch threshold or the waiting time reaches a preset timeout threshold, the accumulated multiple target business data are written into the time series database at once.
6. The data management system for cell and gene therapy according to claim 1, characterized in that, The system also includes a front-end display layer module, which is used to display the device overview screen, device details page, data analysis page and alarm center page, and receive real-time data pushed by the application service layer module through web sockets; The device overview screen displays map markers, key indicator cards, and real-time alarm scroll bars; the device details page displays real-time data curves, historical data queries, and device parameter configurations; the data analysis page displays multi-device comparisons, custom time queries, and report exports; and the alarm center page displays alarm lists and processing records.
7. The data management system for cell and gene therapy according to claim 6, characterized in that, The front-end presentation layer module adopts a front-end and back-end separation architecture, is built using the Vue.js framework with Element UI or Ant Design component library, uses ECharts or Highcharts chart library to display real-time data curves and historical trend charts, and integrates Leaflet or Baidu Map component to display the device's geographical location and operating status on the device overview screen.
8. The data management system for cell and gene therapy according to claim 1, characterized in that, The data processing layer performs the following processing through a hybrid stream processing engine combining Kafka Streams and Apache Flink: Data cleaning is used to remove outliers from standardized business data that exceed the preset normal range, add the current system timestamp to data with missing timestamps, and perform format validation on the data types of each field. Data that fails the format validation is forwarded to the dead letter topic of the message queue. Protocol conversion processing is used to find the corresponding parsing script based on the device model and parse the reported binary data or non-standard field names into structured fields; Data augmentation processing is used to retrieve the static information of the corresponding device from the relational database based on the device identifier in the data, and store the associated static information into the data stream. The static information is pre-cached in the GlobalKTable of Kafka Streams. The rule-based judgment process is used to match data according to preset threshold rules. Single-event threshold rules are executed by Kafka Streams, while multi-event time-series pattern matching is executed by Apache Flink CEP. When data triggers an alarm rule, alarm data is generated and sent to the third topic of the message queue. Business data that does not trigger an alarm rule continues to flow to the next stage. Real-time computation processing is used to perform real-time calculations on business data using a sliding window averaging algorithm, filtering out instantaneous noise and glitches in the data to extract stable signal features and generate target business data. Data splitting processing is used to send the target business data to a second topic of the message queue for consumption by the storage layer, and to send the alarm data to a third topic of the message queue for consumption by the real-time push channel.
9. The data management system for cell and gene therapy according to claim 1, characterized in that, The application service layer module is built on the Spring Boot framework and is used to provide RESTful APIs for the front end to call, and to perform device management functions, data query functions, alarm management functions, real-time data push functions, and user authentication and permission management functions. The device management functions include device registration, device editing, device deletion, and device list and details query; The data query function includes aggregating and querying historical data from the time-series database based on device identifier and time range; The alarm management functions include alarm history query, alarm confirmation, and alarm rule configuration; The real-time data push function pushes real-time device data or alarm data to the client via the WebSocket protocol. The user authentication and permission management function integrates the Spring Security and SaToken frameworks to implement role-based access control.
10. A data management method for cell and gene therapy, characterized in that, include: The device access layer receives heterogeneous device data reported by various heterogeneous devices, converts the heterogeneous device data of each protocol into standardized business data in a unified format through multiple protocol adapters, and sends the standardized business data to the first topic of the message queue. The standardized business data is pulled from the first topic through the data processing layer. The standardized business data is then subjected to data cleaning, protocol conversion, data enhancement, rule judgment and real-time calculation processing in sequence to generate target business data and alarm data. The target business data is sent to the second topic of the message queue and the alarm data is sent to the third topic of the message queue. The target business data is retrieved from the second topic through the data storage layer and written into the time series database; And a publish-subscribe channel that pulls the alarm data from the third topic, writes the alarm data into a relational database, and pushes it to a cache database; The application service layer provides a data query interface and a real-time data push interface. Data can be queried from the relational database, the time-series database, or the cache database through the data query interface, and real-time data can be pushed to the client through the real-time data push interface.
