An internet of things multi-tenant data processing method and system

Abstract

The application discloses a kind of Internet of Things multi-tenant data processing method and system, belong to communication technical field, method includes: by management layer, request information from entry layer server is received, and request information is parsed to obtain request content and request type, request type includes the business request of the second node server communication representation;When request type is business request, node query request is generated according to the tenant identification information parsed from request content, and node query request is sent to central server;Receive target node address information from central server, and request information is forwarded to the second node server corresponding to target node address information by proxy unit according to target node address information, to make the second node server execute request content.Each node server of the application is configured with management layer and node layer, can provide independent calculation and data resource, realize the complete isolation of calculation and data.

Description

Technical Field

[0001] This application relates to the field of communication technology, specifically to a data processing method and system for multi-tenant Internet of Things (IoT) applications. Background Technology

[0002] Multi-tenant SaaS (Software as a Service) platforms need to provide services to multiple enterprises or tenants simultaneously while ensuring data isolation between them. With the explosive growth of IoT businesses, platforms need to connect to massive numbers of devices and process massive amounts of time-series data, which places extremely high demands on system scalability.

[0003] Currently, the main approach to implementing a scalable multi-tenant SaaS architecture is to build a data architecture consisting of a master database and independent tenant databases. This involves creating an independent database for each tenant at the data routing layer and using a weighted random algorithm and a dynamic data source switching mechanism to achieve data isolation and load balancing for the database servers. While this solution addresses the isolation issue at the data level, the computing resources of all tenants on the platform remain in a shared deployment mode, which cannot adapt to the business needs of massive device access and data processing.

[0004] Therefore, how to achieve dual isolation between computing resources and data resources is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a data processing method and system for multi-tenant Internet of Things (IoT) applications, aiming to solve the above-mentioned problems.

[0006] In a first aspect, a multi-tenant data processing method for the Internet of Things (IoT) is provided, applied to a first node server. The first node server is communicatively connected to an entry layer server, a central server, and a second node server. The central server is communicatively connected to the second node server. The first node server is configured with a management layer and a node layer. The management layer includes a proxy unit. The method includes: receiving request information from the entry layer server through the management layer, parsing the request information to obtain request content and request type, wherein the request type includes a business request representing communication with the second node server; when the request type is a business request, generating a node query request based on the tenant identification information parsed from the request content, and sending the node query request to the central server so that the central server obtains the target node address information based on the node query request; receiving the target node address information from the central server, and forwarding the request information to the second node server corresponding to the target node address information through the proxy unit so that the second node server executes the request content.

[0007] In some possible implementations, the request type includes a management request that represents communication with the central server, and the management layer also includes a management business interface unit; the method further includes: when the request type is a management request, generating a data interaction request based on the request content of the request information, and sending the data interaction request to the central server through the management business interface unit so that the central server executes the request content.

[0008] In some possible implementations, the request information includes a new tenant request, which contains the tenant identification information of the new tenant. The request information is forwarded to the central server through the management service interface unit, so that the central server executes the requested content. This includes: sending a configuration query request to the central server through the management service interface unit, so that the central server responds with configuration information from each node server based on the configuration query request; the node servers include a first node server and a second node server; receiving the configuration information from the node servers from the central server, and determining a target node server that meets preset conditions from among the node servers based on the configuration information; generating update information based on the node address information of the target node server and the tenant identification information of the new tenant, and sending it to the central server through the management service interface unit, so that the central server updates its built-in database based on the update information.

[0009] In some possible implementations, the configuration information includes load information and preset weight information; determining the target node server that meets the preset conditions from each node server according to the configuration information includes: obtaining the load coefficient of each node server according to the load information and weight information; determining the node server that meets the preset conditions from each node server according to the load coefficient, and using it as the target node server to bind with the new tenant.

[0010] In some possible implementations, the node layer includes a local database; the method further includes: the management layer obtains the load information of the first node server through the management business interface unit, and determines the tenant to be migrated when the load information indicates that the load of the first node server exceeds a preset load threshold; the node layer integrates the migration data corresponding to the tenant to be migrated in the local database into a migration data packet, and transmits the migration data packet to an external shared storage through the management layer; the management layer sends a tenant migration request to the central server through the management business interface unit, so that the central server executes the tenant migration request.

[0011] In some possible implementations, the node layer includes a local database; the method further includes: the management layer receiving a tenant migration instruction sent by the central server through a management business interface, the tenant migration instruction including address information of the migration data packet corresponding to the tenant to be migrated; the management layer downloading the migration data packet from an external shared storage according to the address information and writing the migration data of the migration data packet into the local database; and sending migration feedback information to the central server so that the central server updates the built-in database according to the migration feedback information.

[0012] Secondly, this application also provides a data processing method for multi-tenant IoT applications, applied to a central server. The central server is communicatively connected to an entry-level server and node servers, and the node servers include a first node server and a second node server. The method includes: receiving and parsing a node query request from the first node server to obtain tenant identification information; querying a built-in database based on the tenant identification information to obtain target node address information corresponding to the tenant identification information; and sending the target node address information to the first node server so that the first node server forwards the business request to the second node server corresponding to the target node address information based on the target node address information.

[0013] In some possible implementations, the database includes a management database and a cache database; querying the built-in database based on the tenant identification information to obtain the target node address information corresponding to the tenant identification information includes: querying the corresponding target node address information in the cache database based on the tenant identification information; when the target node address information is not found, querying the corresponding target node address information in the management database based on the tenant identification information.

[0014] In some possible implementations, the central server is configured with a management database and a cache database; the method further includes: receiving a data interaction request generated by a first node server based on the request content of the request information, and performing data interaction with the first node server according to the data interaction request to execute the request content; wherein, the request information includes a new tenant request, and the new tenant request contains the tenant identification information of the new tenant; receiving a data interaction request generated by a first node server based on the request content of the request information, and performing data interaction with the first node server according to the data interaction request to execute the request content includes: receiving and parsing a configuration query request from the first node server, querying the configuration information of each node server from the built-in database according to the configuration query request, and sending the configuration information to the first node server, so that the first node server determines the target node server that meets the preset conditions from the node servers according to the configuration information, and feeding back update information; receiving and parsing the update information from the first node server to obtain the node address information of the target node server and the tenant identification information of the new tenant, and updating the built-in database.

[0015] Thirdly, this application also provides an IoT multi-tenant data processing system, which includes an entry layer server, node servers communicatively connected to the entry layer server, and a central server communicatively connected to both the entry layer server and the node servers; the entry layer server is used to receive request information sent by the administrator server and the tenant server; the node servers are used to execute the above-described IoT multi-tenant data processing method; or, the central server is used to execute the above-described IoT multi-tenant data processing method.

[0016] This application employs multiple independent and autonomous node servers, each configured with a management layer and a node layer, which can provide independent computing and data resources to achieve complete isolation between computing and data. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the structure of the IoT multi-tenant data processing system provided in the embodiments of this application; Figure 2 This is an interactive schematic diagram of the IoT multi-tenant data processing system provided in the embodiments of this application; Figure 3 This is an interactive illustration of the IoT multi-tenant data processing method provided in the embodiments of this application. Figure 1 ; Figure 4 This is an interactive illustration of the IoT multi-tenant data processing method provided in the embodiments of this application. Figure 2 ; Figure 5 This is an interactive illustration of the IoT multi-tenant data processing method provided in the embodiments of this application. Figure 3 ; Figure 6 This is an interactive illustration of the IoT multi-tenant data processing method provided in the embodiments of this application. Figure 4 . Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified. "A and / or B" includes the following three combinations: A only, B only, and a combination of A and B.

[0021] The use of "applies to" or "configured to" in this application implies open and inclusive language, which does not preclude applicability to or configuration to devices performing additional tasks or steps. Furthermore, the use of "based on" implies openness and inclusivity, because processes, steps, calculations, or other actions "based on" one or more conditions or values ​​may in practice be based on additional conditions or values ​​beyond those conditions.

[0022] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0023] Before introducing the IoT multi-tenant data processing method and system provided in the embodiments of this application, the relevant technical solutions will be described first so that those skilled in the art can fully understand the technical content of this application.

[0024] Creating a separate database for each tenant at the data routing layer solves the tenant isolation problem at the data layer, but the computing resources of all tenants on the platform still adopt a shared deployment model. Specifically, the following problems exist.

[0025] First, the resource isolation capability is insufficient. This solution only achieves physical isolation at the data level. Core computing resources such as web servers and IoT access services are still shared by all tenants, failing to achieve comprehensive resource isolation. Furthermore, when a single tenant experiences a business peak or a massive number of devices report data, it will consume a large amount of the platform's shared core computing resources such as the central processing unit and memory, causing the "noisy neighbor" problem. This leads to a decline in the service performance of other tenants and affects the overall service stability of the platform.

[0026] Second, horizontal scaling has bottlenecks. Although this solution can achieve a certain degree of horizontal scaling at the data routing layer, the application layer is still a shared monolithic or clustered model, and the performance improvement is limited by the processing capacity of a single cluster.

[0027] Third, the overall architecture is highly complex. As the number of platform access nodes continues to increase, the connection limit and transaction processing capability of the centralized database will become the core bottlenecks to system performance. At the same time, building distributed transaction mechanisms and global caching mechanisms is technically challenging, resulting in high implementation and maintenance costs.

[0028] Based on this, embodiments of this application provide a data processing method and system for multi-tenant IoT, which deploys multiple independent node servers. Each node server includes a management layer and a node layer, and can provide independent computing resources and data resources to achieve dual isolation of computing and data resources.

[0029] Firstly, this embodiment provides a multi-tenant data processing system for the Internet of Things, such as... Figure 1 As shown, the system includes: an entry layer server, multiple node servers, and a central server. The entry layer server is used to receive request information sent by the administrator server and the tenant server. The multiple node servers communicate with the entry layer server. The central server communicates with both the entry layer server and each node server. Each node server and the central server execute the IoT multi-tenant data processing method.

[0030] The entry-level server, serving as the unified entry point for access request information, can be equipped with a load balancing component to distribute the received request information, forwarding it to the proxy unit of any node server to ensure the stability and reliability of the overall access service. The request information can be HTTPS (Hypertext Transfer Protocol Secure) messages, and the load balancing component can employ technologies such as Nginx (a high-performance HTTP and reverse proxy web server) and SLB (Server Load Balancer), etc., without limitation in this application.

[0031] The central server, serving as a centralized data sharing layer accessible to all node servers, incorporates a management database and a cache database. The management database stores global data such as a list of node servers, node server configuration information, tenant routing tables, tenant configuration information, and tenant migration task records. The cache database stores tenant routing tables for quick lookup of tenant-node server binding relationships, reducing the frequency of queries to the management database and alleviating its data access pressure. Specifically, the tenant routing table records the mapping between tenant identification information and node address information, representing the binding relationship between tenants and node servers.

[0032] The node server consists of a management layer and a node layer. The management layer includes proxy units and management business interface units. The proxy unit parses the request type of the request information, including business requests and management requests. Based on the tenant identifier information corresponding to the business request, it retrieves the tenant routing table from the central server to transparently forward the tenant's business request to the corresponding node server for processing. Management requests are routed to the local management business interface unit for handling. The management business interface unit is responsible for global management tasks such as tenant operation and maintenance, node control, and node creation. It can read and write the management database and cached data in the central server. Each node server constitutes an independent autonomous unit and does not communicate directly with each other to avoid cross-node data read and write operations and ensure reliable isolation of tenant data.

[0033] The node layer comprises a tenant business interface unit, a device data processing unit, and a local database. The tenant business interface unit receives business requests from its tenant and retrieves the local database to process these requests. The device data processing unit is compatible with various IoT communication protocols and performs tasks such as IoT (Internet of Things) device access, data parsing and cleaning, time-series data storage, and business rule triggering. The local database can include both relational and time-series databases, independently storing the business data and device time-series data of its tenant. The IoT communication protocols compatible with the device data processing unit may include MQTT (Message Queuing Telemetry Transport) and CoAP (Constrained Application Protocol). Relational databases may include MySQL (Relational Database Management System) and PostgreSQL (Object-Relational Database Management System). Time-series databases may include InfluxDB (an open-source time-series database developed by InfluxData) and TimescaleDB (an open-source time-series database based on PostgreSQL). This application does not impose any restrictions.

[0034] like Figure 2As shown in the embodiment of this application, the IoT multi-tenant data processing system distributes the access request information to the management layer of any node server according to the built-in load balancing algorithm. The management layer receives the request information, parses the request information through the proxy unit, and sends a node query request to the centralized data sharing layer to obtain the target node address information returned by the query. This allows the business request to be transparently forwarded to the node layer of the corresponding node server for processing. The management request is directly routed to the management business interface unit, which reads and writes the database of the centralized data sharing layer and receives the returned read / write confirmation to complete the processing of the management request. The node layer receives the business request, retrieves data from the local database through the tenant business interface unit to complete the processing of the business request, and returns the processing result. At the same time, the node layer interacts directly with the IoT devices bound to the tenant through the device data processing unit, returning responses or downlink instructions to the IoT devices.

[0035] In the above implementation, the system decomposes the complex and large distributed system into multiple independent and autonomous node servers. Each node server is equipped with a management layer and a node layer, which can provide independent computing and data resources and achieve complete isolation between computing and data. Furthermore, it can achieve near-unlimited horizontal expansion by adding node servers.

[0036] Secondly, this embodiment also provides a data processing method for multi-tenant IoT, applied to a first node server. The first node server is communicatively connected to an entry layer server, a central server, and a second node server. The central server is communicatively connected to the second node server. The first node server is configured with a management layer and a node layer. The management layer includes agent units.

[0037] The second node server can be any node server connected to the central server, and the second node server and the first node server can be the same node server. The first node server and the second node server operate on the same principle and are both node servers. The second node server can also execute the IoT multi-tenant data processing methods executed in the first node server. The naming of the second node server and the first node server is only for the convenience of distinguishing the interaction relationship between the two node servers.

[0038] like Figure 3 As shown, the data processing method for multi-tenant IoT includes steps S301 to S303, which will be described in detail below.

[0039] Step S301: The management layer receives request information from the entry layer server and parses the request information to obtain the request content and request type. The request type includes a business request that represents communication with the second node server.

[0040] The entry layer server is the entry layer of the system. It can support HTTPS protocol request information and can be equipped with load balancing components such as Nginx and SLB to distribute request information from tenant servers or administrator servers evenly to any node server according to a preset load balancing algorithm. For ease of description, it is referred to as the first node server.

[0041] The management layer of the first node server receives the request information and parses it through the proxy unit to obtain the request content and request type. The request type includes business requests and management requests. Business requests mainly involve querying and deleting / modifying tenant business data, while management requests involve data modification of the entire system, such as adding nodes, adding tenants, and tenant migration. The processing steps for business requests are described here.

[0042] Step S302: When the request type is a business request, a node query request is generated based on the tenant identification information parsed from the request content, and the node query request is sent to the central server so that the central server can obtain the target node address information based on the node query request.

[0043] The central server serves as a centralized data sharing layer shared by all node servers. Deployed independently of all node servers, it includes a built-in management database and a cache database. The management database stores global configuration data, including node lists, tenant routing tables, tenant configurations, migration task records, etc. The tenant routing table records the mapping relationship between tenant identification information and node address information. The cache database can utilize Redis (Remote Dictionary Server) technology to quickly respond to node query requests.

[0044] The first-node server generates a node query request based on the tenant identification information contained in the request content. Through the proxy unit, it can first retrieve the target node address information corresponding to the tenant identification information from the central server's cache database. If the cache database does not find the address, it further queries the management database, effectively reducing the access pressure on the management database. This tenant identification information is the unique identity information of the tenant carried in the request content, used to match the corresponding target node address information.

[0045] Step S303: Receive the target node address information from the central server, and forward the request information to the second node server corresponding to the target node address information through the proxy unit, so that the second node server executes the request content.

[0046] In this context, the node server pointed to by the target node address information is referred to as the second node server for ease of description. The first node server and the second node server cannot communicate directly; instead, they need to use a proxy unit to forward communication. As an example, the first node server transparently forwards the request information to the second node server pointed to by the target node address information through the proxy unit. The node layer of the second node server receives and processes the service request. In a special case, when the target node address information points to the first node server, the second node server and the first node server are the same node server. In this case, the service request is forwarded to the local node layer for processing.

[0047] The node layer comprises a tenant business interface unit, a device data processing unit, and a local database. The tenant business interface unit handles core tenant business requests such as device creation, device list querying, and device command issuance. It only accesses the node's private database and does not perform cross-node data operations. The device data processing unit supports mainstream IoT protocols such as MQTT and CoAP, completing tasks such as device access, data communication, data parsing and cleaning, time-series data storage, and rule engine triggering. The local database can include relational databases such as MySQL and PostgreSQL, as well as time-series databases such as InfluxDB and TimescaleDB, used to store tenant business data and device time-series data reported by the devices, respectively. The data on each node is independent and does not interfere with each other.

[0048] In the above implementation, steps S301 to S303 provide a unified entry point and distribute request information evenly through the entry layer server, the central server provides globally shared routing and scheduling information, and the node servers parse, forward, and execute the request information to complete the processing of the request information. At the same time, each node server has independent business processing and data storage capabilities, which can achieve dual isolation of computing resources and data resources. By adding node servers, near-unlimited horizontal expansion can be achieved.

[0049] In some possible implementations, the request type includes management requests that characterize communication with the central server, and the management layer also includes management business interface units.

[0050] The management business interface unit is used to carry out functions related to global data, such as tenant management, node monitoring, and node creation. It can perform read and write operations on the management database and cache database in the central server.

[0051] like Figure 3 As shown, the data processing method for multi-tenant IoT also includes step S304, which will be described in detail below.

[0052] Step S304: When the request type is a management request, a data interaction request is generated based on the request content of the request information, and the data interaction request is sent to the central server through the management business interface unit so that the central server executes the request content.

[0053] The management requests cover operation types such as adding tenants and monitoring node status. During the processing, global data such as the list of node servers, node server configuration information, tenant routing table, and tenant configuration information stored on the central server can be retrieved and modified.

[0054] In the above implementation, in step S304, the node server directly processes the management request through the management business interface unit. The management business interface unit initiates read and write interaction operations with the central server according to the request content of the management request in order to execute the request content of the management request.

[0055] In some possible implementations, the request information includes a new tenant request, which contains the tenant identification information of the new tenant. When a new tenant request is made, step S304 includes steps S3041 to S3043, which will be described in detail below.

[0056] Step S3041: Send a configuration query request to the central server through the management business interface unit, so that the central server can provide the configuration information of each node server according to the configuration query request; the node server includes the first node server and the second node server.

[0057] The central server provides feedback on the configuration information of the registered node servers, providing data support for the selection of target node servers.

[0058] Step S3042: Receive configuration information from the node servers of the central server, and determine the target node server that meets the preset conditions from each node server according to the configuration information.

[0059] The configuration information may include load information and preset weight information.

[0060] As a possible implementation of step S3042, determining the target node server that meets the preset conditions from each node server according to the configuration information may include: obtaining the load coefficient of each node server according to the load information and weight information; determining the node server that meets the preset conditions from each node server according to the load coefficient, and using it as the target node server to bind with the new tenant.

[0061] The load information can include operating parameters such as the number of node tenants and hardware resource utilization. It can be combined with the weight parameters of each node server in the weight information to calculate the load coefficient of each node server, so as to comprehensively evaluate the carrying capacity of each node server and select the node server with the best carrying capacity as the target node server for allocating new tenants.

[0062] Step S3043: Based on the node address information of the target node server and the tenant identifier information of the new tenant, generate update information and send it to the central server through the management business interface unit so that the central server updates the built-in database according to the update information.

[0063] The central server updates the tenant routing table in the management database based on the updated information and simultaneously updates the cache database. At the same time, it issues instructions to the target node server to initialize the business data table for that tenant. All subsequent business requests from that tenant will be forwarded to the target node server bound to it for processing. Furthermore, the target node server can generate and assign a unique device access address for that tenant. IoT devices belonging to that tenant can directly connect to the target node server and report data through this device access address. Data processing and storage for these IoT devices are all completed on the target node server.

[0064] In the above implementation, steps S3041 to S3043 query the configuration information of all registered node servers, select the node server with the best carrying capacity as the target node server, and synchronously update the tenant routing table to bind the new tenant to the node server, thereby ensuring a balanced distribution of cluster resources.

[0065] In some possible implementations, the node layer includes a local database.

[0066] For example, a local database can contain relational databases and time-series databases, used to independently store the business data and device time-series data of each tenant bound to the node server.

[0067] like Figure 4 As shown, the method also includes step S305 tenant migration. The first node server can be the initiating node for tenant migration. Step S305 can specifically include steps S3051 to S3053, which will be described in detail below.

[0068] Step S3051: The management layer obtains the load information of the first node server through the management business interface unit, and determines the tenant to be migrated when the load information indicates that the load of the first node server exceeds the preset load threshold.

[0069] The load information can include indicators such as the number of node tenants and the utilization rate of hardware and software resources. In the node operation and maintenance scenario, the first node server can monitor the load information of the node server in real time through the management business interface unit, and actively trigger the tenant migration operation when the load of the first node server exceeds the preset load threshold.

[0070] Before proactively triggering a tenant migration operation, management can determine the tenant to be migrated and the receiving node through the management business interface unit. As an example of determining the tenant to be migrated, the tenant with the highest hardware and software resource utilization can be selected to quickly alleviate the load pressure on the node servers. As another example of determining the receiving node, the management business interface unit can read the configuration information of each node server stored on the central server and select the node server with the best carrying capacity as the receiving node to achieve load balancing.

[0071] Step S3052: The node layer integrates the migration data corresponding to the tenant to be migrated in the local database into a migration data package, and transmits the migration data package to the external shared storage through the management layer.

[0072] The migration data includes business data stored in a relational database and device time-series data stored in a time-series database. The two are integrated and compressed into a migration data package and then uploaded to an external shared storage device. The shared storage device can be an object storage device, a distributed file system, or a centralized backup server, etc. This application does not impose any restrictions. At the same time, during the migration, the status of the tenant to be migrated is changed from normal to abnormal in order to lock the migration data of the tenant to be migrated and prevent data from being written during the migration.

[0073] Step S3053: The management layer sends a tenant migration request to the central server through the management business interface unit, so that the central server executes the tenant migration request.

[0074] The tenant migration request may include information such as the storage address of the migration data packet and the tenant identification information, which are used to synchronize the migration status and subsequent route adjustments.

[0075] like Figure 5 As shown, in some possible implementations, the first node server can also serve as the receiving node for tenant migration. Step S305 may further include steps S3054 to S3056, which will be described in detail below.

[0076] Step S3054: The management layer receives a tenant migration instruction from the central server through the management business interface. The tenant migration instruction includes the address information of the migration data packet corresponding to the tenant to be migrated. As an example, the central server executing a tenant migration request may include generating a tenant migration instruction based on the tenant migration request and sending the tenant migration instruction to the node server receiving the migration data.

[0077] The first node server receives and parses the tenant migration instruction, determines the storage address of the migration data packet, tenant identification information and other relevant information, in order to prepare for writing to the local database.

[0078] Step S3055: The management layer downloads the migration data packet from the external shared storage based on the address information and writes the migration data packet's migration data into the local database.

[0079] Once the migration data is written, the system can automatically verify the data integrity. If the verification is successful, the system will proceed to the next step.

[0080] Step S3056: Send migration feedback information to the central server so that the central server updates the built-in database based on the migration feedback information.

[0081] The central server modifies the tenant routing table in the management database and updates the cache database synchronously. After the tenant routing table update takes effect, the sending node automatically releases the database resources corresponding to the tenant in the local database to clear the locally retained data.

[0082] In the above implementation, steps S3051 to S3056 complete cross-node data migration based on external shared memory and, in conjunction with the central server's tenant routing table update, achieve seamless hot migration for tenants, which can effectively balance node load.

[0083] Thirdly, this embodiment also provides a data processing method for IoT multi-tenancy, applied to a central server, which is communicatively connected to an entry layer server and node servers, including a first node server and a second node server.

[0084] The central server is deployed independently of all node servers, yet is shared and accessed by all node servers. The central server is configured with a management database and a cache database, with the cache database utilizing a remote dictionary service. The management database stores global configuration information for the platform, specifically including a list of node servers, node server configuration information, tenant routing tables, tenant configuration information, and tenant migration task records. The tenant routing table records the mapping relationship between tenant identification information and node address information. The cache database caches the tenant routing table to accelerate route lookup efficiency and reduce the access pressure on the management database.

[0085] like Figure 3 As shown, the data processing method for multi-tenant IoT includes steps S401 to S403, which will be described in detail below.

[0086] Step S401: Receive and parse the node query request from the first node server to obtain the tenant identification information.

[0087] The node query request is generated by the proxy unit of the first node server after parsing the tenant's business request. The business request contains tenant identification information. After receiving the node query request, the central server extracts the tenant identification information contained therein for subsequent route matching queries.

[0088] Step S402: Query the built-in database based on the tenant identification information to obtain the target node address information corresponding to the tenant identification information.

[0089] One possible implementation of step S402 is to query the built-in database based on the tenant identification information to obtain the target node address information corresponding to the tenant identification information, including: querying the corresponding target node address information in the cache database based on the tenant identification information; and when the target node address information is not found, querying the corresponding target node address information in the management database based on the tenant identification information.

[0090] Specifically, by adopting a two-level query mechanism that first caches the database and then manages the database, the tenant routing table is queried quickly through the cached database first. In the case where the cached database is not found, the management database is then accessed to minimize the access pressure on the management database and improve the response speed of requests.

[0091] Step S403: Send the target node address information to the first node server so that the first node server forwards the service request to the second node corresponding to the target node address information based on the target node address information.

[0092] After obtaining the target node address information, the first node server can transparently route and forward business requests containing the tenant identifier to the corresponding second node server. The target node server can then independently complete the business data processing without needing to access data across nodes, thus ensuring the independence of the tenant's business processing.

[0093] In the above implementation, steps S401 to S403 maintain a globally shared tenant routing table through the central server, and combine a two-level query architecture of cache database and management database to achieve fast and accurate query of the target node address corresponding to the tenant. With the transparent proxy forwarding mechanism of the node server, the tenant's business requests are forwarded to the corresponding node server for processing.

[0094] In some possible implementations, the IoT multi-tenant data processing method also includes step S404, which will be described in detail below.

[0095] Step S404: Receive a data interaction request generated by the first node server based on the request content of the request information, and perform data interaction with the first node server according to the data interaction request to execute the request content.

[0096] The request information includes a new tenant request, which contains the tenant identifier information of the new tenant; step S404 specifically includes steps S4041 to S4042, which will be described in detail below.

[0097] Step S4041: Receive and parse the configuration query request from the first node server, query the configuration information of each node server from the built-in database according to the configuration query request, and send the configuration information to the first node server so that the first node server can determine the target node server that meets the preset conditions from each node server according to the configuration information and return the update information.

[0098] The configuration information obtained from the query can include the load information and preset weight information of each registered node server. The load information can include operating parameters such as the number of node tenants and hardware resource utilization. The weight information can include the weight parameters of each node server, so that the first node server can determine the carrying capacity of each node server and select the node server with the best carrying capacity as the target node server corresponding to the new tenant.

[0099] Step S4042: Receive and parse the update information from the first node server to obtain the node address information of the target node server and the tenant identifier information of the newly added tenant, and update the built-in database.

[0100] The central server enters the mapping relationship between the tenant identifier information of the new tenant and the node address information of the target node server in the tenant routing table of the management database, and synchronously updates the routing data table of the cache database. After the update is completed, it instructs the target node server to perform the initialization of the business data table of the corresponding new tenant.

[0101] In the above implementation, in steps S4041 to S4042, the central server provides the node server with the configuration information of all registered node servers, and updates the tenant routing tables of the management database and cache database according to the returned update information, so as to complete the node allocation and information filing of the new tenant and ensure that the business requests of the new tenant can be accurately forwarded to the corresponding node server.

[0102] like Figure 6As shown, in some possible implementations, before step S401, the IoT multi-tenant data processing method further includes a registration step S405 for the first node server; after step S405, the first node server completes registration with the central server and can communicate directly with the central server. Each node server can establish a communication connection with the central server through step S405, which will not be elaborated here.

[0103] As an example, step S405 may include steps S4051 to S4052, which will be described in detail below.

[0104] Step S4051: Receive and parse the node registration request sent by the first node server through the management business interface unit to obtain the first node identification information, the first node address information and the first node configuration information.

[0105] The first-node server is pre-deployed and configured with standardized parameters by operations and maintenance personnel. These personnel can pre-configure agent units, management business interface units, tenant business interface units, device data processing units, etc., and simultaneously build local databases such as relational databases and time-series databases. The operations and maintenance personnel pre-configure the first-node's address information, port, and other network parameters, as well as connection information to the central server's management database and cache database. After the first-node server starts and initializes, it initiates a node registration request to the central server through its own management business interface unit. This node registration request includes the first-node identifier information, the first-node address information, and the first-node configuration information. The first-node identifier information represents the unique identity of the first-node server; the first-node address information can include the first-node server's IP address or domain name; and the first-node configuration information can include the first-node server's load information and preset weight information.

[0106] Step S4052: Update the management database and cache database based on the first node address information and the first node configuration information.

[0107] After receiving a node registration request, the central server enters core parameters such as the first node's identifier, address, and configuration information into the node list in the management database. The first node's server status can be set to normal by default. Furthermore, the relevant information of the first node can be synchronously updated to the cache database to pre-warm the first node's routing information cache, improving the efficiency of subsequent node queries and tenant allocation.

[0108] After the central server responds to the node registration request, the operations and maintenance personnel add the first node's address information to the load balancer's backend server group in the pre-configured configuration file on the entry layer server to perform hot configuration reloading. Subsequently, the entry layer server expands its distribution scope for request information to all registered node servers, including the first node server, enabling stateless horizontal scaling of the entry layer. The first node server immediately begins receiving some request information and starts handling tenant business requests and IoT device access.

[0109] In the above implementation, in steps S4051 to S4052, the central server receives the node registration request from the newly added node server and updates the management database and cache database. While retaining the original tenant routing mapping method, it realizes the rapid network registration of the newly added node server, achieves seamless expansion of computing and storage resources, and the expansion is almost unlimited, which can cope with the access of massive numbers of devices.

[0110] The IoT multi-tenant data processing method and system provided in this application deploy multiple independent and autonomous node servers. Each node server is equipped with an independent management layer and node layer to achieve dual isolation of computing resources and storage resources, thereby avoiding resource contention and business interference issues in multi-tenant scenarios. Furthermore, the aforementioned data processing method and system also have the following advantages: First, the operation and maintenance mode adopts a combination of multiple independent node servers, eliminating the need for direct communication between the node servers and reducing system development, testing, and operation and maintenance costs. Second, the system's computing and storage capabilities can be linearly expanded through the aforementioned registration steps, effectively supporting the concurrent access and high-frequency data processing needs of massive IoT devices. Third, tenants can achieve independent allocation and hot migration based on node servers, and the migration process does not affect other business data, minimizing the impact of migration on tenant business. Fourth, the upgrade, restart, or failure of a single node server only affects the tenant business requests within that node and does not affect the operation of other servers. Finally, IoT devices can communicate directly with the corresponding node server through the device access address, and the access, parsing, and storage of IoT device time-series data are all completed within the node server, eliminating cross-node communication overhead and adapting to business scenarios with massive IoT devices, high concurrency, and high traffic.

[0111] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0112] The above provides a detailed description of an IoT multi-tenant data processing method and system provided by the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data processing method for multi-tenant Internet of Things (IoT) applications, characterized in that, It is applied to a first node server, which is communicatively connected to an entry layer server, a central server, and a second node server; the central server is communicatively connected to the second node server. The first node server is configured with a management layer and a node layer; the management layer includes a proxy unit and a management business interface unit; the method includes: The management layer receives request information from the entry layer server and parses the request information to obtain the request content and request type. The request type includes a business request that represents communication with the second node server and a management request that represents communication with the central server. When the request type is the business request, a node query request is generated based on the tenant identification information parsed from the request content, and the node query request is sent to the central server so that the central server can obtain the target node address information based on the node query request; The system receives the target node address information from the central server and forwards the request information to the second node server corresponding to the target node address information through the proxy unit, so that the second node server executes the request content. When the request type is the management request, a data interaction request is generated based on the request content of the request information, and the data interaction request is sent to the central server through the management business interface unit so that the central server executes the request content.

2. The IoT multi-tenant data processing method according to claim 1, characterized in that, The request information includes a new tenant request, which contains the tenant identifier information of the new tenant; The step of forwarding the request information to the central server through the management business interface unit, so that the central server executes the request content, includes: The management service interface unit sends a configuration query request to the central server, so that the central server can respond with the configuration information of each node server according to the configuration query request; the node server includes the first node server and the second node server; The system receives configuration information from the node servers from the central server and determines a target node server that meets preset conditions from among the node servers based on the configuration information. Based on the node address information of the target node server and the tenant identifier information of the new tenant, update information is generated and sent to the central server through the management business interface unit, so that the central server updates the built-in database according to the update information.

3. The IoT multi-tenant data processing method according to claim 2, characterized in that, The configuration information includes load information and preset weight information; The step of determining the target node server that meets the preset conditions from each of the node servers according to the configuration information includes: Based on the load information and the weight information, the load coefficient of each node server is obtained; Based on the load factor, a node server that meets the preset conditions is determined from each of the node servers and used as the target node server to be bound to the new tenant.

4. The IoT multi-tenant data processing method according to claim 1, characterized in that, The node layer includes a local database; the method further includes: The management layer obtains the load information of the first node server through the management business interface unit, and determines the tenant to be migrated when the load information indicates that the load of the first node server exceeds a preset load threshold. The node layer integrates the migration data corresponding to the tenant to be migrated in the local database into a migration data package, and transmits the migration data package to an external shared storage through the management layer; The management layer sends a tenant migration request to the central server through the management business interface unit, so that the central server executes the tenant migration request.

5. The IoT multi-tenant data processing method according to claim 1, characterized in that, The node layer includes a local database; the method further includes: The management layer receives a tenant migration instruction sent by the central server through the management business interface unit. The tenant migration instruction includes the address information of the migration data packet corresponding to the tenant to be migrated. The management layer downloads the migration data packet from an external shared storage device based on the address information and writes the migration data packet's migration data into the local database; The migration feedback information is sent to the central server so that the central server updates its built-in database based on the migration feedback information.

6. A data processing method for multi-tenant Internet of Things (IoT) applications, characterized in that, The method is applied to a central server, which is communicatively connected to an entry-level server and node servers, wherein the node servers include a first node server and a second node server; the method includes: The system receives and parses a node query request from the first node server to obtain tenant identification information. The node query request is generated by the first node server receiving request information from the entry layer server through the management layer and parsing the request information to extract the request content and request type. When the request type is a business request, the system generates the tenant identification information based on the tenant identification information parsed from the request content. The built-in database is queried based on the tenant identification information to obtain the target node address information corresponding to the tenant identification information; The target node address information is sent to the first node server, so that the first node server forwards the service request to the second node server corresponding to the target node address information according to the target node address information; The system receives a data interaction request from the first node server and executes the requested content; wherein the data interaction request is generated by the first node server based on the request content of the request information when the request type is a management request.

7. The IoT multi-tenant data processing method according to claim 6, characterized in that, The database includes a management database and a cache database; The step of querying the built-in database based on the tenant identifier information to obtain the target node address information corresponding to the tenant identifier information includes: Based on the tenant identification information, query the corresponding target node address information in the cache database; If the target node address information is not found, the corresponding target node address information is queried in the management database based on the tenant identifier information.

8. The IoT multi-tenant data processing method according to claim 6, characterized in that, The central server is configured with a management database and a cache database; the method further includes: Receive a data interaction request generated by the first node server based on the request content of the request information, and perform data interaction with the first node server according to the data interaction request to execute the request content; The request information includes a new tenant request, which contains tenant identification information of the new tenant; receiving a data interaction request generated by the first node server based on the request content of the request information, and performing data interaction with the first node server according to the data interaction request to execute the request content includes: The system receives and parses a configuration query request from the first node server, queries the configuration information of each node server from the built-in database according to the configuration query request, and sends the configuration information to the first node server so that the first node server can determine the target node server that meets the preset conditions from each node server according to the configuration information and feed back update information. The system receives and parses the update information from the first node server to obtain the node address information of the target node server and the tenant identifier information of the newly added tenant, and updates the built-in database.

9. A multi-tenant data processing system for the Internet of Things, characterized in that, include: An entry-level server, node servers that are communicatively connected to the entry-level server, and a central server that is communicatively connected to both the entry-level server and the node servers; The entry layer server is used to receive request information sent by the administrator server and the tenant server; The node server is used to execute the IoT multi-tenant data processing method as described in any one of claims 1-5; or... The central server is used to execute the IoT multi-tenant data processing method as described in any one of claims 6-8.

Images