Data management and analysis system and application thereof

Abstract

The invention provides a data management and analysis system and application thereof, and the system comprises one or more variable communication groups which are used for collecting and pre-processing dispersed data, and one or more routers are used for receiving and outputting the processed data meeting the requirements. A data center configured with a learning algorithm neural network receives the data transmitted by the route and manages and analyzes the data based on a distributed data collection and distribution architecture, and an application server outputs a control result based on an analysis result; the method is applied to management and analysis of composite monitoring data in a network environment. According to the method, high-quality data are effectively screened, the reliability of the obtained information is high, the characterization capability is strong, and decision making is effectively and efficiently assisted; reasonable data distribution and feature extraction logic is established, data islands are prevented, data alignment and circulation are realized, and effective information can be extracted and effectively shared; the system reveals the relationship between the data, and is suitable for various fields including but not limited to health monitoring, environmental regulation, remote medical comprehensive systems and the like.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing, and in particular to a data management and analysis system and its applications. Background Technology

[0002] With the development of information technology, a large amount of data is generated in more and more situations. Data is the form and carrier of information. Information can be obtained by processing data, and this data and the information generated can influence decision-making.

[0003] The data generated during the informatization process are categorized and intersected. They will be processed differently and applied in corresponding scenarios. Even the same data will yield different information and learn different features in different scenarios.

[0004] The main challenges in managing and applying massive amounts of data lie in:

[0005] (1) The data quality is not high, which is specifically manifested in the difficulty to guarantee the accuracy, completeness and consistency of the data, resulting in differences in the information obtained from the data, which in turn leads to decision-making errors and inefficiency;

[0006] (2) Inconsistent data standards can easily lead to data silos, making it impossible to achieve data flow and effective sharing;

[0007] (3) The lack of efficient data processing tools and technologies leads to low data processing efficiency and makes it impossible to obtain effective information and characteristics from the data;

[0008] (4) Data from different dimensions are independent of each other, forming data silos, which leads to waste of resources and information blockage;

[0009] (5) Data governance technology cannot meet business needs and cannot obtain effective data information.

[0010] Currently, with smart cities being given new meanings and requirements, the inevitable path of development is to process large amounts of information and data, collect various data in the context of smart cities, respond to people's needs, and achieve scientific and efficient management. This places higher but more accurate demands on researchers. Summary of the Invention

[0011] To address the aforementioned technical problems, this invention provides a data management and analysis system and its applications.

[0012] The technical solution adopted in this invention is a data management and analysis system, the system comprising:

[0013] One or more variable communication groups are used to collect and preprocess distributed data;

[0014] One or more routes are used to receive processed data that meets the requirements and output it.

[0015] A data center is equipped with a learning algorithm neural network to receive data transmitted via routing and to manage and analyze the data based on a distributed data collection and distribution architecture.

[0016] An application server is used to output control results based on the analysis results.

[0017] Preferably, any of the variable communication groups includes one or more variable communication nodes; the variable communication nodes calculate the corresponding variable communication groups based on data transmission time and data transmission energy consumption.

[0018] Preferably, the preprocessing includes data cleaning, data classification, and data labeling; the data labeling is a data weight coefficient.

[0019] Preferably, any of the variable communication groups calculates the loss based on the route idleness and route transmission cost, and selects the corresponding route based on the minimum loss.

[0020] Preferably, the data center includes a data receiving server, and the output end of the data receiving server, which is used in conjunction with data tagging, is equipped with a data management server and an emergency database server. The output end of the data management server, which is used in conjunction with data classification, is equipped with one or more classification processing servers. The output ends of the emergency database server and all the classification processing servers are configured with an analysis center.

[0021] Preferably, a deep neural network is constructed in conjunction with the analysis center;

[0022] The deep neural network includes at least two branches arranged in parallel, each branch including a classification input layer and several LSTM modules arranged in sequence; the output of the last LSTM module of all branches is connected to a fully connected layer and an output layer;

[0023] A cross-attention layer is provided between the corresponding LSTM modules of any two related branches.

[0024] Preferably, a confidence module is provided in conjunction with any of the branches, and the confidence module is configured in conjunction with one or more LSTM modules of the corresponding branch.

[0025] Preferably, at least one branch is an accompanying training branch.

[0026] An application of the aforementioned data management and analysis system is used for the management and analysis of composite monitoring data in a network environment.

[0027] Preferably, the composite monitoring data includes physiological monitoring data and environmental monitoring data.

[0028] This invention provides a data management and analysis system and its application. The system includes one or more variable communication groups to collect and preprocess dispersed data. One or more routers receive and output the processed data that meets the requirements. A data center equipped with a learning algorithm neural network receives the data transmitted by the routers and manages and analyzes the data based on a distributed data collection and distribution architecture. An application server outputs control results based on the analysis results. The system is applied to the management and analysis of composite monitoring data in a network environment.

[0029] The beneficial effects of this invention are as follows:

[0030] (1) Effectively screen high-quality data to ensure that the information obtained from the data is highly reliable and has strong representational ability, and effectively and efficiently assists in decision-making;

[0031] (2) Establish reasonable data distribution and feature extraction logic to prevent the generation of data silos, realize data alignment and circulation, and extract effective information and share it effectively;

[0032] (3) Through systems for managing and analyzing data, the relationships between data are revealed, applicable to various fields including but not limited to health monitoring, environmental regulation and telemedicine integrated systems. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the system structure of the present invention;

[0034] Figure 2 This is a system topology diagram of the present invention;

[0035] Figure 3 This is a schematic diagram of the data center structure in this invention;

[0036] Figure 4 This is a schematic diagram of the deep neural network structure in this invention. Detailed Implementation

[0037] The present invention will be further described in detail below with reference to embodiments, but the scope of protection of the present invention is not limited thereto.

[0038] This invention provides a data management and analysis system, which will be described below with reference to embodiments.

[0039] In an embodiment of the present invention, a spatial information collection system is configured, and the collected information includes environmental data and human body data. The environmental data includes, but is not limited to, indoor and outdoor temperature, humidity, particulate matter concentration, etc., and the human body data includes, but is not limited to, any natural person's heart rate, body temperature, blood oxygen, humidity, etc.

[0040] The system includes:

[0041] One or more variable communication groups are used to collect and preprocess distributed data;

[0042] One or more routes are used to receive processed data that meets the requirements and output it.

[0043] A data center is equipped with a learning algorithm neural network to receive data transmitted via routing and to manage and analyze the data based on a distributed data collection and distribution architecture.

[0044] An application server is used to output control results based on the analysis results; in practical applications, the application server outputs the control results and reflects them on the user / terminal.

[0045] In practical implementation, low-power wireless communication methods such as Zigbee, RoLa, and BLE are adopted. Each independent acquisition space is set as a node, i.e., a variable communication group. Typically, multiple variable communication groups are networked through a single router. The data sent over the network is repackaged and sent to the data center via TCP / IP protocol. The data center learns and analyzes the data to enable the invocation of application services. For data with low real-time requirements, the data volume can be accumulated to a threshold or after a preset time before transmission. Generally, the controller and RF transmitter are in silent mode, operating with extremely low power consumption. For sensors with high real-time requirements, data can be uploaded directly in advance according to the set task priority. This can be achieved through a predetermined transmission strategy or through dynamic adjustment.

[0046] In this invention, this system can also be further expanded as a new data node.

[0047] Each of the variable communication groups includes one or more variable communication nodes; the variable communication nodes calculate the corresponding variable communication group based on data transmission time and data transmission energy consumption.

[0048] The preprocessing includes data cleaning, data classification, and data labeling; the data labeling is the data weight coefficient.

[0049] Each of the aforementioned variable communication groups calculates the loss based on the route idleness and route transmission cost, and selects the corresponding route based on the minimum loss.

[0050] In this invention, the transmission strategy is further elaborated.

[0051] Specifically, a variable communication cluster refers to a communication cluster that is variable in size, primarily because the communication nodes within it can be adjusted. This adjustment can effectively output data waiting to be transmitted with the highest efficiency. During implementation, the variable communication node calculates the corresponding variable communication cluster based on data transmission time and energy consumption. For example, if any variable communication node needs to output data within time t, the energy consumption is E. i Then, n variable communication nodes are selected to form a variable communication cluster, satisfying that the expected output time of the current variable communication cluster is no greater than t, and the currently available energy E group satisfy

[0052]

[0053] Here, α is an adjustment coefficient used to ensure the energy consumption of the variable communication group.

[0054] Accordingly, when a variable communication group selects a route, it is also necessary to comprehensively evaluate the route's idle time, the route transmission cost, and calculate the loss. Specifically, the transmission time of a variable communication group consisting of n variable communication nodes at any route k is T. k ±t', where T k The differences are comprehensively evaluated based on factors such as bandwidth and network response time, and are adjusted using t' as the variable. The selection is based on the existence of T. k Routing during the idle time of ±t', and in T k Based on ±t', calculate the energy loss E corresponding to each route k. route_k Choose the route that minimizes the loss.

[0055] In this embodiment, considering the special nature of the data, in addition to data cleaning, the preprocessing also requires data classification and data labeling. The classification here includes, but is not limited to, environmental data and human body data. The weight of the data label is adjusted by the credibility.

[0056] In this invention, for data with high dispersion, core data can be extracted using clustering algorithms, adaptive neural networks, or other algorithms, which is easily understood by those skilled in the art.

[0057] The data center includes a data receiving server. The output end of the data receiving server, which is used in conjunction with data tagging, is equipped with a data management server and an emergency database server. The output end of the data management server, which is used in conjunction with data classification, is equipped with one or more classification processing servers. The output ends of the emergency database server and all the classification processing servers are configured with an analysis center.

[0058] A deep neural network is constructed in conjunction with the aforementioned analysis center;

[0059] The deep neural network includes at least two branches arranged in parallel, each branch including a classification input layer and several LSTM modules arranged in sequence; the output of the last LSTM module of all branches is connected to a fully connected layer and an output layer;

[0060] A cross-attention layer is provided between the corresponding LSTM modules of any two related branches.

[0061] A confidence module is provided for each of the branches, and the confidence module is configured in conjunction with one or more LSTM modules of the corresponding branch.

[0062] At least one branch is an accompanying training branch.

[0063] In this invention, a data receiving server receives data from different locations within the city around the clock and distributes the data according to real-time requirements. Specifically:

[0064] Data with high complexity and low real-time requirements is sent to the data management server for further subdivision and reassembly. The main task of the data management server is to subdivide and reassemble data with high complexity. This data often contains a large amount of broad data with a large volume, such as environmental information and other related indoor and outdoor information. The subdivision and reassembly of this data takes a long time, so it needs to be separated from the data receiving server. The separated data is then allocated to the corresponding classification processing server based on its classification information. The server processes the data according to rules and outputs it to the analysis center for analysis and processing.

[0065] Data that is singular and has high real-time requirements is directly dispatched to the analysis center through the emergency database server to ensure that the data can be analyzed and processed as soon as possible and sent to the application server so that users can receive feedback.

[0066] In this invention, an improved deep neural network is configured in the analysis center. It processes data through two branches. Data is input through the classification input layer of each branch. Multiple long short-term memory network layers (LSTM) are used to obtain the temporal dependencies of the data. The fully connected layer calculates the weights through the linear layer to obtain the vector of the corresponding layer number. The sigmoid activation function is used to ensure that the weights are between 0 and 1. Finally, the features of each layer are weighted and fused and output by the output layer.

[0067] To better capture the relationships between related branches (corresponding to different data categories), a cross-attention layer is set between the LSTM modules of any two related branches. This layer is used to acquire the features of a single data category in time series while simultaneously obtaining the feature relationships of other data categories. This allows the system to learn more about the relationships of the current data category type, thereby achieving better prediction results.

[0068] Furthermore, by configuring a confidence module within the branch, which is related to data labeling and classification, the weights of different branches and different layers of LSTM modules can be adjusted by regulating the parameters of the confidence module.

[0069] In this invention, to ensure that the training process is always controllable, at least one branch is used as an accompanying training branch, and the data input to this branch is reliable data.

[0070] The present invention also relates to an application of the aforementioned data management and analysis system, which is used for the management and analysis of composite monitoring data in a network environment.

[0071] The composite monitoring data includes physiological monitoring data and environmental monitoring data.

[0072] The following is an application embodiment of the present invention. It should be noted that the present invention does not involve the treatment and diagnosis of diseases, but only the collection of data and the management and analysis of the data itself.

[0073] In this embodiment, a distributed data collection and analysis architecture and a deep neural network are constructed. The distributed data collection and analysis architecture is used to acquire composite monitoring data, including physiological monitoring data and environmental monitoring data, and a parameter deep learning module is used to learn the data.

[0074] In the distributed data collection and analysis architecture, a dedicated classification and processing server for physiological monitoring data is constructed to store users' physiological monitoring data and signals. A dedicated classification and processing server for environmental monitoring data is also constructed to store indoor and outdoor environmental monitoring data. The analysis center is equipped with a deep neural network, enabling continuous self-learning while processing data. This assists the application server in outputting data results and controlling the output of results, including the invocation of applications, services, and microservices. During implementation, the application server primarily provides relevant parameters based on the analysis results to users, healthcare providers, city managers, city planners, and third-party analysts. It pushes user physiological monitoring information to achieve synchronization of classified information among healthcare service providers, urban environmental monitoring units, urban ecological management units, and third-party operators, providing conditions for collaborative work.

[0075] In this embodiment, multidimensional physiological data is monitored as the data source. Data from multiple sensors is combined and optimized in multiple forms and at multiple levels. The information fusion algorithm model of multiple combined physiological parameter data is fully utilized to realize the processing of multiple combined physiological parameter data. The algorithm model is trained with big data to complete the discrete analysis of each sensor and the combined analysis of multiple combined sensors. In the process, multidimensional environmental data is combined to extract the environmental state features of indoor and outdoor spaces from different environmental monitoring sensors. The feature values ​​of physiological parameters are matched with the feature values ​​extracted from the environment, and the decision tree makes a decision for control.

[0076] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0080] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0081] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A data management and analysis system, characterized in that: The system comprises: One or more variable communication groups for collecting scattered data and pre-processing; One or more routes, used to receive and output the processed data that meets the requirements; a data center, configured with a learning algorithm neural network, for receiving data transmitted by the routing and managing and analyzing the data based on a distributed data collection and distribution architecture; An application server is used to output control results based on the analysis results.

2. A data management and analysis system according to claim 1, characterized in that: Any of the variable communication groups includes one or more variable communication nodes; the variable communication nodes calculate the corresponding variable communication group based on data transmission time and data transmission energy consumption.

3. A data management and analysis system according to claim 2, characterized in that: The pre-processing includes data cleaning, data classification and data labeling; the data labeling is the data weight coefficient.

4. A data management and analysis system according to claim 1, characterized in that: Any of the variable communication groups calculates the loss based on the idleness of the route and the route transmission cost, and selects the corresponding route based on the minimum loss.

5. A data management and analysis system according to claim 3, characterized in that: The data center includes a data receiving server. The output end of the data receiving server is equipped with a data management server and an emergency database server for data labeling. The output end of the data management server is equipped with one or more classification processing servers for data classification. The output ends of the emergency database server and all the classification processing servers are configured with an analysis center.

6. A data management and analysis system according to claim 5, characterized in that: Cooperate with the analysis center to build a deep neural network; The deep neural network includes at least two branches arranged in parallel, and any branch includes a classification input layer and a plurality of LSTM modules arranged in sequence; the output end of the last LSTM module of all branches is connected to a fully connected layer and an output layer; There is a cross attention layer between the corresponding LSTM modules of any two associated branches.

7. A data management and analysis system according to claim 6, characterized in that: A confidence module is provided in conjunction with any of the branches, and the confidence module is provided in conjunction with one or more LSTM modules of the corresponding branch.

8. A data management and analysis system according to claim 6, characterized in that: At least one branch is a companion training branch.

9. An application of the data management and analysis system according to any one of claims 1 to 8, characterized in that: Applied to the management and analysis of composite monitoring data in a network environment.

10. The use according to claim 9, characterized in that: The composite monitoring data includes physiological monitoring data and environmental monitoring data.

Images