System for visual analysis of equipment data based on Internet of Things

Abstract

The invention belongs to the technical field of Internet of Things, and particularly relates to an Internet of Things equipment data visualization analysis system which is composed of an analysis configuration module, a data receiving module, a protocol analysis module and a data display module. The analysis configuration module enables a user to configure an analysis rule through a webpage interface, the data receiving module receives equipment data and performs conversion transmission, the protocol analysis module performs data conversion, rule query updating and analysis operation, and the data display module displays the analysis data in a table form and supports historical data query. According to the method, various problems of traditional data analysis are solved, the method has the advantages of being high in development efficiency, low in error rate, user-friendly, high in flexibility and expandability and the like, and the requirement for equipment data analysis in the Internet of Things environment can be met.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Things, and in particular relates to a system based on visualization and analysis of Internet of Things device data. Background Art

[0002] In the data processing process of IoT devices, device data analysis is a key link. Traditional data analysis methods usually include the following main components: Data receiving module: The main function is to receive data from the device, covering a variety of communication interfaces, such as serial port, network, etc. As the data entry, this module receives the original data sent by the device, which is generally in binary form and requires subsequent parsing and processing.

[0003] Protocol parsing module: Developers manually write parsing logic based on protocol documents to convert binary data into readable data types. This process requires developers to have a deep understanding of the device's communication protocol and have proficient programming skills, which takes a lot of time and effort.

[0004] Output display module: presents the parsed data to users in a visual way, usually in the form of charts, tables, etc., so that users can view and analyze the data.

[0005] However, traditional data analysis methods have many disadvantages: Inefficiency: Developers need to manually consult device protocol documents and write parsing code one by one, which leads to a lengthy and inefficient development process that cannot meet the rapidly growing data processing needs of IoT devices.

[0006] High error rate: Due to differences in manual input and understanding of the protocol, errors are prone to occur in the process of writing parsing code, affecting the accuracy of data parsing and making it difficult to ensure data quality.

[0007] Poor user-friendliness: Existing solutions require developers to have a certain technical background. For users without in-depth technical knowledge, the threshold for participating in data analysis is high, which limits the participation of non-technical personnel and makes the scope of use of the system narrow.

[0008] Lack of flexibility: Once the device protocol changes, developers must re-modify the parsing code. The system has poor adaptability, resulting in high maintenance costs. This problem is particularly prominent when the device protocol is frequently updated or new devices are connected.

[0009] Complex coding implementation: For complex data protocols, developers need to have a deep understanding of the data structure. For new developers, the learning cost is high, which is not conducive to team collaboration and system development and maintenance. Summary of the invention

[0010] To solve the problems raised in the above background technology, the present invention provides a system based on the visualization and analysis of IoT device data, and the system architecture is as follows: Parsing configuration module: Function: This module provides users with a web interface that enables users to define data parsing rules for each IoT device. Users can define the parameters required for data parsing in detail on this interface, including the byte length of data parsing, the selection of data type (such as integer, floating point number) and byte order (big endian / little endian).

[0011] Storage and notification: Store user-defined parsing rules in JSON format, use device type and parsing rule version number as tags, and store them persistently in the database. At the same time, this module will notify the protocol parsing module of changes in parsing rules to ensure that the system uses the latest rules when parsing data.

[0012] Data processing: The parsing rules selected by the user are saved and accurately passed to the protocol parsing module as the basis for subsequent data parsing operations, so that subsequent data parsing strictly follows these rules, ensuring the consistency and accuracy of data parsing.

[0013] Data receiving module: Function: This module has strong compatibility and supports multiple communication protocols, such as TCP, UDP, HTTP, etc. It can receive real-time data from various IoT devices. As the data source of the system, it ensures that the system can receive data sent by devices with different protocols.

[0014] Data processing: Convert the received device binary data into a hexadecimal string for subsequent data transmission and processing. The converted hexadecimal string is sent to the protocol parsing module through protocols such as Kafka, MQTT, and HTTP, and these protocols are used to achieve reliable data transmission within the system.

[0015] Protocol parsing module: Function: After receiving the data from the data receiving module, the hexadecimal string is converted into a byte array in Java for further processing in the Java program environment. According to the device type reported by the data, the latest data parsing rules of the device type are queried from the database in real time and cached in the program memory to improve the parsing efficiency and ensure the timeliness of the parsing rules.

[0016] Receive the parsing rule change notification from the parsing configuration module. After receiving the notification, it will check whether the parsing rules in the memory are the latest version according to the device type and parsing version number. If it is not the latest rule, it will delete the old rules in the memory and obtain the latest rules from the database again to ensure that the latest parsing rules are used.

[0017] Data processing: According to the acquired data parsing protocol, the byte order of the data (big endian / little endian) is set, and according to the field information in the parsing rules, the data is traversed from the beginning in a loop, and the data is parsed according to the byte length, data type, and byte order of each field. The parsed data is stored in JSON according to the structure of parameter name, length, type, and parsed value. During the parsing process, the parsed length is accumulated. When the accumulated length is greater than or equal to the received data length, it indicates that the data has been fully parsed. Finally, the parsed data is stored in the database, and the complete parsed data stored in JSON is sent to the data display module through protocols such as Kafka, MQTT, and HTTP.

[0018] Data display module: Function: Mainly responsible for presenting the parsed data to users in a visual way, using web table format to allow users to view and analyze data intuitively.

[0019] Data processing: After receiving the JSON data sent by the protocol parsing module, the parameter names and parsed values ​​are extracted and displayed in the web page list. At the same time, the module supports users to query historical parsed data, providing users with historical data tracing and analysis functions.

[0020] Method flow: The user configures the data parsing rules on the web page interface of the parsing configuration module, including setting information such as byte length, data type and byte order, storing the configured rules in the database, and notifying the protocol parsing module of rule changes.

[0021] The data receiving module receives the real-time data from the device, converts it into a hexadecimal string and sends it to the protocol parsing module.

[0022] After receiving the data, the protocol parsing module first converts the data format, then queries and caches the latest parsing rules for the corresponding device type, parses the data according to the rules, stores the parsed data in JSON format, stores it in the database, and sends the parsing results to the data display module.

[0023] The data display module receives the parsed data, displays it in the form of a web page table, and provides historical data query function to facilitate user viewing and analysis.

[0024] Compared with the prior art, the present invention has the following beneficial effects: 1. Improve development efficiency: The present invention uses a visual parsing configuration interface to allow developers or non-technical users to complete the configuration of data parsing rules through simple interface operations, without the need to manually write complex parsing codes, which greatly shortens the development cycle and can quickly respond to market needs for data parsing.

[0025] 2. Reduce the error rate: The use of a graphical interface and parameter settings to automatically generate parsing logic reduces the intervention of manual code writing, avoids errors caused by developers' manual input and deviations in their understanding of the protocol, and significantly improves the accuracy of data parsing.

[0026] 3. Improved user-friendliness: It provides users with an intuitive and easy-to-operate interface, allowing them to configure data analysis rules and view data without having to possess deep technical knowledge. This lowers the threshold for use, enhances the popularity and usability of the system, and makes it suitable for a wider user group.

[0027] 4. Enhance system flexibility: Users can dynamically adjust parsing rules through a visual interface to adapt to changes in device protocols and the access of new devices, avoiding the tedious work of rewriting code due to protocol changes and reducing the system maintenance burden.

[0028] 5. Improve scalability: The system adopts a modular design, with each module having a clear division of labor and cooperating with each other, which facilitates the subsequent addition of new functional modules or expansion of existing modules. It is easier to support new devices and new protocols and can quickly adapt to the development and changes of the Internet of Things industry.

[0029] 6. Improved cost-effectiveness: It reduces manual operations, lowers error rates, and shortens development time, significantly reducing software development and maintenance costs. At the same time, increased user participation reduces reliance on external technical support, further improving overall cost-effectiveness.

[0030] 7. Real-time monitoring and feedback: The system can display the analysis results in real time, and users can instantly view the data status and achieve real-time monitoring, which helps users to discover and deal with problems in a timely manner and enhances the user's control over the device status. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 is a system flow chart of the present invention; Figure 2 It is a parameter configuration diagram of the present invention; Figure 3 Flowchart of the prior art. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example

[0033] like Figure 1-2 As shown; A system based on visualization and analysis of IoT device data.

[0034] Implementation of parsing configuration module: Use modern web development technologies, such as HTML5, CSS3, and JavaScript to build user interfaces, so that users can easily configure data parsing rules on web pages. The backend can use Python's Django or Java's Spring Boot framework to convert the parsing rules entered by users into JSON format and store them in the database using SQL statements (for relational databases, such as MySQL, PostgreSQL) or MongoDB's operation statements (for non-relational databases). Use message queue services, such as Apache Kafka or RabbitMQ's client library, to send rule change notifications to the protocol parsing module.

[0035] Implementation of data receiving module: Use Java network programming libraries, such as Netty or Java's built-in network classes, to support communication protocols such as TCP, UDP, and HTTP, and ensure that real-time data from different IoT devices is received. Use Java's data conversion function to convert the received binary data into a hexadecimal string, and use Apache Kafka, MQTT, or HTTP's client library to send the hexadecimal string to the protocol parsing module.

[0036] Implementation of protocol parsing module: After receiving the hexadecimal string data, use Java's data processing function to convert it into a byte array, such as using the ByteBuffer class. Use Java's JDBC or JPA technology to query the latest parsing rules from the database, and use Java's cache library (such as Caffeine, Ehcache) to store it in memory. When receiving the rule change notification from the parsing configuration module, check and update the parsing rules in memory according to the device type and parsing version number. During the data parsing process, according to the parsing rules, use Java's data operation and type conversion functions to parse, and use JSON libraries such as Jackson or Gson to store the parsing results in JSON format and store them in the database. Finally, use Apache Kafka, MQTT, or HTTP client libraries to send the parsed JSON data to the data display module.

[0037] Implementation of data display module: Use a web server framework, such as Node.js's Express or Python's Flask, to build a web service. Use HTML, CSS, and JavaScript on the front end to develop web tables, and use JavaScript libraries such as jQuery or Vue.js to display the JSON data received from the protocol parsing module in the web table. For the historical data query function, use the back-end database query statement (such as SQL or MongoDB query statement) to return the query results to the front end for display.

[0038] Alternatives Scripted parsing solution: Description: Developers use scripting languages ​​such as Python and JavaScript to write parsing logic, encapsulate the parsing code into scripts, and users specify parsing parameters through configuration files.

[0039] Advantages: The flexibility of the scripting language allows users to expand functions by themselves, and is suitable for users with certain programming skills.

[0040] Disadvantages: It requires users to have programming knowledge, has a high threshold for use, and requires additional environment settings for configuration and operation, resulting in low efficiency.

[0041] Customized analysis tools: Description: Develop customized parsing tools for specific devices or protocols. Users directly use this tool for parsing. It is only applicable to specific data formats.

[0042] Advantages: Easy to use, may be faster to parse for certain protocols.

[0043] Disadvantages: Lack of versatility, unable to support multiple protocols, need to redevelop tools when protocols change, poor adaptability.

[0044] Template-based parsing solution: Description: Users select preset parsing templates according to their needs. Common protocol formats and parsing methods are predefined in the templates, and users can customize new templates.

[0045] Advantages: Quick to configure, user-customizable templates for added flexibility.

[0046] Disadvantages: Limited number and types of templates, insufficient support for special protocols, users may need to modify templates, and there is a learning curve.

[0047] Analysis solution based on rule engine: Description: Users define rules to parse data through the rule engine and configure data extraction and transformation processes according to specific rules and conditions.

[0048] Advantages: High flexibility, user-defined rules, applicable to a variety of data types, and visual rule configuration improves ease of use.

[0049] Disadvantages: The initial configuration of the rule engine is complex, the learning cost is high, and it is difficult for non-technical personnel to understand and apply the rules.

[0050] Traditional hierarchical analysis scheme: Description: Use a layered approach to parse the message header first, then parse the message body layer by layer, and finally summarize it into structured data.

[0051] Advantages: Suitable for multiple protocol analysis, clear logical structure, suitable for complex hierarchical data analysis.

[0052] Disadvantages: The parsing process is cumbersome, increasing development and maintenance costs. When the protocol changes, all levels of parsing logic need to be revised.

[0053] Based on the above alternatives, the present invention overcomes their shortcomings through innovative system architecture and data processing flow, and provides a more efficient, flexible, user-friendly and scalable solution for IoT device data parsing.

[0054] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A system based on visualization and analysis of IoT device data, characterized in that: include: A parsing configuration module, which provides a user interface to enable users to define data parsing rules for IoT devices, including the byte length, data type and byte order of data parsing, stores the parsing rules in a database with device type and parsing rule version number as tags, passes the parsing rules to the protocol parsing module, and notifies the protocol parsing module of changes in the parsing rules; A data receiving module, which supports communication protocols such as TCP, UDP, HTTP, etc., receives binary data from IoT devices, converts the binary data into a hexadecimal string, and sends the hexadecimal string to a protocol parsing module; The protocol parsing module receives the hexadecimal string from the data receiving module, converts it into a Java byte array, queries and caches the latest data parsing rules from the database according to the device type, receives the rule change notification from the parsing configuration module, parses the data according to the cached parsing rules, stores the parsed data in JSON format and stores it in the database, and sends the parsed JSON data to the data display module; The data display module receives the JSON data from the protocol parsing module, displays the JSON data to the user in the form of a web page table, and supports the user to query the historical parsing data.

2. The system based on visual analysis of IoT device data according to claim 1 is characterized by: The parsing configuration module uses JSON format to store parsing rules.

3. The system based on visual analysis of IoT device data according to claim 1 is characterized in that: When parsing data, the protocol parsing module sets the byte order of the data according to the parsing rules, traverses the fields in the parsing rules, stores the parsing results in JSON, and completes the parsing operation when the cumulative length of the parsing is greater than or equal to the length of the received data.

4. The system based on visual analysis of IoT device data according to claim 1 is characterized in that: The data receiving module sends the received binary data to the protocol parsing module via Kafka, MQTT or HTTP protocol).

5. The system based on visualization and analysis of IoT device data according to claim 1 is characterized in that: The protocol parsing module sends the parsed JSON data to the data display module through Kafka, MQTT or HTTP protocol.

Images