A rapid construction method and system based on IOC intelligent visualization platform

Abstract

The present invention relates to a rapid construction method and system based on an IOC intelligent visualization platform, belonging to the technical field of visualization platform construction. The method comprises: extracting heterogeneous data from heterogeneous data sources according to project functional requirements and integrating them into metadata groups, standardizing the metadata groups and adding them to a data warehouse; the data warehouse stores the metadata groups through a distributed file system and adds project function tags, thereby optimizing the data warehouse; configuring output conversion formats and visualization projects according to the metadata groups, setting a cache mechanism to upload project resources according to the visualization projects, and constructing a visualization component library and a form component library; extracting project components from the visualization component library to the operation area according to the project function tags and the load balancing mechanism, configuring the corresponding data sources, and integrating visualization pages. This allows the visualization platform to quickly respond to changing market demands and technical environments during construction.

Description

Technical Field

[0001] The present invention belongs to the technical field of visualization platform construction, and specifically relates to a rapid construction method and system based on an IOC intelligent visualization platform. Background Art

[0002] With the development of informatization, IOC visualization platforms are used to monitor, manage, and optimize Internet of Things (IoT) systems. By integrating advanced technologies such as big data, IoT, and artificial intelligence, they enable real-time monitoring and analysis of data from various devices and sensors, thereby improving system operational efficiency and management. However, due to the short timeline for informatization planning, the entire process from project launch to system completion is relatively short, which makes the implementation and organization of visualization platforms very challenging. On the one hand, it is difficult to provide constructive suggestions for system implementation when the business is not yet fully clarified. On the other hand, the development of IOC platforms is typically long, involves a wide range of areas, involves large amounts of data, and requires constant upgrades and iterations, which increases the difficulty of scalability and maintenance. In actual applications, the platform may not be able to connect to front-end devices such as cameras. In many cases, data between different departments or systems is isolated, making it impossible to integrate and share data, resulting in a serious information silo phenomenon. This makes comprehensive data analysis and decision support difficult.

[0003] Traditional interaction methods require users to learn the interaction design rules of a specific system, which requires a high level of professional expertise. The learning curve for interactive interfaces is high, and there are challenges such as high barriers to entry for visual analysis and low interaction efficiency. Furthermore, due to a lack of experience in presenting complex data, designers within the team also face difficulties during the design and implementation process.

[0004] When it comes to large-scale data visualization, existing hardware and computing power are often insufficient to support 3D visualization and operations on user devices. This limits the platform's performance and efficiency when processing large-scale data. Different visualization methods, data models, APIs, and interaction methods are not unified, making it difficult to standardize system design and functional requirements. In addition, there is a lack of relevant components or tool support for the analysis and presentation of non-traditional report data types (such as spatiotemporal data and relational data). In terms of data collection, association, and digital experience management, users face difficulties in accessing and using data, which affects the efficiency and effectiveness of the platform. Summary of the Invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a rapid construction method and system based on the IOC intelligent visualization platform.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A rapid construction method based on the IOC intelligent visualization platform, including:

[0008] S1: Obtain project functional requirements, extract heterogeneous data from heterogeneous data sources according to the project functional requirements and integrate them into metadata groups, standardize the metadata groups and then add them to the data warehouse;

[0009] S2: The data warehouse stores the metadata group through a distributed file system and adds project function tags to optimize the data warehouse;

[0010] S3: configuring the output conversion format and visualization project according to the metadata group, setting a cache mechanism to upload project resources according to the visualization project, and building a visualization component library and a form component library through the project resources;

[0011] S4: extracting project components from the visual component library to the operation area according to the project function labels and the load balancing mechanism and configuring the corresponding data source to complete the entire visual page, and integrating the visual page into a project configuration page according to the form component.

[0012] Specifically, the optimization method of the data warehouse is as follows: extracting the data with the most recent storage change by incremental extraction, and compressing the input data: maintaining a fixed-size sliding window and a fixed-size search buffer, starting from the current calculation position, searching the search buffer for the longest repeated string that matches the string at the current calculation position in the sliding window, using the starting position and length of the longest repeated string to represent the position and length of the repeated string, compressing the repeated characters in the input data by outputting a triple, the triple including the offset of the repeated string in the sliding window, the length of the repeated string, and a character after the repeated string; updating the sliding window and search buffer according to the triple, adding the repeated string to the sliding window, and continuing to process the input data; performing Huffman coding on the symbols in the input data, decoding the Huffman code when the input data is called, and then reading the match and length information according to the decoding result and the triple to obtain the original data.

[0013] Specifically, the cache mechanism is as follows: establishing a connection with a cache server in the application; when accessing cached data, first checking whether the cached data exists in the cache server; if so, obtaining the data from the cache server; if not, accessing the underlying data source and storing the data in the cache server; the cache server sets a cache expiration time based on the frequency of data access and the frequency of data changes, and adopts a TTL memory elimination strategy; establishing a cache monitoring mechanism, monitoring the cache hit rate and memory usage through the monitoring mechanism, and regularly clearing expired cache to free up memory space.

[0014] Specifically, the load balancing mechanism is: configuring a listening port and a virtual computing group, the virtual computing group is used to execute the task of moving the visualization component, setting a weight value according to the size of the visualization component, calculating the weight sum of the virtual computing group that processes the visualization component according to the weight value, and organizing the virtual computing group into a server list through the project function tag, setting the server polling index, and setting the index initial value; traversing the server list according to the moving task, calculating the cumulative weight sum during the traversal process, if the cumulative weight sum is greater than or equal to the product of the initial value and the weight sum, then replacing the initial value with the weight value corresponding to the server traversed at the current moment, and after the traversal is completed, selecting the server corresponding to the initial value as the virtual computing group that processes the moving task.

[0015] Specifically, a single virtual computing server in the virtual computing group calculates the movement task of a single visualization component.

[0016] Specifically, the metadata group data is transmitted to the data warehouse by using URI to identify resources and HTTP methods to define operations on resources, so that each metadata group corresponds to a unique identifier.

[0017] Specifically, the visualization page is constructed by constructing a model instance and setting a behavior response for the model instance. The model instance is used to receive a content message of a corresponding visualization component, and the model instance triggers a corresponding behavior response according to the content message.

[0018] A rapid construction system based on the IOC intelligent visualization platform, including: data acquisition and integration module, data storage and management module, data processing and analysis module, and visualization display module;

[0019] The data acquisition and integration module is used to obtain project functional requirements, extract heterogeneous data from heterogeneous data sources according to the project functional requirements and integrate them into metadata groups, standardize the metadata groups and then add them to the data warehouse;

[0020] The data storage and management module is used for the data warehouse to store the metadata group through a distributed file system and add project function tags to optimize the data warehouse;

[0021] The data processing and analysis module is used to configure the output conversion format and visualization project according to the metadata group, and set a cache mechanism to upload project resources according to the visualization project, and build a visualization component library and a form component library through the project resources;

[0022] The visualization display module is used to extract project components from the visualization component library to the operation area according to the project function label and load balancing mechanism and configure the corresponding data source to complete the entire visualization page, and integrate the visualization page into a project configuration page according to the form component.

[0023] The beneficial effects of the present invention are:

[0024] The data acquisition module acquires information from various sensors and data sources to ensure data freshness and timeliness. Data is cleaned, deduplicated, and standardized to eliminate format differences and inconsistencies, ensuring the reliability and analytical value of the integrated data. The processed data is efficiently loaded into a data warehouse or data lake to support subsequent analysis and processing. Multi-threading technology is used to significantly improve the concurrency and efficiency of data processing. Through real-time monitoring and the visualization capabilities of the IOC platform, key parameters are monitored and alarmed in real time, identifying potential risks in a timely manner. The processed information is intuitively presented to decision makers, supporting them in making more scientific decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0026] Figure 1 This is a flow chart of a rapid construction method based on the IOC intelligent visualization platform of the present invention. DETAILED DESCRIPTION

[0027] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0028] See also Figure 1 , a rapid construction method based on the IOC intelligent visualization platform, including:

[0029] S1: Obtain project functional requirements, extract heterogeneous data from heterogeneous data sources according to the project functional requirements and integrate them into metadata groups, standardize the metadata groups and then add them to the data warehouse;

[0030] S2: The data warehouse stores the metadata group through a distributed file system and adds project function tags to optimize the data warehouse;

[0031] S3: configuring the output conversion format and visualization project according to the metadata group, setting a cache mechanism to upload project resources according to the visualization project, and building a visualization component library and a form component library through the project resources;

[0032] S4: extracting project components from the visual component library to the operation area according to the project function labels and the load balancing mechanism and configuring the corresponding data source to complete the entire visual page, and integrating the visual page into a project configuration page according to the form component.

[0033] This example utilizes advanced encryption technology for data storage and transmission security. All data stored on the platform is encrypted using the AES-256 algorithm. The front-end framework utilizes Vue3, Vite, and TS, while the back-end utilizes C++ and Java. Databases utilize PgSQL and Elasticsearch. Data visualization libraries utilize Chart.js, Echarts, and Element Plus. Version control is achieved through Git, and project management is managed through ZenTao.

[0034] Specifically, the optimization method of the data warehouse is as follows: extracting the data with the most recent storage change by incremental extraction, and compressing the input data: maintaining a fixed-size sliding window and a fixed-size search buffer, starting from the current calculation position, searching the search buffer for the longest repeated string that matches the string at the current calculation position in the sliding window, using the starting position and length of the longest repeated string to represent the position and length of the repeated string, compressing the repeated characters in the input data by outputting a triple, the triple including the offset of the repeated string in the sliding window, the length of the repeated string, and a character after the repeated string; updating the sliding window and search buffer according to the triple, adding the repeated string to the sliding window, and continuing to process the input data; performing Huffman coding on the symbols in the input data, decoding the Huffman code when the input data is called, and then reading the match and length information according to the decoding result and the triple to obtain the original data.

[0035] Specifically, the cache mechanism is as follows: establishing a connection with a cache server in the application; when accessing cached data, first checking whether the cached data exists in the cache server; if so, obtaining the data from the cache server; if not, accessing the underlying data source and storing the data in the cache server; the cache server sets a cache expiration time based on the frequency of data access and the frequency of data changes, and adopts a TTL memory elimination strategy; establishing a cache monitoring mechanism, monitoring the cache hit rate and memory usage through the monitoring mechanism, and regularly clearing expired cache to free up memory space.

[0036] Specifically, the load balancing mechanism is: configuring a listening port and a virtual computing group, the virtual computing group is used to execute the task of moving the visualization component, setting a weight value according to the size of the visualization component, calculating the weight sum of the virtual computing group that processes the visualization component according to the weight value, and organizing the virtual computing group into a server list through the project function tag, setting the server polling index, and setting the index initial value; traversing the server list according to the moving task, calculating the cumulative weight sum during the traversal process, if the cumulative weight sum is greater than or equal to the product of the initial value and the weight sum, then replacing the initial value with the weight value corresponding to the server traversed at the current moment, and after the traversal is completed, selecting the server corresponding to the initial value as the virtual computing group that processes the moving task.

[0037] Specifically, a single virtual computing server in the virtual computing group calculates the movement task of a single visualization component.

[0038] Specifically, the metadata group data is transmitted to the data warehouse by using URI to identify resources and HTTP methods to define operations on resources, so that each metadata group corresponds to a unique identifier.

[0039] Specifically, the visualization page is constructed by constructing a model instance and setting a behavior response for the model instance. The model instance is used to receive a content message of a corresponding visualization component, and the model instance triggers a corresponding behavior response according to the content message.

[0040] In this embodiment, the system uses a combination of full backup and incremental backup to regularly back up important data and store the backup data off-site to prevent data loss due to single point failures. At the same time, the system adopts a high-availability architecture design, and through technical means such as master-slave database synchronization and load balancing, ensures that the system has high availability and rapid recovery capabilities. Specifically, the system sets a disaster recovery time objective (Recovery Time Objective, RTO) of 15 minutes and a data recovery point objective (Recovery Point Objective, RPO) of 5 minutes to ensure that the system can recover quickly after an emergency occurs and minimize data loss.

[0041] A rapid construction system based on the IOC intelligent visualization platform, including: data acquisition and integration module, data storage and management module, data processing and analysis module, and visualization display module;

[0042] The data acquisition and integration module is used to obtain project functional requirements, extract heterogeneous data from heterogeneous data sources according to the project functional requirements and integrate them into metadata groups, standardize the metadata groups and then add them to the data warehouse;

[0043] The data storage and management module is used for the data warehouse to store the metadata group through a distributed file system and add project function tags to optimize the data warehouse;

[0044] The data processing and analysis module is used to configure the output conversion format and visualization project according to the metadata group, and set a cache mechanism to upload project resources according to the visualization project, and build a visualization component library and a form component library through the project resources;

[0045] The visualization display module is used to extract project components from the visualization component library to the operation area according to the project function label and load balancing mechanism and configure the corresponding data source to complete the entire visualization page, and integrate the visualization page into a project configuration page according to the form component.

[0046] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0047] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0048] The program code included in the computer-readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF or the like, or any suitable combination thereof. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or a combination thereof, and the programming language includes an object-oriented programming language such as Java, Smalltalk, C++, and also includes a conventional procedural programming language such as "C" language or similar programming language. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, utilizing an Internet service provider to connect through the Internet).

[0049] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A rapid construction method based on IOC intelligent visualization platform, characterized in that: include: S1: Obtain project functional requirements, extract heterogeneous data from heterogeneous data sources according to the project functional requirements and integrate them into metadata groups, standardize the metadata groups and then add them to the data warehouse; S2: The data warehouse stores the metadata group through a distributed file system and adds project function tags to optimize the data warehouse; The data warehouse optimization method is as follows: extracting the data most adjacent to the data where storage changes occur by using an incremental extraction method, and compressing the input data: maintaining a fixed-size sliding window and a fixed-size search buffer, starting from the current moment calculation position, searching the search buffer for the longest repeated string that matches the string at the current moment calculation position in the sliding window, using the starting position and length of the longest repeated string to represent the position and length of the repeated string, compressing repeated characters in the input data by outputting a triple, the triple including the offset of the repeated string in the sliding window, the length of the repeated string, and a character after the repeated string; updating the sliding window and search buffer according to the triple, adding the repeated string to the sliding window, and continuing to process the input data; performing Huffman coding on the symbols in the input data, decoding the Huffman code when the input data is called, and then reading the match and length information according to the decoding result and the triple to obtain the original data; S3: configuring the output conversion format and visualization project according to the metadata group, setting a cache mechanism to upload project resources according to the visualization project, and building a visualization component library and a form component library through the project resources; S4: extracting project components from the visual component library to the operation area according to the project function labels and the load balancing mechanism and configuring the corresponding data source to complete the entire visual page, and integrating the visual page into a project configuration page according to the form component.

2. The method according to claim 1, characterized in that The cache mechanism is as follows: establishing a connection with a cache server in the application, and when accessing cached data, first checking whether the cached data exists in the cache server, and if so, obtaining the data from the cache server; if not, accessing the underlying data source and storing the data in the cache server; The cache server sets the cache expiration time according to the access frequency and data change frequency of the data, and adopts the TTL memory elimination strategy; Establish a cache monitoring mechanism to monitor the cache hit rate and memory usage, and regularly clear expired cache to free up memory space.

3. The method according to claim 1, characterized in that The load balancing mechanism comprises: configuring a listening port and a virtual computing group, wherein the virtual computing group is used to execute the task of moving the visualization component, setting a weight value according to the size of the visualization component, calculating the weight sum of the virtual computing groups that process the visualization component according to the weight value, and forming a server list of the virtual computing groups according to the project function tag, setting a server polling index, and setting an initial value of the index; The server list is traversed according to the mobile task, and the accumulated weight sum is calculated during the traversal process. If the accumulated weight sum is greater than or equal to the product of the initial value and the weight sum, the initial value is replaced by the weight value corresponding to the server traversed at the current moment. After the traversal is completed, the server corresponding to the initial value is selected as the virtual computing group for processing the mobile task.

4. The method according to claim 3, characterized in that A single virtual computing server in the virtual computing cluster computes movement tasks for a single visualization component.

5. The method according to claim 1, wherein The metadata group data is transmitted to the data warehouse by using URI to identify resources and HTTP methods to define operations on resources, so that each metadata group corresponds to a unique identifier.

6. The method according to claim 1, characterized in that The visualization page is constructed by constructing a model instance and setting a behavior response for the model instance. The model instance is used to receive a content message of a corresponding visualization component, and the model instance triggers a corresponding behavior response according to the content message.

7. A rapid construction system using the rapid construction method based on the IOC intelligent visualization platform according to claim 1, characterized in that: include: Data acquisition and integration module, data storage and management module, data processing and analysis module, and visualization display module; The data acquisition and integration module is used to obtain project functional requirements, extract heterogeneous data from heterogeneous data sources according to the project functional requirements and integrate them into metadata groups, standardize the metadata groups and then add them to the data warehouse; The data storage and management module is used for the data warehouse to store the metadata group through a distributed file system and add project function tags to optimize the data warehouse; The data processing and analysis module is used to configure the output conversion format and visualization project according to the metadata group, and set a cache mechanism to upload project resources according to the visualization project, and build a visualization component library and a form component library through the project resources; The visualization display module is used to extract project components from the visualization component library to the operation area according to the project function label and load balancing mechanism and configure the corresponding data source to complete the entire visualization page, and integrate the visualization page into a project configuration page according to the form component.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for quickly building an IOC intelligent visualization platform based on any one of claims 1 to 6 is implemented.

9. A storage medium containing computer-executable instructions, characterized in that: When executed by a computer processor, the computer executable instructions are used to execute the rapid construction method based on the IOC intelligent visualization platform as described in any one of claims 1 to 6.

Images