Method for generating dynamic device digital twin model

Abstract

The present invention relates to the technical field of data generation and storage, and specifically relates to a method for generating a dynamic device digital twin model, which includes the following steps: storing and running according to the default data storage rules; dynamically configuring the device model according to the requirements of the business device model; reading the configuration of the device model based on the data storage module, creating a real-time device model, and creating a corresponding data storage repository table; when the data storage module receives new data, according to the configuration of the device model, storing the data into the corresponding real-time data model repository table to generate a complete device digital twin model; when the real-time device model data in the real-time data model repository table receives an update, actively pushing it to the business layer. In the present invention, the flexible data processing and storage method enables the system to adapt to changing application requirements and conduct in-depth analysis on real-time monitoring data that requires quick response.

Description

Technical Field

[0001] The present invention relates to the technical field of data generation and storage, and particularly to a method for generating a dynamic device digital twin model. Background Art

[0002] As one of the indispensable infrastructure in modern society, the power industry is undergoing a wave of digital transformation. With the rapid development of technology, the multi-device model and large-scale data collection and upload have become one of the focuses of the power industry. In this trend, in order to achieve more efficient and intelligent power system management, power companies are actively exploring advanced technical means, including the multi-device model and large-scale data collection and upload. First of all, the multi-device model is one of the key components of the digital transformation of the power industry. In the power system, there are numerous different types of devices, including generators, substations, transmission lines, etc. The data generated during the operation of these devices is huge, and this data is an important basis for realizing the monitoring, operation, and maintenance of the power system. The establishment of the multi-device model aims to integrate the operation status, performance parameters, and other information of these devices to form a comprehensive and accurate power system model. In this way, power companies can more comprehensively understand the operation status of the power system and improve the reliability and stability of the system.

[0003] To realize the effective application of the multi-device model, the power industry needs to collect and upload a large amount of data. The application of big data in the power system involves a huge amount of data, and this data needs to be collected and transmitted in real time. Seconds-level upload and rapid storage have become difficult problems that need to be solved urgently. In the power system, the real-time nature of data is crucial for the stable operation of the system. Only when the data can be uploaded in seconds can power companies timely obtain the operation information of each device in the system and realize the timely monitoring and scheduling of the power system.

[0004] To achieve seconds-level upload processing of a large amount of data, an efficient data collection system needs to be established first. This includes installing sensors and monitoring devices on power equipment to obtain important parameters such as the operation status, current, and voltage of the equipment in real time. At the same time, a stable and high-speed data transmission channel needs to be established to ensure that the data from the device to the data center can be transmitted within seconds. For the data center, it needs to have powerful data processing and storage capabilities to handle the huge amount of data. Adopting advanced big data technologies, such as distributed computing and storage, can effectively improve the efficiency and speed of data processing.

[0005] In addition, to address the challenge of a massive number of data point numbers, the power industry also needs to adopt advanced data compression and optimization technologies. Data compression can reduce the burden of data transmission and improve the efficiency of data transmission. By performing refined processing and compression on the data, the data volume can be reduced while ensuring data accuracy, thus better meeting the requirements of large-data volume collection and upload.

[0006] At the technical level, during the process of achieving second-level upload of a large amount of data, data security is also a crucial task. The data in the power system involves important aspects such as user privacy and system security. Strict security measures must be taken to ensure that the data is not damaged, tampered with, or leaked during the processes of collection, transmission, and storage. By using means such as encryption technology and access control, the security of the data can be effectively guaranteed. Summary of the Invention

[0007] Based on the above purposes, the present invention provides a method for generating a dynamic device digital twin model.

[0008] The method for generating a dynamic device digital twin model includes the following steps:

[0009] S1: Store and run according to the default data storage rules, use Kafka as the data access layer, and read data in a data-driven manner through Flink technology;

[0010] S2: Dynamically configure the device model according to the requirements of the business device model, create a real-time device model, and update the configuration to the central management system;

[0011] S3: Based on the data storage module, read the configuration of the device model and create corresponding data storage repository tables, and the data storage tables include real-time data model repository tables;

[0012] S4: When the data storage module receives new data, store the data into the corresponding real-time data model repository table according to the configuration of the device model to generate a complete device digital twin model;

[0013] S5: When the real-time device model data in the real-time data model repository table receives an update, actively push it to the business layer.

[0014] Further, in S1, Flink reads data from Kafka Source, performs data analysis and data processing on the data through a custom Process operation, and finally writes the data into the database through a custom Sink.

[0015] Further, the data processing includes writing the data of the same channel into the same database table and performing regularization processing on the data of different device point numbers in the same channel.

[0016] Further, S1 further includes a data flow structure. The data flow structure first collects the section data in the IEC104 manner by the collection end. The collection end preliminarily processes and encapsulates the data, and then writes the data into the message queue Kafka in the data access layer based on the Json method.

[0017] Further, S2 and S3 are synchronized. S3 includes continuously reading the configuration of the device model and creating the corresponding data repository table structure information.

[0018] Further, S4 further includes a data analysis and processing module. The data analysis and processing module dynamically distributes the destination of data storage according to the configuration of the device model, including:

[0019] Sending the data to the real-time data model library table, aggregating and storing the specified data according to the configuration;

[0020] Sending the data to the message bus, and sending the data of the real-time device to the advanced application in units of the real-time device model data;

[0021] Storing the data into the default time series database and storing the raw data of all data.

[0022] Further, the advanced application directly queries the database table generated by the configuration to obtain the changed data related to the dynamic device model, supporting the development of the business layer.

[0023] Advantages of the present invention:

[0024] In the present invention, by dividing the data into a real-time data model and a long-term device model, the efficiency of data processing and storage is effectively improved. The system can be flexibly adjusted according to different business requirements to ensure instant acquisition and processing of real-time data, while ensuring effective storage and access to data. This flexible data processing and storage method enables the system to adapt to changing application requirements, whether it is real-time monitoring that requires quick response or in-depth analysis of historical data.

[0025] In the present invention, in terms of the storage of long-term device model data, this method ensures high accessibility and persistence of the data, supports long-term dynamic modeling storage of the device, which not only helps with long-term trend analysis and device performance evaluation, but also provides key data to support predictive maintenance and fault diagnosis of the device. Through efficient data storage and management, this method provides an advanced and reliable data storage solution for the power industry field, greatly enhancing the stability of the system and the security of the data. Description of the Drawings

[0026] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only those of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0027] Figure 1 Schematic diagram of the data flow architecture of the embodiment of the present invention;

[0028] Figure 2 Underlying data storage logic diagram of the default long-term storage of the embodiment of the present invention;

[0029] Figure 3 Schematic diagram of the data model configuration of the embodiment of the present invention;

[0030] Figure 4 Schematic diagram of the data model library representation of the embodiment of the present invention. Detailed implementation manners

[0031] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the following further details the present invention in combination with specific embodiments.

[0032] It should be noted that unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meaning understood by those of ordinary skill in the field to which the present invention belongs. The "first", "second", and similar terms used in the present invention do not indicate any order, quantity, or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or objects appearing before this term cover the elements or objects listed after this term and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left", "right", etc. are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0033] As Figures 1-4 shown, for the method for generating a dynamic device digital twin model, first, the data of each underlying device is tagged by different acquisition devices and then sent to the unified device access layer. Secondly, the data in the device access layer is obtained, and the data is dynamically analyzed according to different data tags. The specific architecture data flow diagram is as Figure 1 shown.

[0034] The specific steps are as follows:

[0035] Step 1: The system stores and operates according to the default data storage rules.

[0036] Step 2: According to the requirements of the business device model, dynamically configure the device model configuration, distribute and update it to the system, and set the data model type (real-time data model) according to the standard in the configuration.

[0037] Step 3: The data storage module reads the dynamically distributed device model configuration, differentiates between the real-time data model and the long-term device model, and generates corresponding data storage repository tables according to the device model type.

[0038] Step 4: When the data storage part receives new data, it stores the data into the specified real-time data model database table according to the data model configuration, thereby generating a complete device digital twin model.

[0039] Step 5: When the real-time device model data receives an update, it actively pushes the real-time device data to the business layer, thereby completing the process of the device digital twin model from generation and perfection to use.

[0040] The following details the data storage logic as follows:

[0041] Step 1 specifically includes: The system stores and operates according to the default data storage rules.

[0042] In this solution, the mainstream message middleware component Kafka is used as the data access layer. Kafka has good performance and scalability, and can ensure data security and high availability.

[0043] Using the mainstream streaming processing framework Flink technology to read the data from the data access layer in a data-driven manner. Flink is a distributed stream processing framework. In Flink, data processing can be divided into three main parts: Source (data source), Transformation (transformation operation), and Sink (data output).

[0044] Source is the input end of the Flink program, responsible for reading data from external systems or data streams. The data source can be a Kafka topic, file system, socket, etc. Flink provides various implementations of Source to support different data source types. This solution uses the Kafka Source of Flink to obtain data from the data access layer.

[0045] Transformation defines the processing operations to be performed on the input data. This includes operations such as Map, Filter, Reduce, GroupBy, etc. Flink provides a rich API that allows developers to apply these operations on data streams to implement complex data processing logic. This solution uses a custom Process operation to perform custom analysis and processing on the data.

[0046] Sink is the output end of a Flink program and is responsible for sending the processed data to an external system or storage. The data sink can be a Kafka topic, a file system, a database, etc. Flink provides multiple implementations of Sinks to support writing data to different targets. This solution uses a custom Sink to write data into the database.

[0047] The data flow structure is as follows: First, the acquisition end collects the section data in the IEC104 manner. Secondly, the acquisition end performs preliminary processing and encapsulation on the data, and then writes the data into the message queue Kafka in the data access layer in the form of organized Json. In Kafka, the collected data from different power stations is distinguished by different KafkaTopic topics.

[0048] Finally, start the Flink real-time program, use the Kafka Source provided by Flink official to read the Topic topic data in the specified Kafka, and perform streaming processing on the data inside Flink.

[0049] Flink reads and stores each piece of data in the data access module internally. The specific processing is as follows:

[0050] #1 Define the basic storage table

[0051] Define the basic structure of the table. The table name is the device model, such as SEL-751A. The columns are the basic column information required for storage in the table, including: ts time, point number (100 points for each device model), point value (100 values for each device model), channel label chennel, device entity_id, reserved label tag1, reserved label tag2, reserved label tag3, reserved label tag4, reserved label tag5, reserved label tag6, reserved label tag7

[0052] #2 Define the table name where the point data is located

[0053] Name the table according to the device model where the point is located, such as the SEL-751A

[0054] #3 Specify the label values of the columns where the point is stored

[0055] Here is an example. If the point number of this point is 16385 and the point number attribute is voltage, then the value stored in the point column is 16385, and the value stored in the point_name column is: voltage. The device to which this point number belongs is the 15th instance of the device model SEL-751A. Then the entity_tag of the device is SEL-751A=15, and the device id is the entity_id column, with the value being the id of the device

[0056] According to the attributes of each point number, find the database table and column where it is located. The specific data storage logic principle is as Figure 1 shown

[0057] Concatenate the processed data into a long SQL form and write it into the database in the way of JDBC connection

[0058] The data query and write logic are the same. The data to be queried needs to provide the point number and information data (the data written in Figure 1). According to the rules in #1, calculate the table name and column name of the point to be queried (the actual storage table in Figure 1) to query the specific data

[0059] Step 2 specifically includes: Dynamically configure the device model configuration according to the requirements of the business device model. See specifically Figure 1

[0060] Step 3 specifically includes: The data storage module creates the corresponding data storage repository table according to the device model type

[0061] In this solution, the real-time device model database table takes Redis as an example, and the historical device model database table takes TDengine as an example. See specifically Figure 1

[0062] Step 3 and Step 2 are carried out simultaneously. Step 3 needs to persist the device configuration information and create the corresponding database table structure information

[0063] Step 4 specifically includes: Completing the data of the digital twin model

[0064] After the data is collected and regularized, it is sent to the data analysis and processing module through kafka. The data analysis and processing module dynamically distributes the destination of data storage by obtaining the device model data configuration, which is mainly divided into four parts

[0065] The first part sends the data to the historical device model database, and aggregates and stores the specified data according to the configuration

[0066] The second part sends the data to the real-time device model database, and aggregates and stores the specified data according to the configuration

[0067] ​​The third block sends the data to the message bus and sends the data of the real-time device to the advanced application in units of real-time device model data.

[0068] The fourth block stores the data in the default time series database and stores the raw data of all data.

[0069] The process of data storage and distribution is shown in Figure 1 .

[0070] Step 5 specifically includes: the use of the digital twin model data.

[0071] The advanced application directly queries the database table generated by the configuration to obtain the change data related to the dynamic device model to support the research and development of the business.

[0072] Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the present invention is limited to these examples; under the idea of the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the present invention as described above, and they are not provided in detail for the sake of brevity.

[0073] The present invention aims to cover all such substitutions, modifications, and variations that fall within the broad scope of the claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for generating a digital twin model of a dynamic device, characterized in that It includes the following steps: S1: Store and run according to the default data storage rules. Use Kafka as the data access layer and read data in a data-driven manner through Flink technology. S1 also includes a data flow structure. First, the acquisition end collects section data in the IEC104 manner, preliminarily processes and encapsulates the data, and then writes the data into the message queue Kafka of the data access layer based on the Json method. The collected data of different power stations in Kafka are named and distinguished by different KafkaTopic topics; In S1, Flink reads data from Kafka Source, performs data analysis and data processing on the data through custom Process operations, and finally writes the data into the database through custom Sink; In Flink, data processing is divided into three main parts: Source data source, Transformation transformation operation, Sink data output; Source is the input end of the Flink program and is responsible for reading data from external systems or data streams; Transformation defines the processing operations performed on the input data; Sink is the output end of the Flink program and is responsible for sending the processed data to external systems or storage; Read the Topic topic data in the specified Kafka by Kafka Source and perform streaming processing on the data inside Flink; Flink reads and stores each piece of data in the data access module internally. The specific processing is as follows: Define the basic storage table: Define the basic structure of the table, and the table name is the equipment model; Define the table name where the data of this point is located: Name the table according to the equipment model where this point is located; Specify the label values of the columns stored at this point; According to the attributes of each point number, find the database table and columns where it is located, including: Concatenate the processed data into a long SQL form and write it into the database in the way of JDBC connection; The data query and write logic are the same. The queried data needs to provide the point number and information data. According to the rules for defining the basic storage table, calculate the table name and column name where the point to be queried is located. S2: Dynamically configure the equipment model according to the requirements of the business equipment model, create a real-time equipment model, and update the configuration to the central management system; S3: Based on the data storage module, read the configuration of the equipment model and create the corresponding data storage repository tables, including real-time data model repository tables; S4: When the data storage module receives new data, store the data into the corresponding real-time data model repository tables according to the configuration of the equipment model to generate a complete digital twin model of the equipment. It also includes a data analysis and processing module. The data analysis and processing module dynamically distributes the destination of data storage according to the configuration of the equipment model, including: Send the data to the real-time data model repository table, aggregate and store the specified data according to the configuration; Send the data to the message bus and send the data of this real-time equipment to the advanced application in units of real-time equipment model data; Store the data into the default time series database to store the raw data of all data; The data analysis and processing module dynamically distributes the destination of data storage by obtaining the device model data configuration, which is mainly divided into four parts: The first part sends the data to the historical device model database, aggregates and stores the specified data according to the configuration; The second part sends the data to the real-time device model database, aggregates and stores the specified data according to the configuration; The third part sends the data to the message bus, and sends the data of the real-time device to the advanced application in units of the real-time device model data; The fourth part stores the data into the default time series database to store the raw data of all data; S5: When the real-time device model data in the real-time data model library table receives an update, it actively pushes it to the business layer; The data processing includes writing the data of the same channel into the same database table and regularizing the data of different device point numbers in the same channel; S2 and S3 are synchronized, and S3 includes continuously reading the configuration of the device model and creating the corresponding data storage repository table structure information; The advanced application directly queries the database table generated by the configuration to obtain the change data related to the dynamic device model to support the development of the business layer.

Images