Digital twin configuration system based on AI drive and data processing method

Abstract

The invention provides a digital twin configuration system based on AI drive and a data processing method, and the system comprises the digital twin configuration system based on AI drive, and the system comprises a data extraction layer which is used for extracting data from at least one data source; the data conversion layer is used for data cleaning and preprocessing, data aggregation and calculation and AI analysis and prediction; the data loading layer is used for loading the processed data to a storage system; the scheduling layer is used for managing the scheduling layer and scheduling execution of the ETL task; and the data monitoring and management layer is used for monitoring the execution state and the data quality of the ETL process. According to the scheme, the AI large model technology and the digital twinborn technology are combined, the system achieves intelligent data analysis and prediction capacity, meanwhile, real-time monitoring and simulation of a complex system are supported, and therefore the intelligent management level and decision optimization capacity of the system are improved.

Description

Technical Field

[0001] The present invention relates to the field of AI technology, and in particular to an AI-driven digital twin configuration system and a data processing method. Background Art

[0002] With the rapid development of the Internet of Things (IoT) and artificial intelligence (AI), digital twin technology has been widely used in industrial manufacturing, smart cities, medical health and other fields.

[0003] Traditional digital twin systems usually rely on simple rule-driven or physical models, which makes it difficult to handle complex big data environments and fail to fully leverage the advantages of intelligence and automation.

[0004] Therefore, there is an urgent need for a digital twin system that can combine AI big models to achieve more efficient and intelligent system management and optimization. Summary of the invention

[0005] The present invention provides an AI-driven digital twin configuration system and data processing method to achieve more efficient and intelligent system management and optimization.

[0006] According to one aspect of the present invention, there is provided an AI-driven digital twin configuration system, comprising:

[0007] A data extraction layer, for extracting data from at least one data source;

[0008] Data conversion layer, used for data cleaning and preprocessing, data aggregation and calculation, and AI analysis and prediction;

[0009] Data loading layer, used to load processed data into the storage system;

[0010] The scheduling layer is used for scheduling layer management and scheduling the execution of ETL tasks;

[0011] Data monitoring and management layer is responsible for monitoring the execution status and data quality of the ETL process.

[0012] Optionally, the data source includes: sensor data, database data and log data;

[0013] Correspondingly, the data extraction layer is used to collect the sensor data in real time through the configuration interface of the sensor; obtain the database data regularly or according to the acquisition requirements; and obtain the log data from the system log file.

[0014] Optionally, the data conversion layer is used to perform:

[0015] Extract data through the data extraction layer and perform data cleaning and preprocessing;

[0016] Perform feature engineering, extract and select features, transform features and generate feature vectors. For time series data, create time windows and generate feature data sets suitable for model input.

[0017] Use the TensorFlow framework to build an LSTM model, define the model structure, input the preprocessed data into the model for training, and optimize the model performance by adjusting the model parameters;

[0018] Use training data and validation data to evaluate model performance, measure model effects, and optimize the model;

[0019] The trained model is saved and deployed to the production environment, and the real-time data is predicted in real time through the model. Real-time monitoring, alarm and decision support are performed based on the prediction results.

[0020] Optionally, the data loading layer is used to perform:

[0021] Store time series data in a time series database;

[0022] Store structured data in a relational database;

[0023] Store metrics data in the metrics database.

[0024] Optionally, the scheduling layer is used to perform:

[0025] Manage and schedule the execution of ETL tasks through scheduling tools Apache Airflow and Luigi.

[0026] Optionally, the data monitoring and management layer is used to perform:

[0027] Monitoring and log management through Prometheus, Grafana and ELK Stack to ensure data accuracy and integrity;

[0028] Track the execution of ETL tasks, handle exceptions, and provide data query and management through the data management interface.

[0029] Optionally, the system further includes: a user interface layer;

[0030] The user interface layer is used to add graphics and models on the model graphics management page, drag and drop models in the configuration editor to edit the configuration process and save it, view real-time data or simulation data in the monitoring panel, and define and monitor key performance indicators on the indicator system page.

[0031] According to another aspect of the present invention, there is provided a data processing method, comprising:

[0032] The data extraction layer extracts data from at least one data source;

[0033] The data conversion layer performs data cleaning and preprocessing, data aggregation and calculation, and AI analysis and prediction;

[0034] The data loading layer loads the processed data into the storage system;

[0035] Scheduling layer The scheduling layer manages and schedules the execution of ETL tasks;

[0036] The data monitoring and management layer is responsible for monitoring the execution status and data quality of the ETL process.

[0037] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0038] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data processing method described in any embodiment of the present invention.

[0039] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data processing method described in any embodiment of the present invention when executed.

[0040] According to another aspect of the present invention, a computer program product is provided. The computer program product comprises a computer program. When the computer program is executed by a processor, the data processing method according to any embodiment of the present invention is implemented.

[0041] The technical solution of the embodiment of the present invention combines AI big model technology and digital twin technology, and the system realizes intelligent data analysis and prediction capabilities, while supporting real-time monitoring and simulation of complex systems, thereby improving the system's intelligent management level and decision-making optimization capabilities.

[0042] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 1 is a schematic diagram of the structure of a digital twin configuration system based on AI drive provided in the first embodiment of the present invention;

[0045] Figure 2 1 is a schematic diagram of the structure of a digital twin configuration system based on AI drive provided in the first embodiment of the present invention;

[0046] Figure 3 This is an architecture diagram of an AI-driven digital twin configuration system provided in Embodiment 1 of the present invention;

[0047] Figure 4 is a flow chart of a data processing method provided by Embodiment 2 of the present invention;

[0048] Figure 5 It is a structural schematic diagram of an electronic device for implementing the data processing method of an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0050] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0051] Embodiment 1

[0052] Figure 1 : is a structural diagram of an AI-driven digital twin configuration system provided in Embodiment 1 of the present invention, the system comprising:

[0053] The data extraction layer 110 is used to extract data from at least one data source.

[0054] In an embodiment of the present invention, the data source includes: sensor data, database data and log data;

[0055] Correspondingly, the data extraction layer 110 is used to collect the sensor data in real time through the configuration interface of the sensor; obtain the database data regularly or according to the acquisition requirements; and obtain the log data from the system log file.

[0056] Extract data from various data sources, including sensor data, database data, and log data. Sensor data is collected in real time through the configuration interface, database data is extracted periodically or on demand, and log data is obtained from system log files. Use extraction tools such as Apache NiFi and Logstash to ensure efficient data collection.

[0057] The data conversion layer 120 is used for data cleaning and preprocessing, data aggregation and calculation, and AI analysis and prediction.

[0058] In the embodiment of the present invention, the data conversion layer 120 is specifically used to perform:

[0059] Extract data through the data extraction layer and perform data cleaning and preprocessing;

[0060] Perform feature engineering, extract and select features, transform features and generate feature vectors. For time series data, create time windows and generate feature data sets suitable for model input.

[0061] Use the TensorFlow framework to build an LSTM model, define the model structure, input the preprocessed data into the model for training, and optimize the model performance by adjusting the model parameters;

[0062] Use training data and validation data to evaluate model performance, measure model effects, and optimize the model;

[0063] The trained model is saved and deployed to the production environment, and the real-time data is predicted in real time through the model. Real-time monitoring, alarm and decision support are performed based on the prediction results.

[0064] Among them, data cleaning and preprocessing: adding the mutate plug-in to Logstash can remove noise, process missing values, and convert formats.

[0065] Data aggregation and calculation: The Hadoop MapReduce distributed computing framework can be used to perform data aggregation operations, such as calculating the average and sum by time window.

[0066] Data aggregation and calculation: The Hadoop MapReduce distributed computing framework can be used to perform data aggregation operations, such as calculating the average and sum by time window.

[0067] AI analysis and prediction are the core functions of the AI-driven digital twin configuration system. By applying machine learning and deep learning models, advanced analysis and prediction are performed on the collected data. The implementation process of the AI ​​big model is as follows:

[0068] 1. Data: Data such as sensor databases are extracted through the data extraction layer, and then the data is cleaned and preprocessed through the data conversion layer to preprocess the data.

[0069] 2. Feature engineering: extract and select important features, perform feature transformation and generate feature vectors according to task requirements; for time series data, it is necessary to create time windows and generate feature data sets suitable for model input.

[0070] 3. Model selection pre-training: Use the TensorFlow framework to build an LSTM model, define the model structure (input layer, LSTM layer, Dropout layer and output layer, the number of neurons in each layer, activation function, etc.), input the preprocessed data into the model for training, and optimize the model performance by adjusting the model parameters (such as learning rate, batch size, and number of training rounds).

[0071] 4. Model evaluation and pre-optimization: Use training data and validation data to evaluate model performance, and use cross-validation, confusion matrix, AUC and other indicators to measure model effects. Optimize the model by adjusting parameters, increasing data volume, adjusting model structure, etc. to achieve the best prediction effect.

[0072] Model deployment and prediction: The trained model is saved and deployed to the production environment, and can be called using the interface; real-time data is processed through the stream processing framework Apache Kafka, and the deployed model is called for real-time prediction; real-time monitoring, alarms, and decision support are performed based on the prediction results to ensure timely response to changes.

[0073] The data loading layer 130 is used to load the processed data into the storage system.

[0074] In the embodiment of the present invention, the data loading layer 130 is specifically used to perform:

[0075] The processed data is loaded into the appropriate storage system. Time series data is stored in the time series database (TDengine), structured data is stored in the relational database (MySQL), and indicator data is stored in the indicator database (Prometheus). The tools Sqoop and Kafka Connect are used to efficiently load data.

[0076] The scheduling layer 140 is used for scheduling layer management and scheduling the execution of ETL tasks.

[0077] In the embodiment of the present invention, the scheduling layer 140 is specifically used to perform:

[0078] Manage and schedule the execution of ETL tasks through scheduling tools Apache Airflow and Luigi.

[0079] The data monitoring and management layer 150 is responsible for monitoring the execution status and data quality of the ETL process.

[0080] In the embodiment of the present invention, in the embodiment of the present invention, it is specifically used to perform:

[0081] Monitoring and log management through Prometheus, Grafana and ELK Stack to ensure data accuracy and integrity;

[0082] Track the execution of ETL tasks, handle exceptions, and provide data query and management through the data management interface.

[0083] The data monitoring and management layer is responsible for monitoring the execution status and data quality of the ETL process. The system uses Prometheus, Grafana, and ELK Stack (Elasticsearch, Logstash, Kibana) for monitoring and log management. Data quality monitoring ensures the accuracy and integrity of the data, data flow monitoring tracks the execution of ETL tasks, log and alarm management handles abnormal situations, and provides query and management of data through the data management interface.

[0084] like Figure 2 As shown, the system further includes: a user interface layer 160;

[0085] The user interface layer 160 is used to add entities and models in the model entity management page, drag and drop models in the configuration editor to edit the configuration process and save it, view real-time data or simulation data in the monitoring panel, and define and monitor key performance indicators in the indicator system page.

[0086] Users interact with the system through an intuitive user interface. It mainly uses vuejs+threejs front-end technology. First, add the required graphics and models to the model graphics management page. The added graphics can be used in the configuration editor. The configuration editor can drag and drop the model to edit the configuration process and save the dragged configuration. The saved configuration can be seen in the monitoring panel. There are two options for the monitoring panel, one is real-time data and the other is simulation data. When selecting real-time data, you can view information such as the current operating status of the device. If you choose simulation data, you can generate high-quality simulation data through the AI ​​large model. Indicator system page: Users can define and monitor key performance indicators (KPIs) to evaluate the operating status of the system.

[0087] Figure 3 This is an architecture diagram of an AI-driven digital twin configuration system provided in Embodiment 1 of the present invention. Specifically, it includes:

[0088] Data extraction layer: Extract data from data sources such as sensor data, database data, and log data, and use tools such as Apache NiFi and Logstash to ensure efficient collection.

[0089] Data conversion layer:

[0090] Data cleaning and preprocessing: Add the mutate plug-in to Logstash to remove noise, process missing values, convert formats, etc.

[0091] Data aggregation and computation: Use Hadoop MapReduce distributed computing framework to perform data aggregation operations.

[0092] AI analysis and prediction: Use machine learning and deep learning models to perform advanced analysis and prediction of data. The AI ​​large model implementation process includes data extraction and preprocessing, feature engineering, model selection pre-training, model evaluation pre-optimization, model deployment and prediction.

[0093] Data loading layer: The processed data is loaded into the time series database (TDengine), relational database (MySQL), and indicator database (Prometheus), using Sqoop and Kafka Connect for efficient loading.

[0094] Scheduling layer: Use scheduling tools such as Apache Airflow and Luigi to manage and schedule the execution of ETL tasks to ensure that the process proceeds as planned.

[0095] Data monitoring and management layer: Use Prometheus, Grafana, and ELK Stack for monitoring and log management to ensure data accuracy and integrity, track ETL task execution, handle exceptions, and provide data query and management interfaces.

[0096] User interface layer: Users interact with the system through the user interface of vuejs+threejs front-end technology, add graphics and models in the model graphics management page, drag and drop models in the configuration editor to edit the configuration process and save it, view real-time data or simulation data in the monitoring panel, and define and monitor key performance indicators in the indicator system page.

[0097] The key points of the solution of the embodiment of the present invention are concentrated on its innovative AI-driven digital twin technology and multi-layer data processing architecture. By combining AI big model technology and digital twin technology, the system realizes intelligent data analysis and prediction capabilities, while supporting real-time monitoring and simulation of complex systems, thereby improving the system's intelligent management level and decision-making optimization capabilities. The key points to be protected include the design of the system architecture, covering multiple levels such as data extraction, conversion, loading, scheduling, monitoring and management; as well as the specific technical methods used in each layer, such as data extraction methods, AI model implementation and optimization, real-time prediction and fault diagnosis technology, as well as user interface design and interactive functions. These innovations comprehensively protect the system's comprehensive technical advantages in the fields of intelligent operation management and data processing.

[0098] The embodiments of the present invention mainly focus on its innovative data processing and AI-driven technologies, including efficient data extraction methods, the implementation of intelligent data processing and prediction models, and multi-level data management and monitoring mechanisms in the system architecture. Through meticulous technical implementation details and the use of proprietary technical tools such as Logstash, Hadoop MapReduce, TensorFlow, etc., the system is ensured to be technologically advanced and innovative in key links such as data cleaning, feature engineering, model selection and optimization. In addition, the patent also covers the design and interactive functions of the user interface, such as the configuration editor and the real-time monitoring panel, which further enhances the unique value of the system in terms of user experience and ease of operation. Combining these aspects, patent protection ensures the comprehensive innovation and competitive advantage of the system in intelligent management and technical implementation.

[0099] Embodiment 2

[0100] Figure 4 : is a flow chart of a data processing method provided by Embodiment 2 of the present invention. Figure 4 As shown, the method includes:

[0101] S410: The data extraction layer extracts data from at least one data source.

[0102] S420, the data conversion layer performs data cleaning and preprocessing, data aggregation and calculation, and AI analysis and prediction.

[0103] S430: The data loading layer loads the processed data into the storage system.

[0104] S440, Scheduling Layer The scheduling layer manages and schedules the execution of ETL tasks.

[0105] S450, Data Monitoring and Management The data monitoring and management layer is responsible for monitoring the execution status and data quality of the ETL process.

[0106] As the information interaction, execution process and other contents between the units in the above method are based on the same concept as the system embodiment of the present invention, the specific contents can be found in the description of the system embodiment of the present invention and will not be repeated here.

[0107] Embodiment 3

[0108] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0109] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0110] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0111] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a data processing method.

[0112] In some embodiments, the data processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0113] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0114] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0115] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0116] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0117] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0118] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0119] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0120] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. AI-driven digital twin configuration system, characterized by: include: A data extraction layer, for extracting data from at least one data source; Data conversion layer, used for data cleaning and preprocessing, data aggregation and calculation, and AI analysis and prediction; The data loading layer is used to load the processed data into the storage system; The scheduling layer is used for scheduling layer management and scheduling the execution of ETL tasks; Data monitoring and management layer is responsible for monitoring the execution status and data quality of the ETL process.

2. The AI-driven digital twin configuration system according to claim 1 is characterized in that: The data source includes: sensor data, database data and log data; Correspondingly, the data extraction layer is used to collect the sensor data in real time through the configuration interface of the sensor; obtain the database data regularly or according to the acquisition requirements; and obtain the log data from the system log file.

3. The AI-driven digital twin configuration system according to claim 1 is characterized in that: The data conversion layer is used to perform: Extract data through the data extraction layer and perform data cleaning and preprocessing; Perform feature engineering, extract and select features, transform features and generate feature vectors. For time series data, create time windows and generate feature data sets suitable for model input. Use the TensorFlow framework to build an LSTM model, define the model structure, input the preprocessed data into the model for training, and optimize the model performance by adjusting the model parameters; Use training data and validation data to evaluate model performance, measure model effects, and optimize the model; The trained model is saved and deployed to the production environment, and the real-time data is predicted in real time through the model, and real-time monitoring, alarm and decision support are performed based on the prediction results.

4. The AI-driven digital twin configuration system according to claim 1 is characterized in that: The data loading layer is used to perform: Store time series data in a time series database; Store structured data in a relational database; Store metrics data in the metrics database.

5. The AI-driven digital twin configuration system according to claim 1, characterized in that: The scheduling layer is used to perform: The execution of ETL tasks is managed and scheduled through the scheduling tools Apache Airflow and Luigi.

6. The AI-driven digital twin configuration system according to claim 1, characterized in that: The data monitoring and management layer is used to perform: Monitoring and log management through Prometheus, Grafana and ELK Stack to ensure data accuracy and integrity; Track the execution of ETL tasks, handle exceptions, and provide data query and management through the data management interface.

7. The AI-driven digital twin configuration system according to claim 1, characterized in that: Further including: User interface layer; The user interface layer is used to add graphics and models on the model graphics management page, drag and drop models in the configuration editor to edit the configuration process and save it, view real-time data or simulation data in the monitoring panel, and define and monitor key performance indicators on the indicator system page.

8. A data processing method for an AI-driven digital twin configuration system according to any one of claims 1 to 7, characterized in that: include: The data extraction layer extracts data from at least one data source; The data conversion layer performs data cleaning and preprocessing, data aggregation and calculation, and AI analysis and prediction; The data loading layer loads the processed data into the storage system; Scheduling layer The scheduling layer manages and schedules the execution of ETL tasks; The data monitoring and management layer is responsible for monitoring the execution status and data quality of the ETL process.

9. An electronic device, characterized in that The electronic device comprises: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data processing method described in claim 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data processing method of claim 8 when executed.

11. A computer program product, characterized in that The computer program product comprises a computer program which, when executed by a processor, implements the data processing method according to claim 8 .

Images