Multi-modal intelligent technology supervision integrated management system deployed at plant side edge end

Abstract

The application provides a multimodal intelligent technology supervision integrated management and control system deployed at a plant side edge end, and relates to the technical field of equipment maintenance. The system comprises an edge end hardware layer, a multimodal data access layer, an edge computing and AI engine layer, and an integrated management and control application layer. The edge end hardware layer is deployed at a plant side site and is used to provide monitoring data and computing resources at the edge side. The multimodal data access layer is used to realize standardized access of multimodal heterogeneous data from the edge end hardware layer. The edge computing and AI engine layer is used to provide AI engines and processing models. Based on the processing task call of the integrated management and control application layer, the corresponding AI engine or processing model is called, and based on the multimodal data access layer, the associated data of the processing task is provided to obtain a fusion decision result. The integrated management and control application layer is used to provide system functions and the like to a user. The embodiments provided in the application realize integrated whole-process management and control of a plant side technology supervision and management and control system.

Description

Technical Field

[0001] This application relates to the field of equipment maintenance technology, specifically to a multimodal intelligent technology supervision and integrated management and control method deployed at the edge of the plant, a multimodal intelligent technology supervision and integrated management and control system deployed at the edge of the plant, and an electronic device. Background Technology

[0002] With the deepening of Industry 4.0 and the construction of smart power plants, the requirements for intelligent, real-time, and integrated equipment technical supervision, safety management, and operation and maintenance decision-making in plant-side production scenarios such as thermal power generation, new energy power generation, and industrial manufacturing are continuously increasing. Currently, plant-side technical supervision and management systems generally suffer from the following technical pain points: The problem of data silos is prominent: Multimodal data such as equipment operation data, inspection images / videos, test reports, and maintenance records are scattered across independent platforms such as DCS, SIS, MIS, inspection systems, and testing systems, lacking a unified access, governance, and sharing mechanism, making it difficult to fully explore the value of the data. Edge-cloud collaboration capabilities are insufficient: A large number of monitoring, diagnostic, and control tasks with high real-time requirements rely on cloud processing, resulting in significant network latency, bandwidth pressure, and data security risks, failing to meet the millisecond-to-second real-time control requirements of the plant side. Multimodal perception and intelligent analysis capabilities are weak: Traditional platforms mainly process structured time-series data, lacking the ability to analyze, extract features, and fuse unstructured multimodal data such as images, videos, audio, text, and infrared thermal imaging, making it difficult to achieve full-dimensional equipment status perception and accurate fault identification. Technical supervision and control processes are fragmented: Functions such as equipment supervision, safety monitoring, hazard warning, maintenance decision-making, ledger management, and report generation are scattered across different systems, resulting in cumbersome operation processes, a lack of integrated, visualized, and closed-loop control capabilities, low maintenance efficiency, and high human error. Low level of intelligence at the edge: Plant-side edge nodes are mostly for data collection and simple forwarding, lacking edge computing, AI inference, and lightweight model deployment capabilities, making it impossible to achieve real-time intelligent analysis and local autonomous control. Insufficient standardization and scalability: Existing platforms are mostly customized developments with inconsistent interfaces, incompatible protocols, and closed architectures, making it difficult to adapt to the rapid expansion and iterative upgrades of different plants, equipment types, and monitoring scenarios.

[0003] The aforementioned problems result in low efficiency of plant-side technical supervision, delayed discovery of potential hazards, reliance on experience for decision-making, high operation and maintenance costs, and difficulty in managing safety risks. Therefore, a new type of platform is needed that is deployed at the plant edge, supports multimodal data fusion, and has intelligent analysis and integrated management capabilities to achieve intelligent supervision and efficient safety management of the entire plant production process. Summary of the Invention

[0004] The purpose of this application is to provide a multimodal intelligent technology monitoring and integrated control system deployed at the edge of the plant, which improves the completeness of data monitoring and the level of intelligent processing, so as to at least solve some of the problems in the background technology.

[0005] To achieve the above objectives, this application provides a multimodal intelligent technology monitoring and integrated management system deployed at the plant edge, comprising: an edge hardware layer, a multimodal data access layer, an edge computing and AI engine layer, and an integrated management and control application layer; the edge hardware layer is deployed at the plant site to provide edge-side monitoring data and computing resources; the multimodal data access layer is used to achieve standardized access of multimodal heterogeneous data from the edge hardware layer; the edge computing and AI engine layer is used to provide an AI engine and processing model, and to call the corresponding AI engine or processing model based on the processing task of the integrated management and control application layer, and to obtain fusion decision results based on the associated data of the processing task provided by the multimodal data access layer; the integrated management and control application layer is used to provide system functions to users, and to generate processing tasks based on the user's selection of the system functions and send them to the edge computing and AI engine layer, and to obtain the fusion decision results corresponding to the processing tasks and present them to the user.

[0006] Optionally, the system further includes an edge-cloud collaboration layer and a security and operation and maintenance support layer; the edge-cloud collaboration layer is used for data synchronization and model adaptation between the edge and the cloud; the operation and maintenance support layer is used to provide data security, system security and operation and maintenance support within the system.

[0007] Optionally, the edge hardware layer includes: an edge computing gateway, an industrial control computer, an intelligent data acquisition terminal, sensors, cameras, infrared thermal imagers, and audio acquisition devices.

[0008] Optionally, the multimodal data access layer includes: a structured data processing module for processing structured data, including: DCS / SIS / PLC real-time time-series data, electrical energy data, and environmental monitoring data; an unstructured data processing module for processing unstructured data, including: inspection images, videos, infrared thermal images, audio, inspection reports, and maintenance text; a protocol adaptation module for adapting and converting different industrial protocols and different communication interfaces; and a data preprocessing module for preprocessing multimodal data, including: cleaning, format conversion, normalization, feature extraction, data alignment, and noise reduction and enhancement.

[0009] Optionally, the edge computing and AI engine layer includes: an edge computing management module for performing lightweight real-time computing, streaming data processing, local caching, rule engine, and linkage control; a multimodal AI engine including various AI models that support image recognition, video analysis, audio detection, text extraction, time series prediction, and anomaly diagnosis; a model management module for calling and managing models, including lightweight compression, distributed deployment, online updates, and incremental learning; and a fusion decision module for multimodal data processing, including feature fusion, evidence reasoning, confidence fusion, and joint decision-making.

[0010] Optionally, the integrated management and control application layer includes: an equipment technical supervision module for performing equipment status monitoring, performance calculation, life assessment, metal supervision, insulation supervision, chemical supervision, and thermal supervision; a safety intelligent monitoring module for performing video intelligent analysis, area intrusion detection, personnel violation identification, equipment abnormal temperature measurement, and fire hazard monitoring; a hazard early warning and fault diagnosis module for performing multi-dimensional abnormal early warning, fault tracing, root cause analysis, early warning level classification, and coordinated response; an intelligent operation and maintenance decision-making module for performing maintenance strategy generation, maintenance plan optimization, spare parts management, work order generation, and closed-loop tracking; a ledger and document management module for unified management of equipment lifecycle ledgers, technical data, test reports, and operation and maintenance records; a visualization large screen module for multi-dimensional data visualization, real-time monitoring large screen control, trend analysis, and statistical reports; and an automatic report generation module for automatically generating technical supervision reports, monthly reports, quarterly reports, and annual reports according to specifications, supporting data export and data reporting.

[0011] Optionally, the edge-cloud collaboration layer includes: an edge autonomy module for local real-time processing of critical tasks and continuous operation in the event of a network outage; an edge-cloud collaboration module for on-demand uploading of edge data, cloud model distribution and optimization, and remote monitoring and maintenance; a data synchronization module for incremental synchronization, breakpoint resumption, and data consistency assurance; and a remote upgrade module for remote configuration of edge nodes, model updates, and software upgrades.

[0012] Optionally, the security and operation and maintenance support layer includes: a data security module for encrypting data transmission, storing data, managing permissions, and desensitizing data within the system; a system security module for performing identity authentication, access control, firewall, intrusion detection, and log auditing within the system; and an operation and maintenance support module for implementing system monitoring, fault alarms, status self-checks, remote diagnostics, and configuration management.

[0013] This application also provides a method for integrated supervision and management of multimodal intelligent technology deployed at the plant edge, based on the aforementioned integrated supervision and management system for multimodal intelligent technology deployed at the plant edge. The method includes: acquiring user tasks based on the integrated management and control layer; determining an AI engine or processing model and associated data required to complete the user tasks based on the user tasks by the edge computing and AI engine layer; acquiring the required associated data from the multimodal data access layer; processing the acquired associated data through the determined AI engine or processing model to obtain a fusion decision result; and presenting the fusion decision result to the user at the integrated management and control layer.

[0014] This application also provides an electronic device, including: at least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the aforementioned multimodal intelligent technology supervision and integrated control method deployed at the edge of the plant by executing the instructions stored in the memory.

[0015] This application also provides a machine-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the aforementioned multimodal intelligent technology supervision and integrated control method deployed at the edge of the plant.

[0016] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned multimodal intelligent technology supervision and integrated control method deployed at the edge of the plant.

[0017] The above technical solution has the following beneficial effects: This system offers a novel architecture based on the fusion of multimodal data across the entire domain, unifying access to various data sources to achieve comprehensive perception and sharing. Real-time edge processing and analysis reduce latency and cloud dependency, ensuring rapid response and on-site safety. Leveraging multimodal AI fusion modeling, the system significantly improves the accuracy of anomaly, fault, and potential hazard identification, reducing false alarms and missed alarms. The platform encompasses full-process functions including monitoring, early warning, decision-making, operation and maintenance, and record keeping, achieving one-stop management. The edge side possesses high autonomy, ensuring high overall system reliability. The lightweight architecture allows for rapid deployment and adaptability to various plant environments; the modular design facilitates flexible expansion for different scenarios and business needs, effectively reducing manual inspection and analysis costs and improving operational efficiency. Localized data processing enhances privacy protection, complies with industrial safety and regulatory requirements, provides strong support for the digital and intelligent upgrades of plants, and improves operational management levels.

[0018] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings: Figure 1 This illustration schematically shows a structural diagram of a multimodal intelligent technology monitoring and control system deployed at the edge of a plant according to an embodiment of this application. Figure 2 This illustration schematically shows the steps of a multimodal intelligent technology-based integrated monitoring and control method deployed at the edge of a plant according to an embodiment of this application. Figure 3 The diagram schematically illustrates the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0020] The specific embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the embodiments of this application.

[0021] Figure 1 This illustration schematically shows a structural diagram of a multimodal intelligent technology-based integrated monitoring and control system deployed at the edge of a plant according to an embodiment of this application. For example... Figure 1 As shown, a multimodal intelligent technology-based integrated monitoring and control system deployed at the edge of a plant includes: The system comprises an edge hardware layer, a multimodal data access layer, an edge computing and AI engine layer, and an integrated management and control application layer. The edge hardware layer is deployed at the plant site to provide edge-side monitoring data and computing resources. The multimodal data access layer enables standardized access to multimodal heterogeneous data from the edge hardware layer. The edge computing and AI engine layer provides an AI engine and processing models, and invokes the corresponding AI engine or processing model based on the processing tasks from the integrated management and control application layer. It also provides associated data with the processing tasks from the multimodal data access layer to obtain fusion decision results. The integrated management and control application layer provides system functions to users, generates processing tasks based on user selections of these functions, sends them to the edge computing and AI engine layer, and obtains the fusion decision results corresponding to the processing tasks for presentation to the user.

[0022] Through the integrated management and control system with the above hierarchical structure, based on the full-domain fusion of multimodal data, it uniformly accesses various data sources to achieve full-dimensional perception and sharing. Real-time processing and analysis at the edge reduces latency and cloud dependence, ensuring response speed and on-site safety.

[0023] Deployment in real-world scenarios can be designed according to requirements. For example: Option 1: Pure edge architecture, without cloud collaboration, suitable for factory scenarios with no external network and high security requirements; Option 2: Multi-level edge collaboration architecture (field edge → workshop edge → factory edge), suitable for large factories, multi-branch factory scenarios, etc.

[0024] In some embodiments of this application, the system further includes an edge-cloud collaboration layer and a security and operation support layer. The edge-cloud collaboration layer is used for data synchronization and model adaptation between the edge and the cloud. The operation support layer provides data security, system security, and operation support within the system. By setting up the edge-cloud collaboration layer, data can be processed and analyzed locally at the edge, reducing latency, decreasing cloud pressure, improving response speed, and ensuring real-time production and security. By setting up the security and operation support layer, local data processing is prioritized, privacy protection is strong, industrial information security and industry regulatory requirements are met, and core support is provided for the digital, intelligent, and secure upgrades of the plant, improving the overall operation and management level.

[0025] In some embodiments of this application, the edge hardware layer includes: an edge computing gateway, an industrial control computer, an intelligent data acquisition terminal, sensors, cameras, infrared thermal imagers, and audio acquisition devices. The edge hardware layer integrates computing, control, and sensing devices; each sensing device is used to acquire multi-dimensional monitoring data from visual, temperature, acoustic, and physical quantities; the intelligent data acquisition terminal is used for the fusion and forwarding of multi-source signals; and the edge computing gateway and industrial control computer, as core computing nodes, are responsible for data aggregation, protocol parsing, and real-time decision-making.

[0026] In some embodiments of this application, the multimodal data access layer includes: a structured data processing module for processing structured data, including: DCS / SIS / PLC real-time time-series data, electrical energy data, and environmental monitoring data; an unstructured data processing module for processing unstructured data, including: inspection images, videos, infrared thermal images, audio, inspection reports, and maintenance text; a protocol adaptation module for adapting and converting different industrial protocols and different communication interfaces; and a data preprocessing module for preprocessing multimodal data, including: cleaning, format conversion, normalization, feature extraction, data alignment, and noise reduction and enhancement.

[0027] In some embodiments of this application, the edge computing and AI engine layer includes: an edge computing management module for performing lightweight real-time computing, streaming data processing, local caching, rule engine, and linkage control; a multimodal AI engine including various AI models supporting image recognition, video analysis, audio detection, text extraction, time series prediction, and anomaly diagnosis; a model management module for model invocation and management, including lightweight compression, distributed deployment, online updates, and incremental learning; and a fusion decision module for multimodal data processing, including feature fusion, evidence reasoning, confidence fusion, and joint decision-making. In this edge computing and AI engine layer, a lightweight federated learning model can also be used to achieve joint modeling of multiple edge nodes, protecting data privacy. A large model lightweight fine-tuning scheme is adopted to improve multimodal understanding and decision-making capabilities in complex scenarios. A rule engine + AI hybrid inference mode is used to balance real-time performance and inference accuracy.

[0028] In some embodiments of this application, the integrated management and control application layer includes: an equipment technical supervision module for performing equipment status monitoring, performance calculation, life assessment, metal supervision, insulation supervision, chemical supervision, and thermal supervision; a safety intelligent monitoring module for performing video intelligent analysis, area intrusion detection, personnel violation identification, equipment abnormal temperature measurement, and fire hazard monitoring; a hazard early warning and fault diagnosis module for performing multi-dimensional abnormal early warning, fault tracing, root cause analysis, early warning level classification, and coordinated handling; an intelligent operation and maintenance decision-making module for performing maintenance strategy generation, maintenance plan optimization, spare parts management, work order generation, and closed-loop tracking; a ledger and document management module for unified management of equipment lifecycle ledgers, technical data, test reports, and operation and maintenance records; a visualization large screen module for multi-dimensional data visualization, real-time monitoring large screen control, trend analysis, and statistical reports; and an automatic report generation module for automatically generating technical supervision reports, monthly reports, quarterly reports, and annual reports according to specifications, supporting data export and data reporting.

[0029] In some embodiments of this application, the edge-cloud collaboration layer includes: an edge autonomy module for local real-time processing of critical tasks and continuous operation in the event of a network outage; an edge-cloud collaboration module for on-demand uploading of edge data, cloud model distribution and optimization, and remote monitoring and maintenance; a data synchronization module for incremental synchronization, breakpoint resumption, and data consistency assurance; and a remote upgrade module for remote configuration of edge nodes, model updates, and software upgrades.

[0030] In some embodiments of this application, the security and operation and maintenance support layer includes: a data security module for encrypting data transmission, storing data, managing permissions, and desensitizing data within the system; a system security module for performing identity authentication, access control, firewall, intrusion detection, and log auditing within the system; and an operation and maintenance support module for implementing system monitoring, fault alarms, status self-checks, remote diagnostics, and configuration management.

[0031] In some practical scenarios, corresponding professional monitoring modules can be set up according to industry differences to adapt to different industries such as new energy, chemical industry, and metallurgy. A digital twin module can also be added to achieve 3D visualization of equipment, virtual debugging, and simulation. Furthermore, an intelligent linkage control module can be added to achieve a fully automatic closed loop of early warning, linkage, control, and feedback.

[0032] Taking the deployment of a multimodal intelligent technology supervision and integrated management and control platform at the edge of a thermal power plant as an example, this application describes the multimodal intelligent technology supervision and integrated management and control system deployed at the edge of the plant.

[0033] The thermal power plant production area includes key equipment such as boilers, steam turbines, generators, and auxiliary equipment. It is equipped with 3 edge computing gateways, 12 intelligent acquisition terminals, 26 high-definition cameras, 8 infrared thermal imagers, and 6 audio acquisition devices, and is connected to DCS, SIS, inspection system, chemical detection system, metal detection system, etc.

[0034] The system deployment process is as follows: Hardware Deployment: Deploy edge gateways, data acquisition terminals, and sensing devices in the plant control room, equipment layer, and key areas, and conduct network debugging. Data Access: Access time-series data such as temperature, pressure, vibration, and flow; camera video and infrared images; multimodal data such as detection report text and maintenance records. Model Deployment: Deploy lightweight image recognition models (equipment defect identification, personnel violation identification), time-series prediction models (equipment status trends), anomaly diagnosis models, and text extraction models. Application Configuration: Configure business functions such as boiler metal monitoring, turbine vibration monitoring, insulation monitoring, chemical monitoring, safety monitoring, hazard warning, maintenance decision-making, ledger management, and report generation. Joint Testing: Data access testing, AI model inference testing, functional linkage testing, edge autonomy testing, and edge-cloud collaboration testing. Online Operation: Officially put into use, enabling real-time monitoring, intelligent analysis, and integrated management.

[0035] The following operational results were ultimately achieved: 100% unified access rate for multimodal data, data processing latency <500ms; equipment defect identification accuracy >95%, hazard warning accuracy >93%; 100% automatic generation rate of technical supervision reports, improving operation and maintenance efficiency by more than 60%; independent operation for ≥24 hours even when the edge network is down, ensuring production safety; 50% reduction in manual inspection workload, resulting in a significant decrease in operation and maintenance costs.

[0036] Based on the same inventive concept, this application also provides a multimodal intelligent technology supervision and integrated management method deployed at the edge of the plant, which is implemented based on the aforementioned multimodal intelligent technology supervision and integrated management system deployed at the edge of the plant. Figure 2 This illustration schematically depicts the steps of a multimodal intelligent technology-based integrated monitoring and control method deployed at the plant edge according to an embodiment of this application. For example... Figure 2 As shown, the method includes: S01. Obtain user tasks based on the integrated management and control layer; S02. The edge computing and AI engine layer determines the AI ​​engine or processing model based on the user task, as well as the associated data required to complete the user task. S03. Obtain the required associated data from the multimodal data access layer; S04. After the acquired relevant data is processed by the determined AI engine or processing model, the fusion decision result is obtained. S05. The integrated management and control layer presents the fusion decision results to the user.

[0037] In step S02, the edge computing and AI engine layer determines the AI ​​engine or processing model based on the user task using the following steps: 1. According to the category of the user task, obtain the execution resource allocation scheme and execution duration of historical user tasks of the same category; 2. Determine the cloud model with the highest complexity in the execution resource allocation scheme, and obtain the cloud processing duration and edge processing duration in the execution resource allocation scheme corresponding to the cloud model with the highest complexity; 3. Determine the execution resource allocation scheme with the shortest execution duration, and determine the cloud processing duration and edge processing duration in the execution resource allocation scheme with the shortest execution duration; 4. When the cloud models in the execution resource allocation schemes in steps 2 and 3 are the same, then select that cloud model as the AI ​​engine for processing the user task. 5. When the two are different, determine the difference in cloud processing time and the difference in edge processing time between the two schemes to identify whether the computing bottleneck is in the cloud or the edge, and clarify the optimization focus; 6. When the difference in cloud processing time is large, obtain the current resource usage in the cloud. When the cloud resource usage rate is lower than the preset threshold, use the cloud model in the execution resource allocation scheme with the shortest execution time as the AI ​​engine or processing model to process the user task. When the cloud resource usage rate is not lower than the preset threshold, use the cloud model with the highest complexity in the execution resource allocation scheme to process the current user task. 7. When the difference in edge processing time is large, obtain the current communication load of the system. When the communication load is lower than the preset threshold, use the cloud model in the execution resource allocation scheme with the shortest execution time as the AI ​​engine or processing model to process the user task. When the communication load is not lower than the preset threshold, use the cloud model with the highest complexity in the execution resource allocation scheme to process the current user task. 8. Determine the required associated data based on the input requirements of the determined AI engine or processing model.

[0038] The specific limitations of each functional module in the aforementioned multimodal intelligent technology supervision and integrated control system deployed at the plant edge can be found in the limitations of the multimodal intelligent technology supervision and integrated control method deployed at the plant edge mentioned above, and will not be repeated here. Each module in the aforementioned system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0039] In some embodiments of this application, an electronic device is also provided, comprising: at least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor executes the aforementioned multimodal intelligent technology-based integrated supervision and control method deployed at the factory edge. Its internal structure diagram can be shown as follows. Figure 3 As shown. Figure 3 The diagram schematically illustrates the internal structure of an electronic device according to an embodiment of this application. The electronic device includes a processor A01, a network interface A02, a memory (not shown), and a database (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A04. The network interface A02 is used for communication with external terminals via a network connection. When the computer program B02 is executed by the processor A01, it implements a multimodal intelligent technology-based integrated supervision and control method deployed at the plant edge.

[0040] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0041] In one embodiment of this application, a machine-readable storage medium is provided, on which instructions are stored. When executed by a processor, the instructions cause the processor to be configured to perform the aforementioned multimodal intelligent technology supervision and integrated control method deployed at the edge of the plant.

[0042] In one embodiment provided in this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the aforementioned multimodal intelligent technology supervision and integrated control method deployed at the edge of the plant.

[0043] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0044] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0045] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0046] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0047] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0048] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0049] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0050] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0051] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A multimodal intelligent technology-integrated monitoring and control system deployed at the edge of a plant, characterized in that, The system includes: an edge hardware layer, a multimodal data access layer, an edge computing and AI engine layer, and an integrated management and control application layer; The edge hardware layer is deployed on the plant side and is used to provide edge-side monitoring data and computing resources; The multimodal data access layer is used to enable standardized access to multimodal heterogeneous data from the edge hardware layer; The edge computing and AI engine layer is used to provide AI engines and processing models, and to call the corresponding AI engines or processing models based on the processing tasks of the integrated management and control application layer, and to provide associated data with the processing tasks based on the multimodal data access layer to obtain fusion decision results. The integrated management and control application layer is used to provide system functions to users, generate processing tasks based on the user's selection of the system functions and send them to the edge computing and AI engine layer, and obtain the fusion decision results corresponding to the processing tasks and present them to the user.

2. The system according to claim 1, characterized in that, The system also includes an edge-cloud collaboration layer and a security and operation and maintenance support layer; The edge-cloud collaboration layer is used for data synchronization and model adaptation between the edge and the cloud. The operation and maintenance support layer is used to provide data security, system security, and operation and maintenance support within the system.

3. The system according to claim 1, characterized in that, The edge hardware layer includes: edge computing gateway, industrial control computer, intelligent data acquisition terminal, sensor, camera, infrared thermal imager, and audio acquisition device.

4. The system according to claim 1, characterized in that, The multimodal data access layer includes: The structured data processing module is used to process structured data, including: real-time time-series data of DCS / SIS / PLC, power energy data, and environmental monitoring data; The unstructured data processing module is used to process unstructured data, including: inspection images, videos, infrared thermal images, audio, inspection reports, and maintenance texts; The protocol adaptation module is used to adapt and convert different industrial protocols and different communication interfaces; The data preprocessing module is used to preprocess multimodal data, including cleaning, format conversion, normalization, feature extraction, data alignment, and noise reduction and enhancement.

5. The system according to claim 1, characterized in that, The edge computing and AI engine layer includes: The edge computing management module is used to perform lightweight real-time computing, streaming data processing, local caching, rule engine, and linkage control. A multimodal AI engine, including various AI models, supports image recognition, video analysis, audio detection, text extraction, time series prediction, and anomaly diagnosis; The model management module is used for calling and managing models, including: lightweight compression, distributed deployment, online updates, and incremental learning; The fusion decision module is used for multimodal data processing, including feature fusion, evidence reasoning, confidence fusion, and joint decision-making.

6. The system according to claim 1, characterized in that, The integrated management and control application layer includes: The equipment technical supervision module is used to perform equipment condition monitoring, performance calculation, life assessment, metal monitoring, insulation monitoring, chemical monitoring, and thermal monitoring. The security intelligent monitoring module is used to perform intelligent video analysis, area intrusion detection, personnel violation identification, abnormal equipment temperature measurement, and fire hazard monitoring. The hidden danger warning and fault diagnosis module is used to perform multi-dimensional anomaly warning, fault tracing, root cause analysis, warning level classification, and coordinated response. The intelligent operation and maintenance decision-making module is used to generate maintenance strategies, optimize maintenance plans, manage spare parts, generate work orders, and track closed loops. The ledger and document management module is used to uniformly manage equipment lifecycle ledgers, technical data, test reports, and operation and maintenance records; The visualization dashboard module is used for multi-dimensional data visualization, real-time monitoring and control of the dashboard, trend analysis, and statistical reports. The automatic report generation module is used to automatically generate technical supervision reports, monthly reports, quarterly reports, and annual reports according to specifications, and supports data export and data reporting.

7. The system according to claim 2, characterized in that, The edge-cloud collaboration layer includes: Edge autonomy module is used for local real-time processing of critical tasks, as well as continuous operation in the event of network outage; The edge-cloud collaboration module is used for on-demand uploading of edge data, cloud model deployment and optimization, and remote monitoring and maintenance. The data synchronization module is used for incremental synchronization, breakpoint resumption, and data consistency assurance. The remote upgrade module is used for remote configuration, model updates, and software upgrades of edge nodes.

8. The system according to claim 2, characterized in that, The security and operation support layer includes: The data security module is used for encrypted transmission, encrypted storage, access control, and privacy desensitization of data within the system. The system security module is used to perform identity authentication, access control, firewall, intrusion detection, and log auditing within the system. The operation and maintenance support module is used to realize system monitoring, fault alarm, status self-check, remote diagnosis, and configuration management.

9. A multimodal intelligent technology-based integrated monitoring and control method deployed at the edge of a plant, characterized in that, The method, implemented based on the multimodal intelligent technology supervision and integrated control system deployed at the edge of the plant as described in any one of claims 1 to 8, includes: User tasks are obtained based on an integrated management and control layer; The edge computing and AI engine layer determines the AI ​​engine or processing model based on the user task, as well as the associated data required to complete the user task; Obtain the required associated data from the multimodal data access layer; The acquired relevant data is processed by a determined AI engine or processing model to obtain the fusion decision result; The integrated management and control layer presents the fusion decision results to the user.

10. An electronic device, characterized in that, include: At least one processor; A memory connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the multimodal intelligent technology supervision and integrated control method deployed at the plant edge as described in claim 9 by executing the instructions stored in the memory.

Images