Intelligent control and decision-making method based on intelligent laboratory data cockpit

Abstract

The invention provides an intelligent control and decision-making method based on an intelligent laboratory data cockpit. The intelligent control and decision-making method comprises the following sub-steps: S1, constructing a visual three-dimensional laboratory model based on digital twinning; s2, collection, storage, preprocessing and semantic modeling of laboratory full-dimension data; s3, deploying and training a large language model LLM adaptive to a laboratory scene, generating a decision and replying by using a natural language; on the basis of technologies such as digital twinning, a large language model and AI-agent, the efficiency and the intelligent level of laboratory management are greatly improved through an intelligent control and decision-making method; seamless butt joint of the subsystems is realized by constructing a unified data middle station; information in the laboratory can be collected in real time and concentrated to the data platform, so that the operation of the laboratory is more efficient and transparent; based on introduction of a large language model LLM and an AI-agent technology, interaction with a user is carried out, and an intelligent decision is generated according to real-time data.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an intelligent control and decision-making method based on a smart laboratory data cockpit. Background Technology

[0002] With the rapid development of IoT, big data and AI technologies, traditional laboratories are transforming into digital and intelligent "smart laboratories". Smart laboratories achieve massive collection of environmental parameters (such as temperature, humidity and pressure difference), equipment status (such as instrument on / off and running time), energy consumption, security monitoring (such as access control and video) and laboratory process data through large-scale deployment of sensors, smart devices and information systems. Currently, the management and control of smart laboratories generally rely on traditional centralized monitoring platforms (SCADA) or laboratory information management systems (LIMS). Although these systems have achieved data collection and display to a certain extent, they still have problems such as information silos and complex interactions, insufficient data value mining, unfriendly human-computer interaction, and lack of knowledge management and empowerment. In recent years, digital twin technology has made it possible to construct virtual mappings of physical entities, initially realizing three-dimensional visualization of laboratories. At the same time, large language models (LLM) and AI-agent technologies have demonstrated powerful capabilities in natural language understanding, knowledge reasoning, and human-computer dialogue. However, they cannot achieve efficient integration, comprehensive interaction, intelligent decision-making, and knowledge management across multiple domains and systems. How to utilize digital twin technology, large language models, and AI-agent technology to construct an intelligent control and decision-making method for a smart laboratory data cockpit with intelligent control, knowledge reasoning, and a global perspective is a key issue that urgently needs to be addressed in current technological development. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by proposing an intelligent control and decision-making method based on a smart laboratory data cockpit.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: The intelligent control and decision-making method based on the smart laboratory data cockpit includes the following sub-steps: S1: Construct a visualized 3D laboratory model based on digital twins; Includes the following sub-steps: S11: Collect 3D point cloud data of the laboratory and perform 3D modeling; A laser scanner was used to scan the entire laboratory area, collecting 3D point cloud data of the building structure and fixed facilities in the laboratory; the building structure includes the building exterior, floors, rooms, corridors, doors and windows, etc.; the fixed facilities include laboratory benches, fume hoods, ceilings, etc. A 3D model of the laboratory is constructed at a 1:1 scale based on the collected 3D point cloud data of the laboratory using 3D modeling software. The 3D model of the physical laboratory contains multiple 3D model objects, including rooms, corridors, etc. S12: Divide laboratory 3D model objects into attribute sets and IDs and construct a knowledge graph; Obtain the attribute set of the laboratory 3D model, the attribute set including static attributes and dynamic attributes; assign the attribute set to the corresponding laboratory 3D model object and assign an ID to each laboratory 3D model object, the ID being a unique identifier; Specifically, static attributes such as equipment name, model, manufacturer, installation location, system, and subsystem are obtained from relevant laboratory files, while dynamic attributes such as real-time data source address and control interface API address are obtained from laboratory data interface documents. Extract laboratory entities from the 3D model of the laboratory, define the logical relationships between laboratory entities using a graph database, and construct a knowledge graph of the relationships between laboratory entities. The laboratory entities include spatial entities (such as rooms, fume hoods, etc.) and equipment entities; The logical relationships between laboratory entities include spatial relationships (e.g., room 101 contains fume hood 110), system affiliation (e.g., the fume hood belongs to the equipment management system), and data associations (e.g., incubator output temperature data). S13: Build a unified data platform and connect it with the existing subsystems of the laboratory; A unified data platform is built using a distributed architecture and interfaces with various subsystems of the laboratory through multiple adapter interfaces. These subsystems include a safety management system, an environmental monitoring system, an instrument management system, a materials management system, a facility management system, and an energy consumption management system. S14: Integration of 3D visualization engine and data-driven rendering; Select a suitable 3D visualization engine based on the actual needs of the laboratory scenario. The 3D visualization engine includes Unreal Engine, Unity, etc. In the 3D visualization engine, each laboratory 3D model object is associated with a corresponding ID. Real-time data from the data platform (such as temperature, humidity, and equipment status) is pushed to the 3D visualization engine via the WebSocket protocol. The 3D visualization engine receives the push from the data platform in real time and updates the visual effect of the laboratory 3D model when the data changes, thereby realizing data-driven rendering.

[0005] S2: Collection, storage, preprocessing, and semantic modeling of full-dimensional laboratory data; Includes the following sub-steps: S21: Collect laboratory-level data and save it to a database or knowledge base; The laboratory obtains comprehensive data through methods such as direct procurement of laboratory hardware equipment, extraction from subsystems, and manual sorting and import. This comprehensive laboratory data includes environmental monitoring data, equipment operation data, video security data, business process data, and professional knowledge data. S22: Preprocess and semantically model the full-dimensional data of the laboratory; The laboratory data is preprocessed in all dimensions, including data cleaning, unit conversion, and timestamp alignment. A unified data ontology for laboratories is constructed using semantic web technology. This unified data ontology contains all laboratory entities. Preprocessed, multi-dimensional laboratory data from different sources are associated with the corresponding laboratory entities. In other words, the unified data ontology for laboratories is associated with the relevant data of all laboratory entities. S3: Deploy and train a large language model (LLM) adapted to laboratory scenarios, generate decisions and respond using natural language; Includes the following sub-steps: S31: Construct a domain knowledge base and perform preprocessing; Acquire structured and unstructured data related to the laboratory, synchronize them with equipment status information, and save them as knowledge base data to the knowledge base; The unstructured data in the knowledge base is parsed using a text parsing script to remove irrelevant content such as headers and footers; long document blocks in the unstructured data are cut into multiple document blocks, and each document block is converted into a high-dimensional vector using an embedding model such as an embedding model and stored in a vector database; instruction-answer pairs are manually constructed to simulate real user questions and saved to the instruction fine-tuning dataset. The instruction response pair includes the task objective and the operation steps; S32: Fine-tuning the general large language model LLM; This includes domain-adaptive fine-tuning and instruction fine-tuning; The preprocessed unstructured data is used as training corpus and input into the large language model LLM. The LLM is trained using an autoregressive language modeling task. After training, the LLM is fine-tuned using a manually constructed instruction fine-tuning dataset to obtain a large language model LLM that can generate relevant decisions based on user instructions. S33: The AI-agent constructs the context and sends it to the Large Language Model (LLM); Artificial intelligence agents (AI-agents) receive user commands through voice recognition or text input interfaces. The current user command is integrated with the recent dialogue history to construct a complete context. The context includes five requirement categories: knowledge question and answer, real-time data requirement, cockpit display, equipment control, and complex task. The constructed context is sent as input to the Large Language Model (LLM). The LLM generates specific function call instructions for the requirement category of the context, which call the corresponding display interface and data information. During execution, the AI ​​agent sends a request to the data platform (e.g., to obtain real-time device data), retrieves the execution results, and updates the context information based on the execution results. After receiving the execution results, the Large Language Model (LLM) generates a decision based on real data and responds using natural language.

[0006] Furthermore, the specific method for converting user input into user commands is as follows: Based on a unified laboratory data ontology, user input is converted into unified text commands through multimodal intelligent interaction; The multimodal intelligent interaction includes voice interaction mode, text interaction mode, and digital human visualization mode; the voice interaction mode includes automatic speech recognition and speech synthesis. Specifically, Automatic Speech Recognition: When a user initiates an interaction via voice, the system captures the user's audio stream through a microphone and sends the audio stream to an ASR engine deployed in the cloud or locally in real time. The ASR engine then converts the audio stream into text. The ASR engine includes iFlytek, Baidu speech, or open-source models such as Whisper; Speech synthesis: After the AI ​​agent generates a text reply, it selects a customized voice package (e.g., a clear female or male voice) according to different needs and outputs the text in speech form; Text-based interaction modality: Users input commands via keyboard using traditional graphical text input boxes; Digital Human Visualization Modality: During voice interaction, a 3D laboratory model is used to display the scene.

[0007] Preferably, before the Large Language Model (LLM) generates specific function call instructions based on the context requirement categories, it performs verification based on the logical relationships between various laboratory entities through a security rule engine. The verification includes permission verification, logic verification, and interlock verification; Furthermore, when users view any area or device in the 3D laboratory model, they can issue control commands through the AI-agent. The 3D laboratory model provides spatial positioning and visual feedback for the control operation, realizing the linkage between data display and device control.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention, based on technologies such as digital twins, large language models, and AI-agents, significantly improves the efficiency and intelligence of laboratory management through intelligent control and decision-making methods. It solves problems such as data silos, complex interactions, and low decision-making efficiency in traditional laboratory management models. By constructing a unified data platform, it achieves seamless integration of various subsystems. Environmental data, equipment status, energy consumption, and safety monitoring information in the laboratory can be collected in real time through sensors, equipment interfaces, and other channels, and centralized in the data platform, making laboratory operations more efficient and transparent. By constructing a three-dimensional virtual model of the laboratory using digital twin technology, it is possible to achieve comprehensive real-time monitoring and visualization of the laboratory environment and equipment; this not only improves the efficiency of laboratory management, but also helps researchers to intuitively view the real-time status of each area or piece of equipment in the laboratory. With the introduction of Large Language Model (LLM) and AI-agent technology, it can interact with users through natural language and generate intelligent decisions based on real-time data. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the steps of the intelligent control and decision-making method based on a smart laboratory data cockpit according to the present invention. Detailed Implementation

[0010] To provide a further understanding of the purpose, structure, features, and functions of the present invention, detailed descriptions are provided below with reference to specific embodiments.

[0011] like Figure 1 As shown, the intelligent control and decision-making method based on the smart laboratory data cockpit includes the following sub-steps: S1: Construct a visualized 3D laboratory model based on digital twins; Includes the following sub-steps: S11: Collect 3D point cloud data of the laboratory and perform 3D modeling; A laser scanner was used to scan the entire laboratory area, collecting 3D point cloud data of the building structure and fixed facilities in the laboratory; the building structure includes the building exterior, floors, rooms, corridors, doors and windows, etc.; the fixed facilities include laboratory benches, fume hoods, ceilings, etc. A 3D model of the laboratory is constructed at a 1:1 scale based on the collected 3D point cloud data of the laboratory using 3D modeling software. The 3D model of the physical laboratory contains multiple 3D model objects, including rooms, corridors, etc. S12: Divide laboratory 3D model objects into attribute sets and IDs and construct a knowledge graph; Obtain the attribute set of the laboratory 3D model, the attribute set including static attributes and dynamic attributes; assign the attribute set to the corresponding laboratory 3D model object and assign an ID to each laboratory 3D model object, the ID being a unique identifier; Specifically, static attributes such as equipment name, model, manufacturer, installation location, system, and subsystem are obtained from relevant laboratory files, while dynamic attributes such as real-time data source address and control interface API address are obtained from laboratory data interface documents. Extract laboratory entities from the 3D model of the laboratory, define the logical relationships between laboratory entities using a graph database, and construct a knowledge graph of the relationships between laboratory entities. The laboratory entities include spatial entities (such as rooms, fume hoods, etc.) and equipment entities; The logical relationships between laboratory entities include spatial relationships (e.g., room 101 contains fume hood 110), system affiliation (e.g., the fume hood belongs to the equipment management system), and data associations (e.g., incubator output temperature data). S13: Build a unified data platform and connect it with the existing subsystems of the laboratory; A unified data platform is built using a distributed architecture and interfaces with various subsystems of the laboratory through multiple adapter interfaces. These subsystems include a safety management system, an environmental monitoring system, an instrument management system, a materials management system, a facility management system, and an energy consumption management system. The security management system includes information such as real-time monitoring data, personnel access records, and security access lists; The environmental monitoring system is used to access sensor data such as temperature, humidity, pressure difference, and VOC. The instrument management system is used to access relevant information read from smart terminals by analytical instruments (such as mass spectrometers and sequencers), biosafety cabinets, incubators, refrigerators, sterilizers, etc. The material management system is used to connect to the intelligent reagent management terminal and the intelligent consumable management terminal to read relevant reagent, consumable and usage information; The facility management system is used to connect with exhaust systems, fresh air systems, air conditioning systems, waste disposal, wastewater treatment, pure water treatment, etc., and read relevant equipment information; The energy management system is used to connect with the electricity and water energy systems to read water and electricity consumption and distribution information. S14: Integration of 3D visualization engine and data-driven rendering; Select a suitable 3D visualization engine based on the actual needs of the laboratory scenario. The 3D visualization engine includes Unreal Engine, Unity, etc. In the 3D visualization engine, each laboratory 3D model object is associated with a corresponding ID. Real-time data from the data platform (such as temperature, humidity, and equipment status) is pushed to the 3D visualization engine via the WebSocket protocol. The 3D visualization engine receives the push from the data platform in real time and updates the visual effect of the laboratory 3D model when the data changes, thereby realizing data-driven rendering.

[0012] S2: Collection, storage, preprocessing, and semantic modeling of full-dimensional laboratory data; Includes the following sub-steps: S21: Collect laboratory-level data and save it to a database or knowledge base; The laboratory obtains comprehensive data through methods such as direct procurement of laboratory hardware equipment, extraction from subsystems, and manual sorting and import. This comprehensive laboratory data includes environmental monitoring data, equipment operation data, video security data, business process data, and professional knowledge data. The collected environmental monitoring data, equipment operation data, video security data, and business process data are saved to the database, and the collected professional knowledge data is saved to the knowledge base. Specifically, environmental monitoring data is collected periodically using sensors such as temperature and humidity sensors and differential pressure sensors via protocols such as Modbus, LoRaWAN, and ZigBee. This includes data on temperature, humidity, differential pressure, harmful gases, and particulate matter. Data is exchanged using Modbus (RTU / TCP), LoRaWAN, and ZigBee and stored in a database. Equipment operation data: Connect to analytical instruments or experimental equipment such as GC-MS, HPLC, centrifuges, and incubators via a dedicated serial port protocol to collect equipment operation data, including operating status (such as running, stopped, fault) and equipment parameters (such as speed, temperature), and save the equipment operation data to the database in a structured manner; Video security data: Video security data is collected in real time through network cameras (IPC), access controllers, and intrusion alarm sensors, and transmitted and saved to the database through proprietary protocols such as RTSP / RTMP and SDK; Business process data: Business process data, including experimental records, workflows, sample tracking, etc., are collected from the Laboratory Information Management System (LIMS) and electronic lab logbooks and saved to the database; Professional knowledge data: including equipment manuals, SOP documents, safety manuals, etc., are stored in the knowledge base; S22: Preprocess and semantically model the full-dimensional data of the laboratory; The laboratory data is preprocessed in all dimensions, including data cleaning, unit conversion, and timestamp alignment. Specifically, techniques such as interpolation, data smoothing, or filtering are used to clean the laboratory's full-dimensional data, removing outliers and filling in missing values; the laboratory's full-dimensional data is uniformly converted to international standard units; and a uniform timestamp is added to all data to achieve timestamp alignment. A unified data ontology for laboratories is constructed using semantic web technology. This unified data ontology contains all laboratory entities. Preprocessed, multi-dimensional laboratory data from different sources are associated with the corresponding laboratory entities. In other words, the unified data ontology for laboratories is associated with the relevant data of all laboratory entities. S3: Deploy and train a large language model (LLM) adapted to laboratory scenarios, generate decisions and respond using natural language; Includes the following sub-steps: S31: Construct a domain knowledge base and perform preprocessing; Acquire structured and unstructured data related to the laboratory, synchronize them with equipment status information, and save them as knowledge base data to the knowledge base; Specifically, structured data related to the laboratory, such as equipment name and laboratory room ID, is obtained from the equipment metadata database, room layout table, and equipment-room affiliation table. Acquire professional knowledge and historical experience from relevant documents such as equipment manuals (PDF / DOC), standard operating procedures (SOP), safety manuals (MSDS), and academic papers. This professional knowledge and historical experience includes power-on / power-off procedures, parameter ranges, fault code explanations, emergency handling of hazardous materials, and equipment reservation / personnel access rules in laboratory regulations. Device status information, such as temperature, humidity, and device operating status, is periodically retrieved from the data platform via API interface. The unstructured data in the knowledge base is parsed using a text parsing script to remove irrelevant content such as headers and footers; long document blocks in the unstructured data are cut into multiple document blocks, and each document block is converted into a high-dimensional vector using an embedding model such as an embedding model and stored in a vector database; instruction-answer pairs are manually constructed to simulate real user questions and saved to the instruction fine-tuning dataset. The instruction response pair includes the task objective and the operation steps; S32: Fine-tuning the general large language model LLM; This includes domain-adaptive fine-tuning and instruction fine-tuning; The preprocessed unstructured data is used as training corpus and input into the large language model LLM. The LLM is trained using an autoregressive language modeling task. After training, the LLM is fine-tuned using a manually constructed instruction fine-tuning dataset to obtain a large language model LLM that can generate relevant decisions based on user instructions. S33: The AI-agent constructs the context and sends it to the Large Language Model (LLM); Artificial intelligence agents (AI-agents) receive user commands through voice recognition or text input interfaces. For example, users can say "Check the current status of device A" by voice or "Please tell me the operating status of device B" by text input; The current user command is integrated with the recent dialogue history to construct a complete context. The context includes five requirement categories: knowledge question and answer, real-time data requirement, cockpit display, equipment control, and complex task. The constructed context is sent as input to the Large Language Model (LLM). The LLM generates specific function call instructions for the requirement category of the context, which call the corresponding display interface and data information. Specifically, for knowledge-based question-and-answer questions, if the question only involves general or domain-specific knowledge, the Large Language Model (LLM) locates the target device and corresponding data source based on the laboratory's unified data ontology and generates the answer. For data types requiring real-time data, the Large Language Model (LLM) identifies the necessary real-time data (such as the current status of the device), calls a predefined function to obtain the latest real-time data based on the laboratory's unified data ontology, and inserts the real-time data into the response. For cockpit display type, if the task requires displaying a specific module (such as a laboratory equipment panel), the large language model LLM calls the action execution function to display the specific module based on the laboratory unified data ontology; For equipment control issues, if the problem involves equipment control (e.g., switching equipment on and off, adjusting temperature), the Large Language Model (LLM) calls the control function based on the laboratory unified data ontology to determine the control interface of the facility management system and execute the task. For complex tasks, especially multi-step tasks, the Large Language Model (LLM) breaks down instructions into multiple subtasks and executes them by calling multiple functions sequentially. During execution, the AI ​​agent sends a request to the data platform (e.g., to obtain real-time device data), retrieves the execution results, and updates the context information based on the results. After receiving the execution results, the Large Language Model (LLM) generates the final natural language response based on the real data, such as "Device A is currently running, in good condition, and the temperature is 22°C". The AI ​​agent then feeds the natural language response back to the user and continues to follow up based on the user's needs.

[0013] Furthermore, the specific method for converting user input into user commands is as follows: Based on a unified laboratory data ontology, user input is converted into unified text commands through multimodal intelligent interaction; The multimodal intelligent interaction includes voice interaction mode, text interaction mode, and digital human visualization mode; the voice interaction mode includes automatic speech recognition and speech synthesis. Specifically, Automatic Speech Recognition: When a user initiates an interaction via voice, the system captures the user's audio stream through a microphone and sends the audio stream to an ASR engine deployed in the cloud or locally in real time. The ASR engine then converts the audio stream into text. The ASR engine includes iFlytek, Baidu speech, or open-source models such as Whisper; Speech synthesis: After the AI ​​agent generates a text reply, it selects a customized voice package (e.g., a clear female or male voice) according to different needs and outputs the text in speech form; Text-based interaction modality: Users input commands via keyboard using traditional graphical text input boxes; Digital Human Visualization Modality: During voice interaction, a 3D laboratory model is used to display the scene.

[0014] This method supports voice, text, and digital human visualization interaction, enabling users to operate the system more conveniently. Whether interacting through voice control, text input, or digital humans, it ensures seamless integration and efficient communication between users and the system. For example, voice recognition technology allows users to issue commands via voice without manual operation, and the system adjusts the laboratory environment or executes equipment control in real time based on the commands, greatly improving the convenience and flexibility of operation.

[0015] Preferably, before generating specific function call instructions based on the contextual requirement categories, the Large Language Model (LLM) verifies the logical relationships between laboratory entities. The verification includes permission verification, logic verification, and interlock verification; Specifically, permission verification: Before executing the command, check whether the currently logged-in user has permission to control the target device; Logical verification: Check whether the instructions conform to the logical specifications; for example: "Cannot remotely open devices marked as faulty", "Cannot adjust the brightness of the safety cabinet lights to 150%"; Interlock verification: Check whether the command violates the system-level interlock (e.g., when a certain type of experimental equipment is turned on, the room ventilation system must be fully open). If the verification fails, the execution is interrupted and the error reason is reported to the Large Language Model (LLM). The LLM then organizes the language to inform the user (e.g., "Sorry, this device is currently in a fault state and cannot be started remotely. Maintenance personnel have been notified.").

[0016] Furthermore, when users view any area or device in the 3D laboratory model, they can issue control commands through the AI-agent. The 3D laboratory model provides spatial positioning and visual feedback for the control operation, realizing the linkage between data display and device control.

[0017] Through an integrated security rule engine, the system can perform permission and operation verification based on the logical relationships between laboratory entities, ensuring that only authorized personnel can perform relevant control operations. The introduction of logical verification and interlock verification avoids possible erroneous operations or security risks in the system, ensuring the operational safety of the laboratory.

[0018] This invention not only enables real-time control of laboratory equipment but also effectively manages and transfers knowledge. By constructing a laboratory knowledge graph and a domain knowledge base, it can store various information within the laboratory in a structured and semantic manner, providing support for subsequent knowledge reasoning and decision-making. Furthermore, laboratory personnel can quickly access equipment manuals, operating guides, and other documents through an intelligent assistant, improving knowledge acquisition efficiency.

[0019] By combining artificial intelligence and data-driven control strategies, equipment in the laboratory can be intelligently controlled in multiple dimensions. For example, the on / off status and running time of the equipment can be monitored and managed in real time by the intelligent system. The system can automatically adjust the equipment's operating mode based on the real-time collected data, optimize energy consumption, or ensure the normal operation of the equipment. In addition, the AI-agent can automatically arrange and execute tasks according to the actual needs of the laboratory, reduce human intervention, and improve the automation level of laboratory work.

[0020] The present invention has been described in the above-described embodiments; however, these embodiments are merely examples for implementing the present invention. It must be noted that the disclosed embodiments do not limit the scope of the present invention. Conversely, any modifications and refinements made without departing from the spirit and scope of the present invention are within the scope of patent protection of the present invention.

Claims

1. An intelligent control and decision-making method based on a smart laboratory data cockpit, characterized in that: Includes the following steps: S1: Construct a visualized 3D laboratory model based on digital twins; S11: Collect 3D point cloud data of the laboratory and perform 3D modeling; S12: Divide laboratory 3D model objects into attribute sets and IDs and construct a knowledge graph; S13: Build a unified data platform and connect it with the existing subsystems of the laboratory; S14: Integration of 3D visualization engine and data-driven rendering; S2: Collection, storage, preprocessing, and semantic modeling of full-dimensional laboratory data; S21: Collect laboratory-level data and save it to a database or knowledge base; S22: Preprocess and semantically model the full-dimensional data of the laboratory; S3: Deploy and train a large language model (LLM) adapted to laboratory scenarios, generate decisions and respond using natural language; S31: Construct a domain knowledge base and perform preprocessing; S32: Fine-tuning the general large language model LLM; S33: The AI-agent constructs the context and sends it to the large language model (LLM).

2. The intelligent control and decision-making method based on a smart laboratory data cockpit as described in claim 1, characterized in that: The specific details of step S1 are as follows: S11: Collect 3D point cloud data of the laboratory and perform 3D modeling; The entire laboratory area was scanned using a laser scanner to collect 3D point cloud data of the building structure and fixed facilities in the laboratory; a 3D modeling software was used to construct a 3D model of the laboratory based on the collected 3D point cloud data at a 1:1 scale; the 3D model of the physical laboratory contains multiple 3D model objects of the laboratory. S12: Divide laboratory 3D model objects into attribute sets and IDs and construct a knowledge graph; Obtain the attribute set of the laboratory 3D model, the attribute set including static attributes and dynamic attributes; Assign attribute sets to the corresponding laboratory 3D model objects and assign an ID to each laboratory 3D model object, where the ID is a unique identifier; Extract laboratory entities from the 3D model of the laboratory, define the logical relationships between laboratory entities using a graph database, and construct a knowledge graph of the relationships between laboratory entities. The logical relationships between the laboratory entities include spatial relationships, system affiliation, and data association; S13: Build a unified data platform and connect it with the existing subsystems of the laboratory; A unified data platform is built using a distributed architecture and interfaces with various subsystems of the laboratory through multiple adapter interfaces. S14: Integration of 3D visualization engine and data-driven rendering; Select a suitable 3D visualization engine based on the actual needs of the laboratory scenario. In the 3D visualization engine, associate each laboratory 3D model object with a corresponding ID. Push real-time data from the data platform to the 3D visualization engine via the WebSocket protocol. The 3D visualization engine receives the push from the data platform in real time and updates the visual effect of the laboratory 3D model when the data changes, thus realizing data-driven rendering.

3. The intelligent control and decision-making method based on a smart laboratory data cockpit as described in claim 1, characterized in that: The specific details of step S2 are as follows: S21: Collect laboratory-level data and save it to a database or knowledge base; Acquire comprehensive laboratory data, including environmental monitoring data, equipment operation data, video security data, business process data, and professional knowledge data; The collected environmental monitoring data, equipment operation data, video security data, and business process data are saved to the database, and the collected professional knowledge data is saved to the knowledge base. S22: Preprocess and semantically model the full-dimensional data of the laboratory; The laboratory's full-dimensional data is preprocessed, and a unified laboratory data ontology is constructed using semantic web technology. This unified laboratory data ontology contains all laboratory entities. The preprocessed full-dimensional laboratory data from different sources is then associated with the corresponding laboratory entities, meaning that the unified laboratory data ontology is associated with the relevant data of all laboratory entities.

4. The intelligent control and decision-making method based on a smart laboratory data cockpit as described in claim 1, characterized in that: The specific details of step S3 are as follows: S31: Construct a domain knowledge base and perform preprocessing; Acquire structured and unstructured data related to the laboratory, synchronize them with equipment status information, and save them as knowledge base data to the knowledge base; Unstructured data in the knowledge base is parsed using a text parsing script to remove irrelevant content; long document blocks in the unstructured data are cut into multiple document blocks, each document block is converted into a high-dimensional vector and stored in a vector database; instruction-response pairs are manually constructed to simulate real user questions and are saved to an instruction fine-tuning dataset; the instruction-response pairs contain task objectives and operation steps. S32: Fine-tuning the general large language model LLM; This includes domain-adaptive fine-tuning and instruction fine-tuning; The preprocessed unstructured data is used as training corpus and input into the large language model LLM. The LLM is trained using an autoregressive language modeling task. After training, the LLM is fine-tuned using a manually constructed instruction fine-tuning dataset to obtain a large language model LLM that can generate relevant decisions based on user instructions. S33: The AI-agent constructs the context and sends it to the Large Language Model (LLM); Artificial intelligence agents (AI-agents) receive user commands through voice recognition or text input interfaces. The current user command is integrated with the recent dialogue history to construct a complete context. The constructed context is then sent as input to the Large Language Model (LLM). The LLM generates specific function call commands based on the context requirement category, which then call the corresponding display interface and data information. During execution, the AI ​​agent sends a request to the data platform to obtain the execution results and updates the context information based on the results. After receiving the execution results, the Large Language Model (LLM) generates a decision based on real data and responds using natural language. The AI ​​agent then feeds back the natural language response to the user and continues to follow up based on the user's needs.

5. The intelligent control and decision-making method based on a smart laboratory data cockpit as described in claim 1, characterized in that: The specific method for converting user input into user commands is as follows: Based on a unified laboratory data ontology, user input is converted into unified text commands through multimodal intelligent interaction; the multimodal intelligent interaction includes voice interaction modality, text interaction modality, and digital human visualization modality; the voice interaction modality includes automatic speech recognition and speech synthesis.

6. The intelligent control and decision-making method based on a smart laboratory data cockpit as described in claim 1, characterized in that: Before generating specific function call instructions based on the contextual requirement categories, the Large Language Model (LLM) verifies the logical relationships between laboratory entities. The verification includes permission verification, logic verification, and interlock verification; If the verification fails, the execution is interrupted and the error reason is reported to the Large Language Model (LLM). The Large Language Model (LLM) then organizes the language to inform the user. When users view any area or device in the 3D laboratory model, they can issue control commands through the AI-agent. The 3D laboratory model provides spatial positioning and visual feedback for the control operation, realizing the linkage between data display and device control.

Images