A large model driven embodied intelligent crystal growth system and an implementation method thereof

Abstract

The application relates to the technical field of artificial intelligence, in particular to a large model driven embodied intelligent crystal growth system and an implementation method thereof. The system comprises a plurality of core function modules based on a large model, namely, an experiment scheme formulation module, a long-term memory and case management module, an instruction mapping module, an instruction execution control module, a crystal growth state sensing module, a multi-modal information and data fusion module, and a quality evaluation and decision module. The experiment scheme formulation and decision part relies on the powerful natural language understanding ability, knowledge reasoning and analysis ability of a large language multi-modal model, realizes fusion understanding of multi-source heterogeneous data in the crystal growth process, realizes real-time monitoring and prediction of the growth state, realizes autonomous optimization and adjustment of experimental parameters, and finally realizes closed-loop intelligent control of the whole crystal growth process by converting the decision result into a control signal of the crystal growth system with the aid of an embodied intelligent system.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a large model-driven embodied intelligent crystal growth system and its implementation method. Background Technology

[0002] Traditional crystal growth experiments heavily rely on human experience and manual operation. From parameter design and equipment operation to growth process monitoring and result analysis, full participation is required, resulting in low efficiency, poor consistency, and weak repeatability, thus limiting the development speed and quality stability of crystal materials. Existing methods primarily focus on hardware control and process execution, but still require human intervention in experimental design, operation sequence planning, and real-time decision-making and response, lacking effective adaptability to complex and changing experimental environments. Furthermore, the systems lack intelligence and struggle to dynamically adjust based on real-time growth status. Summary of the Invention

[0003] This application provides a large-model-driven embodied intelligent crystal growth system and its implementation method, which enables intelligent perception, autonomous decision-making, and precise execution throughout the crystal growth process. This solves the problems of high dependence on manual labor and extensive process control in traditional crystal growth systems and their implementation methods, thereby improving the quality of crystal growth and R&D efficiency.

[0004] To address the aforementioned technical problems, in a first aspect, embodiments of this application provide a large-model-driven embodied intelligent crystal growth system, comprising seven core functional modules based on large-model-driven operation: an experimental scheme formulation module, a long-term memory and case management module, an instruction mapping module, an instruction execution control module, a crystal growth state perception module, a multimodal information and data fusion module, and a quality assessment and decision-making module. The experimental scheme formulation module is used to transform high-level abstract instructions from the user into executable detailed experimental schemes; the long-term memory and case management module is used to store, index, and manage all historical experimental data, operation records, and successful and failed cases; the instruction mapping module is used to process human-readable natural language experimental data generated by the experimental scheme formulation module. The solution precisely maps to a sequence of control commands that the underlying physical execution devices can recognize and execute. The command execution control module receives and executes standardized control commands issued by the command mapping module. The crystal growth state perception module collects multimodal state data in real time during the crystal growth process, performs preliminary preprocessing, and transmits the real-time multi-source heterogeneous data to the multimodal information and data fusion module via API interface, providing raw, high-quality perception information for state assessment. The multimodal information and data fusion module performs fusion and deep understanding based on the real-time multi-source heterogeneous data uploaded by the crystal growth state perception module, providing comprehensive and accurate situational awareness for final decision-making. The quality assessment and decision-making module makes a final judgment on the growth process and makes autonomous decisions.

[0005] In some exemplary embodiments, the experimental scheme formulation module and the quality assessment and decision-making module rely on the powerful natural language understanding, knowledge reasoning and analysis capabilities of the large language multimodal model to achieve the fusion understanding of multi-source heterogeneous data in the crystal growth process, real-time monitoring and prediction of the growth status, and autonomous optimization and adjustment of experimental parameters.

[0006] In some exemplary embodiments, the long-term memory and case management module includes a vector database and a relational database. It converts unstructured experimental plans, process images, and parameter data into vectors for storage through an embedded model. When the experimental plan formulation module needs to refer to historical experience, the long-term memory and case management module uses a large-model-based retrieval enhancement generation technology to quickly retrieve and provide the most relevant historical cases, experimental parameter ranges, and key precautions for the current task. This provides data support for experimental plan formulation and decision optimization, ensuring the continuous accumulation and evolution of system capabilities.

[0007] In some exemplary embodiments, the experimental scheme formulation module has a built-in large language model of specific prompt words; after receiving the user's input instructions, the experimental scheme formulation module actively calls the long-term memory and case management module to retrieve relevant historical cases and data, and then combines its built-in knowledge of crystal growth to perform comprehensive reasoning and planning, and finally generate a structured scheme.

[0008] In some exemplary embodiments, the large language model implements a specific function through a large language model of a specific prompt word project; the large language model of the specific prompt word project includes one or more of the Deepseek, ChatGPT, Qwen, GLM, Claude, and Gemini series language models.

[0009] In some exemplary embodiments, the experimental scheme formulation module uses prompt word engineering to wrap the relevant data to be executed with tags; the instruction mapping module extracts the data wrapped in the tags through embedded code templates and maps it into standardized instructions that can be executed by the crystal growth equipment, ensuring that high-level strategies can be accurately translated into low-level actions.

[0010] In some exemplary embodiments, the instruction execution control module communicates directly with the hardware device required to implement atomic operations through the device driver interface; the instruction execution control module controls the device to act according to the instruction sequence, monitors the execution status of the instructions in real time and feeds back the execution status to the upstream decision module in real time, and makes dynamic adjustments or safety interruptions when execution fails, forming a closed-loop control at the execution level.

[0011] In some exemplary embodiments, the multimodal information and data fusion module is implemented using a multimodal large model, which can simultaneously process image and numerical data, combine the current crystal growth image with temperature data, comprehensively analyze and determine the current growth stage, identify potential defects and predict future growth trends; the multimodal information and data fusion module generates a structured, semantically rich state description text after fusing multimodal data, providing comprehensive and accurate situational awareness for final decision-making.

[0012] In some exemplary embodiments, the quality assessment and decision-making module receives a fused status report from the multimodal information and data fusion module and compares it with the expected target of the original scheme generated by the experimental scheme formulation module. The quality assessment and decision-making module has a built-in large language model with reinforcement learning thinking, which can make judgments based on preset rules and learned experience to assess whether the current crystal growth quality meets the standards. If the growth is normal, it decides to continue the current process. If an anomaly is detected, it generates an adjustment strategy in real time. If the experiment is determined to be a failure, it automatically archives all data, parameters and reasons for failure of this experiment to the long-term memory and case management module and notifies the researchers, thereby completing a complete closed loop from perception to decision-making to memory.

[0013] Secondly, this application also provides a method for implementing a large model-driven embodied intelligent crystal growth system. This method, based on the large model-driven embodied intelligent crystal growth system described in the above embodiments, achieves closed-loop intelligent control of the entire crystal growth process. The method includes the following steps: The user inputs experimental requirements; the experimental scheme formulation module loads a large language model, parses the user requirements using a preset domain prompt word template, and calls the API of the long-term memory module; the long-term memory module converts the user's experimental requirements and experimental background into vectors using the text-embedding-ada-002 model, calculates cosine similarity, matches them with historical case vectors in the database, and returns the top N cases with the highest similarity and related crystal growth records; the large language model combines the user's input requirements, the retrieval results from the vector database, and its own knowledge to generate a complete experimental scheme, including solution concentration, crystal growth setting temperature, and crystal quality judgment criteria; after the experimental scheme is input into the instruction mapping module, the system uses a built-in Python template code library for parameter extraction and structuring; after extraction, the instruction mapping module fills the extracted parameters into the corresponding device control template and encapsulates them into a unified JSON structured data format; subsequently, the JSON data is sent to the instruction execution control module via MCP. The control module provides an API interface; after receiving JSON from the API, the instruction execution control module parses it and automatically controls the underlying devices to carry out experiments in sequence; after the experiment starts, the crystal growth state sensing module automatically calls the underlying device API according to a preset acquisition cycle to collect data including: macroscopic images, microscopic images, crystal length and width, and real-time temperature of the temperature control device; Python code is used to preprocess the collected data, scaling the image data to a fixed resolution of 500×500, converting temperature values ​​to standardized units, and placing them into a natural language description in a unified template format; the multimodal information and data fusion module uses image data and numerical data... The system uses three types of information—data, instruction prompts, and data—to predict the current growth stage, crystal quality, and growth rate. The quality assessment and decision-making module loads a large language model with reinforcement learning capabilities and compares it with the initial target parameters of the experimental plan. If the deviation is within acceptable limits, the original plan continues to execute; if the deviation exceeds the limit, an adjustment strategy is generated; if a serious defect is detected, the experiment is deemed a failure and terminated. The decision result calls the instruction mapping module to form new underlying instructions, achieving a closed loop of perception-decision-execution. After the experiment, the current experimental record is stored in the database of the long-term memory and case management module, and the generated experimental report is converted into a vector and stored in the vector database.

[0014] The technical solution provided in this application has at least the following advantages: This application provides a large-model-driven embodied intelligent crystal growth system and its implementation method. The system includes seven core functional modules based on a large-model-driven approach: an experimental scheme formulation module, a long-term memory and case management module, an instruction mapping module, an instruction execution control module, a crystal growth state perception module, a multimodal information and data fusion module, and a quality assessment and decision-making module. The experimental scheme formulation module transforms high-level abstract instructions from the user into executable, detailed experimental schemes. The long-term memory and case management module stores, indexes, and manages all historical experimental data, operation records, and successful and failed cases. The instruction mapping module maps the human-readable natural language experimental schemes generated by the experimental scheme formulation module. The system is precisely mapped to a sequence of control commands that the underlying physical execution devices can recognize and execute. The command execution control module receives and executes standardized control commands issued by the command mapping module. The crystal growth state perception module collects multimodal state data in real time during the crystal growth process and performs preliminary preprocessing. It then transmits the real-time multi-source heterogeneous data to the multimodal information and data fusion module via API interface, providing raw, high-quality perception information for state assessment. The multimodal information and data fusion module performs fusion and deep understanding based on the real-time multi-source heterogeneous data uploaded by the crystal growth state perception module, providing comprehensive and accurate situational awareness for final decision-making. The quality assessment and decision-making module makes a final judgment on the growth process and makes autonomous decisions. The large-scale model-driven embodied intelligent crystal growth system provided in this application relies on the powerful natural language understanding, knowledge reasoning, and analysis capabilities of the large-scale multimodal model for its experimental scheme formulation module and quality assessment and decision-making module. This enables the fusion and understanding of multi-source heterogeneous data during the crystal growth process, real-time monitoring and prediction of the growth status, autonomous optimization and adjustment of experimental parameters, and the transformation of decision results into control signals for the crystal growth system through the embodied intelligent system, ultimately achieving closed-loop intelligent control of the entire crystal growth process. Attached Figure Description

[0015] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments, and unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0016] Figure 1 This is a schematic diagram of a large-model-driven embodied smart crystal growth system provided in one embodiment of this application.

[0017] Figure 2 This is a flowchart illustrating a method for implementing a large-model-driven embodied intelligent crystal growth system according to an embodiment of this application. Detailed Implementation

[0018] As can be seen from the background technology, traditional crystal growth experiments suffer from problems such as low efficiency, poor consistency, and weak repeatability, which limit the development speed and quality stability of crystal materials.

[0019] The proposed technology integrates machine learning algorithms to predict solvent evaporation rates in real time, building upon existing flux-controlled crystallization (FRC) systems, resulting in the development of the FRC 2.0 system. This system reduces the standard deviation of crystal linear growth rate fluctuations by more than three times, significantly improving the reproducibility of perovskite single-crystal synthesis and highlighting the crucial role of precise control of the growth environment in optimizing crystal quality. While this research improves the control accuracy of the FRC system by introducing machine learning, the paper explicitly points out that the system still lacks a dynamic humidity controller, and its comprehensive control capability over the crystal growth environment is not yet perfect. Furthermore, this method is currently only applied to the MAPbBr3 perovskite system, and its universality and effectiveness in different material systems still need to be verified. Although this technology proposes using machine learning methods to predict solvent evaporation rates in real time to improve the control accuracy of crystal growth rates, this method relies on historical data calibration, has limited generalization ability for unseen situations, and its control effect is prone to degradation when sudden anomalies occur or when migrating across scenarios. Moreover, the system only optimizes a single control loop and does not construct a multi-task intelligent decision-making system covering the entire crystal growth process.

[0020] In view of the shortcomings of the prior art, the purpose of this application is: (1) Construct an embodied intelligent system driven by a large model that covers the entire crystal growth process. This system integrates multiple links such as knowledge construction, task planning, experimental scheme generation and automated execution. It can autonomously complete the entire process from knowledge extraction to experimental implementation according to user instructions, significantly reducing the dependence on human experience and improving the automation level and repeatability of crystal material research.

[0021] (2) Utilizing the powerful multimodal perception, analysis, reasoning, and autonomous decision-making capabilities of large language models, the system achieves intelligent closed-loop control of the entire crystal growth process. This system can integrate and analyze real-time monitoring data from multiple heterogeneous sources (such as images and temperatures) to accurately diagnose and predict the growth status, dynamically generate and execute optimized process parameter adjustment strategies, thereby significantly reducing the random errors and subjective dependence of traditional manual operations and improving the success rate, consistency, and quality level of crystal growth.

[0022] (3) Construct an embodied intelligent execution system with high autonomy and self-evolution capabilities, which will convert the decision instructions of the large model into control signals of the underlying physical devices in real time and accurately, and drive the execution devices to complete the corresponding experimental operations. This will enable the system to cope with sudden anomalies and dynamic changes in the crystal growth process, achieve robust migration and continuous optimization across scenarios, and promote the evolution of crystal preparation towards intelligence and unmanned operation.

[0023] The embodiments of this application will now be described in detail with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the embodiments of this application to facilitate a better understanding of the application. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.

[0024] See Figure 1 This application provides a large-model-driven embodied intelligent crystal growth system, comprising seven core functional modules based on large-model driving: an experimental scheme formulation module, a long-term memory and case management module, an instruction mapping module, an instruction execution control module, a crystal growth state perception module, a multimodal information and data fusion module, and a quality assessment and decision-making module. The experimental scheme formulation module transforms high-level abstract instructions from the user into executable, detailed experimental schemes. The long-term memory and case management module stores, indexes, and manages all historical experimental data, operation records, and successful and failed cases. The instruction mapping module accurately maps the human-readable natural language experimental schemes generated by the experimental scheme formulation module to... The system consists of a sequence of control commands that the underlying physical execution device can recognize and execute; a command execution control module receives and executes standardized control commands issued by the command mapping module; a crystal growth state perception module collects multimodal state data in real time during the crystal growth process, performs preliminary preprocessing, and transmits real-time multi-source heterogeneous data to the multimodal information and data fusion module via API interface, providing raw, high-quality perception information for state assessment; a multimodal information and data fusion module performs fusion and deep understanding based on the real-time multi-source heterogeneous data uploaded by the crystal growth state perception module, providing comprehensive and accurate situational awareness for final decision-making; and a quality assessment and decision-making module makes a final judgment on the growth process and makes autonomous decisions.

[0025] It should be noted that the experimental design and decision-making process relies on the powerful natural language understanding, knowledge reasoning and analysis capabilities of the large language multimodal model to achieve the fusion and understanding of multi-source heterogeneous data in the crystal growth process, real-time monitoring and prediction of the growth status, autonomous optimization and adjustment of experimental parameters, and transformation of decision results into control signals for the crystal growth system with the help of the embodied intelligence system, ultimately realizing closed-loop intelligent control of the entire crystal growth process.

[0026] The large-model-driven embodied intelligent crystal growth system provided in this application is an embodied intelligent system driven by a large language model. It is used to realize intelligent perception, autonomous decision-making and precise execution throughout the entire crystal growth process, so as to solve the problems of high dependence on manual labor and extensive process control in traditional crystal growth systems and their implementation methods, thereby improving the quality of crystal growth and R&D efficiency.

[0027] Figure 1 A schematic diagram of the architecture of the embodied intelligent crystal growth system is shown. Figure 1 The diagram illustrates the overall structure of the embodied intelligent crystal growth system of this application and the data and control relationships between its modules. The system consists of seven functional modules, namely: (1) Experimental Design Module: This module serves as the starting point for the system's tasks. It understands user needs and generates complete experimental designs through a large language model. It calls upon historical data from the long-term memory and case management module to enhance the knowledge of experimental design. Specifically, this module is responsible for transforming the user's high-level abstract instructions into executable, detailed experimental designs. The core of the module is a large language model equipped with specific prompt word engineering. Large language models with specific prompt word engineering include Deepseek, ChatGPT, Qwen, GLM, Claude, Gemini, and other language models. As an example, the large language model can be Deepseek-v3, GPT-4, or Qwen3. After receiving the user's input instructions, it actively calls the long-term memory and case management module to retrieve relevant historical cases and data. Then, it combines its built-in knowledge of crystal growth to perform comprehensive reasoning and planning, ultimately generating a structured design.

[0028] (2) Long-term memory and case management module: This module stores past experimental cases, operation logs, and parameter records of the system, forming a continuously evolving experience base to provide a basis for subsequent scheme formulation and decision-making. Specifically, this module is the knowledge hub of the system, responsible for storing, indexing, and managing all historical experimental data, operation records, and successful and failed cases. Its core consists of a vector database and a relational database. Through an embedded model, unstructured experimental schemes, process images, parameter data, and other multimodal information are converted into vectors for storage. When the experimental scheme formulation module needs to refer to historical experience, this module uses a large-model-based retrieval enhancement generation technology to quickly retrieve and provide the most relevant historical cases, experimental parameter ranges, and key precautions for the current task, providing data support for experimental scheme formulation and decision optimization, and ensuring the continuous accumulation and evolution of system capabilities.

[0029] (3) Command Mapping Module: This module transforms the natural language descriptions generated by the scheme formulation module into standardized command formats that can be recognized by the underlying control system. Internally, it employs template matching and parameter extraction mechanisms to ensure that high-level decisions are correctly translated into low-level device commands. Specifically, this module is responsible for accurately mapping the human-readable natural language experimental schemes generated by the experimental scheme formulation module into a sequence of control commands that the underlying physical execution devices can recognize and execute. In the experimental scheme formulation module, relevant data to be executed is wrapped with tags (e.g., ...) through prompt word engineering. <temperature>< / temperature>This module extracts the data wrapped in the tags through embedded code templates and maps it into standardized instructions that can be executed by the crystal growth equipment, ensuring that high-level strategies can be accurately translated into low-level actions.

[0030] (4) Instruction Execution Control Module: This module is responsible for parsing the device control instructions output by the instruction mapping module and driving various embodied hardware devices (temperature control station, vacuum drive, robotic arm, camera, etc.) on the crystal growth platform to execute collaboratively. Specifically, this module is the execution end of the system, receiving and executing standardized control instructions issued by the instruction mapping module. This module communicates directly with the hardware devices required for atomic operations, such as the temperature control station, camera, zoom device, vacuum pump, and robotic arm, through the device driver interface. It strictly controls the device actions according to the instruction sequence, monitors the execution status of the instructions in real time, and feeds back the execution status to the upstream decision module in real time. If the execution fails, it will make dynamic adjustments or perform a safety interruption, forming a closed-loop control at the execution level.

[0031] (5) Crystal growth state sensing module: This module monitors the crystal growth images, temperature, and microscopic images in real time through multiple sensors and visual acquisition devices, providing raw sensing data for the system. When this module is responsible for collecting multimodal state data during the crystal growth process in real time, it actively collects macroscopic images and microscopic morphology, temperature, and other data of the current crystal growth through the underlying device control system at fixed intervals, performs preliminary preprocessing, and transmits the data to the multimodal information and data fusion module through the API interface, providing raw, high-quality sensing information for state assessment.

[0032] (6) Multimodal Information and Data Fusion Module: Utilizing a multimodal large-scale model with visual understanding and semantic reasoning capabilities, this module fuses and deeply analyzes multi-source data such as images, temperature, and time series data to form a structured semantic description of the crystal's current state. The multimodal large-scale model includes one or more of ChatGPT-4o, Gemini Pro, and Qwen-VL. Specifically, this module is responsible for fusing and deeply understanding the real-time multi-source heterogeneous data uploaded by the crystal growth state perception module. Its core is a multimodal large-scale model capable of simultaneously processing image and numerical data, combining the current crystal growth image with temperature data to comprehensively analyze and determine the current growth stage, identify potential defects, and predict future growth trends. It generates a structured, semantically rich state description text after fusing multimodal data, providing comprehensive and accurate situational awareness for final decision-making.

[0033] (7) Quality Assessment and Decision-Making Module: As the system's decision-making terminal, this module integrates multimodal data to assess the quality and trend of crystal growth, and generates parameter adjustment or strategy optimization instructions based on feedback results, forming a closed-loop control through information feedback. Specifically, this module is responsible for making the final judgment on the growth process and making autonomous decisions. It receives the fused status report from the multimodal information and data fusion module and compares it with the expected goals of the original scheme generated by the experimental scheme formulation module. This module has a built-in large language model with reinforcement learning thinking, which can make judgments based on preset rules and learned experience. It assesses whether the current crystal growth quality meets the standards. If the growth is normal, it decides to continue the current process. If an anomaly is detected (such as crystallization, excessive defects), it immediately generates an adjustment strategy. If the experiment is judged to be a failure, it automatically archives all data, parameters, and reasons for failure of this experiment to the long-term memory and case management module and notifies the researchers, thereby completing a complete closed loop from perception to decision-making to memory.

[0034] Each module achieves information exchange and process feedback through a unified data communication protocol. Specifically, the seven modules are interconnected through a large language model and multimodal model, vector database, hardware control interface, and feedback loop. The system as a whole forms a four-layer logical closed loop of "perception-decision-execution-evolution", enabling intelligent control and dynamic optimization of the entire crystal growth process.

[0035] Based on their implementation methods, the modules can be divided into three driving modes: large model-driven, rule-driven, and hybrid-driven, with each module undertaking a specific function of the system. Among them, the large model-driven module is based on LLM or VLM as the core processing task, and has powerful semantic understanding, content generation, and complex reasoning capabilities.

[0036] Rule-driven modules, which process based on predefined templates, logic, or code, are highly efficient, have strong deterministic responses, and exhibit predictable behavior.

[0037] Hybrid-driven modules combine the capabilities of large models with preset rule templates. They leverage the intelligence of large models to handle uncertainties and multimodal information while using rules to ensure the accuracy, real-time performance, and controllability of key processes.

[0038] Figure 2 A flowchart of the embodied intelligent crystal growth method is shown.

[0039] After the user inputs their experimental requirements, the large language model of the experimental scheme formulation module calls the API provided by the memory bank to retrieve relevant memory cases. After obtaining the returned results, it combines the user input with its own knowledge to generate an experimental scheme. The parameters in the scheme are mapped into JSON structured data and then passed to the underlying control module for experimental execution. During the experiment, status monitoring is activated. Multimodal data obtained through the API interface provided by the underlying execution device is passed to the visual language model of the multimodal information and data fusion module for data analysis. Based on the analysis results, the current crystal growth state is evaluated, and the decision is made to adjust the scheme or terminate the experiment. If the scheme is adjusted, the experimental scheme is updated and re-executed, achieving a feedback-execution closed loop. If the experiment is terminated, a notification is sent via email, and the experimental data is stored in the memory bank.

[0040] Specifically, the method flow for implementing the embodied intelligent crystal growth system provided in this application is as follows: Users input their experimental requirements, such as crystal type, target size, and growth method (e.g., reverse temperature crystallization), through the system's natural language interactive window.

[0041] The experimental design module loads a large language model. The model parses user needs using preset domain cue word templates and calls the API of the long-term memory module.

[0042] The long-term memory module transforms user requirements and experimental background into vectors, calculates cosine similarity, matches them with historical case vectors in the database, and returns the top N cases with the highest similarity and related crystal growth records.

[0043] The large model combines user input requirements, retrieval results from the vector database, and its own existing knowledge to generate a complete experimental plan, including solution concentration, crystal growth temperature settings, and crystal quality judgment criteria.

[0044] After the experimental design is input into the instruction mapping module, the system uses a built-in Python template library for parameter extraction and structuring. This template library contains preset label recognition rules (e.g., ...). <temperature>< / temperature> , <duration>< / duration> (etc.), which can accurately locate and extract key experimental parameters in natural language schemes.

[0045] After extraction, the instruction mapping module populates these parameters into the corresponding device control template and encapsulates them into a unified JSON structured data format (fields include device ID, instruction type, parameter value, etc.). Subsequently, the JSON data is sent to the API interface provided by the instruction execution control module via the model context protocol.

[0046] Here is a JSON example of one experimental step from the experimental instructions: { "SID": UUID, — The unique identifier for the current experiment. "Stp": 1, — Indicates the current step "Dev": "temperatureCtrl", — Indicates the currently operating device. "Cmd": "set", — This indicates the command currently used to operate the device. "Cmd1": "initialpod", -- Parameter 1 (represents temperature if it is a temperature-controlled device) "Cmd2": "cnt" — Parameter 2 "Ts": "currentTime" — Represents the current timestamp } After receiving the JSON from the API, the instruction execution control module parses it and automatically controls the underlying devices to carry out the experiments in sequence.

[0047] After the experiment begins, the crystal growth state sensing module automatically calls the underlying device API according to the preset acquisition cycle to collect data including: macroscopic images, microscopic images, crystal length and width, and real-time temperature of the temperature control device, etc.

[0048] The data is preprocessed using Python code, which scales the image data to a fixed resolution of 500×500, converts the temperature values ​​to standardized units, and places them into a natural language description in a uniform template format.

[0049] The core of the multimodal information and data fusion module is a multimodal large model, which uses three types of information—image data, numerical data, and instruction prompts—to predict and determine the current growth stage, crystal quality, and growth rate.

[0050] The quality assessment and decision-making module loads a large language model with reinforcement learning capabilities. It compares the model with the initial target parameters of the experimental design (size, transparency, structural integrity).

[0051] If the deviation is within the acceptable range, the original plan continues to be executed; if the deviation exceeds the limit, an adjustment strategy is generated; if a serious defect is detected, the experiment is judged as a failure and terminated.

[0052] The decision result calls the instruction mapping module to form new underlying instructions, realizing a closed loop of perception-decision-execution.

[0053] After the experiment is completed, the current experimental records (i.e., multimodal information such as crystal size, crystal image, real-time temperature of the temperature control station, growth time, etc. during the single crystal growth process) are stored in the database of the long-term memory and case management module, and the generated experimental report is converted into a vector and stored in the vector database.

[0054] In summary, this application provides a closed-loop control method for the entire crystal growth process based on a large language model. This application introduces a large language model as the central decision-making unit into the entire crystal growth process system, with the atomic actions of the underlying devices serving as the execution endpoints. Through a perception-decision-execution closed loop, dynamic response and strategy optimization of the growth process are achieved. Compared with traditional methods, this scheme can achieve autonomous evolution through multi-observation scale fusion, adaptive control, and experimental iteration.

[0055] Furthermore, this application proposes a crystal growth system architecture incorporating feedback control. This application designs a multi-level system architecture consisting of seven core modules: a long-term memory and case management module, an experimental scheme formulation module, an instruction mapping module, an instruction execution control module, a crystal growth state perception module, a multimodal information and data fusion module, and a quality assessment and decision-making module. These modules form an organic whole through information sharing and strategy fusion, enabling flexible responses to external disturbances and complex multimodal perception scenarios.

[0056] It should be noted that the overall system process design and working mechanism are also core innovations of this application. Through multi-module collaboration, the system transforms user instructions into complete experimental plans and performs real-time adjustments based on the growth status, achieving end-to-end intelligent crystal growth from experimental plan generation to quality assessment and strategy archiving.

[0057] The large-model-driven embodied intelligent crystal growth system and its implementation method provided in this application have been proven to be feasible through multiple experiments, simulations and real-world application cases.

[0058] Compared with existing crystal growth methods, this application demonstrates significant advantages in several aspects. These advantages, combined with the specific technical solutions of this application, enable a high degree of automation and intelligent closed-loop control of the entire crystal growth process, effectively improving the quality, efficiency, and repeatability of crystal growth.

[0059] This application constructs a complete closed-loop system covering experimental design, real-time perception, autonomous decision-making, and precise execution through embodied intelligence technology driven by a large language model. Through the collaborative work of seven core modules driven by the large language model, the system can autonomously generate experimental designs based on user instructions and dynamically adjust experimental parameters based on multimodal perception data. This effectively overcomes the problem of traditional methods' high dependence on manual operation in experimental design, process monitoring, and anomaly response, thus improving the automation level and consistency of crystal growth.

[0060] This application achieves synchronous acquisition and semantic-level fusion of multi-dimensional data such as macroscopic images, microscopic morphology, and temperature through the cooperation of a crystal growth state perception module and a multi-modal information fusion module. This enables more accurate identification of crystal defects, prediction of growth trends, and provides a comprehensive and interpretable state assessment for the decision-making module, thereby significantly improving the accuracy and robustness of control.

[0061] Based on the above technical solutions, this application provides a large-model-driven embodied intelligent crystal growth system and its implementation method. The system includes seven core functional modules based on a large-model-driven approach: an experimental scheme formulation module, a long-term memory and case management module, an instruction mapping module, an instruction execution control module, a crystal growth state perception module, a multimodal information and data fusion module, and a quality assessment and decision-making module. The experimental scheme formulation module transforms high-level abstract instructions from the user into executable detailed experimental schemes. The long-term memory and case management module stores, indexes, and manages all historical experimental data, operation records, and successful and failed cases. The instruction mapping module maps the human-readable natural language generated by the experimental scheme formulation module. The experimental scheme is precisely mapped to a sequence of control commands that the underlying physical execution devices can recognize and execute. The command execution control module receives and executes standardized control commands issued by the command mapping module. The crystal growth state perception module collects multimodal state data in real time during the crystal growth process, performs preliminary preprocessing, and transmits the real-time multi-source heterogeneous data to the multimodal information and data fusion module via API interface, providing raw, high-quality perception information for state assessment. The multimodal information and data fusion module performs fusion and deep understanding based on the real-time multi-source heterogeneous data uploaded by the crystal growth state perception module, providing comprehensive and accurate situational awareness for final decision-making. The quality assessment and decision-making module makes a final judgment on the growth process and makes autonomous decisions. The large-scale model-driven embodied intelligent crystal growth system provided in this application relies on the powerful natural language understanding, knowledge reasoning, and analysis capabilities of the large-scale multimodal model for its experimental scheme formulation module and quality assessment and decision-making module. This enables the fusion and understanding of multi-source heterogeneous data during the crystal growth process, real-time monitoring and prediction of the growth status, autonomous optimization and adjustment of experimental parameters, and the transformation of decision results into control signals for the crystal growth system through the embodied intelligent system, ultimately achieving closed-loop intelligent control of the entire crystal growth process.

[0062] Those skilled in the art will understand that the above-described embodiments are specific examples of implementing this application, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of this application. Any person skilled in the art can make their own modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application should be determined by the scope defined in the claims.

Claims

1. A large-scale model-driven embodied intelligent crystal growth system, characterized in that, It includes several core functional modules driven by a large model, namely: The system comprises modules for experimental design, long-term memory and case management, instruction mapping, instruction execution control, crystal growth state sensing, multimodal information and data fusion, and quality assessment and decision-making. The experimental scheme formulation module is used to transform the user's high-level abstract instructions into executable detailed experimental schemes; The long-term memory and case management module is used to store, index, and manage all historical experimental data, operation records, and successful and failed cases. The instruction mapping module is used to accurately map the human-readable natural language experimental plan generated by the experimental plan formulation module into a sequence of control instructions that can be recognized and executed by the underlying physical execution device. The instruction execution control module is used to receive and execute standardized control instructions issued by the instruction mapping module; The crystal growth state sensing module is used to collect multimodal state data in real time during the crystal growth process and perform preliminary preprocessing. It transmits real-time multi-source heterogeneous data to the multimodal information and data fusion module through the API interface, providing raw and high-quality sensing information for state assessment. The multimodal information and data fusion module is used to fuse and deeply understand the real-time multi-source heterogeneous data uploaded by the crystal growth state perception module, so as to provide comprehensive and accurate situational awareness for the final decision. The quality assessment and decision-making module is used to make a final evaluation of the growth process and to make autonomous decisions.

2. The large-model-driven embodied intelligent crystal growth system according to claim 1, characterized in that, The experimental scheme formulation module and the quality assessment and decision-making module rely on the powerful natural language understanding, knowledge reasoning and analysis capabilities of the large language multimodal model to achieve the fusion and understanding of multi-source heterogeneous data in the crystal growth process, real-time monitoring and prediction of growth status, and autonomous optimization and adjustment of experimental parameters.

3. The large-model-driven embodied intelligent crystal growth system according to claim 1, characterized in that, The long-term memory and case management module includes a vector database and a relational database. It uses an embedded model to convert unstructured experimental plans, process images, and parameter data into vectors for storage. When the experimental design module needs to refer to historical experience, the long-term memory and case management module uses a large-model-based retrieval enhancement generation technology to quickly retrieve and provide the most relevant historical cases, experimental parameter ranges, and key precautions for the current task. This provides data support for experimental design and decision optimization, ensuring the continuous accumulation and evolution of system capabilities.

4. The large-model-driven embodied intelligent crystal growth system according to claim 1, characterized in that, The experimental design module has a built-in large language model for specific prompt word engineering; After receiving the user's input command, the experimental scheme formulation module actively calls the long-term memory and case management module to retrieve relevant historical cases and data, and then combines its built-in knowledge of crystal growth to perform comprehensive reasoning and planning, ultimately generating a structured scheme.

5. The large-model-driven embodied intelligent crystal growth system according to claim 4, characterized in that, The large language model achieves specific functions through the large language model of the specific prompt word project; the large language model of the specific prompt word project includes one or more of the Deepseek, ChatGPT, Qwen, GLM, Claude, and Gemini series language models.

6. The large-model-driven embodied intelligent crystal growth system according to claim 1, characterized in that, The experimental scheme formulation module uses prompt word engineering to wrap the relevant data to be executed with tags; the instruction mapping module uses embedded code templates to extract the data wrapped in the tags and map it into standardized instructions that can be executed by the crystal growth equipment, ensuring that high-level strategies can be accurately translated into low-level actions.

7. The large-model-driven embodied intelligent crystal growth system according to claim 1, characterized in that, The instruction execution control module communicates directly with the hardware devices required to perform atomic operations through the device driver interface; The instruction execution control module controls the device to operate according to the instruction sequence, monitors the execution status of the instructions in real time and feeds back the execution status to the upstream decision module in real time. When the execution fails, it will make dynamic adjustments or perform a safety interruption, forming a closed-loop control at the execution level.

8. The large-model-driven embodied intelligent crystal growth system according to claim 1, characterized in that, The multimodal information and data fusion module is implemented using a multimodal large model, which can process image and numerical data simultaneously. It combines the current crystal growth image with temperature data to comprehensively analyze and determine the current growth stage, identify potential defects, and predict future growth trends. The multimodal information and data fusion module generates a structured, semantically rich state description text after fusing the multimodal data, providing comprehensive and accurate situational awareness for the final decision.

9. The large-model-driven embodied intelligent crystal growth system according to claim 1, characterized in that, The quality assessment and decision-making module receives a fusion status report from the multimodal information and data fusion module and compares it with the expected target of the original scheme generated by the experimental scheme formulation module. The quality assessment and decision-making module has a built-in large language model with reinforcement learning thinking, which can make judgments based on preset rules and learned experience to assess whether the current crystal growth quality meets the standards. If growth is normal, then we will continue with the current process; If an anomaly is detected, an adjustment strategy is generated immediately. If an experiment is determined to be a failure, all data, parameters, and reasons for failure will be automatically archived in the long-term memory and case management module, and the researchers will be notified, thus completing a closed loop from perception to decision-making to memory.

10. A method for implementing a large-model-driven embodied intelligent crystal growth system, wherein the method is based on the large-model-driven embodied intelligent crystal growth system as described in any one of claims 1 to 9 to achieve closed-loop intelligent control of the entire crystal growth process, characterized in that, The method includes the following steps: The user inputs the experimental requirements, the experimental plan formulation module loads the large language model, parses the user requirements through the preset domain prompt word template, and calls the API of the long-term memory module; The long-term memory module transforms the user's experimental requirements and background into vectors, uses cosine similarity calculation, matches them with historical case vectors in the database, and returns the top N cases with the highest similarity and related crystal growth records. The large language model combines user input requirements, retrieval results from the vector database, and its own knowledge to generate a complete experimental plan, including solution concentration, crystal growth temperature settings, and crystal quality judgment criteria. After the experimental design is input into the instruction mapping module, the system uses the built-in Python template code library to extract and structure parameters. After extraction, the instruction mapping module fills the extracted parameters into the corresponding device control template and encapsulates them into a unified JSON structured data format; then, the JSON data is sent to the API interface provided by the instruction execution control module via MCP. After receiving the JSON from the API, the instruction execution control module parses it and automatically controls the underlying devices to carry out the experiments in sequence. After the experiment begins, the crystal growth state sensing module automatically calls the underlying device API according to the preset acquisition cycle to collect data including: macroscopic images, microscopic images, crystal length and width, and real-time temperature of the temperature control device; The collected data was preprocessed using Python code, which scaled the image data to a fixed resolution of 500×500, converted the temperature values ​​to standardized units, and placed them into a natural language description in a uniform template format. The multimodal information and data fusion module uses three types of information—image data, numerical data, and instruction prompts—to predict and determine the current growth stage, crystal quality, and growth rate. The quality assessment and decision-making module loads a large language model with reinforcement learning capabilities and compares it with the initial target parameters of the experimental scheme; If the deviation is within the acceptable range, the original plan continues to be executed; if the deviation exceeds the limit, an adjustment strategy is generated; if a serious defect is detected, the experiment is judged as a failure and terminated. The decision result calls the instruction mapping module to form new underlying instructions, realizing a closed loop of perception-decision-execution; After the experiment is completed, the current experimental record is stored in the database of the long-term memory and case management module, and the generated experimental report is converted into a vector and stored in the vector database.

Images