Three-calculation fusion system based on large model and electronic equipment

Abstract

The invention relates to the technical field of intelligent computing, and provides a three-calculation integration system based on a large model and electronic equipment. Comprising a user interaction layer used for receiving multi-modal input of a user and converting the multi-modal input into a structured task request; the intention recognition layer is used for performing semantic analysis and intention understanding on the structured task request and generating structured task description; the task scheduling layer is used for generating a resource allocation scheme according to the structured task description and issuing a task execution instruction; the resource pool layer is connected with the task scheduling layer, comprises a calculation data pool, an algorithm pool and a calculation power pool, and is used for responding to the task execution instruction, providing corresponding data resources, algorithm resources and calculation power resources and executing the calculation task; and the result output module is connected with the task scheduling layer and is used for integrating and outputting task execution results. Three-calculation integration can be realized, intelligent scheduling is supported, and an adaptive optimization capability is possessed.

Description

Technical Field

[0001] This application relates to the field of intelligent computing technology, specifically to a three-in-one computing system and electronic device based on a large model. Background Technology

[0002] With the rapid development of artificial intelligence technology, application scenarios centered on large models are becoming increasingly complex and diverse, placing higher demands on the coordinated scheduling of algorithm models, computing resources, and data resources. Currently, most mainstream intelligent computing platforms adopt a fixed deployment model, isolating and managing algorithms, computing power, and data resources, lacking a unified dynamic scheduling mechanism. This model makes it difficult for the system to achieve elastic resource adaptation when facing multimodal task requests, especially in application scenarios that require simultaneous coordination of heterogeneous algorithm components, distributed computing nodes, and diverse data sources, resulting in significant response latency and low resource utilization efficiency.

[0003] In existing technical solutions, resource scheduling often adopts a single-dimensional optimization strategy, such as allocating tasks solely based on computing load or data readiness, failing to fully consider the inherent relationship between algorithm characteristics and data dependencies. This scheduling method struggles to adapt to the dynamic demands for resource coordination during large-scale model training and inference, leading to frequent instances of both idle computing resources and bottlenecks. Furthermore, traditional systems typically rely on manual configuration for algorithm selection and resource binding, which not only has a high operational threshold but also fails to achieve automated and intelligent resource adaptation based on task characteristics, severely restricting the large-scale application and sustainable development of the system.

[0004] In summary, the existing technologies have the following main shortcomings: First, they lack a joint modeling and collaborative scheduling mechanism for the three elements of algorithms, computing power, and data, resulting in low system resource utilization. Second, they fail to establish an intelligent matching path between task requirements and resource supply, making it difficult for the system to achieve dynamic optimization of resource allocation. Third, they lack an end-to-end automated scheduling process, relying excessively on manual intervention, which affects system response efficiency and scalability.

[0005] Therefore, there is an urgent need for a platform architecture that can achieve the integration of computing, computing, and data processing, support intelligent scheduling, and have adaptive optimization capabilities, in order to solve the technical problems faced by existing systems in complex application scenarios, such as uncoordinated resource scheduling, delayed response, and insufficient intelligence. Summary of the Invention

[0006] In view of this, the embodiments of this application provide a three-component integrated system and electronic device based on a large model, which can realize the integration of three computing, support intelligent scheduling and have adaptive optimization capabilities, and can solve the technical problems of uncoordinated resource scheduling, slow response and insufficient intelligence faced by existing systems in complex application scenarios.

[0007] The first aspect of this application provides a three-component integrated system based on a large model, comprising: The user interaction layer is used to receive multimodal input from users and convert the multimodal input into structured task requests; An intent recognition layer, connected to the user interaction layer, is used to perform semantic parsing and intent understanding on the structured task request, and generate a structured task description containing task type, resource requirements and constraints. The task scheduling layer, connected to the intent recognition layer, is used to generate a resource allocation scheme based on the structured task description and issue task execution instructions to the resource pool layer based on the resource allocation scheme. The resource pool layer, connected to the task scheduling layer, includes a data pool, an algorithm pool, and a computing power pool. It is used to respond to the task execution instructions, provide corresponding data resources, algorithm resources, and computing power resources, and execute computing tasks. The output results module, connected to the task scheduling layer, is used to integrate and output the task execution results.

[0008] In one embodiment, the intent recognition layer includes: The intent recognition large model module is used to perform low-level semantic understanding of the structured task request and output a preliminary intent vector; The knowledge graph auxiliary module interacts bidirectionally with the intent recognition large model module to perform knowledge reasoning and semantic completion on the preliminary intent vector in order to generate the structured task description.

[0009] In one embodiment, the intent recognition large model module includes a general large language model submodule, a fine-tuning submodule, and a first extraction submodule; The general large language model submodule is used to perform low-level semantic modeling of the structured task request; The fine-tuning submodule is connected to the general large language model submodule and is used to optimize the semantic modeling results using domain-specific data and output a preliminary intent vector. The first extraction submodule, connected to the fine-tuning submodule, is used to extract key task information from the initial intent vector.

[0010] In one embodiment, the knowledge graph auxiliary module includes a knowledge storage submodule, a second extraction submodule, and a knowledge automatic update submodule; The knowledge storage submodule is used to store domain knowledge data based on graph structures; The second extraction submodule, connected to the first extraction submodule and the knowledge storage submodule, is used to receive the key task information and perform querying, matching and semantic association in the knowledge storage submodule to semantically complete and verify the key task information. The semantic completion and verification are fed back to the extraction submodule to form the structured task description. The automatic knowledge update submodule is connected to the knowledge storage submodule and is used to periodically update the knowledge nodes in the knowledge storage submodule using differential privacy technology.

[0011] In one embodiment, the task scheduling layer includes: The task scheduling big model module, connected to the intent recognition layer, is used to receive the structured task description and parse it into a task plan containing task sequence and initial resource requirements. The deep learning module is enhanced and connected to the task scheduling model module to obtain the task plan and the real-time status information of the system, and to formulate an optimized resource allocation scheme based on the obtained information. The task execution module, connected to the reinforcement deep learning module, is used to receive the resource allocation scheme and call the interfaces of the data pool, algorithm pool and computing power pool according to the resource allocation scheme, so as to respectively mount data resources, call algorithm components and allocate computing power nodes; The monitoring and feedback module, connected to the task execution module and the resource pool layer, is used to monitor the task execution status and resource usage in the resource pool layer, generate the real-time status information, and feed the real-time status information back to the reinforcement deep learning module to optimize the scheduling strategy.

[0012] In one embodiment, the resource pool layer responds to the task execution instruction and provides resources to execute the computation task in the following manner: The data pool is used to respond to the data instructions in the task execution instructions, provide corresponding data resources according to the metadata description and permission control policy in the instructions, and perform data loading and mounting. The algorithm pool is used to respond to the algorithm-related instructions in the task execution instructions, call the corresponding algorithm components according to the algorithm encapsulation description and access control policy in the instructions, and start the algorithm execution environment; The computing power pool is used to respond to the instructions regarding computing power in the task execution instructions, allocate corresponding computing resources according to the resource scheduling strategy in the instructions, and run the algorithm components called by the algorithm pool on the computing resources to process the data mounted by the data pool in order to execute the computing task.

[0013] In one embodiment, the reinforcement deep learning module employs a near-end strategy to optimize the PPO algorithm; The state space of the enhanced deep learning module includes the CPU utilization, GPU utilization, memory usage, and task queue length of the resource pool layer. The action space of the enhanced deep learning module is a combination of decisions on the allocation of data, algorithms, and computing resources.

[0014] In one embodiment, the monitoring and feedback module is further configured to: Real-time collection of resource node performance metrics, including CPU / GPU utilization, memory usage, disk I / O, and network bandwidth; Provide real-time alerts for resource overload and / or abnormal call conditions; Record audit logs for resource calls.

[0015] In one embodiment, the task scheduling layer is further configured to feed back task execution results, resource utilization efficiency, and abnormal situation information to the intent recognition layer; The intent recognition layer is also configured to optimize the accuracy of intent recognition and the quality of task description based on the feedback information.

[0016] A second aspect of this application provides an electronic device that integrates a system as described in any embodiment of the first aspect of this application.

[0017] The system provided in the first aspect of this application receives multimodal input and converts it into structured requests through a user interaction layer, performs semantic parsing to generate task descriptions through an intent recognition layer, generates resource allocation schemes based on the descriptions and coordinates the execution of tasks through a resource pool layer, and integrates and outputs the results. The clear responsibilities of each layer facilitate system maintenance and expansion. Furthermore, the integration of the resource pool layer enables unified scheduling of data, algorithms, and computing power, improving resource utilization efficiency and task processing throughput. It achieves seamless integration of computing, algorithms, and computing power, supports intelligent scheduling, and possesses adaptive optimization capabilities, thus solving the technical problems of uncoordinated resource scheduling, delayed response, and insufficient intelligence faced by existing systems in complex application scenarios.

[0018] It is understandable that the beneficial effects of the second aspect mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the structure of a three-component integrated system based on a large model, provided in one embodiment of this application; Figure 2 This is a schematic diagram of the structure of the intent recognition layer provided in another embodiment of this application; Figure 3 This is a schematic diagram of the structure of the task scheduling layer provided in another embodiment of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0023] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0026] like Figure 1 As shown, a first aspect of this application provides a three-component integrated system based on a large model, comprising: User interaction layer 1 is used to receive multimodal input from users and convert the multimodal input into structured task requests; The intent recognition layer 2, connected to the user interaction layer, is used to perform semantic parsing and intent understanding on the structured task request, and generate a structured task description containing task type, resource requirements and constraints. Task scheduling layer 3, connected to the intent recognition layer, is used to generate a resource allocation scheme based on the structured task description, and issue task execution instructions to the resource pool layer based on the resource allocation scheme; Resource pool layer 4, connected to the task scheduling layer, includes a data pool, an algorithm pool, and a computing power pool, used to respond to the task execution instruction, provide corresponding data resources, algorithm resources, and computing power resources, and execute computing tasks; Output result module 5 is connected to the task scheduling layer and is used to integrate and output the task execution results.

[0027] In the application, the user interaction layer can receive text, speech, and image input by integrating a speech recognition module and an image parsing module. Through a standardized API gateway, it encapsulates multimodal data into a structured task request in JSON format. This request includes a task identifier and input data payload. The intent recognition layer retrieves this request via RPC and initiates a semantic parsing process. Its intent recognition large model module uses a pre-trained model based on the BERT architecture for semantic vectorization, while the knowledge graph auxiliary module extracts domain entity relationships from the Neo4j graph database for semantic completion. Finally, it generates a structured task description containing task type classification labels, resource requirement parameters, and time constraints. The user interaction layer is primarily built around multimodal input support, multi-terminal scenario adaptation, and a unified task submission interface: First, by constructing a unified multimodal perception and parsing interface, it enables users to input tasks in various forms, including natural language text, images, audio, and video. Then, it utilizes AI technologies such as speech recognition, image and video processing to transform multimodal data into structured task requests. The system employs an intuitive and user-friendly web interface and embedded system command-based input to ensure smooth interaction. It also provides real-time feedback and monitoring of task execution progress based on task status, and finally collects and outputs the final running results.

[0028] The user interaction layer can also receive text, voice, and industrial data stream inputs through an integrated multimodal perception interface. It converts heterogeneous data into structured task requests in Avro format via a RESTful API gateway. These requests include task sequence numbers and payload check codes. The intent recognition layer obtains these requests via the gRPC protocol and initiates the parsing process. Its intent recognition large model module uses a pre-trained model based on the RoBERTa architecture for semantic encoding, while the knowledge graph auxiliary module extracts entity relationships from the Apache Jena graph database for semantic expansion. Finally, it generates a structured task description that includes process type, equipment requirement parameters, and quality constraints.

[0029] The system described in this application is applicable to the generalized application needs of multimodal large models, and is used to solve problems such as fragmented resource scheduling, lagging process response, insecure data management, and high threshold for model invocation in existing systems. It achieves automatic coordination and intelligent fusion of data, algorithms, and computing power, and is task-driven. This system includes five functional layers: a user interaction layer, an intent recognition layer, a task scheduling layer, a resource pool layer, and an output result module. These layers work together to achieve intelligent processing of the entire process from user request parsing to task execution and result output. The user interaction layer receives multimodal input from users and triggers platform application processes. The intent recognition layer includes an intent recognition module, an intent parsing large model module, and a knowledge graph auxiliary module, which can perform semantic recognition, deep parsing, and knowledge reasoning on user requests, providing accurate basis for task generation. The task scheduling layer consists of a task scheduling large model module, a reinforcement deep learning module, a task execution module, and a monitoring and feedback module. It possesses the ability to decompose complex tasks, control execution, and learn feedback. The resource pool layer integrates three basic resources: a data pool, an algorithm pool, and a computing power pool. Through a resource monitoring module, it achieves real-time monitoring and unified scheduling of resource status, thereby ensuring the stability and efficiency of multi-task parallel execution. Finally, the output result module integrates and presents the task execution results, providing users with structured or visualized output content. Through this multi-layered collaborative architecture, the platform achieves intelligent fusion and on-demand access to data, algorithms, and computing power, significantly improving the response efficiency and generalization ability of the multimodal large model in complex application scenarios. To achieve the above objectives, the construction method proposed in this invention includes the following key steps: First, regarding data integration, the platform establishes unified data standards and specifications, combines knowledge graph technology to achieve the association and semantic understanding of multi-source heterogeneous data, and leverages the strong generalization capabilities of large-scale models for automatic data annotation and enhancement, while improving model adaptability through domain-specific fine-tuning. Second, regarding algorithm integration, an automated algorithm design and optimization engine is built based on a general-purpose large-scale model to achieve adaptive evolution under specific tasks. Simultaneously, an algorithm component library and microservice mechanism support cross-domain reuse and rapid deployment of algorithms. Third, regarding computing power integration, the platform constructs a computing power resource pool covering cloud computing, high-performance computing, and edge computing to achieve resource virtualization and elastic scheduling. It also utilizes large-scale models to intelligently analyze task requirements, achieving precise matching and efficient management of computing power. Finally, the platform provides a unified management interface for centralized monitoring and scheduling of data, algorithms, and computing power, and establishes a robust data security and privacy protection mechanism to ensure compliant, controllable, and traceable resource access, fully supporting the development of intelligent applications driven by large-scale models.

[0030] The security of data, algorithms, and computing power in the resource pool platform of the three-in-one computing integration is also a subject of this patent.

[0031] The data pool includes unified and standardized data encapsulation: by constructing a standardized metadata system to provide a structured description of data resources, a unified resource access interface, and an introduction of hierarchical access control strategies, the data access process is made controllable and standardized. In terms of data security, symmetric encryption algorithms are used to encrypt the storage medium, and the data transmission process is encrypted based on the TLS / HTTPS protocol to prevent data leakage and tampering during transmission. At the same time, sensitive information is preprocessed using anonymization and de-identification mechanisms to enhance data privacy protection capabilities.

[0032] The data pool also includes a federated data separation and secure collaboration mechanism: it constructs a federated learning framework to achieve collaborative model training through parameter transmission while ensuring that data does not leave the local area, thereby avoiding direct data exchange and ensuring the separation of data ownership and usage rights; this mechanism can be effectively applied to multi-party collaboration scenarios with data privacy or compliance requirements.

[0033] The data pool also includes data sandboxes and methods for building isolated operating environments: building independent data operating environments based on containerization technology or virtualization platforms to implement logical isolation of data from different sources, preventing cross-domain access and unauthorized calls; this isolation mechanism can be dynamically created and destroyed, supports fine-grained management by task, and effectively reduces data cross-contamination and system security risks.

[0034] Algorithm security is provided externally in a standardized service format, using a unified RESTful or RPC interface to prevent algorithm code leakage, and introducing container security scanning tools to identify potential vulnerabilities. Simultaneously, a role-based access control mechanism is established to ensure that sensitive algorithms can only be invoked in controlled environments, effectively improving the security and controllability of algorithm resources.

[0035] In terms of computing resource management, the research focuses on building a computing resource pool that supports virtualization and elastic scaling. Through a centralized scheduling and monitoring platform, the computing power allocation of different computing tasks is managed in a unified manner, and the principle of security isolation is strictly implemented in the scheduling strategy to ensure the stability and security of operation in a multi-task concurrent environment.

[0036] The overall architecture consists of two core large models. The intent recognition model receives input requests from users or business systems; automatically understands task requirements and identifies task types; extracts the data, algorithms, and computing power requirements of the task, forming a clear description of the task intent. The task scheduling model, based on the intent recognition results, intelligently allocates resources to tasks; automatically combines data resources, algorithm components, and computing power nodes; and achieves automation, efficiency, and dynamic optimization of the task execution process. The two large models work together to form a closed-loop process of accurate intent recognition, intelligent scheduling of the three computing components, efficient resource allocation, and intelligent task execution.

[0037] The two models exchange information through explicit task descriptions, forming upstream and downstream workflows. From the intent recognition model to the three-computation scheduling model: The intent recognition model parses user task requests and outputs a structured, standardized task description. The three-computation scheduling model then uses this task description for resource allocation and task execution planning. From the three-computation scheduling model to the intent recognition model: After task execution, the scheduling model feeds back the task execution results, resource utilization efficiency, and any anomalies to the intent recognition model, serving as a basis for improving the accuracy of intent recognition and the quality of task descriptions.

[0038] Meanwhile, this platform protects against jailbreak attacks during the operation of large language models, including protection against prompt injection attacks and control of model output content. On the one hand, it actively intercepts potential attack behaviors by strictly verifying and filtering prompts input by users, combined with a blacklist mechanism for sensitive words or dangerous operations. On the other hand, it builds a firewall for the output of large models, reviews sensitive information in the generated content, and introduces an artificial intelligence security review model to achieve automatic identification and filtering of sensitive outputs, thereby improving the security and controllability of the model service.

[0039] The system in this application embodiment achieves agile delivery. In submitted natural language tasks, the model can accurately extract key fields such as target, input, output, and constraints. The large model actively generates a list of follow-up questions, controlling the average number of clarification rounds required to complete structured tasks to within 3 rounds.

[0040] It also enables elastic scaling, with each submodule capable of independent horizontal scaling. During peak periods, additional instances are automatically launched and automatically recycled after peak periods, improving CPU / GPU utilization by more than 30%. The data-algorithm-computing power are bound and deeply integrated in one go, reducing multiple rounds of handshakes and data migration, and shortening end-to-end latency by up to 30%. Compliance and security are improved in parallel. Plugin security scanning and signature verification are fully automated, reducing the compliance review cycle by 30%. Data is usable but invisible, ensuring zero raw data leakage during runtime and guaranteeing data security. The misauthorization rate is less than 1%, and the RBAC permission management system achieves precise authorization, preventing data abuse. The jailbreak attack detection rate is greater than 90%, and prompt word filtering and large model content review mechanisms prevent large models from bypassing permissions to access private content.

[0041] In summary, the modular, plug-in, and integrated design not only significantly reduces R&D and operational costs but also delivers quantifiable benefits in performance, elasticity, security, and ecosystem development. Furthermore, the integrated approach provides convenience and a sustainable infrastructure for multimodal, large-scale applications. With modularity ensuring maintainability, plug-in providing scalability, and integrated invocation achieving high efficiency and low barriers to entry, the platform can quickly adapt to new scenarios, continuously integrate new technologies, and stably deliver high-performance services across multimodal tasks.

[0042] In one embodiment, such as Figure 2 As shown, the intent recognition layer 2 includes: The intent recognition large model module 21 is used to perform low-level semantic understanding of the structured task request and output a preliminary intent vector; The knowledge graph auxiliary module 22 interacts bidirectionally with the intent recognition large model module to perform knowledge reasoning and semantic completion on the preliminary intent vector in order to generate the structured task description.

[0043] In application, the intent recognition layer 1 mainly consists of two parts: the intent recognition large model module 21 and the knowledge graph auxiliary module 22. The intent recognition large model module is based on a general large language model and achieves accurate understanding of user input semantics, task type, and operational requirements through fine-tuning and information extraction. This module internally includes a general large language model-based submodule 211, a fine-tuning submodule 213, and a first extraction submodule 213, which are responsible for semantic modeling, domain adaptation, and key information extraction, respectively, forming an efficient mapping mechanism from natural language to system instructions.

[0044] The knowledge graph auxiliary module 22 provides knowledge background and logical support for intent recognition, ensuring the system has continuous learning and reasoning capabilities. This module includes a knowledge storage submodule 221, a second extraction submodule 222, and an automatic knowledge update submodule 223, which respectively implement knowledge storage indexing, information extraction, and dynamic maintenance. The two modules form a collaborative closed loop through bidirectional interaction: the intent recognition result calls the knowledge graph for semantic completion and verification, while knowledge updates inversely optimize the model's understanding capabilities, thereby constructing an intelligent semantic recognition system with continuous evolution and adaptive reinforcement characteristics.

[0045] The workflow of the intent recognition layer is as follows: The system first receives multimodal input data (including text, speech, images, etc.) from the user, and the intent recognition big model module completes semantic parsing and intent extraction. The general big language model submodule is responsible for performing low-level semantic understanding of the input content, the fine-tuning submodule adapts and optimizes the model by combining specific domain knowledge, and the extraction submodule further identifies key information such as task objectives, operation objects, and constraints to generate a structured preliminary intent representation. Subsequently, the intent information is passed to the knowledge graph auxiliary module for semantic enhancement and knowledge association. The knowledge storage submodule provides index support for domain knowledge and semantic nodes, the second extraction submodule matches or expands relevant knowledge nodes according to the preliminary intent, and the knowledge automatic update submodule verifies, corrects, and dynamically completes the results. Finally, the output after knowledge reasoning and consistency verification is passed to downstream modules in the form of task instructions, query results, or decision suggestions, realizing a complete semantic recognition and knowledge enhancement closed loop from multimodal input to intelligent response.

[0046] In the application, the intent recognition large model module uses a 12-layer Transformer encoder to semantically encode the input text, outputting a preliminary 768-dimensional intent vector. The knowledge graph auxiliary module calculates the correlation between this vector and knowledge nodes using a graph attention network.

[0047] The intent recognition large model module can also perform contextual modeling of input commands through a multi-head attention mechanism, outputting a preliminary 1024-dimensional intent vector. The knowledge graph auxiliary module calculates the similarity between this vector and knowledge nodes through a graph neural network.

[0048] For example, when a user submits a part defect detection task, the knowledge graph associates and maps surface defect features with visual detection algorithms. The two parties interact asynchronously through a two-way message queue. The intent recognition big model passes the initially extracted scratch detection keywords to the knowledge graph, and the knowledge graph returns the corresponding optical inspection standards and safety level requirements, which together constitute a complete task description specification.

[0049] The intent recognition layer in this embodiment comprises a dual structure of an intent recognition large model module and a knowledge graph auxiliary module. The large model module is responsible for the underlying semantic understanding and outputting intent vectors, while the knowledge graph module performs knowledge reasoning and semantic completion. This design combines the advantages of data-driven and knowledge-driven methods. The large model provides generalization capabilities to handle diverse inputs, while the knowledge graph injects domain knowledge to ensure logical rigor. The two interact bidirectionally to complement each other, enhancing the accuracy of intent understanding and scenario adaptability.

[0050] In one embodiment, such as Figure 2 As shown, the intent recognition large model module 21 includes a general large language model submodule, a fine-tuning submodule, and a first extraction submodule; The general large language model submodule 211 is used to perform low-level semantic modeling on the structured task request; The fine-tuning submodule 212 is connected to the general large language model submodule and is used to optimize the semantic modeling results using domain-specific data and output a preliminary intent vector. The first extraction submodule 213 is connected to the fine-tuning submodule and is used to extract key task information from the preliminary intent vector.

[0051] In applications, the General Large Language Model submodule can use the GPT-3 architecture to perform semantic modeling of task requests, generating an intermediate representation that includes a syntax tree and semantic roles.

[0052] The general large language model submodule can also use the BART architecture to perform sequence-to-sequence modeling of task requests, generating an intermediate representation with semantic role annotations. The fine-tuning submodule uses AdapterHub technology to inject an industry-specific dictionary to adapt the intermediate representation to the relevant domain.

[0053] For example, the general term surface treatment is specialized as anodizing treatment. The first extraction submodule extracts key parameters such as the allowable defect threshold of the detection standard from the optimized representation based on the BERT-CRF model. These three submodules are executed in series through the workflow engine, and the output of the preceding submodule is transmitted to the subsequent submodule through the message middleware.

[0054] The fine-tuning submodule can be adapted to intermediate representations by injecting medical domain dictionaries through LoRA technology, for example, specializing the general term "thoracic imaging" to "CT plain scan image".

[0055] The first extraction submodule extracts key information such as the examination site and diagnostic urgency level from the optimized representation based on the BiLSTM-CRF model. These three submodules are executed in series in a pipeline mode, with the output of the preceding submodule serving as the input of the subsequent submodule.

[0056] The general large language model submodule of this application performs basic semantic modeling, the fine-tuning submodule performs domain adaptation, and the first extraction submodule extracts key information, realizing the gradual refinement of semantic processing. The general model ensures basic capabilities, the fine-tuning module improves domain relevance, the extraction module accurately locates task elements, and the submodules are connected to ensure data flow continuity, thereby improving the accuracy and efficiency of intent parsing.

[0057] In one embodiment, the knowledge graph auxiliary module 22 includes a knowledge storage submodule 221, a second extraction submodule 222, and a knowledge automatic update submodule 223; The knowledge storage submodule 221 is used to store domain knowledge data based on graph structure; The second extraction submodule 222 is connected to the first extraction submodule and the knowledge storage submodule. It is used to receive the key information of the task and perform query, matching and semantic association in the knowledge storage submodule to perform semantic completion and verification of the key information of the task. The semantic completion and verification are fed back to the extraction submodule to form the structured task description. The automatic knowledge update submodule 223 is connected to the knowledge storage submodule and is used to periodically update the knowledge nodes in the knowledge storage submodule using differential privacy technology.

[0058] In the application, the knowledge storage submodule can use the Nebula Graph database to store industrial knowledge graphs, containing more than 30,000 nodes of material properties and processing technology. The second extraction submodule uses the Cypher query language to match relevant knowledge topologies from the key information output by the first extraction submodule.

[0059] For example, the heat treatment temperature range is expanded based on the extracted aluminum alloy material keywords. The knowledge automatic update submodule obtains production data from the MES system every day, processes the process parameters through differential privacy technology, and then updates the graph nodes. The entire process is decoupled through an event-driven architecture. The semantic completion result is returned to the first extraction submodule through a callback interface to correct entity relationships.

[0060] The knowledge storage submodule of this application maintains graph structure data, the second extraction submodule performs query matching and semantic association, and the knowledge automatic update submodule performs differential privacy updates periodically, ensuring the real-time performance and security of the knowledge base. The storage module provides an efficient retrieval foundation, the extraction module realizes dynamic knowledge expansion, the update module ensures data freshness, and differential privacy technology prevents information leakage, enabling the system to continuously evolve and protect sensitive knowledge.

[0061] In one embodiment, the task scheduling layer is as follows: Figure 3 As shown, it includes: The task scheduling big model module 31 is connected to the intent recognition layer and is used to receive the structured task description and parse it into a task plan containing task sequence and initial resource requirements. The deep learning module 32 is connected to the task scheduling big model module to obtain the task plan and the real-time status information of the system, and to formulate an optimized resource allocation scheme based on the obtained information. The task execution module 33 is connected to the reinforcement deep learning module and is used to receive the resource allocation scheme and call the interfaces of the data pool, algorithm pool and computing power pool according to the resource allocation scheme to respectively mount data resources, call algorithm components and allocate computing power nodes. The monitoring and feedback module 34 is connected to the task execution module and the resource pool layer. It is used to monitor the task execution status and resource usage in the resource pool layer, generate the real-time status information, and feed the real-time status information back to the reinforcement deep learning module to optimize the scheduling strategy.

[0062] In application, the task scheduling layer mainly consists of a task scheduling large model module 31, a reinforcement deep learning module 32, a task execution module 33, and a monitoring and feedback module 34. This forms a closed-loop system from task generation and intelligent scheduling to execution feedback. The task scheduling large model module 31 is responsible for generating executable task plans based on user requirements. Internally, it includes a large language model submodule 311 and a task generation submodule 312: the former generates task descriptions through natural language understanding, while the latter transforms abstract semantic intent into a concrete sequence of executable instructions.

[0063] The deep learning module 32 is responsible for intelligent scheduling and strategy optimization. It includes a reinforcement learning algorithm submodule 321 and a scheduling strategy generation submodule 322. The former continuously trains the strategy model based on resource status and task execution feedback, while the latter dynamically generates the optimal scheduling scheme.

[0064] The task execution module 33 is responsible for the specific task implementation, while the three-computing scheduling and execution submodule 331 completes the unified scheduling and coordinated execution of computing power, algorithms and data resources.

[0065] The monitoring and feedback module 34 runs through the entire task scheduling process, ensuring the controllability and adaptive optimization of system operation. The resource monitoring submodule 341 perceives the usage status of computing power and data resources in real time, and the information feedback submodule 342 sends the execution results and performance indicators back to the upper-layer model for retraining and optimization of the scheduling strategy.

[0066] During operation, the task scheduling layer first receives structured task requests from the intent recognition layer. The task scheduling big model module parses and decomposes the input semantic task, outputting a task plan, execution sequence, and resource requirement description. This data is passed to the reinforcement deep learning module through a unified gRPC interface. The reinforcement deep learning module receives task features and the current system resource status, outputting a policy-optimized task scheduling scheme, including the allocation plan for computing power, algorithms, and data resources. This scheme is pushed to the task execution module in real time. The task execution module calls the three computing resource pool interfaces according to the scheduling scheme, allocating computing power, calling algorithms, and mounting data resources respectively. It also reports the performance indicators and task status during execution to the monitoring and feedback module via the OpenTelemetry interface. The monitoring and feedback module analyzes resource usage and task running status in real time, outputting feedback data and abnormal alarm information. This data is then returned to the reinforcement deep learning module via an internal gRPC stream for policy updates, thus forming a closed data flow loop of "task input—policy generation—execution implementation—monitoring feedback—policy optimization".

[0067] In applications, the task scheduling large model module can use the Seq2Seq model to decode the structured task description into a process step DAG graph, which includes the dependency relationship between the three nodes of the image acquisition feature extraction result output.

[0068] The enhanced deep learning module obtains the TPU utilization index of the computing cluster in real time through the PPO algorithm and dynamically adjusts the computing resource allocation weight of the DAG nodes.

[0069] The task execution module converts the optimized resource allocation scheme into a Kubernetes Operator instruction set, which calls the HDFS interface algorithm pool's MLflow model library and the computing power pool's resource quotas, respectively.

[0070] The monitoring and feedback module collects container operation metrics through the Prometheus monitoring stack. When a memory leak is detected, it immediately triggers a policy update from the reinforcement learning module.

[0071] The task scheduling big model in this application analyzes the task generation plan, the reinforcement learning module optimizes resource allocation, the task execution module calls the resource pool interface, and the monitoring and feedback module monitors the status and provides feedback. This division of labor realizes a closed loop of intelligent decision-making and execution. The big model provides task analysis capabilities, reinforcement learning achieves dynamic optimization, the execution module ensures the issuance of instructions, and the monitoring module ensures system controllability. The collaborative work of each module makes the scheduling process both flexible and reliable, and can cope with complex and ever-changing task requirements.

[0072] In one embodiment, the resource pool layer responds to the task execution instruction and provides resources to execute the computation task in the following manner: The data pool is used to respond to the data instructions in the task execution instructions, provide corresponding data resources according to the metadata description and permission control policy in the instructions, and perform data loading and mounting. The algorithm pool is used to respond to the algorithm-related instructions in the task execution instructions, call the corresponding algorithm components according to the algorithm encapsulation description and access control policy in the instructions, and start the algorithm execution environment; The computing power pool is used to respond to the instructions regarding computing power in the task execution instructions, allocate corresponding computing resources according to the resource scheduling strategy in the instructions, and run the algorithm components called by the algorithm pool on the computing resources to process the data mounted by the data pool in order to execute the computing task.

[0073] In the application, after the data pool receives the dataset hash value in the task execution instruction, it first verifies the access permission and triggers the national cryptographic SM4 algorithm to decrypt the stored data, and then loads it into the shared memory through the RDMA protocol.

[0074] The algorithm pool pulls the corresponding Singularity image from the model repository according to the YOLOv5 model version required by the instruction, and starts the inference service in a secure container.

[0075] The computing power pool translates the computing power requirements in the instructions into Slurm job configurations through a resource abstraction layer, dynamically creates computing environments in the GPU cluster, and finally executes defect detection calculations by the TensorFlow core of the computing power pool. The three pools work together through a service mesh architecture: the data readiness event of the data pool triggers the model loading of the algorithm pool, and the service readiness event of the algorithm pool triggers the task distribution of the computing power pool.

[0076] In application, the construction of the algorithm pool mainly revolves around the standardized encapsulation of algorithm resources, unified interface calls, and secure and controllable management: First, various algorithms are encapsulated in the form of microservices, containers, or serverless functions, and unified calls are achieved through RESTful APIs or RPC remote procedure call interfaces; Second, a unified algorithm metadata description system is established to clarify the algorithm's purpose, input and output specifications, applicable scenarios, and performance indicators, forming a standardized mechanism for algorithm registration, retrieval, and management; Third, fine-grained access control and security management of algorithms are achieved through API gateways, RBAC permission management, and algorithm access auditing mechanisms; In addition, a containerized environment and security scanning mechanism for algorithm execution are built to ensure the stability, scalability, and security of algorithm execution; Finally, an algorithm version control and update mechanism is established to support continuous iterative optimization and automatic deployment of algorithms, thereby achieving efficient management and highly reliable operation of algorithm resources.

[0077] In applications, the construction of computing power pools primarily revolves around the unified scheduling of nodes across cloud, edge, and endpoint platforms. First, a unified encapsulation and abstraction is achieved. Then, through computing power virtualization and cloud-native technologies, a unified computing power resource pool is formed, and a centralized scheduling platform is used for unified scheduling. Computing power nodes from different sources are uniformly encapsulated and abstracted to form standardized computing power resource units. First, resource abstraction is unified, including CPU, GPU, memory, storage, network bandwidth, cost, and the geographical location and latency attributes of computing power nodes. Then, a unified interface encapsulation is implemented, providing a unified RESTful API or RPC interface and supporting automatic node discovery, node registration, and health checks. Strict authentication and permission management, secure transmission channels, resource isolation, and fault tolerance mechanisms are employed for computing power. Secure access and automatic node updates of computing power are supported, achieving efficient load balancing management and operation during invocation.

[0078] In this embodiment of the application, the data pool provides and loads data resources, the algorithm pool calls the algorithm components to start the environment, and the computing power pool allocates resources to perform computation. This response mechanism ensures efficient use of resources. The data pool ensures data accessibility through standardized interfaces, the algorithm pool improves component reusability through encapsulation, and the computing power pool achieves elastic scaling through virtualization. The collaborative work of the three pools makes the task execution process smooth and reduces resource contention and waiting time.

[0079] In one embodiment, the reinforcement deep learning module employs a near-end strategy to optimize the PPO algorithm; The state space of the enhanced deep learning module includes the CPU utilization, GPU utilization, memory usage, and task queue length of the resource pool layer. The action space of the enhanced deep learning module is a combination of decisions on the allocation of data, algorithms, and computing resources.

[0080] In applications, the state space monitor of the enhanced deep learning module continuously collects InfiniBand network traffic data from the high-performance computing cluster, including bandwidth utilization and latency metrics. Decision combinations in the action space are achieved through multi-agent collaborative decision-making. For example, simultaneously adjusting data shard size, model parallelism, and the number of computing nodes; these decisions are translated into actual resource allocation by the Slurm scheduler. When an abnormal network latency is detected on a computing node, the action vector reschedules the node's tasks to a low-latency node group.

[0081] The reinforcement deep learning module in this application uses the PPO algorithm for policy optimization. The state space monitors resource usage indicators, and the action space generates resource allocation decisions, which improves the practicality of the scheduling algorithm. The PPO algorithm ensures the stability of policy updates, the state space covers key performance parameters, and the action space supports multi-dimensional resource coordination, enabling the system to intelligently adjust resource allocation according to real-time load and optimize overall performance.

[0082] In one embodiment, the monitoring and feedback module is further configured to: Real-time collection of resource node performance metrics, including CPU / GPU utilization, memory usage, disk I / O, and network bandwidth; Provide real-time alerts for resource overload and / or abnormal call conditions; Record audit logs for resource calls.

[0083] In the application, the monitoring agent of the monitoring and feedback module is deployed in Sidecar mode on each compute node. It collects container-level system calls and network traffic through the BPF compiler. When the CPU utilization exceeds the threshold for three consecutive sampling periods, it sends an alarm to the network management system via the SNMP protocol. The audit log component uses FluentBit lightweight log collector to record all resource call operations, including timestamps, user identities, and resource fingerprints. These logs support security auditing by operation type after being indexed by OpenSearch.

[0084] In the application, a resource monitoring and feedback module is introduced into the resource pool to provide timely monitoring and feedback on the status of data, algorithms, and computing power at the resource pool layer. It is mainly built around four aspects: resource status awareness, operational behavior monitoring, anomaly detection and early warning, and intelligent optimization. First, in terms of resource status awareness, deploy lightweight monitoring agents or embed them into resource nodes in Sidecar mode to continuously collect key operational indicators such as CPU / GPU utilization, memory usage, bandwidth traffic, disk I / O, call frequency, response latency, and error rate, and synchronize them to the centralized resource monitoring platform to form a comprehensive and real-time resource profile.

[0085] Secondly, in terms of monitoring operational behavior, an audit log system is built to record key operational events, including access time, calling entity, operation content, and result status. Combined with a link tracing mechanism, this enables visualized tracking and source analysis of resource call paths, providing a basis for problem localization and responsibility allocation.

[0086] In terms of anomaly detection and early warning, a rule-based multi-level alarm system is constructed: on the one hand, threshold strategies are formulated to provide real-time early warnings for situations such as resource overload, abnormal calls, and authorization violations; on the other hand, intelligent algorithms such as time series prediction and anomaly pattern recognition are introduced to proactively identify resource usage trends and deviations in system behavior.

[0087] The resource supervision and feedback module provides feedback to the large model, while the resource monitoring module is responsible for real-time perception, early warning, and feedback of the overall system operation status. It is a key component in ensuring system stability, security, and sustainable operation. This module achieves comprehensive supervision from resource status to behavior patterns through a multi-level, multi-dimensional monitoring system, forming a closed-loop feedback mechanism to promote intelligent resource scheduling and optimization. Its sub-modules include: Resource Status Perception: Real-time monitoring of the operational indicators of core resources such as computing power, algorithms, and data comprehensively grasps key parameters such as resource utilization, response time, and task load, ensuring dynamic visibility and traceability of resource status. Operational Behavior Monitoring: Monitoring and analyzing operational behaviors, task execution paths, and resource call chains during system operation to identify potential performance bottlenecks and abnormal behavior patterns, providing data support for system optimization and security protection. Anomaly Detection and Early Warning: Utilizing anomaly detection algorithms and threshold judgment mechanisms, identifying and issuing early warnings for abnormal fluctuations, fault risks, and security vulnerabilities in resource operation, supporting tiered alarm and emergency response mechanisms to improve system robustness and fault tolerance. Resource monitoring and feedback: Link monitoring results with resource scheduling strategies to achieve closed-loop feedback of anomaly handling, performance optimization and strategy adjustment, forming a continuous optimization cycle of "monitoring-analysis-control-remonitoring".

[0088] The monitoring and feedback module in this embodiment collects performance indicators in real time, provides early warnings of anomalies, and records audit logs. This monitoring system enhances the observability of the system, comprehensively covers the status of resource nodes, promptly identifies problems through the early warning mechanism, and supports post-event analysis through audit logs. The module forms an optimization loop through continuous monitoring and feedback, helping the system maintain stable operation and respond quickly to faults.

[0089] In one embodiment, the task scheduling layer is further configured to feed back task execution results, resource utilization efficiency, and abnormal situation information to the intent recognition layer; The intent recognition layer is also configured to optimize the accuracy of intent recognition and the quality of task description based on the feedback information.

[0090] In the application, the task scheduling layer can encapsulate task execution results into Apache Avro format data packets via the Apache Kafka message bus. These packets contain fields such as task ID, execution timestamp, and exception code. The data consumer in the intent recognition layer receives these data packets and parses the deviation between the planned and actual task execution. For example, when a size detection task is detected to be frequently timed out due to lighting conditions, the intent recognition large model module adds environmental factor verification rules. This optimization updates model parameters in real time through an online learning mechanism, forming a closed-loop optimization chain from execution results to intent recognition accuracy.

[0091] In this embodiment, the scheduling layer feeds back the execution results and resource efficiency to the recognition layer. The recognition layer optimizes the accuracy of intent recognition based on the feedback. This cross-layer feedback promotes system self-learning. The execution results help the recognition layer correct task comprehension biases, and the resource efficiency data guides the adjustment of recognition strategies, enabling the system to continuously learn from historical tasks and improve long-term performance.

[0092] A second aspect of this application provides an electronic device that integrates a system as described in any embodiment of the first aspect of this application.

[0093] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A three-in-one computing system based on a large model, characterized in that, include: The user interaction layer is used to receive multimodal input from users and convert the multimodal input into structured task requests; An intent recognition layer, connected to the user interaction layer, is used to perform semantic parsing and intent understanding on the structured task request, and generate a structured task description containing task type, resource requirements and constraints. The task scheduling layer, connected to the intent recognition layer, is used to generate a resource allocation scheme based on the structured task description and issue task execution instructions to the resource pool layer based on the resource allocation scheme. The resource pool layer, connected to the task scheduling layer, includes a data pool, an algorithm pool, and a computing power pool. It is used to respond to the task execution instructions, provide corresponding data resources, algorithm resources, and computing power resources, and execute computing tasks. The output results module, connected to the task scheduling layer, is used to integrate and output the task execution results.

2. The three-component integrated system based on a large model as described in claim 1, characterized in that, The intent recognition layer includes: The intent recognition large model module is used to perform low-level semantic understanding of the structured task request and output a preliminary intent vector; The knowledge graph auxiliary module interacts bidirectionally with the intent recognition large model module to perform knowledge reasoning and semantic completion on the preliminary intent vector in order to generate the structured task description.

3. The three-component integrated system based on a large model as described in claim 2, characterized in that, The intent recognition large model module includes a general large language model submodule, a fine-tuning submodule, and a first extraction submodule; The general large language model submodule is used to perform low-level semantic modeling of the structured task request; The fine-tuning submodule is connected to the general large language model submodule and is used to optimize the semantic modeling results using domain-specific data and output a preliminary intent vector. The first extraction submodule, connected to the fine-tuning submodule, is used to extract key task information from the initial intent vector.

4. The three-component integrated system based on a large model as described in claim 3, characterized in that, The knowledge graph auxiliary module includes a knowledge storage submodule, a second extraction submodule, and a knowledge automatic update submodule. The knowledge storage submodule is used to store domain knowledge data based on graph structures; The second extraction submodule, connected to the first extraction submodule and the knowledge storage submodule, is used to receive the key task information and perform querying, matching and semantic association in the knowledge storage submodule to semantically complete and verify the key task information. The semantic completion and verification are fed back to the extraction submodule to form the structured task description. The automatic knowledge update submodule is connected to the knowledge storage submodule and is used to periodically update the knowledge nodes in the knowledge storage submodule using differential privacy technology.

5. The three-component integrated system based on a large model as described in claim 1, characterized in that, The task scheduling layer includes: The task scheduling big model module, connected to the intent recognition layer, is used to receive the structured task description and parse it into a task plan containing task sequence and initial resource requirements. The deep learning module is enhanced and connected to the task scheduling model module to obtain the task plan and the real-time status information of the system, and to formulate an optimized resource allocation scheme based on the obtained information. The task execution module, connected to the reinforcement deep learning module, is used to receive the resource allocation scheme and call the interfaces of the data pool, algorithm pool and computing power pool according to the resource allocation scheme, so as to respectively mount data resources, call algorithm components and allocate computing power nodes; The monitoring and feedback module, connected to the task execution module and the resource pool layer, is used to monitor the task execution status and resource usage in the resource pool layer, generate the real-time status information, and feed the real-time status information back to the reinforcement deep learning module to optimize the scheduling strategy.

6. The three-component integrated system based on a large model as described in claim 1, characterized in that, The resource pool layer responds to the task execution instruction and provides resources to execute the computation task in the following manner: The data pool is used to respond to the data instructions in the task execution instructions, provide corresponding data resources according to the metadata description and permission control policy in the instructions, and perform data loading and mounting. The algorithm pool is used to respond to the algorithm-related instructions in the task execution instructions, call the corresponding algorithm components according to the algorithm encapsulation description and access control policy in the instructions, and start the algorithm execution environment; The computing power pool is used to respond to the instructions regarding computing power in the task execution instructions, allocate corresponding computing resources according to the resource scheduling strategy in the instructions, and run the algorithm components called by the algorithm pool on the computing resources to process the data mounted by the data pool in order to execute the computing task.

7. The three-component integrated system based on a large model as described in claim 5, characterized in that, The enhanced deep learning module employs a near-end strategy to optimize the PPO algorithm. The state space of the enhanced deep learning module includes the CPU utilization, GPU utilization, memory usage, and task queue length of the resource pool layer. The action space of the enhanced deep learning module is a combination of decisions on the allocation of data, algorithms, and computing resources.

8. The three-component integrated system based on a large model as described in claim 5, characterized in that, The monitoring and feedback module is also configured to: Real-time collection of resource node performance metrics, including CPU / GPU utilization, memory usage, disk I / O, and network bandwidth; Provide real-time alerts for resource overload and / or abnormal call conditions; Record audit logs for resource calls.

9. The three-component integrated system based on a large model as described in claim 1, characterized in that, The task scheduling layer is also configured to feed back task execution results, resource utilization efficiency, and abnormal situation information to the intent recognition layer. The intent recognition layer is also configured to optimize the accuracy of intent recognition and the quality of task description based on the feedback information.

10. An electronic device, characterized in that, The electronic device integrates the system as described in any one of claims 1-9.

Images