34280 results about "Processing" patented technology

The invention discloses a hydraulic engineering equipment data intelligent management system based on digital twinning, and belongs to the technical field of hydraulic engineering. Comprising an intelligent perception and data fusion module for realizing real-time acquisition and standardized processing of cross-modal data; the knowledge graph construction and causal reasoning module is used for constructing an intelligent knowledge system capable of autonomously learning and semantic reasoning; the digital twin modeling and simulation module is used for realizing dynamic simulation and scene deduction of a full life cycle and providing limit working condition simulation and risk assessment support; the intelligent prediction and health management module is responsible for performing real-time monitoring, fault early warning and residual service life prediction on the equipment state, and generating personalized intelligent maintenance strategies for different working conditions; the visualization and decision support module is used for visually presenting the equipment operation data and the analysis result and providing intelligent decision recommendation; and the cloud edge collaboration and system integration module realizes cross-platform interoperation and continuous integration through distributed computing and micro-service architecture.

The invention discloses a computer equipment fault monitoring system and method based on artificial intelligence, and relates to the technical field of computer equipment fault monitoring. The system comprises a data access module, a semantic analysis module, a knowledge graph construction module, a dynamic semantic association module, a data fusion processing module, a decision output module and an adaptive optimization module. The data access module collects and standardizes hardware, software and network data; the semantic analysis module extracts and enhances semantic tags; the knowledge graph construction module forms a data semantic relation network; the dynamic semantic association module screens potential semantic relationships; the data fusion processing module generates a multi-dimensional feature vector; the decision output module triggers fault early warning; and constructing a feedback knowledge graph of the self-adaptive optimization module. According to the method, through event-driven interpolation, dynamic weight fusion, closed-loop feedback optimization and the like, the problems of multi-source data alignment, semantic fusion and dynamic adaptation are solved, the fault monitoring accuracy and the system adaptability are improved, and the method is suitable for fault monitoring and early warning of computer equipment.

The invention discloses a method for enhancing dynamic knowledge retrieval based on a large language model, belongs to the field of knowledge retrieval, and aims to solve the problems of knowledge solidification, insufficient timeliness and illusion of a traditional LLM (Logistics Language Model). A multi-granularity knowledge base is dynamically constructed, and a rule and semantic partitioning technology is combined, so that a text is converted into a normalized vector, and a hybrid index is established; a two-channel retrieval triggering mechanism is adopted, keyword matching scores and BERT semantic probability analysis are fused, and retrieval requirements are intelligently judged; vectorization retrieval is realized through a BGE-M3 model, and candidate results are reordered in combination with a cross encoder to improve the precision. The system supports multi-language adaptive processing, dynamic switching of word segmentation strategies and cross-language retrieval, and introduces real-time knowledge updating and version control. According to the method, the answer timeliness and accuracy are remarkably improved, the context coherence of multiple rounds of dialogues is optimized, the method can be widely applied to the fields of intelligent customer service, professional questions and answers and the like, the LLM illusion risk is effectively reduced, and the knowledge traceability is enhanced.

The invention discloses an electromechanical system fault pre-diagnosis method and system based on digital twinning. The method comprises the following steps of obtaining multi-source data in an electromechanical system operation process; preprocessing the acquired multi-source data, wherein the preprocessing comprises data cleaning, normalization processing and feature extraction; and on the basis of the preprocessed multi-source data, an electromechanical system design drawing, a three-dimensional geometric model, material attributes and a kinetic equation are fused, and a digital twin model is constructed. According to the invention, through a digital twin model dynamic calibration and prediction algorithm, early abnormity of the equipment is identified in advance, the fault probability and the residual life are output, and non-planned shutdown is reduced; by constructing a cross-physical domain fault feature system and fusing model simulation and actual measurement data, the potential fault identification accuracy is improved, and the missed diagnosis rate is reduced; by calibrating parameters of the digital twin model in real time, the method adapts to nonlinear changes of equipment, ensures high-fidelity mapping of the model, and improves fault prediction precision.

A system and method for creating complex, immersive, and interactive digital content is disclosed. The system integrates advanced artificial intelligence, multi-modal input processing, cloud-based shared environments, and immersive hardware to generate, optimize, and deliver rich interactive experiences. The platform supports content mashups, custom scenario generation, and adaptive AI behaviors, enabling the creation of unique and engaging digital environments across various media formats.

The invention provides a decision generation and execution method and system based on an AI agent, and the method comprises the steps: analyzing a user demand document through a natural language processing technology, and extracting key information to construct a structured cue word; then inputting the cue word into a private domain AI agent based on a large model, and generating a preliminary decision scheme in combination with a professional domain database; automatically generating adversarial introspection probe cues, and guiding an AI agent to carry out consistency, risk and constraint conformity evaluation on the preliminary scheme; the system collects feedback response of the AI intelligent agent, analyzes the feedback through a pre-trained graph neural network, and calculates a confidence score of a decision scheme; when the confidence reaches a preset threshold value, automatically generating an execution script according to the decision scheme; and the execution script automatically operates the target system through the preset API and generates an execution document. The whole process realizes a closed-loop intelligent decision-making process from demand understanding, scheme generation, self-verification and automatic execution, and the decision-making efficiency and reliability are remarkably improved.

The invention relates to the technical field of building engineering, in particular to a building data processing method and system based on multiple building specifications, and the method comprises the steps: building model data standardization processing: extracting geometric attributes, material attributes and spatial topological relations of components through a BIM software interface, and generating model data in a standard format; constructing a multi-source specification rule tree, performing semantic analysis on national standard, local standard and industrial standard provisions, extracting triple constraint conditions, and fusing to generate a unified rule tree containing hierarchical relationships and conflict marks; on the basis of a spatial topology mapping relationship between the model and the rule tree, identifying a specification conflict and generating an optimization rule set according to a specification effectiveness level and a spatial attribute priority; and finally, real-time compliance verification is executed, and a three-dimensional compliance report including conflict positioning, article basis and correction suggestions is output. According to the method, multi-source specifications can be automatically adapted, model violation components are accurately recognized, a correction scheme is given, and the specification compliance and the verification efficiency in the design stage are improved.

The invention relates to the technical field of intelligent operation and maintenance and artificial intelligence crossing of a power system, in particular to an intelligent power work order processing method of an RPA fused multi-modal large model, which analyzes multi-modal work order data such as texts, voices, images and the like through a domain adaptation large language model, and realizes fault key information extraction and conflict resolution in combination with a dynamic knowledge graph; performing work order priority scoring and resource allocation by using space-time constraint reinforcement learning; an analysis result is converted into an automatic execution script through an RPA engine, and a whole-process closed loop of order sending, processing and feedback is achieved; meanwhile, a feedback optimization and conflict resolution cooperation mechanism is constructed, and the knowledge graph and the model precision are continuously iterated. The method improves work order processing efficiency and analysis precision, enhances decision scientificity, and is suitable for an intelligent operation and maintenance scene of a power system.

The invention belongs to the technical field of intelligent machining path control, and discloses an intelligent numerical control machine tool automatic programming path optimization method based on workpiece modeling, which comprises the following steps: acquiring a CAD model, machine tool sensor data, tool wear data and historical machining logs, generating a workpiece characteristic parameter set, and fusing a three-level compensation mechanism to generate a dynamic error parameter set; then, dividing a preliminary risk level of the processing area, and performing secondary risk assessment to generate a comprehensive risk level; extracting a risk level conflict area, and determining a final risk level; constructing a static / dynamic cost matrix to obtain a path priority map; thirdly, generating an initial path, smoothing an optimized path trajectory, and performing multi-objective optimization to generate an optimized path planning table; cutting parameters are adjusted in real time, the path feasibility is verified, and a real-time control instruction set is generated; and finally, constructing a quality-process correlation model, generating a global strategy packet, forming closed-loop iteration, and completing system self-evolution.

The invention relates to a cross-modal knowledge reasoning method based on a multi-modal large model. In a cross-modal knowledge reasoning process, an existing model is usually limited by single-modal information extraction and shallow feature fusion, so that deep semantic association among data such as texts, images and videos is difficult to fully capture. In order to solve the problem, the invention provides a model for fusing multi-modal information such as texts, images, videos, documents and the like, and processing of multi-modal data is converted into unified feature extraction, interaction and deep reasoning tasks by fully utilizing a supervision fine tuning strategy, a self-adaptive attention mechanism and a cross-language processing technology. The model adopts a modular design, integrates multi-source data complementary analysis, spatial-temporal feature modeling and emotional semantic analysis, and realizes multi-modal collaborative interaction, dynamic scene understanding, long video key event analysis and man-machine co-emotional response. Through sufficient training, the multi-modal large model shows excellent logical reasoning ability and emotion understanding ability in a complex cognitive task, and a brand new solution is provided for efficient extraction, deep semantic analysis and intelligent response of cross-modal information.

The invention relates to the technical field of program control systems, in particular to a pentahedron machining center precision calibration method and system based on multi-sensor fusion, and the method comprises the steps: a sensor system construction and calibration module is used for field calibration, drift correction and redundancy deployment to ensure data precision, and achieves the whole-course traceability of a calibration process through a block chain technology; the multi-source data preprocessing and fusion module is used for time-space synchronization of heterogeneous data and dynamic fusion of multi-source information; the intelligent modeling and state prediction module is used for performing real-time and multi-task prediction on key states such as tool wear and thermal deformation; based on the prediction result, the adaptive compensation and path optimization module is used for dynamically optimizing the tool path; meanwhile, through an online learning mechanism, the calibration model is continuously updated by utilizing a processing result; the distributed cooperative control module executes data processing and calibration algorithms locally and makes a cooperative decision with a numerical control system, and low-delay and intelligent response to machining abnormity is achieved.

A system and method for adaptive real-time multi-modal compression with dynamic resource allocation provides intelligent compression optimization based on continuously monitored device conditions. The system monitors battery level, CPU utilization, and memory availability while classifying incoming multi-modal data streams comprising image, audio, text, and sensor data to determine processing priorities. Multi-objective optimization balances compression efficiency, reconstruction quality, and energy consumption using evolutionary algorithms that generate optimal parameters for an adaptive variational autoencoder. The autoencoder features dynamically selectable processing complexity, adjustable latent space dimensionality, and modality-specific processing layers. The system automatically switches between operational modes including emergency mode triggered by resource constraints, which applies maximum compression settings and intelligent data triage. Continuous learning adapts compression parameters based on observed performance outcomes, improving future optimization decisions. The system enables homomorphic operations on compressed data and provides enhanced compression performance under varying resource constraints across diverse edge computing applications.

The invention relates to a digital resource sharing method and system based on a pedigree mapping relation, and belongs to the technical field of enterprise digital resource management, and the resource sharing method comprises the steps: obtaining multi-source heterogeneous data from an internal distributed system of an enterprise through a preset interface protocol; performing standardization processing on the multi-source heterogeneous data to generate a structured resource pool; a graph dictionary module is constructed, the resources in the structured resource pool are subjected to association topology analysis, and a resource relation topological graph is output; establishing a pedigree relationship between resources and business scenes based on the three-dimensional mapping model, respectively generating and fusing a business process demand map, a production stage demand map and a product capability matching map, and constructing a visual digital resource capability platform; and outputting the target resource identifier and the associated path in response to a resource calling instruction input by a user. Integration and sharing of internal and external multi-source heterogeneous data of an enterprise can be realized, the problem of data islands is solved, and meanwhile, the intelligent degree of a resource management system is improved.

Disclosed in the present application are a multi-modal data processing method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring a reference image and a reference text; extracting a reference visual feature of the reference image; by means of a multi-modal large language model, determining an embedding of the reference text, an embedding of a start mark of the reference visual feature, an embedding of the reference visual feature, and an embedding of an end mark of the reference visual feature; on the basis of the multi-modal large language model, splicing the embedding of the reference text, the embedding of the start mark, the embedding of the reference visual feature, and the embedding of the end mark into a target embedding sequence, performing attention processing on the basis of the embedding of the start mark, the embedding of the end mark, and an embedding selected by a sliding window in the target embedding sequence, and outputting a predicted sequence; and generating a predicted image and a predicted text on the basis of the predicted sequence.

The invention relates to a multi-modal AI data fusion processing method, device and equipment and a medium, and the method comprises the steps: firstly extracting visual, auditory and text modal features through a pre-training encoder, executing dimension alignment, and generating a standard data feature set with unified dimensions; a cross-modal semantic graph is constructed based on a cosine similarity algorithm, and the problem of semantic mismatch of heterogeneous data is solved; residual enhancement is carried out on the map nodes, and noise interference is eliminated; fusing the optimized features and the semantic topology in combination with a graph convolutional network to generate aggregation graph representation; the fusion features are mapped to a low-dimensional semantic space through a variational auto-encoder, and cross-modal correlation essence is captured; the key dimension contribution degree is quantified, a visual report is generated, and semantic association rules among modals are disclosed, so that the dimension isomerism limitation of a traditional fusion technology is broken through, quantifiable cross-modal semantic mapping is established, the whole process traceability from feature fusion to decision interpretation is realized, and the method is suitable for popularization and application. And the multi-modal decision black box problem in the fields of medical diagnosis, automatic driving and the like is effectively solved.

The invention discloses a multi-modal bill processing method based on dynamic knowledge enhancement. The multi-modal bill processing method comprises the following steps: S1, constructing a dynamic knowledge base containing an aging weight; s2, synchronously processing text, image and format features of the bill by adopting a multi-modal feature fusion network to generate a composite feature vector; s3, semantic-level, format-level and timeliness three-level fusion retrieval is carried out based on the composite feature vector, and a three-level fusion retrieval engine comprises dynamic weighted sorting with timeliness attenuation, a difference degree triggered artificial review mechanism and a policy sensitive slope adjustment algorithm; s4, setting a multi-expert cooperative verification system, wherein the multi-expert cooperative verification system comprises cooperative work of a rule engine, a large language model and a logical reasoning module; s5, implementing a dynamic knowledge updating mechanism, and automatically triggering incremental learning of the knowledge base when policy change or format update is detected; and S6, outputting structured data, and synchronously generating an auditing traceability chain containing a decision path. According to the method, the key field identification accuracy can be improved, and auditing traceability and non-perceptual increment updating in the whole process are realized.

The invention relates to the technical field of data processing, in particular to an intelligent document updating processing method and system.The intelligent document updating processing method comprises the steps that part parameters and coding rules are extracted from engineering design file metadata, a part list and a PDM and PLM system, the semantic association relation between parts is analyzed through a knowledge graph, and the mapping relation between logic identifiers and physical files is constructed; generating an initial version file library; and monitoring file names and version changes in real time based on a micro-service architecture, calling a simulation service in combination with a knowledge graph to verify parameter compatibility, screening an optimal version, updating the optimal version to a main version library, optimizing historical conflict data in a block chain evidence storage index by using a genetic algorithm to generate a standardized code, and outputting a cross-platform parameter mapping table. According to the method, the problems of file name change, non-standard coding conflict, version control missing and cross-platform adaptation incoherence are solved, and the data tracing efficiency, the version management reliability and the system compatibility are improved.

The invention discloses a data knowledge-based method based on semantic fusion, and relates to the technical field of computer information processing.The method comprises the steps that a requirement set serves as input, knowledge requirement analysis, concept modeling, relation modeling and constraint declaration are completed, and a semantic model OB is constructed; taking the original data set Sraw as input, completing standardization processing and structure segmentation under the support of a semantic model OB, and forming an entity corpus and a semantic unit set; based on the semantic unit set, structured and unstructured triple extraction, semantic verification and graph loading are executed, and an initial knowledge graph is constructed; performing entity alignment, relationship merging, rule reasoning and versioning release on the initial knowledge graph to generate a graph; introducing a quality evaluation mechanism, and outputting an optimized atlas and an evaluation document; the KG opt deployment is online, and query packaging, visualization, service arrangement and incremental maintenance are completed. The method aims at solving the problems that multi-source heterogeneous data are not uniform in structure and inconsistent in semantics.

A system and method for efficient natural language processing combines large and small language models with a thought caching architecture. The system includes a router that directs prompts either to a large language model for thought generation or to a thought cache containing previously generated thoughts. When using the large model, generated thoughts are combined with the original prompt and routed through a smaller language model to produce responses. The thought cache stores reasoning patterns that can be retrieved and reused, eliminating the need to regenerate similar thoughts for related prompts. The system supports both local and cloud-based caching, enabling personal and enterprise-wide thought storage and retrieval. This architecture reduces computational overhead while maintaining reasoning capabilities, effectively extends context windows beyond traditional limits, and enables efficient scaling across different deployment scenarios. The system can operate with reduced resources by leveraging cached thoughts without requiring constant access to the large model.

The invention relates to the technical field of large language models, and discloses a voice data interaction feedback control processing method based on a large language model. The method comprises the following steps: acquiring an industrial voice instruction through an AMR main controller, and obtaining a standardized vector through industrial lexicon matching and intention classification; inputting an industrial large language model for reasoning processing, and generating an AMR execution scheme; carrying out distributed coordination and task allocation on the AMR cluster to form a control instruction sequence; and the motion controller executes monitoring, processes exceptions and outputs a feedback strategy. Accurate classification and semantic understanding of complex industrial instructions are achieved, the processing capacity of professional knowledge in the industrial field is improved, and meanwhile high-precision real-time state monitoring is achieved.

The invention relates to an automatic driving large model training optimization method based on multi-scene data balance. Comprising the following steps: (1) constructing a real vehicle high-speed driving scene library; (2) constructing a visual language automatic driving large model, training by adopting an iterative training framework based on a real vehicle high-speed driving scene library, designing a multi-task joint loss function and a weight adaptive adjustment strategy, realizing multi-task target balance, and obtaining a trained visual language automatic driving large model; (3) dynamic simulation is carried out for the automatic driving working condition, and a high-fidelity simulation data test set is constructed based on simulation data; and (4) performing hyper-parameter optimization and lightweight processing on the trained visual language automatic driving large model according to the high-fidelity simulation data test set to obtain a scene data balanced automatic driving large model. According to the method, the large model reasoning speed is increased, and resource occupation is reduced, so that the judgment capability of the large model on dynamic working conditions and high-risk scenes is remarkably improved.

A real-time system for the multimodal generation of virtual reality scenes based on artificial intelligence for the creation of immersive three-dimensional environments from natural language narratives, consisting of: a speech capture module configured to continuously record a user's spoken narrative via one or more directional microphones, preprocesses the captured signal by noise reduction and temporal alignment, and outputs a digital speech stream; A speech-to-text processing unit that is operationally coupled to the speech capture module and configured for real-time speech recognition using a continuous neural transformer model. The unit is trained to transcribe natural language utterances into structured text data while maintaining contextual continuity throughout the evolving narrative. a semantic interpretation processing unit that is communicatively linked to the speech recognition unit and configured to perform natural language understanding techniques to extract contextual entities, spatial references, temporal relationships, and object attributes from the transcribed narrative; the engine includes a large language model that is fine-tuned for spatial reasoning tasks; a scene graph generation module configured to transform the interpreted semantic data into a structured, hierarchical representation that defines nodes for identified entities and edges for corresponding relationships, with each node associated with metadata describing geometry, position, orientation, texture, and linking attributes between objects; a multimodal image-language model processor coupled with the scene graph generation module, wherein the processor is configured to retrieve, adapt, or synthesize appropriate three-dimensional elements from a pre-trained visual-lexical embedding space and align these elements with their semantic and spatial definitions derived from the scene graph; a scene assembly and rendering controller configured to create a cohesive virtual scene from the aligned assets, perform real-time rendering using a GPU-accelerated ray tracing pipeline, and produce a stereoscopic visual output that corresponds to the evolving narrative; A head-mounted virtual reality visualization device connected to the rendering engine and configured to display the generated immersive environment to the user in real time. The device features motion sensors and inside-out tracking cameras to detect head and body movements, dynamically updating viewing angles and perspective within the rendered scene; and a bidirectional feedback module integrated into the head-mounted device and connected to the semantic interpretation processing unit; the module is configured to interpret corrective commands, gestures, or supplementary comments from the user to refine or modify specific scene elements without interrupting the real-time visualization; The system continuously updates the virtual scene as the narrative develops, ensuring temporal synchronization between speech input and rendered output below a defined latency threshold, thus enabling a natural, dialogic construction of complex three-dimensional virtual environments.

The invention discloses a dynamic intention recognition and GraphRAG fusion method and system for open domain questions and answers, and relates to the technical field of intelligent and natural language processing. The method comprises the following steps: constructing and dynamically updating a heterogeneous knowledge graph; performing dual-stage intention recognition on user query; subtasks are disassembled based on an intention result, and cooperative execution is carried out through multiple agents; service standardization integration is realized by means of a model context protocol; and closed-loop optimization is carried out in combination with user feedback. The system comprises an image knowledge base, an intention recognition module, an agent collaboration module, an MCP service integration module and a feedback optimization module. The system is a novel open domain question-answering system integrating structured knowledge modeling, dynamic intention understanding, intelligent collaborative reasoning and standardized service interaction, and through fusion of structured knowledge modeling and intelligent collaborative reasoning, question-answering accuracy and complex intention understanding ability are improved, system adaptability and efficiency are enhanced, multi-system collaboration is promoted, and the system has a wide application prospect. The method is suitable for scenes such as knowledge services and intelligent assistants.

The invention relates to a multi-modal data and RAG technology fused power design knowledge base construction method, and belongs to the technical field of power software development. The method comprises the following steps: carrying out collection and information extraction on multi-source heterogeneous original data; the method comprises the following steps of: constructing a multi-dimensional knowledge element structure containing parameters, specifications and case relationships by carrying out classification, specialized and precise processing and cross-modal association on data; based on a vector, graph and relational database mixed storage architecture, semantic vector efficient retrieval, knowledge graph relation management and business data synchronization are achieved respectively; and a dynamic optimization result is subjected to hybrid retrieval, a dual-drive reasoning mechanism outputs compliance conclusions and bases, and a retrieval enhancement generation service ensures that output contents conform to specifications. And systematic management and intelligent application of the electric power design knowledge are realized.

The invention relates to the technical field of numerical control machine tools, in particular to a five-axis machining center real-time thermal error compensation system based on digital twinning and a medium. Firstly, a data acquisition unit is used for acquiring physical actual measurement data in real time; then, the digital twin processing unit constructs a model set used for representing the thermal dynamic behavior of the five-axis machining center, and the model set is composed of a plurality of virtual thermal model members; thirdly, fusing physical measured data into the model set through a data assimilation algorithm to dynamically correct the state of each model member and predict the thermal error of the machine tool; the prospective compensation decision-making unit receives the thermal error prediction result output by the digital twinning processing unit and generates a prospective compensation instruction; and finally, the compensation execution unit issues the prospective compensation instruction to a numerical control system of the five-axis machining center for real-time adjustment. The control precision of real-time thermal error compensation of the five-axis machining center can be improved.

The invention relates to the technical field of artificial intelligence chips, and provides an operator optimization method, electronic equipment, a storage medium and a program product, and the method comprises the steps: determining the block size of operator data in each dimension based on the batch size and mask mode of the operator data and the hardware parameters of computing equipment, each dimension comprises a batch dimension and a sequence length dimension; segmenting the operator data based on the block size of the operator data in each dimension to obtain a plurality of data blocks; and distributing the calculation tasks corresponding to the plurality of data blocks to a plurality of processing units on the calculation equipment, and performing parallel execution on the calculation tasks corresponding to the plurality of data blocks based on the plurality of processing units. According to the method and the device, the data are segmented in the batch dimension and the sequence length dimension at the same time, so that when the data batch is small, a plurality of processing units on the computing equipment can also participate in computing at the same time, hardware resources are prevented from being idle, the utilization rate of the hardware resources is improved, and the overall computing efficiency is improved.

The invention belongs to the field of artificial intelligence, and provides a dialogue interaction system based on multi-mode emotion perception and knowledge graph dynamic enhancement. A user edge terminal obtains multi-modal data, a lightweight Transform fusion network is adopted, the multi-modal data is converted into a fusion feature vector through cross-modal attention fusion and dynamic weight adjustment, and the fusion feature vector and historical conversations in a preset round are compressed in real time; the cloud service platform inputs the compressed data into DKGE, mining and fusing feature vectors and entity knowledge and emotional relations implied in historical dialogues in real time in the dialogue interaction process to update a dynamic knowledge graph, performing knowledge enhancement processing based on the dynamic knowledge graph, and constructing an initial reply prototype of the current round of interaction of the user; inputting the fusion feature vector and the updated dynamic knowledge graph into a dialogue strategy model, and determining a response strategy and a knowledge calling direction of the current round of dialogue; and the edge terminal generates real-time interaction reply information according to the initial reply prototype, the response strategy and the knowledge calling direction.

The invention relates to the technical field of information retrieval, and discloses a multi-modal fusion and reinforcement learning collaborative retrieval enhancement generation method and system. The method comprises the following steps: receiving an original query input by a user, and generating a sub-query based on a large language model in combination with a multi-modal context of a current iteration step; forming a current state in combination with the sub-query and the multi-modal context, modeling a retrieval enhancement generation task as a Markov decision process, and adaptively selecting an optimal action from a predefined action set in the current state by utilizing a large language model according to a decision strategy; executing a corresponding multi-modal retrieval operation according to the optimal action, fusing the obtained multi-modal information, generating an intermediate answer or a final answer of the sub-query, and updating a multi-modal context by using the intermediate answer; off-line training optimization is carried out on the large language model through imitation learning and a calibration chain, and decision strategies and sub-queries are inferred online through the model after fine adjustment. According to the invention, more efficient and accurate complex query processing is realized.

The invention discloses a multi-modal data real-time identification and cooperative processing system based on edge computing and federated learning. The multi-modal data real-time identification and cooperative processing system comprises a cloud center coordination node, a plurality of edge computing nodes, a cross-modal encryption engine, a federated learning controller and a model updating verification module. The cloud center coordination node executes federated learning model aggregation and dynamic task allocation, and generates a cross-modal encryption strategy; and the edge computing node is configured with a multi-modal data acquisition module, a local model training unit and a co-processing gateway to realize multi-modal data acquisition and local processing. The system encrypts vision, acoustics and text data by using differentiated algorithms such as spatial confusion, frequency domain permutation and homomorphic encryption; the federated learning controller carries out multi-modal feature fusion, hierarchical encryption and dynamic networking at the edge node; and the model updating verification module performs aggregation updating after ensuring parameter consistency by using secure multi-party calculation. According to the method, real-time processing and privacy protection of multi-modal data are realized, and the data co-processing efficiency is improved.

The invention discloses a cross-domain computing task processing method, a program product, equipment and a medium, and relates to the technical field of cloud computing. The method comprises the following steps: determining a computing node for executing a cross-domain computing task in a cross-domain collaborative scene, and generating an identity certificate bound with the computing node based on a hardware credible state of the node; collecting security policies of a plurality of management domains for conflict detection and decision, and generating a conflict-free session policy; scheduling the cross-domain computing task to a target computing node meeting a preset credible requirement, and issuing a revocable dynamic session token; and recording the operation information of the cross-domain calculation task in the whole life cycle as an audit event, and generating a chained audit log which is linked by the hash value and is digitally signed through a preset verifiable audit interface. By means of the technical scheme, it can be ensured that the whole life cycle of any computing task meets the closed-loop safety requirements of identity credibility, permission controllability, execution propriability and behavior traceability in the complex distributed computing environment.