2570 results about "Data type" patented technology

A system and methods for processing diverse financial data types using a jointly trained Vector-quantized Variational Autoencoder (VQ-VAE) and neural upsampler. This system efficiently handles time-series, textual, sentiment, and structured tabular data through specialized encoding modules. A novel fusion module integrates these encodings, capturing cross-modal relationships via attention mechanisms and gated fusion units. The fused representation is compressed into a discrete latent space by the VQ-VAE encoder, then reconstructed and enhanced by the VQ-VAE decoder and neural upsampler, respectively. Joint training optimizes all components simultaneously, using a comprehensive loss function that balances reconstruction quality across modalities with upsampling performance. This approach enables superior data compression, reconstruction, and analysis, leveraging inter-modal correlations to improve financial forecasting, risk assessment, and decision-making. The system's ability to explore the latent space facilitates generation of new, synthetic financial scenarios for robust model testing and strategy development.

The invention provides a science and technology finance platform data sharing method, which comprises the following steps: extracting key field information and a metadata label from a data stream of a science and technology finance platform, the key field information comprising data content and a source identifier; analyzing a data type characteristic value according to the metadata label, and determining a data type and a corresponding sensitivity level by adopting a logistic regression model; querying a source credibility rule base based on the source identifier, and determining a risk level of the data source; according to the sensitivity level, the risk level and the receiver permission identifier, calculating to obtain a dynamic sensitivity score and generating a dynamic sensitivity tag; according to the risk level mapping table, in combination with the sensitivity level, determining a final desensitization mode of the field; and performing format verification on the processed data stream, transmitting the processed data stream, and recording execution information of the dynamic sensitivity label and the desensitization strategy.

The invention discloses a fusion method for automatic coprocessing of multi-source heterogeneous data, which comprises the following steps of: analyzing the format and semanteme of the multi-source heterogeneous data to obtain metadata features of each data source, the metadata features comprising data types, coding modes and semantic tags, and generating a standardized metadata description set; according to the metadata description set, a pre-established protocol template library is adopted, an applicable template is matched, an initial unified exchange protocol is generated, and the initial unified exchange protocol comprises a format conversion rule and a semantic mapping relation; and if the format conversion rule of the initial unified exchange protocol cannot adapt to the metadata features of the newly added data source, dynamically updating the protocol template through a machine learning algorithm to obtain an exchange protocol adaptive to the new data source.

The invention relates to the technical field of natural language processing, in particular to a semantic synthesis and text vectorization-based data lake metadata governance method, which comprises the following steps of: performing vector coding of field content keywords and adjacent context words on the basis of field names, data types and description information of data sources in a data lake; and extracting semantic co-occurrence groups of the fields in the differentiated contexts, identifying distances between semantic vectors, and judging whether the distances are in a similar field grouping range or not to obtain a field semantic collection quantity. According to the method, a semantic co-occurrence relation is extracted through field keywords and context vector coding, a semantic hierarchical structure is constructed, the field division precision and abstract ability are improved, the consistency is judged in combination with a context semantic structure, a high-frequency alternative path is adjusted to optimize a semantic structure, and word vector similarity and a structure retention rate are fused to realize field merging; the continuity and the accuracy of a treatment structure are improved, and the intelligence and the consistency of metadata treatment in the data lake are enhanced.

The invention discloses an electric power field knowledge question-answering system construction method based on a large language model, and relates to the field of electric power field knowledge question-answering, and the method comprises the steps: judging the data type of electric power field knowledge, and carrying out the processing of the electric power field knowledge according to the judgment result through matching with a processing technology, and generating an entity relation triple; constructing a power field knowledge graph; optimizing the power field knowledge graph based on the attention network, outputting an answer causal path of the fault problem by using the optimized power field knowledge graph, and marking a confidence score of the answer causal path; and inputting the solution causal path and the confidence score into a language model to obtain a fault question answering result, and optimizing the question answering result according to the consistency of the fault question answering result and the power field knowledge graph. According to the method, on the premise that the fault diagnosis logic is rigorous and the result is traceable, knowledge in large-scale unstructured literatures in the power industry is activated, so that accurate question and answer services can be provided for operation and maintenance personnel in real time.

A system and method for processing and generating multimodal data using a combination of Vector Quantized Variational Autoencoder (VQ-VAE) and Latent Transformer architectures. The system efficiently handles diverse data types including time-series, textual, sentiment, and structured tabular data through specialized encoding modules. A novel fusion module integrates these encodings, capturing cross-modal relationships. The fused representation is compressed into a discrete latent space, processed by a latent transformer, and then reconstructed and enhanced. This approach enables superior data compression, reconstruction, analysis, and generation of synthetic scenarios.

The invention discloses an intelligent data analysis method and system based on an industry large model, and relates to the technical field of data analysis, and the method comprises the steps: collecting business event data, carrying out the feature vector extraction according to the data type, calculating a zoom dot product attention score matrix through employing a self-attention mechanism, and carrying out the weighted output of a multi-modal fusion vector, constructing an industry knowledge database to determine an embedded vector; and calculating a relevancy value, screening a combination of the multi-modal fusion vector and the knowledge fragment, and forming a gating input vector to carry out fusion vector calculation. According to the method, unified expression and correlation mining of heterogeneous information such as structured data, text information and images can be realized through a multi-modal feature fusion and knowledge fragment retrieval scheme, the attention degree of each modal feature can be adaptively adjusted according to a service scene through a self-attention mechanism and a model, information complementation is realized, and the accuracy of information retrieval is improved. And automatically highlighting the multi-modal dimension most related to the current task.

The invention discloses a data quality intelligent auditing system and method based on a dynamic rule base, and belongs to the technical field of data auditing, and the system comprises a rule base construction module which is used for analyzing business scene parameters through a scene analysis unit according to business scene demands and data type features to generate a rule configuration instruction; the multi-source monitoring engine module is connected to the rule base construction module and is used for collecting multi-source data in real time and loading corresponding checking rules; the automatic verification execution module is used for executing normalized quality verification on the multi-source data based on the verification rule base; and the feedback optimization module analyzes a rule hit rate and a false alarm rate in a verification result through a reinforcement learning algorithm, and dynamically iteratively updates a rule threshold value and a logic combination in the rule base. By constructing a full-automatic process of rule generation, execution, feedback and updating, the problems that a traditional system depends on manual intervention, response is slow, the industry average rule updating period is 3-7 days, and real-time updating is achieved through the scheme are solved.

The invention relates to the technical field of data fusion, and discloses a multi-source heterogeneous data fusion method and system based on edge computing, and the method comprises the steps: obtaining a heterogeneous data stream, carrying out the data type recognition and data feature extraction, and obtaining an original feature set; according to the original feature set, unifying feature dimensions and adjusting a time reference to obtain time sequence vector data; according to the time sequence vector data, filling the feature value of the missing time point to obtain a multi-modal feature; performing block storage on the multi-modal features, verifying the synchronism of adjacent modals, distributing modal synchronization weight coefficients, and finally generating a fusion feature matrix; according to the fused feature matrix, identifying and filtering redundant feature dimensions, establishing a feature association map, and executing feature merging to obtain a simplified feature matrix; according to the simplified feature matrix, feature importance scores are calculated, sorting weight coefficients are arranged and distributed in a descending order according to the scores, and a multi-modal fusion semantic vector is generated. The method improves the accuracy of data analysis and decision.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a machine learning task execution and feedback method, device, equipment and medium. And performing semantic reasoning by using a pre-training language model, generating a decision result in combination with an entity relationship associated with a task type and a data type in the knowledge graph, executing a corresponding machine learning task, converting a task execution process and an execution result into natural language feedback information, and outputting the natural language feedback information to a user side. According to the method, natural language input is structured into machine learning task description, large model semantic reasoning and knowledge graph knowledge association are combined, an executable decision result is generated, automatic closed loop from task recognition, model selection to result feedback is achieved, the user operation threshold is lowered, and the model development efficiency and the interaction intelligence level are improved.

A non-transitory, processor-readable medium storing instructions that when executed by a processor, cause the processor to receive data artifacts, encode the artifacts to a standard data type, and compute, for each artifact, a hash function. The hash functions and encoded documents are stored in a first database. The processor is caused to tokenize the encoded artifacts, to produce tokens associated with natural-language identifiers extracted from the encoded artifacts. The processor is caused to transform, using an embedding model, the tokens to produce vectors that are stored in a second database and classified based on categories. The second database is configured to be queried to perform a semantic search in response to receiving a request from a user operating a user compute device. The processor is caused to retrieve, from the semantic search, a subset of vectors from the second database to be displayed on the user compute device.

Various embodiments of the technology described herein cause an LLM to intelligently process data based on a user query and a schema determined for a data set. Certain embodiments programmatically leverage an LLM and utilize its output based on a user query. In this manner, data is processed without the LLM having to access an entire data set, and instead only utilizes information associated with the user query and the schema. The schema comprises a textual description, such as a string of alphanumeric characters, that describes the data, data types, and / or data structure of the data set. Embodiments of the technology described herein are performed by an LLM interface layer separate from a user device layer and an LLM layer. The LLM interface layer is positioned between an LLM abstraction layer and an application layer by which a user can interface with the LLM interface layer.

The invention discloses a mineral resource intelligent prediction method and system based on multi-source heterogeneous data fusion and deep learning, and the method comprises the steps: collecting and preprocessing multi-source heterogeneous data, and carrying out the standardization processing to form a structured data set; multi-source heterogeneous data fusion: realizing data layer space registration and feature layer weight dynamic allocation through an attention mechanism multi-modal fusion module, and outputting a high-dimensional metallogenic feature vector; constructing a CNN-LSTM mixed deep learning model and completing initialization training, and outputting an initial mineralization probability graph; and establishing a dynamic updating engine, performing model increment training based on transfer learning, correcting the mineralization probability through positive and negative sample reinforcement learning in combination with a newly added data type, and outputting a time sequence dynamic mineralization probability graph. According to the method, mineralization probability dynamic evaluation and risk quantitative updating are realized, the prediction precision and the model updating efficiency are improved, the method is adaptive to a multi-stage exploration scene, and accurate real-time support is provided for exploration decision making.

The invention discloses a method for simulating and forecasting flood in a cold highland area based on hydrological and hydrodynamic coupling, and belongs to the technical field of disaster forecasting. The method specifically comprises the following steps: S1, multi-source basic data collection and preprocessing: collecting multi-type and multi-scale basic data for a target cold and cold mountainous area drainage basin; and S2, deep learning correction and fusion of the satellite rainfall data: aiming at the local overestimation and underestimation problems of the satellite rainfall data, a deep learning algorithm is adopted to carry out hour scale correction and fusion. Four types of core data of satellite remote sensing, reanalysis, ground observation and geographic space are collected, total factors of'rainfall-runoff-terrain-underlying surface 'required by flood simulation in the cold and cold mountainous area are covered, simulation one-sidedness caused by lack of data types in traditional modeling is avoided, rainfall and runoff abnormal values are eliminated by adopting a 3-sigma criterion, data formats and spatial-temporal scales are unified, and the modeling efficiency is improved. A standardized data set is formed, and interference of abnormal values, format incompatibility and space-time mismatching on subsequent model input is avoided.

The invention discloses a virtual power plant energy scheduling method and system under a local data abnormal condition, and the method comprises the steps: recognizing a data type and an incidence relation related to abnormal data through an incidence matrix after the abnormal data of a virtual power plant are collected and recognized; and calculating an abnormal data correction value based on the association relationship function and the real-time association data, and performing weighted average on the abnormal data correction value and the original abnormal data to obtain correction data. And then calculating an abnormal data time difference and a correlation deviation degree, and distributing a credibility weight for the corrected data. And finally, calculating the correlation degree between the scheduling parameter of the preset scheduling scheme and the weighted correction value, and selecting the scheme with the highest correlation degree as a virtual power plant resource scheduling scheme. By implementing the technical scheme provided by the invention, the scheduling accuracy in a data exception state is improved.

The invention discloses a privacy protection data sharing and cooperative training method and system based on federated learning. The method comprises the steps of receiving software development log data, adaptively judging the sensitivity degree according to a data type, dynamically adjusting noise disturbance intensity according to the sensitivity degree to perform data desensitization, and generating a sensitivity index; selecting a feature extraction strategy, extracting time sequence correlation features from the desensitization data, constructing a dynamic graph structure with a weight, and obtaining a time sequence feature vector through iterative fusion; calculating the time sequence correlation of the time sequence feature vector to obtain a data quality score, and setting a contribution weight based on the quality score to perform parameter aggregation; combining sensitivity indexes with data quality scores to construct a security sharing domain, decoupling global training parameters into knowledge fragments in the domain, formulating a recombination rule, and selectively acquiring the required knowledge fragments by all parties for local training. According to the method, deep collaboration is realized on the premise of protecting data privacy, and the collaboration training effect is improved.

The invention belongs to the technical field of multi-source heterogeneous data quality and analysis, and relates to a data management and intelligent analysis method oriented to power multi-source heterogeneous. According to the method, ontology model alignment, temporary ontology generation and multi-mode semantic embedding technologies are adopted, semantic unification of cross-data types is achieved, meanwhile, a spatio-temporal joint indexing mechanism is established, geographic coordinates and timestamps are bound and stored, efficient multi-dimensional query is supported, and data integration efficiency and semantic consistency are improved; the problem of poor adaptability of a fixed threshold value is solved by dynamically adjusting the operation data safety threshold value and evaluating the abnormal confidence coefficient, the detection precision and credibility of the power multi-source heterogeneous data anomaly are improved, the anomaly detection accuracy is improved, and the false alarm rate is reduced; by constructing an abnormity confirmation logic and combining equipment parameters, environment data and communication states, reasons such as equipment faults, environment interference and communication abnormity can be accurately positioned, root causes can be quickly positioned, and time can be shortened.

The invention provides a fault diagnosis question-answering system based on a multi-modal knowledge graph and a large language model, and the system comprises a time sequence data processing module which is used for carrying out the noise reduction of time sequence data, carrying out the feature extraction of the time sequence data after noise reduction, and representing a time sequence data body through a plurality of feature data; the multi-modal knowledge graph module is used for constructing a multi-modal knowledge graph, and the multi-modal knowledge graph comprises a text modal knowledge graph and a time sequence data modal knowledge graph; the large model fine tuning module is used for performing fine tuning on the large model by adopting a LoRA model based on different fault problems and corresponding solutions so as to obtain a question and answer large model; and the RAG question and answer module is used for enhancing the user question based on the multi-modal knowledge graph, inputting the enhanced statement into the fine-tuned large model, and outputting a result. According to the method, the fault diagnosis speed is increased, fault questions and answers are carried out on various data types, and the problem that a large model lacks professional domain knowledge is solved.

The invention discloses a fault diagnosis method and device, a medium and a product, and relates to the technical field of data processing. Multi-dimensional data such as text logs, time series data, topological graph structures, operation records and environmental parameters are obtained, various factors possibly involved when the fault occurs are included, different types of data can complement and verify each other, the fault features are accurately described, the limitation of single data type analysis is avoided, and the analysis efficiency is improved. And the accuracy of fault diagnosis is effectively improved. Then converting different types of multi-dimensional data into target feature representations, matching the target feature representations with data in a historical fault case library, and when a historical fault of which the matching degree is greater than a preset matching threshold value is found, determining a diagnosis conclusion and a repair scheme of the historical fault as a diagnosis conclusion and a repair scheme of the current fault. And each new fault does not need to be analyzed and reasoned, so that the fault diagnosis time is shortened, and the fault diagnosis efficiency is improved.

Predicting the salience of one or more data entities to a particular (target) data entity from among a plurality of data entities may comprise generating a graph of the plurality of data entities and a machine-learned model architecture that predicts the salience of the one or more data entities output by the machine-learned model architecture using the graph. For example, the machine-learned model architecture may comprise a first machine-learned model for generating an embedding using the content of the target data entity, a second machine-learned model for generating a vector using the data type indicated by the target data entity, and a third machine-learned model (e.g., a graph neural network or other feed-forward neural network) for generating a contextual representation of the target data entity to which other contextual representations associated with the plurality of data entities may be compared (e.g., using Euclidean distance, cosine similarity, dot product).

The invention discloses an industrial internet data processing method based on big data, which comprises the steps of designing a visual data processing flow arrangement tool, allowing a user to flexibly define logic and steps of data processing, and realizing common data processing operation through a predefined data processing operator library in the flow arrangement process. A user is allowed to select a required data processing component through dragging operation through a visual configuration interface, and corresponding data types and processing conditions are set to form a complete data processing flow chart; acquiring data type information configured by a user, performing primary condition filtering judgment on the data according to the information, and screening out original data meeting conditions; judging whether data needs to be dynamically calculated along with time or not according to whether conditions are changed or not; a real-time data processing engine is introduced, and data are collected, cleaned, converted and analyzed in real time; through a visual interface, NLP interaction, real-time processing and intelligent scheduling, the flexibility of data processing is improved.

The invention discloses an intelligent data management system and method based on multi-source data collection, and relates to the technical field of data processing. Multi-source data from multiple data sources are obtained, an original data set with credibility marks is generated, data records in the original data set are classified according to data types, and for time series data, data records in the original data set are classified according to credibility marks. Extracting trend features by applying a segmented multi-model time sequence feature extraction framework; for the data of the unstructured data, extraction of unstructured features is carried out through a feature extraction model based on a multi-layer perceptron; for structured data, a knowledge graph technology is adopted to establish an adaptive association relationship, association relationship data is generated, trend features, unstructured features and the association relationship data are used as input of a data value evaluation model, and data asset performance evaluation of dynamic weight is generated; and intelligent management and value evaluation of the multi-source heterogeneous data are realized.

The invention provides a self-adaptive semantic-driven data set field matching method and system, and the system comprises a data preprocessing module which is used for carrying out the cleaning, standardization and preliminary analysis of an input data set, and extracting a field name, a data type, a field description and a data sample; the deep semantic representation modeling module is used for constructing a field-level semantic representation vector; the multi-level similarity calculation module is used for comprehensively calculating the grammatical similarity, the semantic similarity and the statistical similarity among the fields, dynamically adjusting the weight of the similarity of each level by adopting a weighted fusion algorithm, and generating a comprehensive similarity matrix; and the matching result management and application module is used for generating a field matching mapping table and a fusion suggestion according to the comprehensive similarity matrix. According to the method, high-precision automatic matching of data set fields is realized by fusing deep semantic understanding, multi-dimensional similarity calculation and incremental adaptive learning, and the efficiency and accuracy of data set fusion are remarkably improved.

The invention relates to a data verification method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining to-be-verified data, and determining a verification priority corresponding to the to-be-verified data based on a data type of the to-be-verified data; dividing the to-be-verified data into a plurality of to-be-verified data sets according to the verification priority; determining a target verification mechanism corresponding to each to-be-verified data set based on the verification priority corresponding to each to-be-verified data set, and determining a target verification strategy corresponding to each to-be-verified data set based on a corresponding relationship between the verification mechanism and a preset verification strategy; and according to the target verification mechanism and the target verification strategy corresponding to each to-be-verified data set, performing data consistency verification on each to-be-verified data to obtain a data verification result. By adopting the method, the efficiency of data verification can be improved.

An artificial intelligence digital assistant that retrieves and consolidates data from diverse sources is described. After a system initiates a response to an information request received from a user, a language model generates a high-dimensional vector that represents the information request from the user. The language model identities data access tools that access corresponding data types identified for the information request. The system executes the identified data access tools that retrieve, from corresponding data sources, data values from corresponding data records represented by corresponding unique high-dimensional vectors that have similarities, which exceed a confidence level, to the high-dimensional vector that represents the information request. The system sends a consolidation of the data values in the response to the user.

The invention belongs to the technical field of multi-modal intelligent data management, and discloses a multi-modal business data co-processing and intelligent conversion method, which comprises the steps of fusing multi-source data, coupling cross-modal time sequence association, detecting image-text semantic conflicts, and generating an RAG semantic unit and a space-time alignment semantic packet. Constructing a four-dimensional priority label to divide resource pool categories, dynamically allocating resource pool computing resources, generating a low-redundancy feature set, triggering a forced preemption strategy during resource competition, and synchronously generating a resource arbitration log; injecting a dynamic rule node according to a data type, executing confidence arbitration in combination with a conflict arbitration identifier, generating business decision content and generating a rule correction decision packet; constructing a full-link semantic thinking chain, synchronously capturing a system operation track, and aligning to generate a dual-channel tracing report; identifying residual sensitive fields, triggering hot update to load corresponding rules, quantizing effectiveness to generate compliance reports, reversely updating an industry rule base, and optimizing a resource allocation strategy.

A method for using machine learning models to automatically cluster data into meaningful populations and groups and to identify hidden patterns and structures in such data is provided. The method includes: receiving first information that relates to a group of entities; analyzing the first information with respect to a predetermined set of parameters and a predetermined set of data types; selecting, based on a result of the analysis, a first machine learning (ML) model from among a predetermined set of ML models; and using the selected first ML model to generate a report that includes second information that relates to at least one data cluster identified by the first ML model from the first information.

A system and method for compressing and restoring multi-modal data utilizing a variational autoencoder to enable homomorphic compression techniques is disclosed. Multi-modal input data, comprising at least two different data types, is compressed into a unified latent space using an encoder network of a multi-modal variational autoencoder. Homomorphic operations are performed on compressed data in the latent space. The latent space compressed data is decompressed using a decoder network of the multi-modal variational autoencoder. The system utilizes modality-specific layers and cross-modal attention mechanisms to effectively process diverse data types. The homomorphic operations enable performing computations while the data is in a compressed form, preserving results of those operations in the decompressed output. This approach allows for efficient storage, transmission, and analysis of multi-modal data while maintaining privacy and data integrity across different modalities.

The invention discloses a data index classification and intelligent recommendation method and device based on deep learning and a medium, belongs to the technical field of deep learning and big data, and aims to solve the technical problem of how to improve data management efficiency and precision and improve user experience. According to the technical scheme, the method comprises the following steps: data collection and preprocessing: determining a data source and a data type, collecting data corresponding to the data source and the data type, and preprocessing the collected data to obtain metadata information; constructing a deep learning model: selecting a corresponding deep learning model according to the property of the task and the data type, and carrying out training, parameter adjustment, optimization and deployment on the deep learning model; data index classification and label distribution; modeling according to user requirements and performing intelligent recommendation; designing a user interface and displaying an application; and optimizing and iterating the system.

The invention discloses a multi-field business collaborative management system and method based on a block chain, and relates to the technical field of business collaborative management. Standardization rules are introduced, the field name, the data type and the unique identifier of data are processed in a unified manner, and by labeling sources, timestamps and operation records, the business collaborative management efficiency is improved; an anomaly detection algorithm based on statistical distribution, machine learning and rule verification is adopted, consistency verification and repeated data cleaning are combined, key features are extracted through principal component analysis, sparse coding and a deep learning embedding model, semantic alignment and feature fusion of texts, numerical values, time sequences and image data are achieved through multi-modal modeling, and the feature fusion of the text, numerical values, time sequences and image data is achieved. Cross-modal unified data representation is generated; a knowledge graph technology and a block chain technology are combined, a semantic relationship between fields is constructed through semantic alignment, an alliance chain is used for recording data circulation and operation, dynamic access authority management is realized in combination with an intelligent contract, and transparency and non-tampering performance of data are guaranteed.