155 results about "Semi-structured data" patented technology

The invention relates to an enterprise-level simulation knowledge graph construction method based on multi-modal data integration, and belongs to the technical field of knowledge graphs. The method comprises the following steps: integrating structured data, semi-structured data and unstructured data through a multi-modal data warehouse; performing knowledge extraction on the semi-structured data and the non-structured data to obtain entities and relationships, and performing knowledge fusion; storing the fused entities and relationships by using a graph database, and constructing a simulation knowledge graph; a vector database is embedded in combination with a simulation knowledge graph, semantic extension search is realized through multi-modal joint search, similar cases are searched through a simulation result graph, and a simulation scheme comparison matrix is automatically generated. The multi-modal data is effectively integrated, the comprehensiveness and accuracy of knowledge graph construction are improved, more powerful, efficient and intelligent support is provided for simulation analysis of enterprises, and the enterprises can be assisted in rapidly making scientific decisions in complex and changeable business scenes.

A method for product fault root cause analysis based on knowledge graphs includes (i) acquiring product-associated design data, wherein the design data comprises semi-structured data and / or unstructured data, (ii) acquiring product-associated production data, wherein the production data comprises structured data, (iii) performing information extraction on the design data to acquire structured design data, (iv) performing a knowledge fusion of knowledge generated separately based on the production data and the structured design data to construct a target knowledge graph, and (v) and performing product fault root cause analysis tasks based on the target knowledge graph.

The invention relates to the technical field of data processing, in particular to a knowledge graph construction method for multi-source heterogeneous data of power transformation main equipment and a related system.The method comprises the steps that multi-source heterogeneous data collection is carried out, and for structured database data, a database connection tool is adopted, and needed data is obtained by writing SQL query statements; analyzing the semi-structured data by using a corresponding analysis library to extract key information, and converting the key information into a structured data format; for unstructured text data and webpage data, a natural language processing technology and a web crawler technology are adopted; according to the knowledge graph construction method for the multi-source heterogeneous data of the power transformation main equipment, the data utilization rate is increased, and the data can be enhanced and supplemented by introducing a data enhancement algorithm in secondary data processing.

A system for interactive searches based on user queries data and a plurality of Large Language Models (LLMs) including a processor of a Human-Machine Interface (HMI) server node configured to host a network of LLMs and at least one machine learning module (ML) and connected to at least one user-entity node over a network and a memory on which are stored machine-readable instructions that when executed by the processor, cause the processor to: receive a search request input data from the at least one user-entity node; evaluate, by a first dedicated LLM, relevance of the search request input data by discerning between primary and secondary information; responsive to evaluation by the first dedicated LLM, derive classifying features from the primary and secondary information and generate a feature vector based on the classifying features; ingest the feature vector into the ML module configured to extract additional search parameters from a predictive search model based on historical search data associated with the at least one user-entity node; dissect, by a second dedicated LLM, the search request input data and the additional search parameters to separate the data into qualitative and quantitative criteria elements based on the primary and the secondary information; transform, by a third dedicated LLM, the quantitative criteria elements into structured queries for a database; process, by a fourth dedicated LLM, the qualitative criteria elements by searching through a semi-structured data repository; and synthesize, by the fifth LLM, processed search findings into a succinct human-language summary.

The invention discloses a high-performance AIOT real-time data processing method and device based on Flink, Kafka and Dores, and the method comprises the steps: constructing an end-to-end hierarchical architecture, capturing multi-source database change data in real time through Flink CDC, and analyzing semi-structured data in combination with Apache NiFi, and generating a standardized data stream; the method comprises the following steps: realizing message routing based on KafkaTopic division of a service domain, ensuring the consistency of a data structure by using Schema Registry, and optimizing the transmission efficiency through a dynamic compression algorithm; an Flink real-time computing engine is adopted to implement window aggregation, asynchronous dimension table association and incremental state persistence strategies, and the resource fluctuation and state management problems in streaming computing are solved; time pre-partitioning and intelligent cold and hot layering are achieved through a Dores storage layer, the query performance is improved in combination with materialized views and multi-level indexes, and finally a full-link closed loop of collection-transmission-calculation-storage-management is formed. According to the invention, the problems that the real-time performance and the consistency are difficult to balance and the system complexity is high in the prior art are solved.

The invention relates to a mixed expert multi-model task processing method and system based on an AI Agent scene. The method comprises the following steps: constructing a knowledge block set for unstructured data and semi-structured data; constructing an associated document vector set of topN similarity; user input and a reverse decoding document are spliced, and user tasks are divided by using LLM to generate subtask sets; constructing a task allocation data set # imgabs0 # based on the scoring model and the expert model pool; and the dependency aggregation # imgabs1 # for generating the subtask answers is used as a final user task result. According to the method and the system, fine-grained division and route selection of the model pool are carried out on user tasks, so that the accuracy, diversity and authenticity of answers required by users can be effectively improved, and the multi-model parallel utilization rate of an Agent system is enhanced.

Systems and methods for Artificial Intelligence (AI)-driven computer system utilities for generating dynamic User-Interface (UI) elements and loading parsed semi-structured data into a structured data schema, which may then be readily queried utilizing known end-user tools.

The invention discloses a method for performing product fault root cause analysis based on a knowledge graph, which comprises the following steps: obtaining design data associated with a product, the design data comprising semi-structured data and / or non-structured data; production data associated with the product is obtained, and the production data comprises structured data; performing information extraction on the design data to obtain structured design data; performing knowledge fusion on knowledge generated based on the production data and the structured design data to construct a target knowledge graph; and executing a product fault root cause analysis task based on the target knowledge graph.

The invention discloses a personalized learning system based on multiple agents and retrieval enhancement generation. The system comprises a knowledge extraction agent, a retrieval enhancement generation module, a planning teaching agent, a knowledge consolidation agent and a test evaluation agent. The knowledge extraction agent extracts document information from the original learning materials and converts the document information into semi-structured data; the retrieval enhancement generation module is used for storing the semi-structured data into a vector library in a vector form, retrieving information in the vector library according to a request sent by a user, and inputting enhancement information combined by the retrieval information and the request into a downstream agent; and the planning teaching agent, the knowledge consolidation agent and the test evaluation agent are respectively used for constructing a knowledge graph according to the received information to generate a personalized teaching plan, generating exercise questions to consolidate learned knowledge, evaluating a learning effect and providing feedback suggestions. The learning efficiency, the learning effect and the learning experience of the learner can be remarkably improved, and personalized learning can be met.

The invention provides an agricultural park energy supply and consumption system regulation and control strategy generation method driven by a knowledge graph, and the method comprises the steps: extracting time series data feature indexes of structured data, completing the interval tagging processing, and forming a structured data feature tag library according to the complex and diversified scheduling operation demands of an agricultural park energy supply and consumption system; and combining knowledge extraction, fusion and storage of other semi-structured data and non-structured data to construct an agricultural park energy supply and consumption system knowledge graph. According to the method, the input feature information matrix is formed on the basis of the characterization processing of the to-be-optimized operation scene of the agricultural park energy supply and consumption system and in combination with the basic information, so that the corresponding historical strategy stored in the knowledge graph is retrieved layer by layer from the to-be-optimized operation scene, the system information visualization and the interactive query are realized, and meanwhile, the user experience is improved. Intelligent generation of the regulation and control strategy of the energy supply and consumption system in the agricultural park is completed, and support is provided for intelligent decision making of the energy supply and consumption system in the agricultural park.

The present invention discloses a standardized alarm process method, system, electronic device and storage medium, including obtaining alarm data indicators in alarm information; converting alarm data indicators into normalized indicators through a normalization algorithm; fusing normalized indicators with semi-structured data in corresponding alarm information to form a standardized alarm data structure; designing business tags based on systems or businesses, designing response tags based on notification methods and upgrade strategies, and adding business tags and response tags to standardized alarm data structures; receiving alarm event streams; filling business templates with data in alarm event streams based on tag matching rules; generating contextualized alarm data streams based on business templates. The present invention realizes a standardized alarm process method and system through multi-dimensional tool integration in multi-cloud and hybrid environments, and drives the deepening of alarm capabilities based on multi-algorithms, thereby improving the uniformity, efficiency and response speed of alarm management.

The invention provides a resume matching method and device, and is suitable for the technical field of data processing. The method comprises the following steps: analyzing and mining resume unstructured data, resume semi-structured data, resume structured data, post demand unstructured data, post demand semi-structured data and post demand structured data to obtain resume implicit information and post demand implicit information; according to a resume matching calculation parameter, a resume matching parameter optimization step length, resume unstructured data, resume semi-structured data, resume structured data, resume implicit information, post demand unstructured data, post demand semi-structured data, post demand structured data and post demand implicit information; and generating resume matching information. The matching accuracy of the resumes and the post requirements is effectively improved, the resume matching reliability and rationality are enhanced, and therefore the recruitment efficiency and effectiveness are improved.

A system and method are provided for generating a cybersecurity behavioral graph from a log files and / or other telemetry data, which can be unstructured or semi-structured data. The log files are applied to a machine learning (ML) model (e.g., a large language model (LLM)) that generates / extract from the log files entities and relationships between said entities. The entities and relationships can be constrained using a cybersecurity ontology or schema to ensure that the results are meaningful to a cybersecurity context. A graph is then generated by mapping the extracted entities to nodes in the graph and the relationships to edges connecting nodes. To more efficiently extract the entities and relationships from the data file, an LLM is used to generate regular expressions for the format of the log files. Once generated, the regular expressions can rapidly parse the log files to extract the entities and relationships.

The invention provides an intelligent sound image file arrangement method based on an event knowledge graph. The method comprises the following steps: acquiring sound image file data; for structured data in the sound image archive data, directly defining main attributes, other attributes and relationships of archive entities, and completing knowledge extraction; for semi-structured data in the sound image file data, knowledge extraction is completed through rule matching; for unstructured data in the sound image file data, converting audio and video data into unstructured text data, and intelligently extracting entities, relationships and events by utilizing an NLP algorithm and combining with a constructed file event Schema to complete knowledge extraction; and performing knowledge fusion on the knowledge extracted from the structured data, the semi-structured data and the unstructured data, and obtaining a triple based on the knowledge fusion to construct a visual knowledge graph. By constructing the event knowledge graph, archive personnel can be helped to catalog and index the archives more quickly and accurately.

The invention relates to an electric power personalized master data management system combined with a multi-tenant management mode. The electric power personalized master data management system comprises a tenant management layer, an intelligent master data service layer, an intelligent storage layer and an interface layer, the tenant management layer is used for registering, distributing and authenticating tenants, providing unique identifiers of the tenants, and supporting tenant-level permission and personalized configuration management; the intelligent master data service layer comprises an intelligent data cleaning module, a master data intelligent modeling module and an anomaly detection module; the intelligent storage layer is used for storing main data by adopting a relational database MYSQL; the MongoDB is used for storing unstructured or semi-structured data; and the interface layer provides a standardized API (Application Program Interface) to support integration of multiple systems, ensures identity verification and authorization of all requests through a zero-trust method, and minimizes a trust radius. Efficient storage, cleaning, analysis and application of the master data are achieved, and the master data management quality and efficiency are effectively improved.

The invention relates to a supplier behavior real-time sensing and monitoring method and device and electronic equipment. The method comprises the steps of collecting and preprocessing supplier multi-source heterogeneous data in real time, dynamically capturing a Binlog of a MySQL database through a Flink CDC framework, obtaining service behavior data in real time, collecting text log data through a Flume framework, and transmitting network behavior data in real time. Integrating the structured data, the unstructured data and the semi-structured data, and constructing a unified data transmission channel; constructing a multi-dimensional user portrait; comparing the semantic similarity between the bidding file and the bidding requirement, and triggering compliance early warning when the matching degree is lower than a threshold value; analyzing a supplier cooperation network, and marking a cluster group with abnormal transaction frequency and irrelevant business fields as a cross-bidding risk; and analyzing the access behavior sequence, and detecting a malicious access pattern to generate a malicious access alarm. According to the supplier behavior monitoring system and method, real-time performance, accuracy and multidimensional performance of supplier behavior monitoring are achieved through the architecture of multi-source heterogeneous data real-time integration, multi-dimensional portrait construction and intelligent risk identification.

The invention relates to the technical field of knowledge questions and answers, in particular to a large language model knowledge question and answer method and system fused with a multi-modal knowledge graph. The method comprises the following steps: acquiring structured and semi-structured data and unstructured multi-modal data in an enterprise business scene; capturing data change through a real-time synchronization technology and executing targeted preprocessing to ensure the consistency of data quality and modal characteristics; constructing a dynamically updated multi-modal knowledge graph based on the preprocessed data, and integrating a multi-modal embedded model, a reordering model and a fine-tuned generative large model to obtain a knowledge storage-feature matching-semantic generation integrated architecture; according to the method, the defect of knowledge lag of a traditional system is overcome, and seamless fusion of the AI capability and the enterprise core business process is realized.

The invention discloses a multi-modal medical data fusion and analysis method, which is applied to the technical field of medical informatization and comprises the following steps: acquiring multi-modal medical data; wherein the multi-modal medical data comprises structured data, unstructured data and semi-structured data generated in the diagnosis and treatment process of the patient; structural data features are extracted by adopting an optimized Z-score standardization formula; using sparse attention CNN to extract unstructured data features; deepening diagnosis and treatment semantic association by adopting a Transform encoder and entity embedding fusion form, and extracting semi-structured data features; multi-modal data are fused through closed-loop design of coding dimension reduction integration, decoding reconstruction verification, exclusive loss optimization and feature standardization output, and a feature relevance verification mechanism is fused; and matching the fused multi-modal data with a data analysis task, and performing task analysis and mining. According to the invention, efficient and reliable technical support is provided for medical digital transformation and precise diagnosis and treatment.

To provide a method for product fault root cause analysis based on knowledge graphs.SOLUTION: A method disclosed herein comprises acquiring product-associated design data including semi-structured data and / or unstructured data, acquiring product-associated production data including structured data, performing information extraction on the design data to acquire structured design data, performing a knowledge fusion of knowledge generated separately based on the production data and the structured design data to construct a target knowledge graph, and performing product fault root cause analysis tasks based on the target knowledge graph.SELECTED DRAWING: Figure 4

The invention discloses a method for root cause analysis of product defects based on a knowledge graph, comprising: capturing the design data associated with the product, wherein the design data includes semi-structured and / or unstructured data; capturing the production data associated with the product, wherein the production data includes structured data; performing information extraction from the design data to obtain structured data; performing knowledge fusion of the knowledge generated from the production data and the structured design data, respectively, to create a target knowledge graph; and performing the tasks for root cause analysis of product defects using the target knowledge graph.

The invention discloses a semi-structured data compression storage method and device, equipment and a storage medium. Comprising the following steps: acquiring original semi-structured data, and analyzing the original semi-structured data to generate a mapping table; determining a data structure of the column storage file according to the mapping table, and creating header information of the column storage file according to the data structure; and compressing and storing the original semi-structured data into the column storage file based on the header information. By adopting the column storage and the specific compression mode, the value of the specific field can be read without decompressing the whole segment of data, so that the data reading efficiency is greatly improved. The reading inconvenience caused by integral compression is avoided, the data is more flexible and convenient to use, and the readability of the data is improved. The characteristics of the semi-structured data are fully utilized for compression, the compression effect better than that of a traditional general compression algorithm can be achieved, the storage space is saved, and the hardware and time cost of enterprises in the aspects of data storage and processing is reduced.

Disclosed in the present disclosure are a blockchain-based data management method, apparatus, and system and an electronic device. The data management method comprises: on the basis of a plurality of metadata types, generating a plurality of corresponding sub-directories, wherein the plurality of metadata types include an unstructured data type, a semi-structured data type, and a structured data type; on the basis of the metadata types of data, determining data identifiers and storage locations of the data, wherein the storage locations of the unstructured data type, the semi-structured data type, and the structured data type are different; and at least on the basis of the plurality of subdirectories and the data identifiers, storing and managing the data.

The invention provides a health risk grading evaluation method and system based on dynamic feature recognition, and relates to the technical field of data processing, and the method comprises the steps: 1, collecting multi-source heterogeneous data in real time through an information platform, and employing an automatic classification engine to analyze a data structure change to generate a feature grading tag; distributing the data to heterogeneous storage areas according to the feature classification labels, wherein the heterogeneous storage areas comprise a structured data area, a semi-structured data area and an unstructured data area; deploying a dynamic probe group in each type of storage partitions, and capturing a basic evolution index set in real time; and 2, based on the feature grading labels, matching encryption strategies for different grading data, dynamically adjusting a transmission protocol based on the basic evolution index set, and when the feature grading labels meet preset conditions, executing de-identification processing, and outputting a to-be-analyzed data stream. According to the method, through full-process dynamic treatment and closed-loop optimization, accurate recognition, hierarchical management and control and adaptive protection of health data risks are realized, and data security and utilization efficiency are both considered.

The invention provides a mobile terminal intelligent map construction and self-adaptive visualization method and system, and the method comprises the steps: collecting various source data, such as texts, images, audios and videos, through technologies such as crawlers; processing and integrating the obtained different types of heterogeneous data to form structured data and semi-structured data; training a knowledge extraction model, and reading structured and semi-structured data to perform entity recognition and relationship extraction; writing an automatic atlas construction program, and storing a construction result in a database; and acquiring related parameters of the equipment to write a self-adaptive visual program, and displaying associated information of the entity. According to the method, efficient atlas construction and self-adaptive visual display at the mobile terminal are realized through integrated method design and intelligent technical means. The method not only improves the construction efficiency and quality of the knowledge graph, but also provides a more convenient and intuitive information acquisition mode for the user, and improves the understanding and analysis efficiency of the user on complex information.

The invention discloses a pharmaceutical enterprise data backup system and method and a storage device. The pharmaceutical enterprise data backup system comprises a pharmaceutical enterprise data operation system, a data classification method and a data backup storage device. The data operation system comprises a first master server PSM1, a data input device, a data classification module DCM, a first conversion module TCM1, a second server SVR2, a second conversion module TCM2, a data backup management module DBMM and a data memory DM. The data classification method comprises the following steps: S1.1, defining structured data of the pharmaceutical enterprise as a first type of data, S1.2, defining semi-structured data of the pharmaceutical enterprise as a second type of data, S1.3, defining unstructured data of the pharmaceutical enterprise as a third type of data, and S1.4, defining running log type data as a fourth type of data; the memory can be a read-only memory, a magnetic disk or an optical disk. Through the technical means, the safety problem of pharmaceutical enterprise data backup can be solved.

The invention discloses a low-code process automatic arrangement method and system, and particularly relates to the technical field of automatic arrangement, and the method comprises the steps: carrying out the unified collection and verification of an external system, resources and system boundaries before deployment, generating an operation environment fingerprint with a signature, and carrying out the strong binding of the fingerprint with a process instance; process DAG modeling is completed on the visual canvas, system and semantic double validity verification is carried out, and a post-curing parameter snapshot is used as a subsequent unique parameter source; in the online simulation period, observable measurement is collected, a deviation perception index is formed, and steady selection and switching of an execution kernel are achieved by combining hysteresis and a minimum residence strategy; in a production state, a canary release strategy, a degradation strategy and a rollback strategy are used for communicating simulation and real disk, and the SLA and the key KPI are continuously monitored; normalized collection and immutable archiving are carried out on structured and semi-structured data in the operation period, an evidence chain and a minimum reproduction experiment package are generated based on hash and signature, and the comparability and traceability of cross-version results are achieved.

The invention discloses a semi-structured data analysis method and device and a medium, and the method comprises the steps: collecting semi-structured data from a preset data source, recognizing the data type of the semi-structured data, and determining an analysis mode corresponding to the semi-structured data according to the data type; wherein the analysis mode is divided into a direct analysis mode and an indirect analysis mode according to whether the data content can be directly obtained or not; on the basis of a direct analysis mode, determining an organization relationship among content elements in the semi-structured data, and reading data contents corresponding to the semi-structured data according to the organization relationship; based on the indirect analysis mode, performing region division on the semi-structured data, and obtaining data content corresponding to the semi-structured data through each data component obtained after division; and generating an embedded vector corresponding to the data content, and storing the embedded vector into a vector database to realize analysis of the semi-structured data.

The invention relates to the technical field of financial science and technology and big data analysis, and discloses a training method of a financial risk identification model based on big data, and the method comprises the following steps: S1, collecting data sources from structured data, unstructured data and semi-structured data, and carrying out the preprocessing of the data; s2, multi-modal features are extracted and fused, and a multi-modal feature vector is generated through fusion; and S3, providing transparency of a model prediction result by using an explanatory enhancement method of the model, including an SHAP value, LIME local explanation and a rule-based explanatory module. Structured data, unstructured data and semi-structured data are integrated through a multi-modal data fusion technology, data from different sources are used for providing more comprehensive feature information for a model, an Attention mechanism is adopted for weighted fusion, it is ensured that the weight of important features is higher, the model can more accurately capture complex information in a financial market, and the financial market quality is improved. And the risk identification accuracy is improved.

The invention relates to an informatization multi-service data integration method, system and device and a medium, and belongs to the technical field of service data processing. The integration method comprises the steps that a target data source is collected and initialized; structured data and semi-structured data are regularly and automatically extracted from the target data source to a source-attached data source through an ETL (Extract-Transform-Load) engine; performing digital semantic analysis on the unstructured data in the target data source, and performing partial structured processing according to a digital semantic analysis result to obtain partial structured data; taking structured data, semi-structured data and partial structured data in the source data source as a to-be-integrated data source; and performing data integration on the data in the to-be-integrated data source, synchronizing the integrated data to the short-term data center in real time, performing data aggregation, obtaining the aggregated data, and storing the aggregated data to the comprehensive data center. According to the method, data integration is effectively realized in an enterprise multi-source heterogeneous data environment, and the data management and decision-making requirements of an enterprise can be better supported.

A semantic diffusion model may generate semi-structured data using existing character image creations. Form image generation is one area of possible application. Embodiments include both training the diffusion model and using the diffusion model. The model can learn to permute and rearrange character features for different regions. Newly generated forms can be applied to train the semantic diffusion model to provide further improvements to the model's capability and generality. The model can generate high quality character-like images that incorporate geometric properties such as character locations and regions of similar meaning, which humans can check and / or interpret. Embodiments are suitable for semi-structured data such as forms, tables, and aligned keyword text generation, and resolve the issue of generating data for mixed and combined geometries and semantics. There also is applicability to hybrid or multimodality datasets, so long as the raw data can be interpreted and converted as character-like image tensors.