121 results about "Semi-structured data" patented technology

The invention relates to an enterprise-level simulation knowledge graph construction method based on multi-modal data integration, and belongs to the technical field of knowledge graphs. The method comprises the following steps: integrating structured data, semi-structured data and unstructured data through a multi-modal data warehouse; performing knowledge extraction on the semi-structured data and the non-structured data to obtain entities and relationships, and performing knowledge fusion; storing the fused entities and relationships by using a graph database, and constructing a simulation knowledge graph; a vector database is embedded in combination with a simulation knowledge graph, semantic extension search is realized through multi-modal joint search, similar cases are searched through a simulation result graph, and a simulation scheme comparison matrix is automatically generated. The multi-modal data is effectively integrated, the comprehensiveness and accuracy of knowledge graph construction are improved, more powerful, efficient and intelligent support is provided for simulation analysis of enterprises, and the enterprises can be assisted in rapidly making scientific decisions in complex and changeable business scenes.

A system for interactive searches based on user queries data and a plurality of Large Language Models (LLMs) including a processor of a Human-Machine Interface (HMI) server node configured to host a network of LLMs and at least one machine learning module (ML) and connected to at least one user-entity node over a network and a memory on which are stored machine-readable instructions that when executed by the processor, cause the processor to: receive a search request input data from the at least one user-entity node; evaluate, by a first dedicated LLM, relevance of the search request input data by discerning between primary and secondary information; responsive to evaluation by the first dedicated LLM, derive classifying features from the primary and secondary information and generate a feature vector based on the classifying features; ingest the feature vector into the ML module configured to extract additional search parameters from a predictive search model based on historical search data associated with the at least one user-entity node; dissect, by a second dedicated LLM, the search request input data and the additional search parameters to separate the data into qualitative and quantitative criteria elements based on the primary and the secondary information; transform, by a third dedicated LLM, the quantitative criteria elements into structured queries for a database; process, by a fourth dedicated LLM, the qualitative criteria elements by searching through a semi-structured data repository; and synthesize, by the fifth LLM, processed search findings into a succinct human-language summary.

The invention discloses a high-performance AIOT real-time data processing method and device based on Flink, Kafka and Dores, and the method comprises the steps: constructing an end-to-end hierarchical architecture, capturing multi-source database change data in real time through Flink CDC, and analyzing semi-structured data in combination with Apache NiFi, and generating a standardized data stream; the method comprises the following steps: realizing message routing based on KafkaTopic division of a service domain, ensuring the consistency of a data structure by using Schema Registry, and optimizing the transmission efficiency through a dynamic compression algorithm; an Flink real-time computing engine is adopted to implement window aggregation, asynchronous dimension table association and incremental state persistence strategies, and the resource fluctuation and state management problems in streaming computing are solved; time pre-partitioning and intelligent cold and hot layering are achieved through a Dores storage layer, the query performance is improved in combination with materialized views and multi-level indexes, and finally a full-link closed loop of collection-transmission-calculation-storage-management is formed. According to the invention, the problems that the real-time performance and the consistency are difficult to balance and the system complexity is high in the prior art are solved.

The invention relates to a mixed expert multi-model task processing method and system based on an AI Agent scene. The method comprises the following steps: constructing a knowledge block set for unstructured data and semi-structured data; constructing an associated document vector set of topN similarity; user input and a reverse decoding document are spliced, and user tasks are divided by using LLM to generate subtask sets; constructing a task allocation data set # imgabs0 # based on the scoring model and the expert model pool; and the dependency aggregation # imgabs1 # for generating the subtask answers is used as a final user task result. According to the method and the system, fine-grained division and route selection of the model pool are carried out on user tasks, so that the accuracy, diversity and authenticity of answers required by users can be effectively improved, and the multi-model parallel utilization rate of an Agent system is enhanced.

Systems and methods for Artificial Intelligence (AI)-driven computer system utilities for generating dynamic User-Interface (UI) elements and loading parsed semi-structured data into a structured data schema, which may then be readily queried utilizing known end-user tools.

The invention discloses a personalized learning system based on multiple agents and retrieval enhancement generation. The system comprises a knowledge extraction agent, a retrieval enhancement generation module, a planning teaching agent, a knowledge consolidation agent and a test evaluation agent. The knowledge extraction agent extracts document information from the original learning materials and converts the document information into semi-structured data; the retrieval enhancement generation module is used for storing the semi-structured data into a vector library in a vector form, retrieving information in the vector library according to a request sent by a user, and inputting enhancement information combined by the retrieval information and the request into a downstream agent; and the planning teaching agent, the knowledge consolidation agent and the test evaluation agent are respectively used for constructing a knowledge graph according to the received information to generate a personalized teaching plan, generating exercise questions to consolidate learned knowledge, evaluating a learning effect and providing feedback suggestions. The learning efficiency, the learning effect and the learning experience of the learner can be remarkably improved, and personalized learning can be met.

The invention provides a resume matching method and device, and is suitable for the technical field of data processing. The method comprises the following steps: analyzing and mining resume unstructured data, resume semi-structured data, resume structured data, post demand unstructured data, post demand semi-structured data and post demand structured data to obtain resume implicit information and post demand implicit information; according to a resume matching calculation parameter, a resume matching parameter optimization step length, resume unstructured data, resume semi-structured data, resume structured data, resume implicit information, post demand unstructured data, post demand semi-structured data, post demand structured data and post demand implicit information; and generating resume matching information. The matching accuracy of the resumes and the post requirements is effectively improved, the resume matching reliability and rationality are enhanced, and therefore the recruitment efficiency and effectiveness are improved.

The invention relates to a supplier behavior real-time sensing and monitoring method and device and electronic equipment. The method comprises the steps of collecting and preprocessing supplier multi-source heterogeneous data in real time, dynamically capturing a Binlog of a MySQL database through a Flink CDC framework, obtaining service behavior data in real time, collecting text log data through a Flume framework, and transmitting network behavior data in real time. Integrating the structured data, the unstructured data and the semi-structured data, and constructing a unified data transmission channel; constructing a multi-dimensional user portrait; comparing the semantic similarity between the bidding file and the bidding requirement, and triggering compliance early warning when the matching degree is lower than a threshold value; analyzing a supplier cooperation network, and marking a cluster group with abnormal transaction frequency and irrelevant business fields as a cross-bidding risk; and analyzing the access behavior sequence, and detecting a malicious access pattern to generate a malicious access alarm. According to the supplier behavior monitoring system and method, real-time performance, accuracy and multidimensional performance of supplier behavior monitoring are achieved through the architecture of multi-source heterogeneous data real-time integration, multi-dimensional portrait construction and intelligent risk identification.

The invention relates to the technical field of knowledge questions and answers, in particular to a large language model knowledge question and answer method and system fused with a multi-modal knowledge graph. The method comprises the following steps: acquiring structured and semi-structured data and unstructured multi-modal data in an enterprise business scene; capturing data change through a real-time synchronization technology and executing targeted preprocessing to ensure the consistency of data quality and modal characteristics; constructing a dynamically updated multi-modal knowledge graph based on the preprocessed data, and integrating a multi-modal embedded model, a reordering model and a fine-tuned generative large model to obtain a knowledge storage-feature matching-semantic generation integrated architecture; according to the method, the defect of knowledge lag of a traditional system is overcome, and seamless fusion of the AI capability and the enterprise core business process is realized.

The invention discloses a multi-modal medical data fusion and analysis method, which is applied to the technical field of medical informatization and comprises the following steps: acquiring multi-modal medical data; wherein the multi-modal medical data comprises structured data, unstructured data and semi-structured data generated in the diagnosis and treatment process of the patient; structural data features are extracted by adopting an optimized Z-score standardization formula; using sparse attention CNN to extract unstructured data features; deepening diagnosis and treatment semantic association by adopting a Transform encoder and entity embedding fusion form, and extracting semi-structured data features; multi-modal data are fused through closed-loop design of coding dimension reduction integration, decoding reconstruction verification, exclusive loss optimization and feature standardization output, and a feature relevance verification mechanism is fused; and matching the fused multi-modal data with a data analysis task, and performing task analysis and mining. According to the invention, efficient and reliable technical support is provided for medical digital transformation and precise diagnosis and treatment.

The invention discloses a semi-structured data compression storage method and device, equipment and a storage medium. Comprising the following steps: acquiring original semi-structured data, and analyzing the original semi-structured data to generate a mapping table; determining a data structure of the column storage file according to the mapping table, and creating header information of the column storage file according to the data structure; and compressing and storing the original semi-structured data into the column storage file based on the header information. By adopting the column storage and the specific compression mode, the value of the specific field can be read without decompressing the whole segment of data, so that the data reading efficiency is greatly improved. The reading inconvenience caused by integral compression is avoided, the data is more flexible and convenient to use, and the readability of the data is improved. The characteristics of the semi-structured data are fully utilized for compression, the compression effect better than that of a traditional general compression algorithm can be achieved, the storage space is saved, and the hardware and time cost of enterprises in the aspects of data storage and processing is reduced.

The invention provides a health risk grading evaluation method and system based on dynamic feature recognition, and relates to the technical field of data processing, and the method comprises the steps: 1, collecting multi-source heterogeneous data in real time through an information platform, and employing an automatic classification engine to analyze a data structure change to generate a feature grading tag; distributing the data to heterogeneous storage areas according to the feature classification labels, wherein the heterogeneous storage areas comprise a structured data area, a semi-structured data area and an unstructured data area; deploying a dynamic probe group in each type of storage partitions, and capturing a basic evolution index set in real time; and 2, based on the feature grading labels, matching encryption strategies for different grading data, dynamically adjusting a transmission protocol based on the basic evolution index set, and when the feature grading labels meet preset conditions, executing de-identification processing, and outputting a to-be-analyzed data stream. According to the method, through full-process dynamic treatment and closed-loop optimization, accurate recognition, hierarchical management and control and adaptive protection of health data risks are realized, and data security and utilization efficiency are both considered.

The invention discloses a pharmaceutical enterprise data backup system and method and a storage device. The pharmaceutical enterprise data backup system comprises a pharmaceutical enterprise data operation system, a data classification method and a data backup storage device. The data operation system comprises a first master server PSM1, a data input device, a data classification module DCM, a first conversion module TCM1, a second server SVR2, a second conversion module TCM2, a data backup management module DBMM and a data memory DM. The data classification method comprises the following steps: S1.1, defining structured data of the pharmaceutical enterprise as a first type of data, S1.2, defining semi-structured data of the pharmaceutical enterprise as a second type of data, S1.3, defining unstructured data of the pharmaceutical enterprise as a third type of data, and S1.4, defining running log type data as a fourth type of data; the memory can be a read-only memory, a magnetic disk or an optical disk. Through the technical means, the safety problem of pharmaceutical enterprise data backup can be solved.

The invention discloses a low-code process automatic arrangement method and system, and particularly relates to the technical field of automatic arrangement, and the method comprises the steps: carrying out the unified collection and verification of an external system, resources and system boundaries before deployment, generating an operation environment fingerprint with a signature, and carrying out the strong binding of the fingerprint with a process instance; process DAG modeling is completed on the visual canvas, system and semantic double validity verification is carried out, and a post-curing parameter snapshot is used as a subsequent unique parameter source; in the online simulation period, observable measurement is collected, a deviation perception index is formed, and steady selection and switching of an execution kernel are achieved by combining hysteresis and a minimum residence strategy; in a production state, a canary release strategy, a degradation strategy and a rollback strategy are used for communicating simulation and real disk, and the SLA and the key KPI are continuously monitored; normalized collection and immutable archiving are carried out on structured and semi-structured data in the operation period, an evidence chain and a minimum reproduction experiment package are generated based on hash and signature, and the comparability and traceability of cross-version results are achieved.

The invention relates to the technical field of financial science and technology and big data analysis, and discloses a training method of a financial risk identification model based on big data, and the method comprises the following steps: S1, collecting data sources from structured data, unstructured data and semi-structured data, and carrying out the preprocessing of the data; s2, multi-modal features are extracted and fused, and a multi-modal feature vector is generated through fusion; and S3, providing transparency of a model prediction result by using an explanatory enhancement method of the model, including an SHAP value, LIME local explanation and a rule-based explanatory module. Structured data, unstructured data and semi-structured data are integrated through a multi-modal data fusion technology, data from different sources are used for providing more comprehensive feature information for a model, an Attention mechanism is adopted for weighted fusion, it is ensured that the weight of important features is higher, the model can more accurately capture complex information in a financial market, and the financial market quality is improved. And the risk identification accuracy is improved.

A semantic diffusion model may generate semi-structured data using existing character image creations. Form image generation is one area of possible application. Embodiments include both training the diffusion model and using the diffusion model. The model can learn to permute and rearrange character features for different regions. Newly generated forms can be applied to train the semantic diffusion model to provide further improvements to the model's capability and generality. The model can generate high quality character-like images that incorporate geometric properties such as character locations and regions of similar meaning, which humans can check and / or interpret. Embodiments are suitable for semi-structured data such as forms, tables, and aligned keyword text generation, and resolve the issue of generating data for mixed and combined geometries and semantics. There also is applicability to hybrid or multimodality datasets, so long as the raw data can be interpreted and converted as character-like image tensors.

The invention discloses a multi-mode complex event detection and optimization method for semi-structured data, and relates to the field of data stream processing. According to the method, mode matching of the semi-structured data is realized by using a multi-mode normal tree. In order to enable a regular tree mode to be matched with a semi-structured data stream, the patent adopts a method of disassembling the regular tree mode into two parts, namely a tree mode and a regular mode, and realizing matching step by step. The tree mode designs a node relationship in which a user is interested into a logic tree, and a TwigList algorithm is used for matching data meeting conditions in a data stream and storing the data. The regular pattern defines a behavior sequence in which a user is interested, and state transition is realized by using NFA. And carrying out NFA processing on the data stream which is successfully matched with the tree pattern to obtain a final result.

The subject technology receives a first semi-structured object. The subject technology iterates through a list of fields specified by a target object type. The subject technology, for each field, determines whether a field with a same name is present in the first semi-structured object. The subject technology, in response to the field being found in the first semi-structured object, converts a value of the field to a target field type according to defined type conversion rules. The subject technology stores the converted value in a unified representation comprising a data structure that stores both structured and semi-structured data types. The subject technology processes a query using the unified representation.

The invention discloses a document processing method and device, equipment and a storage medium, and relates to the technical field of data processing, the disclosed document processing method comprises the following steps: obtaining a to-be-processed document, and identifying a text region and an image region in the to-be-processed document; performing text extraction on the text area to obtain vector text content; performing character recognition on the image area through a plurality of optical character recognition models and obtaining a recognition result; determining a confidence coefficient weight in the recognition result according to the character recognition result corresponding to each optical character recognition model; fusing each character recognition result with the vector text content according to each confidence coefficient weight to obtain a fused text content; and outputting the structured data or the semi-structured data according to the fused text content, thereby being compatible with the input of the mixed type document, realizing the processing of the mixed type document containing the text region and the image region, and effectively meeting the user requirements.

A vector conversion method, apparatus and system for semi-structured data, and a device and a medium, which relate to the technical field of big data processing. The method comprises: reading a first semi-structured data file, and acquiring data to be converted (S302); parsing a pre-configured rule file, and determining a correlation between said data and a field to be extracted in the rule file, wherein the rule file is a file comprising a first mapping relationship between said field and a data structure of vector storage middleware, and a second mapping relationship between said field and a vector conversion algorithm (S304); and on the basis of the correlation and the second mapping relationship, calling the vector conversion algorithm to perform vector conversion on said data, and performing storage in the vector storage middleware on the basis of the first mapping relationship (S306). The efficiency of vector conversion of semi-structured data can be improved by means of a rule file, and storage is performed in a corresponding vector database, thereby realizing the persistence of the vector database; and direct calling is performed during use, thereby achieving convenience and fast performance.

The invention discloses a zero sample learning multi-source bill call ticket data integration method, system and product, and the method comprises the following steps: importing multi-source bill call ticket data, and classifying the imported multi-source bill call ticket data according to the data format type, dividing the multi-source bill call ticket data into structured data, semi-structured data and unstructured data; extracting structured data features to obtain structured features so as to create a data field, and importing corresponding data; performing analysis and feature extraction on the semi-structured data to obtain semi-structured features, matching the semi-structured features with the structured features to form similar features, and importing corresponding data; extracting the unstructured data to obtain unstructured features, matching the unstructured features with the structured features or similar features, and importing corresponding data; and acquiring data in the data field according to feature matching. According to the invention, zero sample learning automatic data import of the multi-source bill call ticket can be realized.

The invention provides an electric power material detection system based on an improved association rule and a knowledge graph, and belongs to the field of electric power material detection. According to the system, an association analysis model for equipment detection is established by using an association rule algorithm, a hidden association relationship between unqualified detection items is mined, and a judgment basis for the disqualification of the related items is provided. In addition, in combination with information such as a material and equipment detection report and expert knowledge, knowledge such as entities, attributes and relationships is extracted from semi-structured data by adopting a natural language processing technology, a material and equipment detection management knowledge graph is constructed in a graph database Neo4j, a systematic material detection knowledge base is provided, network visualization is carried out, and the material and equipment detection management knowledge graph is established. And information query and management are facilitated. According to the invention, the detection efficiency of the electric power material equipment is improved, and increasing detection requirements are met.

A method (300) of storing semi-structured data (12U) includes receiving user data (12) comprising semi-structured user data from a user (10) of a query system (150). The method includes receiving an indication (14) that the semi-structured user data fails to include a fixed schema. In response, the method further includes parsing the semi-structured user data into a plurality of data paths (210) and extracting a data type (220) associated with each respective data path of the plurality of data paths. The method additionally includes storing the semi-structured user data as a row entry in a table (204) of a database in communication with the query system, wherein each column value associated with the row entry corresponds to a respective one of the plurality of data paths and the data type associated with the respective data path.

The present invention relates to a method for recommending labels and label instances, and belongs to the field of computer data processing technology. In response to the problems of label management and intelligent label management of massive test data, the present invention proposes a method for recommending labels and label instances. The method mainly includes: collecting structured, unstructured and semi-structured data, constructing ontology concepts according to the types of test data, and forming a label library; then, using different methods to extract equipment entities and entity relationships from multimodal data such as images, texts, audio, video, and paper, and constructing a label instance library; using methods such as rule mapping and natural language processing to map the relationship between labels and label instances; finally, mining user personal information and label usage information, combining personal information and label information, and forming intelligent recommendations based on labels and label instances.

The invention relates to a pneumonia data intelligent collection method based on big data. The method comprises the following steps: obtaining pneumonia-related data to obtain to-be-collected data; performing data preprocessing on the to-be-collected data to obtain processed data; classifying the processed data through a pre-trained CNN neural network, and dividing the processed data into structured data, unstructured data and semi-structured data; and storing the structured data in a relational database, and storing the unstructured data and the semi-structured data in a non-relational database. According to the pneumonia data intelligent collection method and device based on the big data, on one hand, the problem of high labor cost of manual processing is solved, and on the other hand, due to the fact that the CNN neural network model is adopted for classification, the problem that final summarized and collected data are disordered, and follow-up analysis is affected is solved.

The application discloses a cardiovascular chronic disease data management method based on artificial intelligence, relates to the technical field of artificial intelligence medical management, and comprises the following steps: collecting patient multi-element heterogeneous medical data, normalizing and converting structured and semi-structured data, extracting key features of unstructured medical image data and vectorizing and encoding the key features, and generating a standardized cardiovascular chronic disease comprehensive data cube; calling a pre-trained multi-modal cardiovascular risk assessment model to analyze the data cube, and generating a comprehensive risk assessment report containing a risk level, a contribution factor, an evolution trend and an individualized early warning threshold; combining a clinical path knowledge base to generate an individualized health management plan, continuously collecting plan execution feedback data, dynamically updating the data cube, and iteratively optimizing the assessment report and the management plan. The method realizes multi-element heterogeneous data regularization, enriches risk assessment dimensions, realizes dynamic adaptation of a health management plan, and improves the standardization and individualization level of cardiovascular chronic disease data management.

The invention relates to the technical field of knowledge extraction, in particular to a domain knowledge extraction method and system for special material safety standards. The method comprises the following steps: collecting special material safety standard data, preprocessing the data, and manually classifying standard terms; carrying out ontology design; extracting structured and semi-structured data knowledge; extracting knowledge of the unstructured data; performing preference fine tuning and optimization on the entity type; the domain knowledge extraction system for the special material safety standard comprises a data collection module, a classification module, an ontology design module, a first data knowledge extraction module and a second data knowledge extraction and preference fine adjustment module. Through the mode, the corpus tagging efficiency of the professional field can be improved, the dependence on artificial resources is reduced, and the conditions that local deployment is difficult, the professional term understanding of the model professional field is insufficient, and the knowledge extraction ability is insufficient are relieved.

The invention discloses a data warehouse construction and application method based on multi-source data fusion, and relates to the technical field of data fusion, and the method comprises the steps: multi-source data integration layer construction: obtaining structured, unstructured and semi-structured data through a heterogeneous data collection module, carrying out the automatic preprocessing and standardization processing of the data, and obtaining a multi-source data integration layer; the standardization processing comprises a unified naming rule, a coding rule and a data type; designing a core model of the data warehouse: constructing a star model based on a business theme, the star model comprising at least one fact table and a plurality of associated dimension tables, the fact table storing business index data, and the dimension tables storing descriptive attribute data; implementing an ETL automatic process: configuring a data extraction and conversion rule and a scheduling strategy through a visual ETL tool, automatically loading the data processed by the integration layer to a star model, and monitoring a task state in real time; and intelligent query and application recommendation.

The application discloses an analysis system and an analysis method for automatically detecting semi-structured data quality problems, the analysis method takes JSON data in semi-structured data as a specific research object, and comprises the following steps: a parsing mode module is used for parsing semi-structured data into an aggregated schema tree; a data quality problem monitoring module is used for automatically detecting potential data quality problems of data according to the aggregated schema tree and a data quality space; a visualization generation module is used for visualizing the aggregated schema tree and the data quality problems, helping users to interactively find and view the data quality problems; and a data cleaning module is used for user configuration and solution of the data quality problems. The system can help users to quickly and effectively locate and clean the data quality problems of semi-structured data.

This invention relates to the field of pipeline inspection and management technology, specifically a long-distance pipeline inspection and management system and method, including a centralized control center, edge processing equipment, cameras, a cloud platform, and manual inspection terminals; the cameras are installed at designated locations for real-time monitoring of target objects. The advantages of this invention are: by introducing a data governance module and adding three-dimensional tags containing timestamps, location coordinates, and associated object IDs to multi-source data, and simultaneously establishing a core index master ledger and associated ledgers bound together by unique data IDs, this system achieves automatic association and fusion of unstructured, structured, and semi-structured data generated during inspections. This solves the "data silo" problem caused by inconsistent data formats and scattered storage in existing technologies, providing a complete and traceable data chain for each potential hazard or equipment status, supporting full-process traceability and analysis.