774results about "Multi-dimensional databases" patented technology

The invention discloses a tunnel blasting quality evaluation and optimization method based on multi-source data fusion, and belongs to the field of tunnel blasting quality evaluation, and the method comprises the steps: collecting and preprocessing multi-source data of a tunnel blasting region; based on the preprocessed multi-source data, performing blasting quality evaluation according to local back break, a blasting contour line, average linear back break and point cloud extraction to obtain a blasting quality evaluation result; according to the blasting quality evaluation result, the blasting quality is graded, and a comprehensive blasting quality score is calculated and graded; and establishing a database containing geological parameters, surrounding rock response parameters and blasting process parameters, training through a convolutional neural network model to generate a blasting parameter optimization scheme, and dynamically adjusting blasting parameters of the next cycle according to the comprehensive blasting quality score. According to the method, the blasting parameter optimization and the quality evaluation process are closely combined to form a closed-loop system, the specific situation in the construction can be reflected in real time, and the accuracy of the blasting effect is ensured.

The invention discloses a data production and application method based on index management, and the method comprises the steps: receiving an index query request containing a target index identifier, a dimension constraint condition and a time range parameter, carrying out the analysis and semantic verification of the target index identifier based on a global index asset library, and obtaining the index definition information; generating a standardized query statement according to the index definition information, and selecting an optimal data calculation engine to execute query; performing multi-level cache optimization and parallel computing acceleration on the query process to obtain an original data set; processing the original data set in real time according to a preset analysis model to generate structured index data containing trend analysis, anomaly detection or attribution inference results; and filtering and desensitizing the data based on a fine-grained permission control strategy, only returning contents in a permission range and recording an audit log. By means of the method, unified management, efficient query and intelligent analysis of the index data are achieved, the automation level and query performance of data production are improved, and meanwhile data safety and access controllability are guaranteed.

The disclosed embodiments provide systems and methods for performing queries via an intelligent query engine. Various embodiments include receiving an input query and parsing the input query into a query representation object; generating an evaluation plan based on the query representation object, wherein the evaluation plan comprises a graph of computation nodes, each computation node specifying a granularity for grouping, associated filters, an aggregation schema, and join dependencies required for node computation, and wherein the evaluation plan is ordered to account for hierarchical dependencies among the computation nodes; translating the evaluation plan into an executable query optimized for a specific target data store, wherein the translation adapts query syntax and join structures to capabilities of a target data store; and executing the translated query on the target data store, resolving dependencies and aggregations according to the evaluation plan to generate query results satisfying the input query.

The invention relates to the technical field of information and data processing, in particular to an enterprise digitalization-oriented data management and application method, which comprises the following steps of: acquiring multi-source heterogeneous data and recording metadata; establishing a cross-department management system to standardize the treatment process; building an asset catalog based on the business section and the data field classification; formulating unified data description of a multi-level data standard system; designing a quality control rule to filter low-quality data; constructing a business index system and embedding the business index system into a business system; identifying a cross-system and cross-type association mode through a metadata association technology and a rule engine; and dynamically adjusting the treatment process by adopting a multi-objective optimization algorithm in combination with service feedback and treatment parameters. According to the method, the defects of existing data governance in comprehensiveness, depth and business adaptability are overcome, standardized governance and potential value mining of multi-source data are achieved, and effective data support is provided for intelligent decision making of enterprises.

The invention discloses a lithium battery full life cycle quality tracing system based on digital twinning, and relates to the technical field of lithium battery quality tracing, and the system comprises a physical entity layer which integrates multiple types of sensors and detection equipment to collect full-link data; the digital twinning modeling layer is used for constructing a multi-scale model to map a battery state; the data acquisition and transmission layer is used for realizing data real-time transmission preprocessing through an edge gateway and an encryption protocol; the data storage and management layer is used for establishing a data warehouse by using a distributed database and performing backup; the intelligent analysis and decision-making layer integrates an algorithm to realize anomaly detection, performance prediction and tracing; the visualization and interaction layer is used for providing a multi-terminal visualization and interaction function; and the block chain evidence storage layer constructs an alliance chain to ensure that the data cannot be tampered. According to the invention, full-life-cycle data connection and credible tracing of the lithium battery are realized, the quality problem positioning efficiency is improved, intelligent early warning and optimization capabilities are provided, the operation threshold is reduced, and industrial quality control and green development are facilitated.

The invention discloses a cultural heritage file multi-dimensional storage system, which relates to the field of cultural heritage digital protection and intelligent storage and comprises a data acquisition and processing module, a double-library management module, a multi-dimensional index retrieval engine, a security and convergence module and a backup and disaster recovery module. According to the method, structured texts and unstructured files are uniformly accessed through various interfaces, a special tool chain is adopted for format standardization, audios and videos are transcoded into an H.264 / AAC format through FFmpeg, images are subjected to resolution adjustment and compression through ImageMagick, plain texts are extracted from texts through ApacheTika and are converted into UTF-8 codes, standardized data are uniformly converted into an intermediate format, and the format of the intermediate format is converted into the intermediate format. According to the technical scheme, the method comprises the following steps of: standardizing input data by adopting a three-layer duplicate removal strategy, facilitating other modules to receive data, performing refined redundancy processing on the standardized input data by adopting a three-layer duplicate removal strategy, remarkably reducing storage redundancy, partitioning a cultural heritage data main library according to three layers of ''project type-medium type-age'', and partitioning an inheritor information library according to two levels of ''province-inheritor number prefix''.

The invention relates to a medical health big data exploration system, and the system comprises a demand understanding module which is used for converting a data exploration demand into a structured exploration task; the compliance review module is used for performing compliance review on the probing task; the code generation module is used for generating a query code according to the probing task and the target database characteristics; the query execution module is used for querying query result data corresponding to the query code from a target database according to the query code; the query result analysis module is used for performing multi-dimensional analysis on the query result data to obtain a multi-dimensional analysis result; a report generation module; and the result presentation module is used for presenting the exploration report in an interactive mode in a user interface and supporting iterative improvement of the report by providing a feedback mechanism, so that the problems of data isomerism, difficulty in quality evaluation, low exploration efficiency, high professional threshold, insufficient privacy protection and the like in the existing medical data exploration process are solved.

The invention discloses a dynamic data docking method based on a rule engine, which comprises the following steps: S1, data source registration and automatic detection: a user inputs various heterogeneous data source information through a system console, and a system automatically detects the data source type, the data structure and the connection availability and generates an adaptive connection configuration file; according to the method, a dynamic data docking method with the rule engine supporting hot updating as the core is introduced, seamless integration and flexible conversion of multi-source heterogeneous data are achieved, and the problem of business interruption caused by code modification and service restarting due to rule change in a traditional scheme is thoroughly solved; through visual rule management, a unified data model and a closed-loop verification feedback mechanism, the development and maintenance cost of the system is remarkably reduced, and meanwhile, the flexibility, reliability and traceability of data processing and the continuous self-adaptive optimization capability of the system are greatly improved.

The invention relates to the technical field of data governance, and provides a data platform metadata annotation completion method, system and equipment, and the method comprises the steps: collecting metadata basic annotations in a multi-source heterogeneous database through a standard application program interface; collecting a logic model definition document and a data standard specification document, and constructing a local knowledge base through vectorization processing; when a user triggers annotation completion, calling a local knowledge base through a large language model, and generating metadata intelligent annotations; and returning the collected basic annotation of the metadata and the generated intelligent annotation of the metadata to the user as annotation completion results. According to the data platform metadata annotation completion method, system and equipment, metadata collection normalization, multi-source knowledge fusion, knowledge base construction and intelligent annotation completion can be achieved, cost reduction and efficiency improvement are achieved, the metadata annotation quality is remarkably improved, and the use threshold is lowered.

The invention provides a network optimization data analysis method and system and electronic equipment, and belongs to the technical field of data processing, and the method comprises the following steps: writing initial column type compressed data into a column type database; the column database is used for constructing an index for column compressed data, and the initial column compressed data is obtained by formatting, analyzing and compressing original network optimization data; performing incremental aggregation on the initial column compressed data through a column database to generate initial aggregated data; after receiving an analysis query request, positioning target column compressed data and target aggregated data related to the analysis query request in the column database according to the index; and performing vectorization instruction set operation on the target column compressed data and the target aggregated data in the column database to obtain an analysis result. According to the method, efficient analysis and processing of large-scale network optimization data are realized by constructing an index-driven multi-stage materialized view incremental aggregation mechanism and a vectorization instruction set operation path.

The invention discloses a simulation data management method based on a servitization integrated architecture, and relates to the technical field of combat simulation and deduction, and the method comprises the steps: building a servitization integrated data management architecture according to a data collection layer, a data storage layer, a data management layer, a data management layer and a data service layer; based on a service integrated data management architecture, data standardization is carried out, a data model is designed, data fusion preprocessing, data integration and data analysis are carried out on collected multi-source heterogeneous data according to a data standard and the data model, data fusion and warehousing management are completed, and meanwhile balance and unification of data security and sharing are guaranteed. According to the invention, a one-stop and all-around data management solution can be provided for simulation application.

The invention relates to a remote sensing image data processing method and system based on a Spark architecture, and belongs to the technical field of remote sensing image data processing. The method comprises the following steps: converting a collected original file into a COG format, constructing a hybrid index through a spatial dimension reduction technology, extracting metadata and establishing a metadatabase; based on the hybrid index, quickly positioning a target tile through spatial query, reading a data block, and constructing a TileLayerRDD of a spatial partition in a Spark cluster; and dynamically assembling and calculating the directed acyclic graph according to the algorithm warehouse template, and submitting the directed acyclic graph to a Spark cluster for distributed memory calculation. According to the method, rapid retrieval is achieved through the COG format and the mixed index, pre-slicing is replaced with Spark distributed computing, the storage pressure is greatly reduced, and on-demand real-time analysis and high-concurrency online service of mass remote sensing data are finally achieved through micro-service and elastic resource scheduling.

The invention belongs to the technical field of artificial intelligence, and discloses a large model memory management and optimization system and method based on a vector data lake. Comprising a multi-dimensional memory vectorization coding and hierarchical management module, an intelligent retrieval and dynamic updating module, and a memory optimization strategy and interactive evaluation and cyclic adjustment optimization module based on a reflection and backtracking mechanism. According to the large model memory management and optimization system and method based on the vector data lake, a multi-dimensional and hierarchical memory storage and dynamic retrieval framework is constructed, collaborative management of long-term / short-term memory is supported, efficient reasoning, knowledge reuse and continuous learning ability of an intelligent agent in a complex dynamic task are achieved, and the system and the method have the advantages that the system and the method are simple and convenient to operate. Therefore, the robustness, the adaptability and the task migration capability of the intelligent system are remarkably improved.

The invention provides a dynamic optimization and visual decision-making system for port and navigation multimodal transport knowledge graph collaboration, and solves the problems of high transport mode conversion cost, difficult information sharing and the like. Aiming at the defects that a traditional static data warehouse (such as Hadoop) is insufficient in real-time performance, a general data fusion framework (such as Apache NiFi) is poor in field adaptability and the like, the system innovatively constructs a multi-source heterogeneous database, integrates port production, economic association, infrastructure and freight data, and realizes accurate path planning and personalized logistics scheme recommendation through an intelligent algorithm. The system adopts a React + ECharts + ShadcnUI technology stack to realize knowledge graph visualization, practice shows that the operation efficiency can be remarkably improved, the transportation cost is reduced, and technical support is provided for sustainable development of ports.

The invention relates to the technical field of big data, and provides a data verification method and device in big data cluster migration, equipment and a medium, and the method comprises the steps: obtaining a source data cluster and a target data cluster; dividing the source data cluster and the target data cluster into a plurality of data layers according to a standard data warehouse layering strategy, and binding different data comparison strategies for each data layer; establishing a fingerprint baseline table and constructing a sampling model feature library; performing data fragmentation processing on each data layer, and controlling each data layer to execute a data comparison task in parallel; and if any data layer of the target data cluster and any data layer of the source data cluster are detected to have difference, recording difference data and performing data restoration. According to the method, the source data cluster and the target data cluster are subjected to hierarchical parallel fragmentation comparison, a plurality of fragments of each layer are subjected to data comparison tasks in parallel at the same time, different comparison strategies are bound to the data layers, the comparison process can be achieved, and the overall data processing efficiency is improved.

The invention provides a heterogeneous data mapping optimization method, and relates to the technical field of software development. A JSON Schema model library containing business and voucher objects is constructed by obtaining heterogeneous source data, the construction process comprises data format analysis and feature extraction, and construction and classified storage are completed by means of DSL rules and visual editing; based on the model library, generating a mapping relation description document by utilizing a preset abstract mapping rule template and an expanded JSON Schema grammar; a mapping engine analyzes the document to construct a field-level dependency graph, data conversion is executed according to the field-level dependency graph, and an inerting execution and caching mechanism is adopted in the conversion process; and verifying the converted data and the mapping process by adopting a mode of combining static pre-verification and incremental verification, dynamically adjusting a mapping rule according to a verification result or model change, and performing version management. Through the steps, the data mapping efficiency, flexibility and accuracy are improved, and the system performance is optimized.

The invention discloses a data lake and warehouse integrated management method and system based on port traffic, and particularly relates to the technical field of data management. The method comprises the following steps: acquiring multi-source heterogeneous traffic data, executing unified timestamp labeling and format standardization to obtain preprocessed multi-source heterogeneous traffic data, and referring to a predefined business ontology and a semantic meta-model, extracting semantic features, performing hierarchical analysis and constructing a semantic mapping rule; generating a semantic drift correction parameter in combination with the historical semantic drift accumulation deviation; generating an alignment strategy by constructing a dynamic semantic alignment and correction mechanism; semantic correction and fusion are carried out based on the alignment strategy, and lake and warehouse fusion data of semantic alignment are output, so that the accuracy of lake and warehouse collaborative management and the consistency of data analysis are improved.

The invention discloses a fire-fighting spatial-temporal trajectory data processing and off-line data warehouse construction method based on a stream batch fusion architecture, and relates to the field of digital fire fighting, and the core steps of the method are as follows: after multi-source data are collected by an edge gateway, high throughput distribution and storage are realized by Kafka; real-time optimization such as track cross repair and offset correction is completed through Flink streaming calculation; performing offline batch processing on the historical data by the Spark to generate an analysis result; generating an optimal rescue path through a distributed Dijkstra algorithm in combination with road topology, real-time traffic and fire fighting truck parameters; the method comprises the following steps: realizing mixed storage by adopting ElasticSearch secondary index and HBase total storage; based on hot and cold data hierarchical management, intelligent applications such as fire-fighting commanding and dispatching, battle comment and review and the like are supported. Practice verifies that the method can shorten the fire control time of the main urban area by 60%, significantly improves the fire emergency response capability, and is suitable for large-scale fire track data scenes.

The invention discloses an enterprise digital operation real-time data analysis method based on big data, and particularly relates to the field of data analys.The enterprise digital operation real-time data analysis method comprises the steps that enterprise operation data are collected in real time through a multi-protocol adapter, abnormity is dynamically cleaned and detected, and data standardization is achieved in combination with statistical process control and BERT sentiment analysis; in the hierarchical stream processing architecture, edge nodes pre-aggregate data and filter anomalies, a center engine dynamically divides time windows to generate minute-level / second-level indexes, and a self-adaptive rolling window and dual-threshold session cutting support inventory monitoring and user behavior analysis; the incremental machine learning model updates prediction in real time based on a sliding window, screens a dynamic optimization decision tree through Hoeffering splitting conditions and feature importance, and triggers fine tuning or full training when data distribution drift is detected; and the cross-source data association engine fuses real-time and historical data, calculates a KPI anomaly score, and triggers hierarchical early warning.

The disclosure improves the efficiency and accuracy of Financial Planning & Analysis functions within large companies. This improvement is facilitated via a chatbot tool, which functions against financial systems in an organization. Key tool components include a chat interface, a presentation component, a visualization component, a forecast component, and an AI component. The AI component provides AI capabilities to the other components (chat, presentation, visualization, and forecast). Behind the scenes, the tool provides data reconciliation functions, which permits information to be gathered from multiple sources, which are reconciled to each other. Data sources include various types of databases relational and non-relational ones, as well as data cube structures. The query functions permit natural language queries and other queries through a chatbot or chat-like interface.

The invention relates to a multi-hospital-area manpower dynamic allocation decision support system based on a hierarchical response mechanism, which integrates hospital information system, electronic medical record and office automation data in real time through a multi-source data fusion module, and constructs a distributed data warehouse through standardized cleaning. During operation of the system, the data acquisition layer captures abnormal events of the courtyard area in real time, and drives the prediction engine to output early warning after cleaning and verification. And when the response threshold value is reached, the decision-making unit automatically matches the optimal scheduling scheme and synchronizes the task instruction through the mobile terminal. Full-chain tracking is implemented in the execution process, and the resource state is dynamically updated. And afterwards, multi-index scoring is carried out based on dimensions such as human efficiency and business load, strategy library iteration and model weight adjustment are driven, and a'monitoring-prediction-decision-execution-evaluation 'closed loop is formed. According to the invention, intelligent collaboration and elastic scheduling of human resources in multiple courtyards are realized, and the emergency scene response efficiency and the medical resource utilization rate are remarkably improved.

The invention discloses a data asset management system and method, and relates to the field of data management, and the system comprises a data collection layer which is used for collecting multi-source heterogeneous data; the data integration layer is used for integrating the multi-source heterogeneous data and constructing a data warehouse; the data storage layer is used for carrying out structured storage on the data in the data warehouse; the data governance layer is used for performing standardization processing and quality management on the data stored in the data storage layer to obtain target data; and the data application layer is used for performing dynamic tracking inventory and classification level-to-level management on the data assets based on the target data. Through cooperative work of five layers of architectures, full-process closed-loop management of the multi-source heterogeneous data from collection to application is realized, and the problems of difficulty in integration of the multi-source data, low management efficiency, uncontrollable quality and the like are effectively solved by the overall scheme.

Described herein are systems and methods for automatically enriching datasets in a data analytics environment, with system knowledge data. The system can operate, upon an analysis of a data set, to automatically enrich the data set. Users of data analytics environments, such as business users preparing data visualizations, may be unaware of additional data and system knowledge data that could be utilized to improve the data visualizations. The systems and methods described herein can provide an automatic enrichment of data from, for example, a knowledge repository, which can be delivered to a data analytics customer using various delivery means.

The invention provides an intelligent file merging method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence and cloud computing, in particular to the technical direction of big data. According to the specific implementation scheme, first-class files and second-class files are recognized from a to-be-processed file set, the file size of the first-class files is smaller than a target size, and the file size of the second-class files is larger than that of the first-class files; splitting the second class file into a plurality of sub-files; determining a to-be-merged file set based on the plurality of sub-files and the first type of files; and carrying out merging operation on the to-be-merged file set.

The invention discloses a data warehouse table structure generation method and device, a medium and a product, and belongs to the technical field of data warehouses. According to the method, the Excel template configured with multiple pieces of source database table structure information is obtained, table field definition is automatically analyzed, the source data type is intelligently mapped and converted into the target data type according to the source database type, meanwhile, partition key fields are automatically recognized based on the preset strategy, and finally a complete and accurate data definition language (DDL) statement is generated. According to the method, remarkable technical breakthrough and value are achieved in practical application, the table building efficiency can be improved by 90% or above, batch processing of hundreds of tables is supported, and manual errors are reduced by 95% or above; mainstream database type conversion is comprehensively supported, and various data models such as Dores are adapted; the method has the capabilities of automatic type detection and conversion, intelligent key field selection and partitioning and bucket dividing strategy optimization.

The invention belongs to the field of artificial intelligence, particularly relates to an integrated artificial intelligence analysis and modeling platform supporting large-scale data processing, and aims to solve the problems of low resource scheduling efficiency, model training reasoning splitting, data algorithm coupling, limited cross-modal modeling and the like under high-dimensional heterogeneous mass data. The platform comprises a distributed data access layer, a unified metadata governance engine, a heterogeneous computing resource scheduling center, a model life cycle management module, a multi-modal fusion modeling framework and a dynamic service orchestration interface, and realizes automatic data access governance, dynamic resource scheduling, model full-cycle management and control and cross-modal intelligent fusion. And the resource utilization rate, the modeling accuracy and the service deployment efficiency are remarkably improved.

The invention discloses an intelligent work order dynamic scheduling processing method based on multi-source data fusion. The method comprises the following steps: collecting Internet of Things data, resource data and personnel data through a data fusion layer; the intelligent decision-making layer analyzes the Internet of Things data in real time, creates a pre-diagnosis work order, or manually creates a fault work order and submits the fault work order to the intelligent decision-making layer; after a work order enters the system, an intelligent decision-making layer calculates a comprehensive scheduling score for each maintenance personnel meeting the condition, automatically distributes the work order to the personnel with the highest score, and pushes all information to a communication terminal of the intelligent decision-making layer; during dispatching, warehouse inventory is automatically checked through the resource collaboration layer; after maintenance is completed, results and time are recorded back to the system, and a self-learning closed loop is formed. According to the method, the mode of finding things by people and finding objects by people is changed into the mode of finding people by things and preparing objects, the method is a fundamental change of an operation and maintenance mode, the per capita efficiency can be improved, the cost can be reduced, and excessive dependence on personal experience can be avoided.

The invention provides a vehicle health examination visualization report generation method and system and a storage medium, and the method comprises the steps: obtaining the feature parameters of a vehicle, constructing a multi-dimensional database according to the feature parameters, and marking the association rules of the feature parameters and detection points in the multi-dimensional database; generating a detection item list through a rule engine and a machine learning model according to the multi-dimensional database; in the detection process, the detection items and the detection points are fused with the multimedia technology, a 3D model of the vehicle part is established, the abnormal part is displayed, the AR technology is fused with the real video of the vehicle part, and a vehicle health examination visual report is generated. Through the technical scheme of the invention, the coverage rate of workshop detection items is improved, the omission ratio is effectively reduced, the conversion rate of the detection items is improved, the understanding efficiency of vehicle owners for vehicle condition problems is enhanced, and vehicle faults or potential problems are displayed more visually.

The invention relates to the technical field of financial credit data evaluation, and discloses a user credit evaluation method based on multi-source credit data. The method comprises the steps of collecting multi-source heterogeneous credit investigation data in real time through a preset API interface group, and performing fusion after cleaning and standardization to form a user credit investigation data cube; extracting static, behavior sequence, association network and time sequence evolution four-dimensional features based on the cube; processing corresponding features by using a gradient boosting tree, a graph neural network and a time sequence convolutional network sub-model by using a multi-modal integrated learning framework, and generating a basic credit score through adaptive weighted fusion; the score is dynamically calibrated in combination with the real-time data flow and the macroeconomic factor, and high-risk user judgment and early warning are carried out based on dynamically adjusted behavior permission parameters by utilizing a real-time risk early warning engine; and finally, model retraining is triggered based on historical and recent default rate differences. According to the method, comprehensive, accurate and dynamic credit assessment is realized, and the assessment accuracy and timeliness are remarkably improved.

Systems and methods are disclosed for converting natural language queries to a query instruction set for searching a data warehouse. To generate a query instruction set from a natural language query, a system iteratively uses a generative artificial intelligence (AI) model and database query tools to generate a query instruction set in a stepwise manner. The system and generative AI model do not require a priori knowledge of data table contents in the data warehouse, which may include sensitive information. In addition, the system does not require access to the data warehouse to generate the query instruction set. Instead, the system is implemented to use structure information from the data warehouse, including table lists (such as table names) and table format information (such as column names) of tables in the data warehouse, and the generative AI model is a generally trained model to generate the query instruction set.