286 results about "Data assessment" patented technology

The invention provides a virtual DPU power plant simulation fault restoration method and system based on digital twinning, and relates to the technical field of digital twinning, and the method comprises the steps: carrying out the preprocessing of collected DPU power plant operation data, including noise reduction, time sequence alignment and abnormal point elimination, carrying out the data quality evaluation, training a fault feature mapping model based on the processed data, and carrying out the fault restoration of the DPU power plant. The model is used for recognizing abnormal clusters in real time, a fault evolution path is searched and determined in combination with a conditional random field and a Monte Carlo tree, an optimal path is determined through a particle filtering algorithm, fault root causes are determined in combination with causal analysis, spectral clustering and a Bayesian network, and a fault diagnosis report is generated.

The invention provides a data fusion sharing method for multi-source heterogeneous power big data, which comprises the following steps: when receiving multi-source heterogeneous power data, carrying out dynamic protocol adaptation, and converting multi-source heterogeneous power data formats of different protocols into a current system internal standard format; performing data quality evaluation on the data converted into the standard format in the current system, and when a quality evaluation result does not meet a quality requirement, performing data restoration; analyzing the data passing the data quality evaluation or restoration, determining the correlation among different data sources, and fusing the data according to the correlation among the different data sources; and performing distributed storage on the fused data for distributed indexing and access. By implementing the method and the system, efficient fusion and sharing of multi-source heterogeneous data can be realized, the intelligent level of data management and analysis of a power system is improved, and the requirements of an intelligent power grid on data instantaneity, accuracy and safety are met.

The invention discloses an AI-based data asset quality assessment and automatic repair method and system, and the method comprises the steps: receiving a to-be-processed target data asset, inputting the target data asset into a multi-dimensional joint quality scoring engine, and generating a comprehensive quality assessment report containing an integrity defect map, a distribution abnormality thermodynamic diagram and a semantic conflict link; based on the comprehensive quality evaluation report, utilizing a rule and learning fused strategy generator to analyze repair requirements and constraint conditions contained in a defect map, an abnormal thermodynamic diagram and a conflict link, and dynamically generating repair operation instruction sets and priority sequences for different types of defects; and inputting the target data assets, the repair operation instruction set and the priority sequence into a multi-expert collaborative repair execution module, repairing defects in the target data assets according to the priority sequence, and outputting the repaired data assets. According to the embodiment of the invention, comprehensive and accurate data quality evaluation can be realized, and the automation level and accuracy of data restoration are improved.

The invention provides a data quality evaluation method and system based on big data analysis. The method comprises the following steps: receiving real-time data from different data sources, the real-time data comprising structured data and unstructured data; performing significance analysis processing on the unstructured data to obtain a business context of the real-time data; determining the integrity and consistency of the real-time data through an intelligent algorithm to obtain an integrity and consistency evaluation result of the real-time data; based on the evaluation result of the logical relationship, combining the integrity and consistency evaluation results to construct a data quality evaluation model, the data quality evaluation model being used for quantifying the accuracy and timeliness of the real-time data; generating a data quality report, and proposing service improvement measures according to the data quality report; and optimizing the data acquisition and processing flow according to business improvement measures. According to the technical scheme provided by the invention, the flexibility and efficiency of data quality evaluation can be improved, and the real-time data analysis demand of an enterprise can be better supported.

The invention provides an active transformer area voltage out-of-limit early warning and photovoltaic bearing capacity evaluation method, which comprises the following steps: acquiring multi-source data of a power distribution transformer area, and generating a standardized multi-source time sequence data set through time alignment processing, data quality evaluation and feature fusion; extracting voltage out-of-limit features based on the time sequence similarity matrix and the node influence matrix; the periodicity, volatility and randomness characteristics of photovoltaic output are analyzed, a coupling relation matrix and a sensitivity vector are established, and an influence evaluation model is constructed; performing out-of-limit early warning based on an early warning index system and a deep learning model; and establishing a multi-dimensional evaluation index system for carrying out bearing capacity evaluation in combination with power flow analysis and the dynamic influence factors. Through multi-source data fusion and multi-dimensional feature analysis, the accuracy of voltage out-of-limit early warning is improved, accurate evaluation of the photovoltaic bearing capacity is realized, and effective technical support is provided for safe operation of the power distribution network.

The invention provides a multi-dimensional quality evaluation method, device and equipment for carbon footprint data and a storage medium, and the method comprises the steps: obtaining multi-source heterogeneous data of a product, the multi-source heterogeneous data comprising basic information of the product and carbon footprint related data of the whole life cycle of the product; generating carbon footprint data of the product based on the multi-source heterogeneous data; and performing multi-dimensional quality evaluation on the carbon footprint data based on a pre-constructed carbon footprint knowledge graph, a time dimension detection model, a space dimension detection model and a cross-dimension detection rule. According to the invention, the carbon footprint knowledge graph, the time dimension detection model, the space dimension detection model and the cross-dimension detection rule are introduced to realize semantic, time, space and cross-dimension multi-dimensional quality evaluation of the carbon footprint data, and the detection dimension can cover the multi-dimensional features required by the carbon footprint calculation. Therefore, it is ensured that the carbon footprint data quality evaluation result can reflect the influence on the carbon footprint calculation reliability more accurately.

The invention discloses an intelligent processing system for enhanced training of legal text small samples, and relates to the field of intelligent processing of legal texts. The system comprises a knowledge base module, a data enhancement module and a data quality evaluation module. The basic synonym library ensures standardization and authority of term replacement through an authoritative dictionary, judgment document mining and co-occurrence frequency calculation, scene accurate matching of term replacement is achieved, the legal relation qualitative change problem is reduced, scene classification is achieved through the context adaptation library based on an LSTM-CRF model, scene key terms are extracted in combination with an attention mechanism, and the scene key terms are extracted through the context adaptation library based on the LSTM-CRF model. The term replacement is more fit with the field context of the legal text, the expansion explanation library maintains the hyponymy of the legal concept through the knowledge graph, it is ensured that the replaced terms do not change the abstract-concrete level of the legal concept, and the preciseness of the legal logic is maintained.

The invention relates to a system, device and method for unsupervised data quality assessment of datasets. The method of labeling a set of data points as anomalous includes the steps of receiving time series data from a plurality of time series data sources, dividing the time series data into partitions, and normalizing the values of the data points in the partition. For each partition, a representative value for each successive pair of data points in the partition is calculated and a two-dimensional distribution mapping of calculated representative values for each partition is generated. Thereafter, cluster analysis is performed on a plurality of distribution mappings and identifying clusters which are anomalous and the data points in each of the partitions of each distribution mapping which is associated with the anomalous cluster is labeled as anomalous.

The embodiment of the invention provides a multivariate data processing method and system.The method comprises the steps that original data are collected from multiple data sources by calling multiple APIs, and multi-source heterogeneous data are obtained; formatting the multi-source heterogeneous data according to a preset format conversion rule, wherein the formatting comprises timestamp alignment, space coordinate system conversion and key frame extraction; performing multi-dimensional feature extraction on the unified data to generate space-time aligned feature vectors, and performing cross-modal data fusion on the feature vectors based on a dynamic weight distribution algorithm to generate a fusion data packet; and performing data quality evaluation on the fused data packet, including verifying the integrity, consistency and real-time performance of the data, and dynamically adjusting a transmission path of a data acquisition interface according to a verification result. According to the method, an efficient and reliable solution is provided for collection, processing and fusion of multi-source heterogeneous data, and the data processing efficiency and the decision support capability are remarkably improved.

The invention discloses a quantum encryption handheld laser gas monitoring method and system, and relates to the technical field of handheld laser gas monitoring. The quantum encryption handheld laser gas monitoring method and system comprise the following steps: S1, collecting environmental parameter information, storing environmental monitoring data, and establishing an environmental monitoring database; s2, performing laser spectrum monitoring on the target gas, and performing data quality evaluation based on a laser spectrum monitoring result; s3, executing quantum key analysis on the laser spectrum monitoring data qualified in data quality evaluation; and S4, performing environmental adaptability analysis by integrating the evaluation results of the environmental monitoring data and the laser spectrum monitoring data and the quantum key analysis result. The problems that an existing handheld gas monitoring device is unstable in operation under the complex environment conditions of high temperature, low temperature, high humidity, dust interference, air pressure fluctuation and the like, and the measurement precision is difficult to guarantee are solved.

The invention discloses a data quality governance method, and relates to the technical field of privacy calculation and data governance. According to the method, various kinds of original data are obtained, one kind of original data comes from a data source, and the data source comprises a hospital, a company and a medicine production enterprise; normalizing the original data to obtain standardized data; inputting the standardized data into a preset federal data quality evaluation model for data quality evaluation to obtain a quality evaluation result corresponding to each kind of standardized data, the quality evaluation results including an integrity evaluation result, a consistency evaluation result and a timeliness evaluation result; aggregating the quality evaluation results to obtain a global quality evaluation report; and determining a corresponding quality optimization strategy according to the global quality evaluation report, and treating data according to the quality optimization strategy. And the data quality treatment effect is improved.

The invention provides a hollow-core optical fiber cable fiber breakage detection method based on a reference optical fiber and an optical cable, and belongs to the technical field of optical fiber communication. The method comprises the following steps: acquiring monitoring data of a reference optical fiber in a hollow-core optical fiber cable, and performing quality evaluation on the data to generate a signal quality comprehensive score; when the score is higher than a preset threshold value, feature extraction and dynamic weighted fusion are carried out to generate a comprehensive feature vector; outputting a fiber breakage probability by using the fiber breakage state recognition model, calculating a fiber breakage risk value through a prediction model in combination with time sequence data, and obtaining a fiber breakage risk index through a risk fusion algorithm; and finally, matching and executing a hierarchical response strategy according to the risk index. According to the invention, multi-modal data fusion, intelligent state identification and active early warning are realized, and the reliability, precision and operation and maintenance efficiency of broken fiber detection are significantly improved.

The invention relates to a data transaction quality evaluation automatic optimization method under multi-source data fusion, and relates to the technical field of data transaction management, and the method comprises the following steps: data collection and preprocessing: collecting multi-source data including data meta-information, third-party verification data and transaction historical data; a data quality evaluation model: constructing a data quality evaluation index system, and obtaining the data quality evaluation model by using the decision tree; and automatically optimizing and executing dynamic optimization, outputting a data quality score according to the evaluation model, and automatically executing a data cleaning, supplementing or encrypting optimization strategy based on a score result. The method is beneficial to improving data quality, reducing manual intervention, improving efficiency and reducing cost.

The invention provides a data quality evaluation method and system, and relates to the technical field of data quality evaluation.The data quality evaluation method is used for coupling type data quality evaluation of knowledge graph construction and specifically comprises the following steps that an entity historical version set of a knowledge graph and attribute time sequence data of the entity historical version set are obtained, calculating a redundancy index and a timeliness entropy change index of each entity, and calculating a coupling quality index based on the coupling coefficient; constructing a global quality evaluation matrix; and outputting a quality evaluation result and a cleaning strategy according to the distribution characteristics of the singular values of the global quality evaluation matrix. According to the method, the accuracy, adaptability and operability of knowledge graph data quality evaluation are remarkably improved, and an effective technical means is provided for constructing and maintaining a high-quality, low-redundancy and high-timeliness knowledge graph.

The invention provides a big data processing method and system and a storage medium, and belongs to the technical field of big data processing and analysis. The method comprises the following steps: acquiring original data and classifying the original data through a data type identification module to generate a data type tag sequence; based on the data type label sequence, performing preliminary cleaning on original data to form a data set after preliminary cleaning; based on the data set after preliminary cleaning, a multi-dimensional data quality evaluation model is adopted to generate a data quality scoring sequence; according to the data quality scoring sequence, abnormal values and noise are identified and repaired, and a repaired data set after quality evaluation is generated; and based on the repair data set, executing multi-source data fusion and deduplication operation to form a fusion data set after deduplication. The method improves the quality and accuracy of the medical big data, realizes efficient data processing and unified storage, and is suitable for processing scenes of complex multi-source data.

The invention relates to a real-time line loss analysis method and system based on cross-modal measurement data fusion, and the method comprises the following steps: S1, collecting multi-source heterogeneous data, carrying out the preliminary quality screening of the collected original data through employing an improved isolation forest algorithm, and obtaining an original data set and a corresponding data quality preliminary evaluation label; s2, preprocessing the original data set according to the data quality preliminary evaluation label to obtain preprocessed data; s3, according to the preprocessed data, constructing a comprehensive feature vector and an importance score matrix of each feature; s4, deeply combining the physical law of the power system with artificial intelligence to construct a hybrid analysis model, and performing hybrid analysis based on the comprehensive feature vector and the importance scoring matrix to obtain an analysis result; and S5, the analysis result is converted into visual and visual display and operable decision suggestions. According to the invention, high-precision real-time analysis of power distribution network line loss is realized, and intelligence and accuracy of line loss management are effectively improved.

The invention discloses a supplier quality evaluation method based on big data analysis, and relates to the technical field of supplier quality data evaluation management. The method comprises the following steps of collecting original heterogeneous data, performing preprocessing, generating a data quality report, performing quantitative evaluation on multi-source heterogeneity and real-time performance of supplier quality data, and triggering an optimization adjustment mode. According to the method, after original heterogeneous data is collected from a supplier historical transaction database and preprocessed, a data quality report is generated, the multi-source isomerism and the real-time performance of the supplier quality data are quantitatively evaluated, the data are compared with judgment values in the database and analyzed, a data integration optimization mode and a resource scheduling optimization mode are triggered, and the quality of the supplier is optimized. The real-time performance of data in supplier quality evaluation is improved, and the problem that the real-time performance of a supplier quality evaluation method is insufficient due to data multi-source heterogeneous in the prior art is solved.

The invention provides an air quality data quality control method based on time series data, and relates to the technical field of air detection, and the method comprises the steps: obtaining multi-source atmosphere monitoring station data, constructing a spatial topological relation network of monitoring stations, and defining a dynamic hierarchical structure of atmosphere pollutant data under different spatial and temporal scales and a cross-hierarchical association rule of the dynamic hierarchical structure; carrying out anomaly detection and traceability analysis on the obtained monitoring data, and carrying out data correction and filling; performing credibility evaluation and grading on the corrected monitoring data by applying a space-time clustering algorithm and combining a site topological relation and a data association rule, and generating a data credibility grading result of a space-time dimension; when the data credibility is detected to be abnormal or the potential failure risk is detected, a graph neural network model integrated with atmospheric diffusion physical constraints is adopted; and generating a visual atmosphere data quality evaluation and control decision report, and automatically triggering an instrument calibration, maintenance or data review instruction for a specific site or region according to the risk level.

The invention provides an industrial data quality evaluation method based on micro-service, a medium and a system, and belongs to the technical field of electrical digital data process.The industrial data quality evaluation method comprises the steps that firstly, multi-source heterogeneous data is collected through a distributed network, and an evaluation system containing deviation rate, completeness rate and timeliness indexes is established; extracting features by using a deep learning model, and generating a feature probability distribution matrix; then, constructing a state transition matrix based on the characteristic entropy and the fluctuation coefficient, deploying an evaluation unit through the micro-service architecture, and establishing a transverse and longitudinal evaluation network to calculate information gain and association strength; and finally, performing feature fusion through an improved pyramid structure, realizing quality scoring by adopting a three-layer evaluation equation set, ensuring scientificity of a scoring result through logarithm mapping, and realizing dynamic evaluation of industrial data quality. The technical problem that in the prior art, an industrial data quality evaluation method is difficult to adapt to dynamic changes of multi-source heterogeneous data is solved.

The invention relates to the technical field of data processing, and discloses a data quality verification and restoration method based on big data, comprising the following steps: S1, setting a data quality assessment rule; s2, defining related rules, indexes, quality templates and quality tasks, and evaluating the accuracy, integrity, consistency and credibility of the big data set; s3, recovering data integrity and accuracy by adopting a proper recovery technology; and S4, generating a data quality report and visual content, presenting a data quality condition for the user, assisting the user in discovering problems and formulating improvement measures, and supporting subsequent data analysis and decision. According to the method, the data quality can be improved, the data credibility can be enhanced, data analysis and decision can be optimized, automatic and intelligent processing can be realized, flexible rule customization and management can be provided, the effects are beneficial to improving the data quality and availability, and more efficient, accurate and reliable data processing and decision support can be brought to users.

The invention discloses a large model training data set construction method based on table type data. The method comprises the steps that 1, data preprocessing is carried out; step 2, constructing an automatic processing method and an extraction template based on header association; step 3, constructing a deformable template model; and step 4, binding question types and fields. According to the method, manual intervention is reduced by applying an automatic processing method based on header association and an extraction template, a question and answer template is preset, key fields are matched by using a regular expression, a table format is automatically adapted through a field identification module, and then data quality is guaranteed through a data quality evaluation and feedback mechanism; a deformable template model is provided to be combined with a dynamic adjustment mechanism, and field identification, dynamic adjustment and mapping modules cooperate to ensure accurate query; heterogeneous table data is standardized, problem types and corresponding fields are automatically bound to improve the processing efficiency, and the diversity of a data set is increased by using a data enhancement strategy based on a natural language.

The invention is suitable for the field of data processing, and discloses a data treatment method, terminal equipment and a storage medium. The method is applied to a data facility, the data facility comprises a data platform, a data system and a data tool, and the method comprises the steps of accessing or collecting to-be-governed data; executing a preset operation corresponding to the full life cycle stage on the to-be-treated data, wherein the full life cycle stage comprises a data production stage, a data processing stage, a data circulation stage, a data application stage, a data maintenance stage, a data display stage and a data destruction stage; multi-dimensional data assessment corresponding to the full life cycle stage is executed on the to-be-governed data, assessment results are obtained, and the assessment results comprise a data value assessment report, a data quality assessment report, a data standardization assessment report and a data safety assessment report; and constructing a matrix type responsibility system according to an evaluation result, wherein the responsibility system maps a data engineer, a data analyst, a data manager, a data security teacher and a data quality evaluator to a corresponding life cycle stage in the full life cycle stage. According to the method, the problems of full-flow fault, evaluation subjectivity, role fuzzification and facility islanding in datamation treatment are solved.

The invention belongs to the technical field of wind power plant data monitoring, and particularly discloses a wind power plant data quality evaluation method based on high-dimensional space anomaly identification, which comprises the following steps of: constructing a high-dimensional space according to wind power plant multi-dimensional parameters monitored by an SCADA (Supervisory Control and Data Acquisition) system; performing correlation analysis by using a Spearman correlation analysis method, combining strong correlation parameters according to an analysis result, and constructing a three-dimensional subspace based on correlation analysis; based on an adaptive clustering analysis method, identifying in the three-dimensional subspace to obtain a first abnormal data set; combining the weak correlation parameters, calculating an Euclidean distance under a time sequence corresponding to the weak correlation parameters, drawing a curve of the Euclidean distance changing along with time, and obtaining a second abnormal data set according to the curve; and performing abnormal cause identification, and outputting an abnormal data set and a corresponding abnormal cause thereof. The problem that an existing abnormal point identification method cannot distinguish the types of abnormal points is solved, and the accuracy of abnormal data identification is improved.

The invention relates to the technical field of data acquisition, and particularly discloses a data acquisition method and system based on edge computing, which are applied to edge computing of the Internet of Things, and comprises the following steps: acquiring multi-source target information according to the Internet of Things, and classifying the multi-source target information to obtain multi-source classification data, the multi-source classification data comprises structured data, unstructured data, time series data and cross-domain data; and establishing a multi-source data index according to the structured data, the unstructured data, the time series data and the cross-domain data, and acquiring a data acquisition frequency set according to the multi-source data index. According to the method, the comprehensiveness and the accuracy of data quality evaluation are ensured by obtaining the first data evaluation value, the second data evaluation value, the third data evaluation value and the fourth data evaluation value of the structured data, the unstructured data, the time series data and the cross-domain data, and the problems that evaluation is not comprehensive and quality changes are not monitored are solved.

The invention discloses a multi-modal data fusion enterprise benefit policy declaration automatic processing system, which relates to the field of enterprise benefit policy declaration and comprises a data acquisition module, a preprocessing module, a fusion module, a policy matching module, a material generation module, a declaration process automation module, an intelligent feedback and optimization module, a user interaction module, a policy recommendation optimization module and a data quality evaluation module. The method comprises the following steps: acquiring enterprise internal and external data marking reliability in a multi-source manner; cleaning and analyzing normalized data by adopting an innovative algorithm; fusing data by combining CNN, RNN and a self-attention mechanism; generating and auditing materials based on a knowledge graph and deep learning matching policy and a template and NLG; and an immersive interface is created to support various interactions. According to the invention, multi-source data is collected efficiently, enterprise benefit policies are matched accurately, declaration materials are generated automatically, enterprise declaration efficiency and success rate are improved, government auditing process is optimized, data security is enhanced, policy accurate landing is promoted, and government and enterprise win-win is assisted.

The invention discloses a medical informatization data intelligent analysis system based on a large model, and belongs to the technical field of medical informatization and artificial intelligence, and the system comprises a data collection and standardization module, a medical knowledge graph construction module, a knowledge enhancement inference analysis module, a data quality evaluation module and a structured output module. The system collects multi-source heterogeneous medical data from an electronic medical record system, a laboratory information system, a medical image information system and a hospital information system, performs standardization processing, automatically constructs a medical knowledge graph, and performs intelligent analysis and reasoning on the medical data by using a knowledge retrieval enhanced medical field large language model. Meanwhile, the data quality is evaluated in four dimensions of integrity, accuracy, timeliness and relevance, and finally a structured analysis report is generated. The multi-source medical data can be effectively integrated, the accuracy and interpretability of medical data analysis are improved, and intelligent support is provided for clinical decision making.

The invention belongs to the technical field of data asset management, and discloses a data asset classification protection method based on data assessment pricing, which comprises the following steps: S1, data collection: collecting data assets to be assessed, priced and subjected to classification protection, the data assets comprise business data, customer data and financial data in an enterprise, and market data and partner data which are acquired from the outside; and S2, data asset assessment: comprehensively assessing the characteristics, usage scenarios and potential risks of the data assets to generate a pricing model of the data assets. According to the scheme, value evaluation is performed on the data assets, the data classification is determined in combination with business and safety sensitivities of the data assets, and corresponding safety protection strategies are formulated for different types of data, so that the management efficiency of the data assets is improved, safety resources are reasonably allocated, the overall safety cost is reduced, and the precision and effectiveness of data protection are improved.

The invention provides a data quality evaluation method and system. The method comprises the following steps: firstly, collecting three types of data sets of three-phase unbalance degree, free charge density distribution and line impedance decomposition; and then, carrying out space topology mapping on the charge density gradient extreme point of the cable joint area and the voltage phase angle, considering charge migration effect compensation, and calculating a corrected three-phase voltage unbalance parameter. Secondly, establishing a coupling factor matrix based on the separated line inherent impedance component and the load fluctuation impedance component, and performing time domain convolution operation on the coupling factor matrix and a voltage unbalance parameter to generate a distortion feature vector; and finally, calculating data credibility weight distribution of the global monitoring nodes according to the nonlinear correlation degree between the three-phase unbalance degree data and the distortion feature vector, and using the data credibility weight distribution to evaluate the quality of the low-voltage treatment data. According to the technical scheme provided by the invention, the accuracy and reliability of data quality evaluation can be improved.

The invention discloses a multi-dimensional large model data quality evaluation method and system, and the method comprises the steps: extracting data features in a data set and a reference data set which meet the lowest inherent quality requirements, and carrying out the evaluation of the quality of data based on the information field effect between data samples in the data set; and feature updating and context semantic association enhancement processing are carried out on each data sample, so that the semantic feature expression ability of each data sample is improved, and more accurate data quality evaluation is realized. Therefore, by introducing the context association information of the data set, the semantic information of each data sample can be more accurately understood, so that the accuracy of overall data quality evaluation is improved.

In general, embodiments of the present invention provide systems, methods and computer readable media for automated dynamic data quality assessment. One aspect of the subject matter described in this specification includes the actions of receiving a data quality job including a new data sample; and, if the new data sample is determined to be added to a reservoir of data samples, sending a quality verification request to an oracle; receiving a new data sample quality estimate from the oracle; and adding the new data sample and estimate to the reservoir. A second aspect of the subject matter includes the actions of receiving, from a predictive model, a judgment associated with a new data sample; analyzing the new data sample based in part on the judgment to determine whether to send a new data sample quality verification request to an oracle; and, if a new data sample quality estimate is received from the oracle, determining whether to add the new data sample and the judgment to the reservoir.