Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

335 results about "Data repair" patented technology

Data asset quality evaluation and automatic repair method and system based on AI

The invention discloses an AI-based data asset quality assessment and automatic repair method and system, and the method comprises the steps: receiving a to-be-processed target data asset, inputting the target data asset into a multi-dimensional joint quality scoring engine, and generating a comprehensive quality assessment report containing an integrity defect map, a distribution abnormality thermodynamic diagram and a semantic conflict link; based on the comprehensive quality evaluation report, utilizing a rule and learning fused strategy generator to analyze repair requirements and constraint conditions contained in a defect map, an abnormal thermodynamic diagram and a conflict link, and dynamically generating repair operation instruction sets and priority sequences for different types of defects; and inputting the target data assets, the repair operation instruction set and the priority sequence into a multi-expert collaborative repair execution module, repairing defects in the target data assets according to the priority sequence, and outputting the repaired data assets. According to the embodiment of the invention, comprehensive and accurate data quality evaluation can be realized, and the automation level and accuracy of data restoration are improved.
Owner:ZHEJIANG COMM SERVICES

Enterprise data link treatment and value management method and system

The invention discloses an enterprise data link management and value management method and system, and the method comprises the steps: carrying out the real-time capturing of multi-source heterogeneous data of all business systems in an enterprise through a distributed data collection engine, and recognizing the types of original data in different formats through a preset data source adapter; if structured data is detected, a relational database connector is adopted for extraction, and if the structured data is recognized as unstructured data, a document analysis module is started for content extraction, and an initial data set containing metadata tags is obtained; performing format conversion and field mapping on the initial data set according to a pre-established data standardization rule base, eliminating duplicate records and abnormal values through a data cleaning algorithm, performing automatic evaluation on a data quality grade by adopting a naive Bayes classifier, if a data quality score is lower than a preset threshold value, triggering a data recovery process, and if the data quality score is lower than the preset threshold value, performing data recovery. And standardized data meeting a unified standard is obtained. The normativity and value utilization efficiency of data management are effectively improved.
Owner:FRIENDSHIP INT ENG CONSULTING CO LTD

Data processing system based on artificial intelligence algorithm

The invention discloses a data processing system based on an artificial intelligence algorithm, and relates to the technical field of artificial intelligence and data processing, and the system comprises a data input interface which is used for receiving a multi-source heterogeneous data stream; and the preprocessing engine is connected with the data input interface and comprises a dynamic metadata sensing unit, an incremental quality evaluation unit, a bidirectional verification self-repairing unit and a closed-loop feedback optimization unit. According to the data processing system based on the artificial intelligence algorithm, through dynamic metadata perception and an incremental quality evaluation mechanism, the manual intervention requirement of a multi-source heterogeneous data preprocessing stage is reduced, and the problem of adaptation stiffness caused by the fact that a traditional method depends on a static rule is solved; by combining the bidirectional verification design of the business rule and the model feedback, the dual reliability of the data recovery strategy in logic rationality and algorithm compatibility is ensured, the risk of overfitting or scene mismatching caused by a single verification mechanism is avoided, and the input data quality and decision accuracy of a downstream artificial intelligence model are improved.
Owner:HANGZHOU XINMI QUANTITATIVE DATA TECHNOLOGY CO LTD

Meteorological data assimilation method and system based on big data

The invention discloses a big data-based meteorological data assimilation method and system, and the method comprises the steps: obtaining historical meteorological data, carrying out the time-space coupling of the historical meteorological data, constructing a self-adaptive assimilation dynamic weight model, and obtaining the data assimilation weight of to-be-assimilated observation data; the method comprises the following steps: assimilating observation data to be assimilated according to a traditional assimilation strategy to obtain a first analysis field, constructing a data depth assimilation model, carrying out data restoration and data assimilation on the observation data to be assimilated to obtain a second analysis field, carrying out weather prediction according to the second analysis field, and calculating assimilation deviation. And adopting a Bayesian network to optimize a meteorological data assimilation process according to the assimilation deviation, and processing meteorological data to be assimilated according to the optimized meteorological data assimilation process to obtain an assimilation analysis field. The method not only can improve the efficiency and accuracy of meteorological data assimilation, but also has good interpretability, and can be directly applied to a meteorological data assimilation system.
Owner:CHAOHU UNIV

Machine learning oriented interactive tabular data quality display systems and methods

Certain example embodiments relate to dashboards that help streamline and automate data quality management processes used with machine learning (ML) models and ML-enabled technology. A clean dataset is initialized from a dirty dataset. A search space is the set of all possible combinations of available error detection algorithms and data repair algorithms. A scoring function measures performance of a given error detection algorithm and data repair algorithm combination on the clean dataset. An ML model is trained using the clean dataset. Best error detection and data repair algorithms are selected, based on an optimization on the set of all possible combinations, and the defined scoring function. The selected best error detection algorithm is applied to the clean dataset, and a repaired dataset is generated using the selected best repair algorithm. The clean dataset is set to the repaired dataset. This procedure is repeated until a condition is met.
Owner:SOFTWARE AG

Plant protection field literature information batch structured extraction method based on AI large model

The invention discloses a plant protection field literature information batch structured extraction method based on an AI large model, and belongs to the technical field of computer data processing and artificial intelligence application. According to the method, a PDF format literature is converted into a Markdown format text through a PDF literature self-adaptive preprocessing module and a Markdown conversion module; then, a double-strategy self-adaptive AI extraction module is adopted, different modes are adopted for processing according to a text length threshold value, and JSON data are extracted in combination with a JSON format data restoration strategy; then, the CPU intensive conversion task and the I / O intensive AI analysis task are processed in parallel through a two-stage parallel scheduling module, and a structured database file is generated through aggregation according to a predefined mapping rule through a multi-dimensional aggregation export module. According to the method, end-to-end automation from literature acquisition to structured data output is realized, and the problem of low extraction efficiency of manual literature reading information is solved.
Owner:SANYA INSTITUTE OF NANJING AGRICULTURAL UNIVERSITY

Soil drought risk early warning system and early warning method based on machine learning

The invention discloses a soil drought risk early warning system and early warning method based on machine learning, and belongs to the field of soil drought early warning, and the system comprises a data collection module, a data restoration and fusion module, a space-time drought coupling prediction model construction module, and a real-time early warning and model updating module. Wherein the data acquisition module is used for acquiring wind cloud meteorological data and MODIS optical image data; the data restoration and fusion module is used for generating an all-weather soil humidity chart; the space-time drought coupling prediction model construction module is used for constructing a space-time drought coupling prediction model; and the real-time early warning and model updating module is used for obtaining the dynamic drought index and dynamically updating the dynamic drought index. By adopting the soil drought risk early warning system and early warning method based on machine learning, high-precision drought risk prediction and decision support are realized through multi-source data fusion, a space-time coupling prediction model and a dynamic grading early warning mechanism.
Owner:SHENYANG INSTITUTE OF ATMOSPHERIC ENVIRONMENT CHINA METEOROLOGICAL ADMINISTRATION (LIAONING INSTITUTE OF METEOROLOGICAL SCIENCES)

Data processing method and device, electronic equipment, storage medium and product

The invention discloses a data processing method and device, electronic equipment, a storage medium and a product. The method comprises the steps of obtaining a disk dynamic mapping request; in response to the disk dynamic mapping request, a data repair migration task is generated, and the data repair migration task is used for triggering disk dynamic mapping and data repair migration processes of the erasure code redundant group; and executing the data restoration migration task.
Owner:CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD +1

Method for identifying abnormal data and repairing missing data of hydroelectric generating set

The invention provides a hydroelectric generating set abnormal data identification and missing data restoration method which is used for monitoring data of a hydroelectric generating set, and the identification method selects a K-means clustering algorithm to detect abnormal values of the monitoring data according to time sequence delay and mode similarity characteristics of the monitoring data. According to the repairing method, a time sequence data repairing model based on a self-attention mechanism is constructed, and the time sequence data repairing model is used as a generator and is matched with a discriminator and a loss function to form a self-attention generative adversarial network SA-GAN for filling the missing monitoring data of the hydroelectric generating set; according to the method, the operation state of the hydroelectric generating set can be analyzed more accurately, more reliable data support is provided for operation and maintenance personnel, and then the operation efficiency and safety of the hydroelectric generating set are improved.
Owner:FUJIAN HUADIAN FURUI ENERGY DEVELOPMENT CO LTD +1

Ammeter historical data restoration method based on multi-dimensional incidence relation

The invention discloses an electric meter historical data restoration method based on a multi-dimensional incidence relation, and relates to the technical field of electric power data management and intelligent restoration, the accuracy and traceability of data restoration are realized by establishing a multi-source exception type library and an element time-varying degradation model, and the accuracy and traceability of data restoration are improved for multi-source data conflicts. The method comprises the following steps: pre-defining a plurality of abnormal mode combinations and corresponding repair strategies, positioning a fault source through mode matching of a real-time index and an abnormal type library, selecting reference data in combination with a historical average deviation rate of a target ammeter and an associated ammeter, avoiding deviation caused by single judgment, and for nonlinear degradation, firstly determining fault elements of a shunt and an ADC chip, a resistance degradation model and a conversion error model are constructed, a theoretical correction value is calculated, a result is optimized through a machine learning model trained by historical normal data, meanwhile, original readings, model parameters and calculation bases are stored in an associated mode, a traceable log is formed, and the repair accuracy is guaranteed.
Owner:NANJING TIANSU AUTOMATION CONTROL SYST CO LTD

Automatic data standardization management method based on large language model, medium and equipment

The invention relates to the technical field of data analysis and processing, in particular to an automatic data standardization management method based on a large language model, a medium and equipment, and the method comprises the following steps: a data receiving step: receiving data from various data sources; a data source analysis step: identifying field semantics and standardization requirements; a standardized rule generation step: based on the semantic understanding result of the large language model, automatically generating a standardized rule suitable for a corresponding data source; a standardization rule application and optimization step: performing application and dynamic optimization; an intelligent monitoring and verification step of data quality: performing real-time quality detection of the data by using a large language model; and a data restoration and suggestion step: automatically deducing a correct value or giving a restoration suggestion when the data does not meet the standardization requirement. According to the method, the problems of non-uniform standards, high manual rule maintenance cost and lack of dynamic adaptivity caused by inconsistent formats and semantics of the multi-source data are solved.
Owner:CHONGQING COMM CONSTR CO LTD

Abnormal data recovery and risk early warning method and system for offshore wind turbine

The invention provides an offshore wind turbine abnormal data recovery and risk early warning method and system, and the method comprises the steps: receiving the multi-modal monitoring data of an offshore wind turbine in real time, the multi-modal monitoring data comprising time domain sensor data, frequency spectrum images and environmental parameters; inputting the multi-modal monitoring data into a pre-trained visual language large model, and inputting the frequency spectrum image into a visual encoder of the visual language large model to extract frequency domain features; time domain sensor data and environmental parameters are input into a language encoder of the visual language large model to generate time sequence features and semantic features; and performing cross-modal fusion on the frequency domain features, the time sequence features and the semantic features through a cross-modal fusion module of the visual language large model to obtain cross-modal fusion features, and synchronously executing abnormal data recovery and risk early warning based on the cross-modal fusion features to obtain recovered complete monitoring data and a risk early warning result. According to the method, time domain and frequency domain data are combined, and the multi-mode large model is utilized to realize high-precision data restoration and intelligent risk early warning.
Owner:SHANGHAI ELECTRIC POWER IND +1

Intelligent denitration control system based on multi-model fusion and self-learning

The invention provides an intelligent denitration control system based on multi-model fusion and self-learning, and belongs to the technical field of industrial denitration control, and the method comprises the following steps: collecting three-dimensional data of a boiler side, a combustion side and a denitration side in a DCS / SIS system in real time through an OPC protocol; s2, an intelligent data preprocessing step, wherein the step comprises the links of anomaly detection, data restoration and dynamic normalization; s3, a spatio-temporal feature engineering step, wherein the step comprises a time delay analysis link, a feature derivation link and a feature screening link; s4, a multi-model dynamic fusion step, wherein the step comprises a base model group and a fusion mechanism link; and S5, an online self-learning optimization step, the step comprises incremental learning, parameter evolution and knowledge distillation links, and aiming at the problem that algorithm optimization adjustment cannot be carried out aiming at the environmental change of the denitration process, self-adaptive evolution of control parameters is realized by constructing a'prediction-control-feedback 'complete chain.
Owner:NANJING GUODIAN ENVIRONMENTAL PROTECTION TECH CO LTD

Sewage treatment time sequence data missing value interpolation method based on random configuration network

The invention discloses a sewage treatment time series data missing value interpolation method based on a random configuration network. The method comprises the following steps: collecting water quality data in a sewage treatment process, and preprocessing the water quality data to obtain univariate time series data; performing seasonal trend decomposition on the single-variable time series data to obtain a seasonal component, a trend component and a residual component; respectively performing component interpolation on the season component, the trend component and the residual component; and performing addition reconstruction on the seasonal component, the trend component and the residual component after interpolation to obtain complete sewage treatment process monitoring data. According to the sewage treatment time sequence data missing value interpolation method based on the random configuration network, original data is decomposed into season, trend and residual components, interpolation is conducted on all the components through self-similarity decomposition and a random configuration network model, finally a complete sequence is reconstructed, the periodicity and trend characteristics of missing data can be effectively recovered, and the accuracy of the missing data is improved. And the restoration precision and the application reliability of the sewage treatment monitoring data are improved.
Owner:XIAN UNIV OF SCI & TECH

Data recovery method and device, computer equipment, storage medium and program product

The invention relates to a data recovery method and device, computer equipment, a storage medium and a program product. The method comprises the steps that a current repairing task of a hybrid tree is determined, the hybrid tree is a hybrid storage structure of a log merging tree LSM tree and a B tree, the hybrid tree comprises multiple layers, the multiple layers comprise the first layer of the LSM tree and all layers of the B tree, ordered character string tables of the first layer of the LSM tree are merged and sunk into the B tree, and each node in the B tree stores at least one ordered character string table obtained through merging and sinking; under the condition that the current repair task indicates that the repaired damaged data storage object belongs to the B tree, obtaining a B tree index of the damaged data storage object from the current repair task; searching a damaged data storage object from the mixed tree according to the B tree index, wherein the damaged data storage object is an ordered character string table or a node; and under the condition that the damaged data storage object is found, repairing the damaged data storage object. By adopting the method, the data recovery efficiency can be improved.
Owner:DAWNING INFORMATION IND (BEIJING) CO LTD +2

Sensor data recovery method and system based on mask perception space-time modeling

The invention provides a sensor data recovery method and system based on mask perception space-time modeling, and belongs to the technical field of Internet of Things and intelligent sensing data processing. The method comprises the following steps: modeling a missing position into a trainable potential vector representation through an adaptive missing representation module, and avoiding noise introduced by traditional zero filling or mean filling; a missing perception space-time decoding module is adopted, and a dynamic weight distribution mechanism of mask constraint is combined in the decoding process, so that information is effectively prevented from being excessively smooth; designing a space-time dual-channel feature aggregation unit, and capturing spatial dependence and time sequence dependence between the sensors at the same time; and finally realizing accurate completion of a large-scale space-time sensor matrix through an output recovery unit. According to the method, the accuracy and robustness of sensor data restoration can be remarkably improved, the reliability of subsequent monitoring, prediction and anomaly detection is enhanced, and the method has wide engineering application value.
Owner:SOUTHEAST UNIV

Financial audit-oriented automatic data cleaning and anomaly identification method

The invention discloses an automatic data cleaning and anomaly recognition method for financial auditing, and relates to the technical field of financial auditing. Extracting fields from the multi-source heterogeneous data by using a regular expression, and unifying the format through semantic mapping; repairing numeric missing data by using LSTM, and checking logic errors by using a rule engine; identifying abnormal transactions based on an isolated forest algorithm in combination with business rules; constructing a risk assessment matrix to divide risk levels; dynamic knowledge graph analysis implicit association can be constructed, an incremental learning optimization model is utilized, and an auditing evidence chain and block chain evidence storage are recorded. According to the invention, automatic processing of financial audit data is realized, the data recovery success rate is improved, and the anomaly identification accuracy is improved; hidden risks are mined through the knowledge graph, audit traceability is guaranteed through block chain evidence storage, manual annotation cost is reduced by 40%, and audit efficiency and reliability are improved.
Owner:HANGZHOU JIAANNIANXIN INFORMATION TECHNOLOGY CO LTD

BDDR backdoor detection and data restoration method and system oriented to large model

The invention discloses a BDDR backdoor detection and data recovery method and system oriented to a large model, and belongs to the field of backdoor defense. Comprising the following steps: constructing a knowledge distillation architecture under federal learning, including an edge server and a plurality of clients, and obtaining distillation data; inputting the distillation data into a randomly initialized model for training, recording the loss change of each batch of data, and screening out abnormal batches to form a backdoor data set; the edge server initializes two independent models, respectively uses a distillation data set and a backdoor data set for training, and guides learning of backdoor features; using probability distribution to calculate and correct a backdoor label, generating a clean data set by adding noise, finely adjusting a large model, detecting residual backdoor feature intensity, and adjusting probability distribution calculation parameters to further weaken backdoor features according to the residual backdoor feature intensity so as to obtain a final repaired data set; while the generalization ability of the large model is improved, backdoor attacks can be effectively identified and defended, the data privacy of the client is protected, and the model security is ensured.
Owner:NANJING UNIV OF POSTS & TELECOMM

Energy storage battery compartment safety monitoring method and device based on multi-source data

The invention provides an energy storage battery compartment safety monitoring method and device based on multi-source data, and relates to the field of battery energy storage systems. The method comprises the following steps: acquiring multi-source parameter data corresponding to an energy storage battery bin through multiple types of sensors configured in the energy storage battery bin, and acquiring environmental parameter data around the energy storage battery bin; inputting the multi-source parameter data and the environment parameter data into the graph neural network model, and performing cross-sensor data restoration according to the graph neural network model; constructing a coupling relationship between the repaired source parameter data and the environment parameter data under the constraint of multiple physical quantities, and constructing an abnormal distribution prediction model according to the coupling relationship; according to the coupling relationship, based on a multi-scale time sequence trend extraction analysis method, obtaining safety anomaly features corresponding to the energy storage battery compartment; and inputting the safety abnormal characteristics into an abnormal distribution prediction model, and outputting a safety monitoring result corresponding to the energy storage battery compartment according to the abnormal distribution prediction model. The problems that potential safety hazards in the energy storage battery cabin are difficult to recognize in time through manual inspection and the possibility of fault spreading is greatly increased are solved.
Owner:HUBEI ELECTRIC POWER EQUIP

Fan assembly process missing data filling method based on semi-supervised learning

The invention relates to a fan assembly process missing data filling method based on semi-supervised learning, and belongs to the technical field of industrial data missing repair. Aiming at the technical problems that the traditional linear interpolation and supervised learning are difficult to process the missing of wind power assembly multi-source heterogeneous data, the dynamic working condition adaptability is poor and the annotation dependence is high, a feature extraction method of combining space-time tensor modeling with a bidirectional space-time attention network is provided, and the reliability of three-dimensional space-time tensor quantitative data is constructed; designing an adversarial domain adaptation mechanism to realize feature space alignment of labeled and unlabeled data; a dynamic confidence threshold strategy is adopted to screen high-credibility samples, and a collaborative loss function optimization model is constructed; the method has the technical effects that the data filling precision and the process conformity under the complex working condition are remarkably improved, the data labeling requirement is reduced, flexible production is adapted, the cross-production-line data island problem is solved, the real-time repair efficiency is optimized, assembly quality tracing and process optimization are enabled, and reliable data support is provided for intelligent wind power manufacturing.
Owner:CHONGQING UNIV

Data recovery method and device, storage medium and electronic device

The embodiment of the invention provides a data recovery method and device, a storage medium and an electronic device.The method comprises the steps that abnormal second data in first data is determined according to a data rule engine, and a data rule applied in the data rule engine is configured according to a data service scene corresponding to the first data; the second data is restored through a preset processing mode, the preset processing mode comprises a data prediction generation algorithm, and the data rule engine and the data prediction generation algorithm are updated in real time through a preset monitoring mode. By the adoption of the technical scheme, the problem that the data restoration effect is poor due to the fact that a data restoration method in the related technology cannot be updated in real time along with changes of the restored data is solved.
Owner:ZTE CORP

Multi-modal power data restoration method based on local-global correlation learning

The invention relates to a local-global association learning-based multi-modal power data restoration method, which comprises the steps of obtaining initial data and performing preprocessing to obtain a power data abnormal value, the initial data comprising power operation data, external association data and multi-modal data; based on the preprocessed power operation data and the power data abnormal value, local association learning is carried out, local data association is mined, and a local restoration scheme of the power data abnormal value is obtained; performing global association learning based on the preprocessed initial data and the abnormal value of the power data, mining a cross-time and cross-source global association rule, and obtaining a reasonable correction range of the abnormal value of the power data; and generating a power operation data restoration result in combination with the local restoration scheme and the reasonable correction range. Compared with the prior art, the method has the advantages that the data exception can be accurately recognized and repaired, and the data repair accuracy is greatly improved.
Owner:STATE GRID SHANGHAI MUNICIPAL ELECTRIC POWER CO

Multi-source meteorological data dynamic fusion wind power plant simulation method

The invention relates to the technical field of wind energy development and utilization, in particular to a multi-source meteorological data dynamic fusion wind power plant simulation method, which comprises the following steps of: accessing satellite remote sensing, a ground observation station, numerical forecasting and wind power plant historical operation data, and outputting standardized meteorological data through format analysis, standardized processing and cleaning; a credibility score is calculated based on multi-dimensional indexes such as data integrity and time consistency, and abnormal data restoration or standby data source switching is achieved; in combination with wind power plant DEM topographic features and a self-adaptive sliding window, the weight of a data source is adjusted in real time through reinforcement learning, and a dynamic weight value is generated; an improved Windpowerlib framework adapts to fusion data, fan parameters are dynamically configured, and an initial simulation result is generated; and collecting real-time data of the wind power plant, and correcting model parameters by using Kalman filtering. The wind power plant simulation precision and timeliness can be improved, the data utilization rate and the model adaptability are enhanced, and the deviation between the simulation result and the actual working condition is reduced.
Owner:POWERCHINA BEIJING ENG CORP

Check disk data recovery method and device, electronic equipment and readable storage medium

The invention discloses a check disk data recovery method and device, electronic equipment and a computer readable storage medium. The method comprises the steps that a check disk corresponding to a diagonal check chain in a target redundant array is determined; for each verification block in the verification disk, grouping all target data blocks corresponding to the verification block to obtain each data block group; wherein all the target data blocks in each data block group belong to the same data disk; for each data block group, calculating all target data blocks in the data block group by using a data disk corresponding to the data block group to obtain an initial check value corresponding to the data block group; sending the initial verification value corresponding to each data block group to a verification disk, and calculating each initial verification value by using the verification disk to obtain a verification value corresponding to the verification block; and obtaining recovery data of the check disk based on the check values corresponding to all the check blocks, and completing data recovery of the check disk. According to the technical scheme, the data recovery of the check disk can be realized more quickly and efficiently.
Owner:SHANDONG YUNHAI GUOCHUANG CLOUD COMPUTING EQUIP IND INNOVATION CENT CO LTD

Aero-engine damaged blade data repairing and modeling method

The invention provides an aero-engine damaged blade data restoration and modeling method, which comprises the following steps: processing finite element grid files of a prototype blade and a damaged blade, and extracting grid topology information and node information of the blade; comparing node information of the prototype blade and the damaged blade, and determining multiple types of damaged nodes in the damaged blade; estimating the correct node position of the damaged node through three-dimensional data interpolation, and repairing the damaged area; the intermediate state shape of the blade is reconstructed by setting a repair reduction coefficient; and carrying out secondary reconstruction on nodes of the front edge area and the tail edge area of the blade, arranging blade profile node coordinates, and outputting a blade surface blade profile geometric file supporting pneumatic simulation. According to the method, the automatic conversion process from the finite element simulation grid to the pneumatic simulation blade profile file can be efficiently achieved, the blade damage analysis and repair design efficiency is greatly improved, and subsequent pneumatic performance simulation is facilitated.
Owner:HARBIN INST OF TECH +1

Traffic flow state data time gathering method based on Gaussian mixture probability

The invention discloses a traffic flow state data time gathering method based on Gaussian mixture probability, and belongs to the technical field of calculation, reckoning or counting. The method comprises the following steps: acquiring traffic flow state data and vehicle trajectory data of each lane in a target road section; carrying out probability diagnosis on the traffic flow data by adopting a Gaussian mixture model, and identifying and deleting abnormal points; introducing a model based on Transform and graph neural network fusion, and realizing space-time repair of abnormal data through joint modeling of an attention mechanism and a space-time topology dependency relationship; and carrying out time window collection on the repaired data based on wave walking and stopping prediction, and calculating a traffic flow rate, an average speed and an occupancy rate. According to the method, the joint characteristics of the traffic flow state data and the trajectory data are fully utilized, meanwhile, the abnormal data repairing precision and the space-time consistency are improved, adaptive collection of the traffic operation dynamic characteristics is achieved by introducing walking and stopping wave prediction, and the robustness and reliability of traffic flow data processing are remarkably improved.
Owner:HEBEI PROVINCIAL COMM PLANNING & DESIGN INST +1

Data synchronization and comparison method and device and electronic equipment

The invention relates to a data synchronization and comparison method and device, and aims to efficiently and accurately solve the problem of data consistency of a source end and a target end. The method comprises the following steps: (1) data partitioning: partitioning data of a source end and a target end according to a preset rule; (2) hash calculation: generating a hash value for each data block; (3) hash comparison: rapidly screening inconsistent data blocks by comparing hash values; (4) recording and comparing: comparing inconsistent data blocks item by item and field by field to determine specific differences; and (5) data restoration: restoring the target end data according to the comparison result. According to the method, the comparison data volume is reduced through data partitioning and Hash calculation, and the comparison efficiency is remarkably improved; hash comparison and record comparison are combined, so that the comparison accuracy is ensured; the scheme does not depend on external factors, is suitable for various complex data structures, and has wide applicability.
Owner:CHINA ELECTRONICS CLOUD DIGITAL INTELLIGENCE TECH CO LTD

Traffic data space-time restoration method based on time domain-frequency domain evolution

The invention relates to the technical field of data restoration, in particular to a traffic data space-time restoration method based on time domain-frequency domain evolution, which can accurately capture dynamic characteristics of a traffic situation on different time scales through the method based on time domain-frequency domain evolution, and improves the modeling capability for a complex traffic situation. According to the method, the inverse Fourier transform technology and the wavelet transform technology are utilized, the fitting capacity for actual time fluctuation can be guaranteed during time signal reconstruction, the diversity characteristics of the traffic situation can be reserved, and the quality of time signal reconstruction under emergencies is effectively improved. According to the method, the GAT model and the Transform model are combined, missing data are repaired in the space-time dimension, the space-time interaction characteristics of the traffic flow can be fully utilized, missing area data can be accurately deduced, time and space dynamic characteristics are captured, the accuracy and integrity of a repair result are further optimized, and a more reliable basis is provided for traffic flow prediction and management.
Owner:CHONGQING UNIV OF POSTS & TELECOMM

Distributed new energy data scheduling and processing method based on cloud-edge-end collaboration

This application discloses a distributed new energy data scheduling and processing method based on cloud-edge-end collaboration, which relates to the field of distributed new energy data analysis technology. Specifically, the application discloses the following steps: the cloud computing center outputs a task scheduling strategy adapted to the task processing capabilities of each edge node to the terminal device based on edge node information, terminal device information, and the edge node task scheduling model; the terminal device uploads the distributed new energy operation data to the corresponding edge node according to the task scheduling strategy and encryption strategy; the edge node performs abnormal data identification and correction on the distributed new energy data based on the abnormal data detection model and the abnormal data repair model. The beneficial effects of this application are: reducing transmission time, improving data processing efficiency, and enhancing the timeliness and accuracy of data output.
Owner:STATE GRID ZHEJIANG ELECTRIC POWER CO LTD NINGBO POWER SUPPLY CO

Data online modeling and data management method for dimension modeling

The invention relates to the technical field of data processing, and discloses a data online modeling and data management method for dimension modeling. The method comprises the following steps: performing dimension characteristic baseline acquisition on a multi-source heterogeneous data stream to generate a dimension attribute characteristic set; performing dimension hierarchical modeling based on the feature set, and outputting a result containing an atomic layer, an aggregation layer and an application layer; calculating a dimension correlation degree according to the topological relation of the service entities, and generating a correlation constraint parameter; detecting dynamic modeling conflicts based on the parameters, and outputting a conflict dimension identification set; distributing a query direction optimization weight according to the identifier set to obtain a dimension query optimization strategy; incremental data loading is controlled based on a strategy, and an incremental loading sequence divided according to time windows is generated; performing closed-loop consistency verification on the sequence, and outputting a verification result set containing data integrity and association consistency indexes; recording a modeling state snapshot according to the result set, and generating a snapshot with a version mark; and analyzing the abnormal backtracking path based on the snapshot, and outputting a dimension data recovery path instruction.
Owner:SHANDONG LUNENG SOFTWARE TECH