43 results about "Data heterogeneity" patented technology

The invention relates to a big data technology, and aims to provide a credit risk control system and method based on a federation mode. The system comprises a heterogeneous data access layer used foraccessing and converting data, a data preprocessing layer used for preprocessing original data, and a sample alignment layer used for keeping training samples of different data providers aligned, anda federated learning layer used for training a local model by utilizing the participant local data and forming a global model after gradient aggregation. The invention provides a unified data access format, data preprocessing and a risk prediction model based on federated learning, and solves the challenge problem brought by data heterogeneity and privacy leakage to risk control. A central serverdoes not need to participate in the model training and learning process, and it can be guaranteed that user privacy is not eavesdropped. Risk control modeling can be carried out by combining a plurality of different participants, the modeling process is standardized, the risk control capability is finally improved, and the cost is reduced for enterprises.

The invention discloses a crowdsourcing task distribution method based on user reliability. The method comprises: S1, inputting a user set W and a task set T; S2, starting task distribution and carrying out initialization on a correlated task and the user set; S3, for each task ti in the to-be-distributed task set T, selecting a user meeting a condition and adding the user into a user set Wr,I of the task ti; S4, starting a first round of user distribution and carrying out ergodic processing on each task in the to-be-distributed task set T; S5, carrying out user distribution form a second round to the first round and carrying out ergodic processing on each task ti in the to-be-distributed task set T; and S6, completing the task distribution and updating a reputation value of the ser according to the quality of task completion by the user in the set Wr. Therefore, reliability and heterogeneity of the user and task heterogeneity can be guaranteed; and the effective yield of the task can increase and thus more user can be simulated to complete tasks with high quality.

The invention relates to an image data expansion method for deep learning model training and learning, and belongs to the technical field of computer medical image calculation. The method comprises the following steps: firstly, judging a data type, and identifying CT or MRI image data; For the image data, judging whether an ROI (Region Of Interest) is defined or not, and selecting a correspondingmethod to complete the construction of an image data set in combination with the size of a tumor region; Training the image data set by adopting a basic image transformation method to obtain a preliminary training data set; And finally, carrying out data expansion on the preliminary training data set, carrying out deep training by adopting a network model, and finally, carrying out probability prediction. Based on artificial intelligence deep learning, a series of data expansion methods are applied to learning of deep model training in the field of medical image processing, the influence of medical image data heterogeneity on abnormal data is solved, computer-aided diagnosis is facilitated, and the diagnosis efficiency and accuracy are improved.

The invention relates to a data management service system based on large data. The data management service system comprises a heterogeneous data normalized-description module, a data semantization module, a data storage performance module, a data logic-management module, a data scenarization and service matching module and a data display module. The data management service system based on scenes solves the problems as follows: first, the data volumes are large at present, the data variety is large, the data is from various data sources, and the data categories and formats are rich; as a result, the problem of difficult storage is formed; second, the description of data heterogeneity: the data with multi-source large data forms data islands; different data structures exit in each data source, and at the same time, different designation systems also exist in each data source; the homogeneous data can also not interoperate; third, the data matching problem: the data matching problem is formed as the structures of data categories are different, the precision ratio and the recall rate are low, and the query cost is high.

The invention discloses a shale heterogeneity classification and comparative method, which comprises:(1) building the isochronous stratigraphic comparison lattice for the vertical heterogeneity classification of the shale reservoir by the high-frequency sequence classification and comparation; (2) studying the macro heterogeneity in the isochronous stratigraphic comparison lattice and analyzing and establishing the vertical and horizontal change characteristics of the macro heterogeneity of different high-frequency sequences of the shale reservoir in the isochronous stratigraphic comparison lattice; (3) studying the micro heterogeneity in the isochronous stratigraphic comparison lattice and establishing the vertical and horizontal change characteristics of the micro heterogeneity of different high-frequency sequences of the shale reservoir; (4) classifying and comparing small layers, and conducting detailed classification of small layers in the isochronous stratigraphic comparison lattice according to the heterogeneity characteristics of the shale reservoir in research results of the macro heterogeneity and the micro heterogeneity. The method selects the shale reservoir with high heterogeneity both in vertical and lateral directions as stable and continuous output selecting layer and provides geological basis for shale gas' exploiting vertical selecting layer, plane division and development technique policy in the future.

The invention discloses a traffic accident severity prediction method applied to a regional road network. The method comprises the following steps: 1, acquiring and preprocessing regional road networktraffic accident data; 2, based on the regional road network traffic accident data, establishing a potential category analysis model; 3, respectively establishing a CART decision tree model for eachsub-category according to a potential category analysis result; and 4, respectively establishing an accident severity model (considering independent variables and interaction items) based on binary logistic regression for each subcategory, and taking intersection points of sensitivity and specificity curves as model prediction classification thresholds. According to the method, the adverse effectof accident data heterogeneity on an analysis result can be reduced, the problems that a traditional traffic accident severity prediction model ignores interaction items and the comprehensive prediction effect of unbalanced data is poor are solved, and the prediction precision and goodness of fit of the accident severity model are improved.

Disclsoed is a predictive measurement model of connectivity strength between vehicle nodes based on spatio-temporal data analysis of vehicle networking in urban scenarios. Aiming at the connectivity problems caused by spatio-temporal data heterogeneity and topology frequent changes in vehicle networking, a neural network model based on tensor factor aggregation is constructed to predict the connectivity strength between vehicle nodes, thus providing model support for the connectivity research of vehicle networking without infrastructure and with infrastructure.

The invention discloses a suspicious event detection method and system based on URL heterogeneity. The method includes the steps of firstly, capturing network data packets in the user sending direction; secondly, analyzing the network data packets, and extracting URLs in the network data packets; thirdly, judging whether the URLs are highly-suspicious events or not based on the detection rule in a knowledge base, conducting deep detection if the URLs are highly-suspicious events, and otherwise, conducting detection through a preset detection scheme; fourthly, adding or deleting the detection rule as required, judging whether server ports required by the URLs are system reserved ports or not, judging the URLs to be security events if the server ports required by the URLs are the system reserved ports, and judging the URLs to be the highly-suspicious events if the server ports required by the URLs are not the system reserved ports; fifthly, judging whether domain names of the URLs are meaningful words or not, judging the URLs to be security events if the domain names of the URLs are meaningful words, and judging the URLs to be the highly-suspicious events if the domain names of the URLs are not meaningful words. The suspicious event detection method and system solve the problem that a traditional detection method has an effect on known malicious URLs and has no effect on unknown or non-captured URLs.

The invention discloses a CA-NARX water quality prediction method based on meteorological factors, and belongs to the technical field of intelligent water quality prediction data application. The method comprises the following steps: 1, ; data standardization, 2, creating a sample matrix, 3, determining an initial clustering center according to the quantile; (4) initial clustering is carried out according to the Euclidean distance; (5) the mean value of each class is used as a new clustering center; (6) clustering is carried out in batches according to the Mahalanobis distance between each sample and the clustering center; (7) clustering number screening is carried out; (8) the best clustering number is selected; according to the method, the problems of high cost and low prediction accuracy of water quality prediction of small and medium-sized reservoirs are mainly solved. Meanwhile, the problem that a traditional clustering algorithm is inapplicable to data heterogeneity and differentvariances is solved, and the training accuracy of the NARX neural network is improved to a certain extent.

The invention discloses a data integration system for grid state detection, and an implementation method of the data integration system. The method comprises the following steps of: classifying grid state detection systems according to detection objects; establishing a special database for storing original detection data of various state detection systems; establishing a standard database for storing unified grid equipment ledger information; establishing standard data formats of various state detection systems, converting the original detection data into the standard data formats, and associating the standard data formats with the corresponding grid equipment ledger information; and publishing the data. By the invention, the problems of inconvenience in use, difficulty in integration and the like caused by difference in transmission modes of different manufacturers of the grid state detection systems and data heterogeneity are solved. Meanwhile, a unified and integrated data storage and publishing mode is provided, so that a user can acquire all data through a platform. By establishing the unified grid equipment ledger information in the standard database, the equipment ledger maintenance workload of a maintainer is reduced, working efficiency is improved, and labor cost is reduced.

The invention relates to the field of atmospheric pollutant space-time distribution calculation, and discloses a fine particulate matter (PM2.5) comprehensive domain space-time calculation inference method based on multi-source city big data. According to the method, PM2.5 concentration data of a fixed station and a sensor, satellite remote sensing aerosol optical thickness and other environment covariants are collected, an iteration vacancy filling-machine learning model is established, and the problems of data heterogeneity, space-time insufficient matching, sampling deviation and the like existing in multi-source data fusion are effectively solved. According to the method, multi-source data of a fixed station, a sensor, satellite remote sensing and the like are flexibly and efficiently fused, high-resolution spatial-temporal distribution of PM2.5 can be more accurately reconstructed, a high-resolution spatial-temporal distribution result of PM2.5 hourly concentration of a 1km grid is formed, the method is an important technical basis for realizing fine management and control of air quality, high potential pollution sources can be explored and positioned in real time, and finally, pollution discharge can be monitored and controlled in a targeted manner.

The invention relates to the field of federated learning, in particular to a heterogeneous model aggregation method and system based on federated learning, and the method comprises the steps of initializing a neural network model; the method also includes that each client contributes a part of local data and uploads the local data to the server to form a shared data set, and a CGAN model is trained; the client uses a local data set and a data set generated by the CGAN model to train a local model, predicts each data in the shared data set and uploads a prediction score to the server; the server calculates the prediction score deviation degree of each client, takes the reciprocal of a calculation result as a weight, calculates a global prediction score, and uses the global prediction score to perform knowledge distillation on the server model; the client downloads the prediction scores of other client models from the server for cooperative training; and model convergence is performed after multiple iterations. According to the invention, the problem of data heterogeneity of the client side can be solved, the client side model uploads and downloads the prediction score of the shared data set, and the communication traffic between the client side and the server side is reduced.

The invention belongs to the technical field of big data, and particularly relates to a research and judgment platform based on big data analysis. The research and judgment platform based on big dataanalysis comprises a big data analysis system, and the big data analysis system comprises a data acquisition and access part, a data storage part, a computing service part, an algorithm library, an analysis model, a system assembly, a business application part, a standard and safety system and an operation and maintenance management platform. The current situations of data heterogeneity, sharing difficulty, data chimney, application island and the like are solved; the problems of fuzzy construction specifications, incapability of intercommunicating systems, different standards, different dataformats, disordered name use and the like are solved; the problems that the information sharing amount and the sharing convenience degree cannot be guaranteed, the information amount is too small, theinformation sharing amount is influenced in turn, and a vicious circle of information sharing is formed are solved. The problems that data applications such as data cleaning, data management and dataanalysis are lacked, and functions such as intelligent analysis, intelligent early warning and intelligent handling of data are lacked are solved.

The present invention discloses systems and methods for measuring intra-tumor heterogeneity based on genetic information of a tumor. Such systems and methods may indentify genetic information of mutation specific to the tumor, determine a mutant-allele fraction for each mutated locus, calculate mutant-allele tumor heterogeneity (MATH), and measure the distribution of mutant-allele fractions among tumor-specific mutated loci of the tumor.

The invention relates to the technical field of document reading, in particular to a heterogeneous legal data-oriented multitask reading system and method, and the system comprises the following modules which are sequentially connected: a data input module which is used for inputting statistical and textual legal data; a data preprocessing module which is used for carrying out data cleaning and data conversion on the law data; a data analysis module which is used for analyzing the preprocessed data; a reading result processing module which is used for integrating the analyzed data to form structured reading result data; and a result pushing module which is used for feeding back the reading result data to the law researcher. Statistical analysis and machine reading understanding technologies are used at the same time, structured data such as statistical yearbook and unstructured data such as judgment documents, file materials and interview text records can be processed at the same time,and the problem of data heterogeneity is solved.

The invention provides a method and system for detecting an overflow loophole based on format heterogeneity and a storage medium. The method includes the steps that a network data packet is acquired and restored into files, the files are distributed to each detection branch according to file types, structure features of the file in each detection branch are extracted, target detection features aredetermined through the calculation of the difference degree, then clustering analysis is conducted on the features of each target detection feature, and the obtained file corresponding to the small part of the features is the file with an abnormal structure. According to the method, through a clustering algorithm of the structure features, a small part of abnormal files in a large number of filescan be quickly distinguished, deep detection is conducted only on the abnormal files, the efficiency of detecting the overflow loophole is effectively improved, and an inspired detection capability for unknown overflow loopholes is achieved.

The invention belongs to the technical field of dual-redundancy servo control systems, andparticularly relates to a real-time redundant communication system based on RS422 and CAN bus heterogeneity, which is used for communication of different intelligent single machines of a servo system for rocket flight control, realizes high-reliability communication with relatively low cost, has self-diagnosis and self-recovery functions, and ensures high reliability of flight control systems such as a rocket. The system comprises an intelligent single machine I and an intelligent single machine II, the intelligent single machine I comprises an RS422 interface and a CAN interface, and the intelligent single machine II comprises an RS422 interface and a CAN interface; An RS422 bus is arranged between the two RS422 interfaces, and a CAN bus is arranged between the two CAN interfaces; And combining the effective data of the RS422 into a message frame, each frame of data comprising a frame header, a frame length, a frame identifier, a data area, a check and a frame tail, the frame header and the frame tail being fixed byte bytes, and the frame length representing the total length of the effectivedata needing to be sent.

The invention relates to the field of credit risk assessment, and aims to provide a financial risk control cold start modeling method based on unsupervised field self-adaption. The method comprises the steps of data input and preprocessing, variational auto-encoder mapping, domain alignment based on adversarial, sample adaptive weighting based on weighted adaptation, pseudo-tag iterative optimization, parameter optimization and result output. The risk control modeling algorithm can be applied to the cold start stage of new service online lack of label samples, is suitable for a cold start scene without label samples, and is higher in precision and better in effect in a financial risk control scene. The problem of negative migration in existing migration learning and the problem of data heterogeneity in the financial risk control field can be effectively solved. The model training and deployment framework is high in generalization, can be effectively applied to other service scenes, and has good adaptability and mobility.

The invention discloses an Agent-based heterogeneous geographical information public service platform operation and maintenance data collection method. The method comprises the steps of establishing operation and maintenance data structure standards and operation and maintenance information classification standards; according to a geographical information public service platform operation and maintenance interface, establishing an exclusive adaptive Agent, standardizing data, and providing operation and maintenance data in a REST service form; calling an adaptive Agent service to obtain the operation and maintenance data by a report Agent, adding data tags for the data, and reporting the data to an aggregate Agent; and receiving the reported operation and maintenance data by the aggregateAgent, determining operation needed to be performed for the operation and maintenance data according to the data tags, collecting the data to a database, and returning a collection result to the report Agent. For the situations of platform heterogeneity, operation and maintenance data heterogeneity, different network environments where a platform is located and the like in geographical informationpublic service platform construction, unified data structure standards are established, and unified collection of the heterogeneous geographical information public service platform operation and maintenance data is realized.

The invention provides a system and a method for drawing a visual data analysis graph based on gplot2, and the system comprises a user instruction module which is used for receiving a user instruction sent by a user; the import module is used for importing input data according to the user instruction; the drawing module is used for drawing a data analysis graph and comprises a drawing unit; the drawing unit comprises a single graph model and a combined model; the single graph model comprises a scatter diagram, a box plot and a violin graph; the combined model is a combination of two single-graph models, and comprises a scatter diagram and a box plot, a scatter diagram and a violin graph, and a box plot and a violin graph; the drawing module takes the received user instruction as a parameter to call each unit for drawing; and the export module is used for exporting the data analysis graph. According to the method, breakthroughs are made in multiple aspects of graph combination, form diversification, color specialization, data heterogeneity and the like, and richer and more professional choices are provided for data analysis visualization.