55 results about "Pooled data" patented technology

The invention provides a data association maintenance processing method and system based on file reference, and relates to the technical field of data association maintenance, which comprises the following steps: constructing a directed reference graph, identifying direct and indirect reference paths, and constructing a reference dependence intensity matrix in combination with a time sequence, an interaction amount and a reference propagation probability. And carrying out file clustering based on the matrix, and determining a core reference file set. And when the core file is updated, acquiring incremental update data by adopting data block level difference comparison, and merging the data blocks based on the content similarity and the semantic association degree. After the merged data blocks are segmented according to threshold values and semantic boundaries, updating is propagated to a target node through a multi-path updating propagation network, and the data consistency is ensured through version consistency check and a compensation updating mechanism. According to the method, data association between files can be effectively maintained, and the data updating efficiency and consistency are improved.

The invention provides a performance optimization method and system for cross-data-source paging query, and relates to the technical field of data processing, and the method comprises the steps: calculating a data distribution weight and a data source routing table, determining an optimal data access path, and splitting a paging query request into sub-query tasks; dynamically adjusting the number of parallel query threads based on the load state and distributing sub-query tasks; establishing a local index and adaptively adjusting an index structure; dynamically adjusting the connection number of the connection pool and acquiring data in batches; performing feature analysis according to a query result, determining the size of a data fragment, and selecting an optimal merging strategy to merge the data; constructing a cache access mode graph, and aggregating and storing related data based on the access association degree; and optimizing a performance strategy set through reinforcement learning, adjusting a cache updating interval, a data access path and a feature matrix, and finally obtaining a query result. According to the method, the cross-data-source paging query efficiency can be effectively improved, the query delay is reduced, and the overall performance of the system is improved.

The invention relates to the technical field of data management, and discloses a data management platform and management method for tumor immunoassay. The method comprises the following steps: performing standardized acquisition and preprocessing on multi-source tumor immune data to obtain a standardized data set; performing wavelet decomposition and dimension reduction processing on the standardized data set to obtain a dimension reduction feature set; performing cross-center data calibration on the dimension reduction feature set to obtain a calibrated merged data set; performing annotation expansion on the calibrated merged data set to obtain a complete annotation data set; and performing immune feature weight calculation and hierarchical data management according to the complete annotation data set, and generating a hierarchical tumor immune data management structure with differentiated storage priorities. The method is different from a traditional unified storage mode, the heterogeneity of the tumor immune microenvironment is considered, differentiated storage priorities are distributed according to the immune value of the sample, and the computing resource distribution and data access efficiency is optimized.

The invention discloses a technical trade measure data automatic sharing method and system based on artificial intelligence, and belongs to the technical field of data sharing, and the method comprises the steps: a control center receives a technical trade measure new version data entry signal, carries out the judgment of a target data sharing reliability decision pool, and carries out the data sharing; if the judgment result of the target data sharing reliability decision pool is a secondary judgment pool, re-judgment of the target data sharing reliability decision pool is carried out, and if the re-judgment result of the target data sharing reliability decision pool is an experimental sharing pool, the formal sharing qualification of the target data is confirmed, and the automatic sharing process of the target data is executed. The method can guarantee the safety and stability of data sharing, accurately judges the actual application value of the data through experimental sharing and sharing value scoring analysis, ensures that the data entering a wide sharing link can provide an accurate basis, continuously monitors the data of the observation pool, reasonably cleans invalid data, improves the data management efficiency, and is suitable for popularization and application. And the accuracy, reliability and practicability of data sharing are improved.

The present invention relates to the technical field of FPGA bitstream compression, and a method and system for fast compression of FPGA bitstreams, including: traversing line numbers in a target multi-layer index structure, identifying a data block group, a feature value group, and the number of data blocks corresponding to the line numbers, determining whether the number of data blocks is a preset number, if it is the preset number, then determining whether there is a subsequent single data block group set, if there is, then merging the data block group with the subsequent single data block group set to obtain a merged data block group, if not, then using the data block group as the original data block group, if it is not the preset number, then determining whether there are the same feature values in the feature value group, if there are, then compressing the data block group to obtain a compressed data block group, if not, then using the data block group as the original data block group, and performing bitstream recombination on the merged data block group set, the compressed data block group set, and the original data block group set. The present invention can solve the problems of poor overall compression effect of FPGA bitstream data and relatively large consumption of computing resources.

The invention discloses a distributed storage method and device based on a time sequence database, and relates to the technical field of data storage. Obtaining original time sequence data and carrying out time clustering segmentation to obtain a time sequence data fragment set; performing data merging on the time sequence data fragments in the time sequence data fragment set to obtain a merged data fragment set; determining a storage position of each merged data fragment in the merged data fragment set; constructing a time query index for the data in each merged data segment; and for the data in each merged data segment, obtaining a storage address of the data, enabling the storage address to correspond to the time query index of the data, and storing the data in the storage address. According to the method, data of a time sequence is clustered and then segmented, then the segmented data is merged, storage fragmentation is reduced, finally, a storage address and a time query index are associated by constructing a query tree, efficient storage is still met in the face of mass data, and the data management efficiency is improved.

The invention relates to the technical field of disability care modeling, and discloses a disability crowd care consultation digital twinborn modeling system based on a Longson scale. The system comprises seven functional modules, namely a data acquisition module, a data processing module, a condition screening module, a condition merging module, a time identification module, a digital twin modeling module and a decision-making module. The data acquisition module acquires the evaluation data of the Lungson scale of the disabled crowd, and the data processing module performs abnormal value processing and data filling on the data; the condition screening module is used for screening data meeting initial conditions and termination conditions of the care transient process from the processed data; the data merging module merges the two types of screened data, generates merged data and identifies the end time of the care transient process; the time identification module determines start time by using the merged data; the digital twinborn modeling module constructs a care process digital twinborn model, and the decision making module matches care decisions according to the start and end time of a care transient process, and assists in improving the care consultation accuracy and normalization of disabled people.

The present invention belongs to the field of distributed data storage and synchronization in information engineering, and specifically relates to a distributed experience feedback system data offline synchronization system and method. It mainly includes the following steps: application of system eigenvalue algorithm; data set export process based on JSON format; data encryption and decryption process based on eigenvalue algorithm. Its beneficial effect is: adopting a JSON-based data set mode without using SQL mode, supplemented by a data recovery algorithm, each data in the JSON data set is restored and compared with the target system data, so as to achieve the effect of securely merging data and avoid accidental deletion and overwriting of data. The data validity period is also set when the file is exported, and the system also generates a check code based on the validity period and the target system feature code. When importing, the check code can be used to judge the validity period. If the file exceeds the validity period, it cannot be imported, which improves the fault tolerance during a certain import.

The invention belongs to the technical field of artificial intelligence, and relates to a data exception monitoring method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining compensation data in a preset time period; constructing a graph model based on the claim data; performing feature extraction on the graph model to obtain graph features, and merging the graph features with the compensation data to obtain merged data; splitting the merged data based on the time sequence to obtain target data; performing clustering analysis on the target data based on a target clustering algorithm to obtain a plurality of clustering groups; performing index analysis on all the clustering groups based on the target index, and judging whether an abnormal claim group exists in all the clustering groups or not; and if yes, executing alarm processing corresponding to the abnormal claim group. In addition, the invention also relates to a block chain technology, and the target clustering algorithm can be stored in a block chain. The method can be applied to case abnormity monitoring scenes in the fields of financial science and technology and medical health insurance, and the recognition accuracy of repeated claim is effectively improved.

According to examples, an apparatus may include a processor and a memory on which are stored machine-readable instructions that when executed by the processor, may cause the processor to receive an updated data record from a data source and may determine a first conflated data record. The first conflated data record may be associated with the updated data record and include data records in a first grouping from among a plurality of data sources. The processor may identify the data records included in the first conflated data record and may generate a second conflated data record that updates conflations among the updated data record and the identified data records. The second conflated data record may include a second grouping of data records. The processor may replace the first conflated data record with the second conflated data record to incrementally update a set of conflated data records.

This invention discloses an adaptive multi-table data normalization method based on RPA and OCR technologies. Using RPA tools and OCR technology, this invention can synchronously adjust the corresponding data content in the "Floating Summary" and "Normal Transfer" sub-tables of the summary table after the client modifies the data. Then, it performs data aggregation and merging on the two sub-tables separately. Finally, after confirming that the data does not need modification, the data in the summary table is finally aggregated and merged into the group's audit standard notes to the financial statements, thereby generating an audit report. This invention achieves automated merging and aggregation processing of massive amounts of single-file data through RPA and OCR technologies. Under the premise of intelligently identifying basic data table information, quickly matching and capturing data, and accurately summarizing data, it achieves the normalization and reverse writing of audit standard notes to the financial statements, solving the problems of traditional manual data aggregation and merging methods, which may result in aggregation errors, large workloads, and high labor costs due to repeated data adjustments and modifications.

When a merging condition is present, first processing element circuitry is responsive to control signals received the decoding of N data update instructions that each specify updated data to be made available for stashing to buffer the updated data. When the merging condition is no longer present in response to a stashing specific merge end trigger, the first processing circuitry issues a merged data update stashing transaction to interconnect circuitry specifying, as merged updated data, the updated data, to trigger stashing control circuitry to cause the merged updated data to be made available for stashing in an associated storage structure of at least one further processing element.

The embodiment of the invention provides a rendering method and system for large-scale data in a port scene. The method comprises the following steps: performing preliminary screening on obtained large-scale data about a target port to obtain preliminary screening data, and classifying the preliminary screening data into structured data and unstructured data; performing standardization processing on the structured data and the unstructured data to obtain standard structured data and standard unstructured data; in response to the standard structured data and the standard unstructured data, completing merging, and performing optimization processing on the merged data to generate optimized data; in response to completion of verification of the optimization data, performing three-dimensional modeling and rendering based on the optimization data passing verification to generate a three-dimensional model; and pushing the visual information representing the three-dimensional model to a display terminal. By utilizing the rendering method disclosed by the embodiment of the invention, the overall efficiency of large-scale data processing and the system stability can be remarkably improved.

The present invention discloses a verification method, device, equipment and storage medium for data processing tasks, the method comprising: obtaining source full data of a source server and source incremental data generated thereon; performing a data processing task on the source full data and the source incremental data according to the source data processing logic of the source server to obtain source result data; performing a merge data processing task on the source full data and the source incremental data according to the target data processing logic in the target server to obtain target result data; and verifying whether the two result data are consistent. In the present invention, by making the basic data and change data of the target server and the source server consistent, the data results are calculated and verified respectively, which refines the verification method, reduces the possibility of data errors, and makes the verification method applicable to more scenarios.

The invention provides a merged task scheduling method and device, and relates to the technical field of cloud computing, in particular to the technical field of distributed table storage. A specific embodiment of the method comprises the steps of obtaining to-be-processed data; in response to triggering of the merging task, grouping the to-be-processed data according to the merging period, and generating a to-be-processed data group; based on the data volume of the to-be-processed data group and the merging period, determining the to-be-processed data volume of the current time period; selecting to-be-processed data matched with the to-be-processed data size from the to-be-processed data set according to the priority; and merging the selected to-be-processed data in the current time period to generate merged data.

The disclosure generally shows systems and methods for optimizing MVNO subscriber mappings to pooled data pricing plans offered by MNOs with different tiered data pools are variously linked to different MVNO subscribers. The system receives and processes raw CDR data provided by the MNO. The system may then poll a backend server to receive data concerning subscriber ID assignments and subscriber activations / deactivations for the MVNO. Then the system determines optimal plan changes for each subscriber based on expected usage and pushes those plan changes to a Wholesale API Proxy that communicates with the MNO to implement the mapping changes to the pooled data pricing plan.

The invention provides a non-contact offline data transmission method, system and equipment in a power system, and the method comprises the steps: obtaining a plurality of data files of different types at a coding end, carrying out the merging processing of the plurality of data files, and obtaining a merged data file; performing fragmentation processing on the merged data file to obtain a plurality of animation groups matched with the sampling frequency of the code scanning ends and the display refresh rate of the coding end, wherein the number of the animation groups corresponds to the number of the code scanning ends; performing data acquisition on the visual codes in each animation group by using a plurality of code scanning ends to obtain acquisition data, and sending target transmission data determined based on the acquisition data to a decoding end; and at the decoding end, decoding the target transmission data sent by the different code scanning ends to restore a plurality of data files. By adopting the non-contact off-line data transmission method, system and equipment in the power system, the reliability and fault tolerance of large file transmission under the condition of no network and no contact are improved.

The invention discloses an automatic verification method and device for MEC cloud pool data. The method comprises the following steps: firstly, acquiring CMDB data of the distributed cloud; acquiring real-time data of the distributed cloud; comparing the CMDB data with the real-time data to obtain a comparison result; and automatically synchronizing the real-time data to the CMDB data, which is inconsistent with the real-time data, in the comparison result, so as to obtain CMDB synchronous data, then detecting the CMDB synchronization data to obtain a detection result; and according to the real-time data, carrying out automatic restoration on an abnormal value in the detection result to obtain CMDB restoration data, and finally, integrating the CMDB synchronous data and the CMDB repair data to obtain CMDB automatic check data. According to the method, automatic verification of the CMDB data is realized, and the efficiency of resource management in the cloud pool and the accuracy of the data are effectively improved.

The invention provides a data merging method and a related device. The data merging method comprises the following steps: firstly, dividing received block data into at least one group of to-be-merged data; for each group of to-be-merged data, data conflict detection can be carried out according to the node path indexes recorded in the to-be-merged data, so that whether data node information with the same node path index exists in each group of to-be-merged data or not is detected. If yes, according to a source data timestamp corresponding to the to-be-merged data and a preset data priority, one piece of node information is determined from multiple pieces of data node information with the same node path index to serve as insertion node information. And carrying out data merging according to the determined inserted node information and the corresponding data type description. Therefore, the problem of data conflict in a data merging scene can be solved, so that the data can be accurately and completely restored according to the block data, and the data quality of the merged data can be improved.

The invention relates to a method for validating and combining data sets (4, 5, 6) for a driver assistance system (9) of a vehicle (8). The method comprises the steps: - generating a plurality of data sets (4, 5, 6), each of the data sets (4, 5, 6) originating from an external data source (1, 2, 3) located outside a vehicle (8) having a driver assistance system (9) for which the data sets (4, 5, 6) are intended, - Checking the data records (4, 5, 6) for their data integrity, - Validate the data records (4, 5, 6) if the data integrity check has resulted in a positive result, - generating a combined data set (23) by combining at least a selection of the data sets (4, 6) when a predetermined minimum number (m) of the plurality (n) data sets (4, 6) has been validated, and - transmitting the combined data set (23) to the vehicle (8) for use of the combined data set (23) in its driver assistance system (9).

The invention discloses a dimension reduction and equalization method for high-dimensional unbalanced data of a wind turbine, and the method comprises the steps: removing singular values in a to-be-processed high-dimensional unbalanced data set, and obtaining an updated data set; performing normalization processing on the updated data set to obtain a normalized data set; performing oversampling on the icing data set in the normalized data set to obtain an oversampled data set; carrying out undersampling on an unfrozen data set in the normalized data set to obtain an undersampled data set; and merging the oversampling data set and the undersampling data set to obtain a merged data set, and performing standardization and dimension reduction processing on the merged data set in sequence to obtain a dimension-reduced balanced data set. The dimension reduction equilibrium data set obtained by the invention can effectively identify the early characteristics of icing of the wind turbine blade of the wind turbine generator, equipment maintenance can be carried out in time, the occurrence risk of disastrous accidents such as blade breakage and gearbox failure is reduced, stable operation and economic benefits of a wind power plant are ensured, and meanwhile, the difficulty of data processing and analysis is reduced.

The invention provides a form data incremental interaction method and device under a front-end and rear-end separation architecture, and relates to the field of incremental data processing.The method comprises the steps that if a received data processing request needs to be subjected to rear-end incremental processing, local cache data are extracted according to a cache serial number in the data processing request; combining incremental data in the data processing request with the local cache data to obtain combined data; performing back-end increment processing on a service data processing result obtained by performing service processing based on the merged data, and returning a compressed back-end increment processing result to the client; wherein the business data processing operation comprises business data extraction, business data calculation and business data statistics. The invention can provide an efficient, reliable and safe incremental interaction data processing method so as to solve the problems of large bandwidth consumption, long processing time, insufficient security and the like in the existing full-quantity synchronization technology.

Various embodiments described herein support or provide operations for facilitating the reconciliation and storage of conflicting user data across multiple platforms. Specifically, an audit record that includes a plurality of data records of an attribute is identified. A consolidated data record of the attribute in a summary record is generated. A current value of the attribute is determined based on the consolidated data record of the attribute in the summary record. The current value of the attribute is stored in a final record for downstream processing.

The invention relates to the technical field of data processing, can be applied to business scenes such as financial science and technology and medical health, and discloses a storage optimization method, device and equipment based on file generation analysis and a medium. Constructing a to-be-merged data set and writing the to-be-merged data set into a merged file according to a partitioning strategy, monitoring newly-added small files and extracting features, generating a feature vector set, training an analysis model to generate a file generation trend analysis result, and adjusting the partitioning strategy based on the result to optimize a subsequent writing operation. According to the method, the trend analysis model based on feature vector training is introduced, the future small file generation mode is dynamically predicted, the partition strategy is adjusted accordingly, accurate write-in control over the to-be-merged data set is achieved, and therefore the number of small files is remarkably reduced, storage overhead is reduced, and query performance and storage efficiency are improved.

The present invention provides systems and methods for auditable data. Embodiments of the data management system facilitate the storage and management of various data (e.g., information about transactions, customers, infrastructure, geographical locations, medical records, etc.) to support data creation, manipulation, storage, and / or sharing by different parties. Users can create new and / or update data, digitally sign the data, and store the digital signature and the data on an immutable storage using the data structures of the data management system. Using a data structure in which data and associated metadata are stored on an immutable storage in association with corresponding data and user digital signatures can be used to provide data traceability and audibility. Snapshots of merged data can be implemented for data with many updates to avoid latency when querying the data by merging the updated data. A collaborative rating mechanism can provide a measure of trust in the data.

The present application relates to the technical field of audit data integration, and specifically relates to a heterogeneous data integration method for off-site audit, a computer device and a medium. For different data groups in a single data type, the method identifies data groups with missing collection and combines the data groups with missing collection, and identifies data groups with repeated collection and combines the data groups with repeated collection, to obtain merged data corresponding to each data type. Based on the merged data, the method performs natural language processing and one-hot encoding on data of different data types, forms column vector combinations between different data categories of each data type, analyzes the matching degree between any two column vectors in the column vector combinations corresponding to different data types, obtains matching column vectors of the column vectors, and completes the integration between the same data categories in different data types. When performing off-site audit on the integrated data, data classification analysis can be more convenient, and the audit efficiency and accuracy can be improved.

The application discloses a data synchronization distribution method and device and a storage medium. The method comprises the following steps: using an open source framework to synchronize and take out to-be-processed data from a relational database, wherein the open source framework is a Canal open source framework developed by using a programming language java; performing grouping processing on the to-be-processed data to obtain a to-be-processed data group; using multi-threading to perform merging processing on the to-be-processed data in the to-be-processed data group, to obtain a merged data group; and performing synchronous distribution processing on the merged data group. The application solves the technical problem that the data distribution method in the prior art cannot guarantee data sequence in a high concurrency case.

The invention relates to a multidimensional data index optimization method for hydropower big data storage. The method comprises the following steps: data access and feature extraction: extracting physical dimension and service dimension attributes of time sequence data; dynamic weight calculation: calculating the comprehensive weight of the dimension combination in real time based on the query frequency, the data selectivity and the service priority; constructing and updating a hierarchical fusion index, constructing an adaptive grid partition R-tree main index and an inverted auxiliary index according to the weight, and establishing bidirectional pointer association; routing and execution are intelligently queried, an optimal index path is selected according to the weight, and data are quickly positioned and merged through a pointer; online monitoring and self-adaptive adjustment are carried out, data distribution and query performance are monitored in real time, and weight recalculation and index increment recombination are triggered; the method further comprises storage management of life cycle perception, and differential index strategies are implemented on hot, warm and cold data; the method can dynamically adapt to query load changes, improve the multi-dimensional query efficiency and reduce the storage cost.

Provided are techniques for identifying and correcting model focus drift during model training. A model is trained using an original dataset with data, and the is classified into clusters. A representation is assigned to each of the clusters. A first visualization dashboard with visualizations is generated, where each visualization represents a first data distribution of an associated cluster using the representation assigned to that cluster. The model is fine-tuned using a fine-tune dataset. A second visualization dashboard is generated by updating each visualization, where each visualization represents a second data distribution of the associated cluster. It is determined that a cluster of the clusters has focus drift based on changes between the first data distribution and the second data distribution. The focus drift is corrected by: adding the data of the cluster to the fine-tune dataset to form a combined dataset and fine-tuning the model using the combined dataset.