150 results about "Data sanitization" patented technology

A multi-source multi-tenant reference data utility and methods for forming and maintaining the same, delivering high quality reference data in response to requests from clients, implemented using a shared infrastructure, and also providing added value services using the client's reference data. Included are data cleansing and quality assurance of the received data with full tracking of the sourcing of each value, storage of resulting entity values in a repository which allows retrievals and enforces source based entitlements, and delivery of retrieved data in the form of on demand datasets supporting a wide range of client application needs. An advantageous implementation has additional services for reporting on data quality and usage, a selection of value adding data driven computations and business document storage. By using a shared infrastructure and amortizing the costs of data quality assurance across a plurality of clients, while ensuring that clients only receive values from data sources to which they are licensed, better quality data at lower cost is delivered.

The invention relates to a system and a method for privacy preservation of sensitive attributes stored in a database. The invention reduces the complexity and enhances privacy preservation of the database by determining the distribution of sensitive data based on Kurtosis measurement. The invention further determines and compares the optimal value of k-sensitive attributes in k-anonymity data sanitization model with the optimal value of l sensitive attributes in l diversity data sanitization model using adversary information gain. The invention reduces the complexity of the method for preserving privacy by applying k anonymity only, when the distribution of the sensitive data is leptokurtic and optimal value of k is greater than the optimal value of l.

The invention discloses a remote storage service method and system based on SGX. The method comprises remote storage service system initialization, data storage, data access and data cleaning operations; the remote storage service system initialization operation comprises the steps of creating a credible space for a key generation function, a key expansion function and an encryption and decryptionfunction by virtue of the SGX and generating a parent key in the credible space; the data storage operation comprises the steps of transmitting to-be-stored data to a remote storage server by a client, generating a sub key of the to-be-stored data in the credible space and encrypting the to-be-stored data; the data access operation comprises the steps of decrypting accessed data in the credible space by the remote storage server and transmitting to the client by virtue of an encrypted channel; and the data cleaning operation comprises the steps of collecting to-be-cleaned data and the sub keyby the remote storage server and deleting. The method disclosed by the invention enables data encryption and decryption processes as well as storage and access processes not to be snooped or interfered by a rogue program or other prerogative software.

A network operator intermediates a cashless transaction by sending a payment device issuer an outbound clearing data file enriched with cleansed merchant data. Issuers that choose to participate in the merchant data cleansing service will be provided with the cleansed, and optionally augmented, merchant information. Providing this data to the issuer could be used to help reduce costs associated with “do not recognize” calls by cardholders, disputed charges, and charge-backs. With cleansed data provided in the clearing record, a device holder would be given access to this cleansed information about the merchant where a transaction took place through their statement. The present disclosure also provides for augmenting the clearing record data to include merchant-specific add-on and/or aggregate data not presently included in the transaction clearing record, yet useful to the issuer both in form and content.

The invention discloses an online processing method and system oriented to streamed data. The method comprises the following steps: 1, establishing an online memory cache layer, extracting the property of the streamed data according to a key value structure, and storing in the online memory cache layer; 2, establishing a hybrid index structure for the streamed data in the memory cache layer; 3, adding an access flag bit to each streamed datum for which the index structure is established, and recording the access state of the streamed data by each analysis program, wherein the flag bits are used for flagging the register condition of the streamed data by different analysis programs; 4, cleaning data: if a certain streamed datum is accessed by all specified analysis programs in the memory cache layer, cleaning the streamed datum. By adopting the method and the system, the data reading and writing stress in a streamed processing procedure is reduced greatly, the stress of a database in a large-scale streamed data processing system can be effectively relieved, and the real-time processing speed of the streamed data can be increased.

The embodiment of the invention discloses a client side file cleaning method. The client side file cleaning method comprises the steps that a catalog of files on which data cleaning needs to be carried out is obtained; types of the files in the catalog of the files are read; whether assigned file types exist in the types of the files or not is judged; cleaning operation is carried out on the files in the catalog of the files based on judgment results. The invention further provides a client side file cleaning device. The client side file cleaning method and device aim to provide a cleaning strategy capable of cleaning user data without influencing the user, and the storage space of a cell phone is more greatly released.

The invention discloses a data consistency cloud auditing system and an implementation method. The data consistency cloud auditing system comprises a cloud view management subsystem and a distributed auditing subsystem, and is characterized in that the cloud view management subsystem and the distributed auditing subsystem perform data interaction through a protocol interaction module. The implementation method comprises the steps that each service bearing network element acquires original data files according to format specification requirements; the distributed auditing subsystem starts data comparison of corresponding auditing service/network element in a time mode according to configuration dispatching information, completes difference data analysis and reports a result of data comparison and difference analysis to the cloud view management subsystem; and the cloud view management subsystem sends a synchronous cleaning operation request to the distributed auditing subsystem so as to implement and feed back a difference data cleaning result. The data consistency cloud auditing system disclosed by the invention is flexible in use, high in resource utilization rate and high in information aggregation ability while ensuring the service data consistency among the network elements through the above principles.

The embodiments of the invention provide a method and a device for cleaning petroleum geology software data. The method comprises the steps: acquiring a cleanup strategy file, wherein the cleanup strategy file comprises a plurality of cleanup tasks for different database users and the default execution time, corresponding user information and user information corresponding cleanup manner of each cleanup task; determining a cleanup task, of which the default execution time is current time, in the cleanup strategy file; executing the determined cleanup task in the related user information corresponding cleanup manner by using the related corresponding user information. The method and the device have the advantages that manual data cleanup can be avoided, the time and labor are saved, and the accidental deletion rate is lowered; meanwhile, the cleanup tasks can be executed in an automatic and timed manner, and data of the different database users can be subjected to different cleanup respectively according to user information, so that the different database users can clean respective data in an independent and targeted manner, and the improvement on data security is facilitated.

The invention provides a remote sensing image data archiving management system and method, and the system comprises a data integration unit which is configured to integrate received spaceflight and aviation remote sensing image data; a data archiving unit which is configured to classify the data, wherein different types of data are stored in different file forms; a data processing unit which comprises a plurality of task processing nodes, wherein the task processing nodes process corresponding tasks in parallel, and the tasks comprise a plurality of data query, data cleaning, data scheduling,data feature extraction, three-dimensional image global optimization matching, suspicious matching detection, block adjustment and digital surface model extraction and splicing; and a data storage unit which is configured to store the original data and processing results of the aerospace and aviation remote sensing images. According to the invention, unified and efficient management and release ofmassive multi-source heterogeneous remote sensing image data can be realized, and the data sharing service level is effectively improved.

The invention discloses a cache data cleaning method and device, cache data cleaning equipment and a computer readable storage medium, and relates to the technical field of internet. The predicted cache growth amount of the circular queue can be predicted, the current total data amount can be counted, and when the sum of the predicted cache growth amount and the total data amount is greater than the cleaning threshold, the target cache nodes with low importance coefficients are cleaned, so that the lowest importance of the cleaned cache data is ensured, the utilization value of the cache spaceis improved, and the user viscosity is relatively high. The method comprises the steps of predicting a predicted cache growth amount of a circular queue according to a historical cache growth amountof the circular queue in a historical time period; counting the total data amount of the cache data currently stored in the circular queue, and calculating the sum of the total data amount and the predicted cache increment; when the sum is greater than or equal to the clearing threshold value, calculating an importance coefficient of each cache node based on the node position and the node access rate of each cache node in the circular queue; and cleaning the cache data of the target cache node in the circular queue.

The invention relates to a method and a device for cleaning data. The method comprises the following steps: determining whether the difference between a latest access time and a current time of to-be-cleaned data stored in a current scan node in a data structure, which comprises a plurality of nodes and is used for saving latest access time and document information of to-be-cleaned data, is greater than a preset threshold or not; determining whether the to-be-cleaned data stored in the current scan node in the data structure is in a white list or not if the difference is greater than the threshold; continuing to scan the next node in the data structure if the corresponding to-be-cleaned data in the current scan node in the data structure is in the white list; and cleaning the corresponding to-be-cleaned data in the current scan node if the corresponding to-be-cleaned data in the current scan node in the data structure is not in the white list. By adopting the technical scheme, the memory space of a system can be released, and the running speed of the system can be promoted.

A method for determining present or future trends, and providing a recommendation based on those trends is provided. The method includes receiving raw data, where the raw data is data which has not been cleaned or normalized, cleaning and normalizing the raw data, creating historic data via machine learning, comparing the cleaned and normalized data with the historic data, generating a model based on the compared cleaned and normalized data and the historic data, wherein the model generates one or more determinations, and providing the one or more determinations for use by a recommendation engine or a user. Additionally, a method of collecting specific data is provided. The method includes receiving a survey and additional information provided by a panelist on a mobile application, filtering and organizing the panelists, storing the collected information, providing the stored data to a server for cleaning and normalizing, and providing the panelist with rewards.

The invention belongs to the field of communication, and provides a buffer data management method and device, computer equipment and a storage medium. The method is applied to an intelligent terminal.The method comprises the following steps of obtaining buffer data removal reference data; judging whether or not the buffer data removal reference data conforms to preset buffer data removal conditions; when it is determined that the buffer data removal reference data conforms to the preset buffer data removal conditions, removing the buffer data of the intelligent terminal according to preset rules. According to the embodiment, when the buffer reaches a certain value or other preset conditions are met, the intelligent terminal can remove buffer automatically, a user does not need to operatemanually, the user's time is shortened, and the user experience is improved.

The invention discloses an investment and financing data management method and device, computer equipment and a storage medium. The investment and financing data management method comprises the stepsof crawling investment and financing news data on the Internet and business data of a preset enterprise through a preset web crawler; performing data cleaning processing on the crawled data, and storing the data subjected to the data cleaning processing into a Hive data warehouse; extracting news text data from the Hive data warehouse, and extracting entities from the news text data through an NLPmodel; establishing an investment and financing knowledge graph according to entities extracted from the news text data, and storing the investment and financing knowledge graph into a Neo4j graph database; storing the statistical data and the redundant attribute data in the Hive data warehouse into a Mysql relational database; and storing the complex data in the Hive data warehouse into an ES database. The investment and financing data management method and device can visually display the relationship between investment and financing institutions through the knowledge graph, are high in theinformation extraction efficiency, and are fast in the query response.

The invention provides a business data processing method and device, electronic equipment and a storage medium, and the method comprises the steps of obtaining first business data which is business data to be written into a target database; determining a first target index matched with a target business attribute, wherein the target business attribute is the business attribute of the first business data, and each index in a target database is only used for storing business data of one business attribute; and writing the first business data into the first target index. Through the invention, the problem that unreasonable data cleaning is easily caused by cleaning business data by taking an index as a unit in an index management mode based on time granularity in related technologies is solved.

The invention discloses a big data cleaning method and device, computer equipment and a storage medium, and belongs to the technical field of big data. According to the method, the mapping relation between the service data types and the cleaning rules is configured, the cleaning rule matching table is generated, the distributed cluster is constructed, the distributed cluster comprises a plurality of sub-servers, each sub-server correspondingly processes one kind of service type data, the cleaning rules are distributed to the corresponding sub-servers according to the service data types, and the cleaning rules are distributed to the corresponding sub-servers according to the service data types. a target service data type is determined corresponding to the original data, a target sub-server corresponding to the target service data type is searched and the data cleaning is performed on the original data in the target sub-server to obtain cleaned data. In addition, the invention also relates to a block chain technology, and original data can be stored in a block chain. According to the method, automatic cleaning of different types of service data is achieved by constructing the distributed cluster, the method has high universality and adaptability, and meanwhile unified management of data cleaning rules is facilitated.

The invention provides a method and a device for cleaning cache data, and a terminal. The method comprises the following steps: regularly acquiring the use frequency of applications preset in the terminal and a reading record of the cache data of each application within a preset duration, wherein generation time of read cache data of each application within the preset duration is recorded in the reading record; determining cache data reservation days corresponding to each application according to the user frequency and the reading record; when system time reaches a preset cleaning time point, seeking to-be-cleaned data in the cache data of each application according to the cache data reservation days, and cleaning. According to the method, the device and the terminal, disclosed by the invention, self-adaptive cleaning based on an individual data cleaning model is realized, cleaning is carried out in time, flexibility and pertinence are obtained, and actual demands of a user are met, so that more intelligentization is obtained.

The invention discloses a method and system for cleaning and synchronizing data for multiple levels of platforms, and the method comprises the steps: S1, triggering data cleaning service and data synchronization service of a lower-level platform after a database of the lower-level platform is changed; S2, judging whether the data of the lower-level platform has dirty data or not, and if not, starting a data synchronization service; if the dirty data exists, performing data cleaning on the dirty data, and then starting the data synchronization service; S3, detecting online states of the lower-level platform and an upper-level platform, and if two platforms are both in a normal online state, performing data synchronization; and S4, transmitting the cleaned lower-level platform data to the upper-level platform, writing the uploaded data into a database by the upper-level platform, and completing data cleaning and synchronization for the two levels of platforms. According to the invention,reliable effectiveness of data can be ensured, and influence of dirty data of the lower-level platform on the upper-level platform is avoided; a data synchronization compensation mechanism is utilized to reduce the influence caused by network obstruction and data synchronization failure and improve the timeliness and accuracy of data synchronization.

The invention relates to an Internet of Things data acquisition system capable of continuously acquiring data. The Internet of Things data acquisition system comprises physical acquisition equipment, a cache server and a cloud server; the cache server comprises a plurality of equipment modules corresponding to the physical acquisition equipment, each equipment module comprises a data reading unit, a data storage unit, a data uploading unit and a data cleaning unit, the data reading unit reads acquired data acquired by the physical acquisition equipment and stores the acquired data to the data storage unit; the data uploading unit uploads the collected data to the cloud server, the data storage unit marks the collected data which is uploaded and not uploaded to the cloud server as an uploaded state and a non-uploaded state respectively, and the data cleaning unit periodically deletes the collected data in the uploaded state. Compared with the prior art, the system has the advantages of continuous data uploading, good security and the like.

The application discloses data cleaning methods, equipment and a computer-readable medium. One of the methods includes: sending a data cleaning request by a user end, wherein the data cleaning requestincludes file path information and file fingerprint information of to-be-cleaned files; receiving the data cleansing request by network equipment, and determining and sending a data cleansing strategy on the basis of the file path information and the file fingerprint information; and receiving and executing the received data cleaning strategy by the user end. Therefore, cleaning errors can be reduced, reliability and efficiency of data cleaning can be improved, and user experience is improved.

The embodiment of the invention provides a data life cycle management method and device. The method comprises the following steps that: first equipment calculates the data active duration of a data table based on the access condition of the data table; when the creation duration of the data table is greater than the data active duration, the first equipment determines a target processing strategybased on the type of the data table; and the first equipment performs data life cycle management on the data table according to the target processing strategy. Therefore, the operation of manual intervention is reduced, the processing efficiency of data life cycle management is improved, the security of data archiving or data cleaning is ensured, and the cost pressure of enterprise storage is reduced.

Digital data sanitization is disclosed. An indication that a data sanitization process should be performed is received. The data sanitization process is performed. Performing the data sanitization process includes determining an amount of free space on a storage device. Performing the data sanitization process further includes performing a set of one or more write operations, where performing the write operations decreases the amount of free space on the storage of the device.

The invention discloses an equipment information management method and device, electronic equipment and a storage medium, which are used for solving the technical problems that data cleaning cannot be carried out in time and the data cleaning timeliness is poor in the prior art. The method comprises the following steps: receiving a reporting request of a preset device sent by a client, wherein the reporting request carries equipment information; verifying the validity of the equipment information; after the verification is passed, obtaining a storage period of the equipment; obtaining weighted packet information corresponding to the storage cycle; calculating the survival validity period of the equipment information according to the weighted packet information and the storage period; storing the equipment information to a database according to the survival validity period, wherein the database is used for performing expiration marking and data cleaning on the equipment information according to the survival validity period.