57 results about "Data sanitization" patented technology

Systems and methods for intelligent data sanitization employing PCM and AI / ML are provided. The idea uses AI / ML to detect specific facts that needs sanitization rather than full properties in incoming records. Data sanitization is optimized using this focused method, saving computational resources. To properly manage changing data volumes, PCM shifts between Logical 0 and Logical 1 states. Logical 0 processes smaller volumes with high resistance and low conductivity, while Logical 1 processes large volumes with low resistance and high conductivity. The AI / ML module organizes and directs data to maximize resource and processing efficiency. The PCM processes data in-memory and directly overwrites, eliminating erasure. AI / ML and PCM integrate to sanitize data quickly, efficiently, and securely, improving system performance and data integrity without a central repository. The system dynamically adjusts to changing data patterns, protecting and optimizing data.

The invention relates to the technical field of thermal power plant abnormal data processing, in particular to a thermal power plant production monitoring system network security data set abnormal data identification and cleaning method, which comprises the following steps: mapping a spatial-temporal feature vector to a preset thermal power plant multi-level causal graph to obtain a data causal graph; performing information propagation and node updating on the data causal graph, and calculating an abnormal score of each node in a preset thermal power plant multilevel causal graph; identifying abnormal data according to the abnormal score and the network security data set; and cleaning the abnormal data according to the abnormal type and a preset data cleaning strategy. According to the method, correlation modeling of data on time and a system topological structure is realized through a multi-level cause and effect graph; an abnormal score is calculated through information spreading and node updating, and an abnormal event in the network security data can be accurately identified; the abnormal data is cleaned based on the preset data cleaning strategy, redundant, wrong and abnormal data can be effectively removed, and meanwhile key risk information is reserved.

Methods and systems are described herein for leveraging artificial intelligence to sanitize sensitive data and prevent the data from leaving the mobile device and / or be exposed to unauthorized third parties. More specifically, methods and systems are described for a novel and unconventional architecture for a data sanitization application, a novel and unconventional delivery format for the data sanitization model, and a novel and unconventional output format of the data sanitization model.

A system and method for automated bulk creation of Universal Resource Locators (URLs) with embedded Urchin Tracking Module (UTM) parameters is disclosed. The method processes input data, including endpoint URLs and UTM parameters, through pre-processing steps such as data cleansing and format standardization. The method appends UTM parameter-value pairs to endpoint URLs, optionally encoding the parameters. The generated tracking URLs are stored in a structured database for seamless integration with analytics tools, enabling reliable tracking and actionable insights into web traffic and user behavior.

A system for the near real-time exchange of regulatory-compliant and privacy-compliant consumer credit data between two parties (i.e., a credit bureau and a financial services institution) that facilitates the financial services institutions (i.e., lenders) to directly identify audiences of pre-screened consumers (i.e., prospects) to offer credit products without requiring the intervention of a third party, such as an agent of the bureau, employs a data clean room to facilitate the necessary data exchange. The clean room enables and expedites the presentment of firm offer of credit pre-screened offers across authenticated or authenticatable offline and digital online channels, thereby making credit available to consumers in a more timely and efficient manner while protecting consumer privacy.

The invention discloses a data processing method and device, equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: judging whether a current service table meets a preset table rotation condition or not based on the service table information of the current service table; under the condition that the current service table meets a preset table rotation condition, judging whether data cleaning of the data in the empty table is completed or not; and under the condition that data cleaning of the data in the vacant table is completed, taking the vacant table as a new current service table to respond to the service request. According to the invention, under the condition that the current service table meets the preset table rotation condition and the data in the vacant table is cleaned, the vacant table is used as the new current service table to respond to the service request. Compared with an existing mode of cleaning the data table in the service low-peak period, the method has the advantage that the table data can be cleaned under the condition that the online service is not paused by taking the cleaned empty table as the new current service table to respond to the service request.

The application discloses a systematic color SVG icon dataset construction method and device, and the method comprises the following steps: collecting SVG files, and performing data cleaning on the collected SVG files to obtain initial files; wherein the SVG files comprise SVG icons of different categories, and the initial files are color SVG files; performing standardization processing on the boundary box size of the initial files to obtain sample files; classifying the sample files, and dividing training set, verification set and test set according to the classified sample files to obtain SVG datasets. The application can improve the construction quality of color SVG datasets.

Systems and methods for intelligent data sanitization employing PCM and AI / ML are provided. The idea uses AI / ML to detect specific facts that needs sanitization rather than full properties in incoming records. Data sanitization is optimized using this focused method, saving computational resources. To properly manage changing data volumes, PCM shifts between Logical 0 and Logical 1 states. Logical 0 processes smaller volumes with high resistance and low conductivity, while Logical 1 processes large volumes with low resistance and high conductivity. The AI / ML module organizes and directs data to maximize resource and processing efficiency. The PCM processes data in-memory and directly overwrites, eliminating erasure. AI / ML and PCM integrate to sanitize data quickly, efficiently, and securely, improving system performance and data integrity without a central repository. The system dynamically adjusts to changing data patterns, protecting and optimizing data.

There is provided a data preparation function in a wireless communication network, the data preparation function comprising: one or more processors arranged to: collect data from one or more data sources in the wireless communication network; analyse the collected data to derive one or more data characteristics and to identify whether the collected data face one or more quality issues or irregularities; and prepare the collected data based on the analysis, including performing one or more of the following: data recovery to recover data missing from the collected data; data cleaning of the collected data; formatting of the collected data; or separation of the collected data into different data sets for one or more training tasks.

The embodiment of the present application provides a kind of data cleaning method and device, the method includes determining data attribute field from each business field of business scene, by data attribute field, determine the search field set from each database table, and determine the database table set matched with search field set, the source data associated with business scene is analyzed, determine the business data value of the business field to be cleaned, according to search field set and database table set, the structured query language sql data cleaning script corresponding to the business data value of the business field to be cleaned is constructed, and the relevant business data in the database table set associated with the business data value of the business field to be cleaned is cleaned by sql data cleaning script.It is thus, the scheme can realize the accurate cleaning of dirty data automatically by the constructed sql data cleaning script, so as to effectively reduce the time consumed by test personnel for cleaning dirty data.

The present invention relates to a computer implemented method for data cleansing. In the method, a data pre-processing module of a first computing instance (3) obtains an at least two-dimensional un-shuffled data structure (10). The data structure (10) comprises healthcare patient data. The data structure (10) comprises data on a plurality of patient records. Each patient record comprises one or more fields wherein each field has a field type and wherein the patient records have a pre-shuffled order. The data pre-processing module of the first computing device shuffles fields of a first field type by moving fields having the first field types from one patient record to another patient record. Fields with the first field type have a post-shuffled order after the shuffling thereby creating a shuffled data structure(lO). A communications module transmits (5) the shuffled data structure (10) comprising the healthcare patient data over a communications network (7) from the first computing instance to a second computing instance.

We provide the system. [Solution] The information processing device is a means of collecting diverse data related to multiple businesses, A means of cleaning up and standardizing inaccurate data using collected data, A means for training and updating generative artificial intelligence models based on cleaned and standardized data, A means of automatically determining whether a business can continue using a trained model, A means for generating and reporting a report that includes the judgment results and recommendations, A system that includes this.

Embodiments of the disclosed technologies receive a first-party trained model and a first-party data set from a first-party system into a protected environment, receive a first third-party data set into the protected environment, and, in a data clean room, joining the first-party data set and the first third-party data set to create a joint data set for the particular segment, tuning a first-party trained model with the joint data set to create a third-party tuned model, sending model parameter data learned in the data clean room as a result of the tuning to an aggregator node, receiving a globally tuned version of the first-party trained model from the aggregator node, applying the globally tuned version of the first-party trained model to a second third-party data set to produce a scored third-party data set, and providing the scored third-party data set to a content distribution service of the first-party system.

The application discloses a data cleaning method and a terminal of a database, wherein slow query logs in a first database are monitored, and when the slow query logs exceed a preset number to generate an abnormal database table list, it indicates that there is a possibility of abnormal data in the database. A test database table is created in a second database according to the abnormal database table list, and test data corresponding to the data amount of the abnormal database table is added to the test database table. In this way, the deviation value of the size of the abnormal list can be determined according to the size of the abnormal list and the test list. When the deviation value reaches a threshold value, it is considered that there is a data hole in the list, and data cleaning is automatically performed for the abnormal list, thereby reducing the abnormal data stored in the database table and improving the overall system performance.

The embodiment of the application belongs to the field of artificial intelligence, and relates to a title generation method based on artificial intelligence, comprising the following steps: acquiring a pre-acquired title data set; cleaning titles in the title data set based on multiple data cleaning rules to obtain target text data; using the target text data as training data to train a pre-trained language model based on an adversarial training strategy and a noise reduction strategy, to obtain a title generation model; acquiring a target text to be recognized; inputting the target text into the title generation model, and processing the target text by the title generation model to generate a corresponding target title. The application also provides a title generation device based on artificial intelligence, a computer device and a storage medium. In addition, the application also relates to blockchain technology, and the target title can be stored in the blockchain. The application uses the title generation model to quickly and accurately generate the target title corresponding to the target text to be recognized, and ensures the accuracy of the generated target title.

The invention provides a database archiving method, system, equipment and program product applied to an online tourism transaction platform, which are applied to the technical field of computer data processing, and the method comprises the following steps: generating a data copy from to-be-archived business data in an online production database, executing archiving calculation in an offline analysis data storage area to determine a record identifier, and storing the record identifier in an offline analysis data storage area; the record identifier is sent to the asynchronous message queue, the record identifier is obtained through the data cleaning service, and the data cleaning operation is executed in the online production database. According to the scheme, data cleaning of the online production database can be achieved on the premise that the performance of an online transaction system is not affected, and the data cleaning efficiency is improved. And furthermore, the archiving integrity of the order associated data is ensured through audit-driven cascade calculation, so that data cleaning of an online production database can be realized on the premise of not influencing the performance of an online transaction system.

The application relates to a data cleaning method and device, computer equipment, a storage medium and a computer program product, and relates to the technical field of cloud computing. The method comprises the following steps: determining a cleaning cache space corresponding to each to-be-cleaned data table according to a data type corresponding to each to-be-cleaned data table, and storing index information of each to-be-cleaned data table in the corresponding cleaning cache space; determining a target cleaning cache space from a plurality of cleaning cache spaces, deleting data in a to-be-cleaned data table corresponding to the index information in the database based on the index information in the target cleaning cache space, and determining the target cleaning cache space from the remaining cleaning cache spaces, jumping to the step of deleting data in a to-be-cleaned data table corresponding to the index information in the database based on the index information in the target cleaning cache space until there is no target cleaning cache space. The method can improve the data cleaning efficiency.

Methods and systems for restricting queries to a database according to a privacy budget. The technology includes assigning a first privacy allowance to first data in a database table, the first privacy allowance being an amount of a privacy currency, and assigning a second privacy allowance to second data in the database table; receiving a query from a database user, the query including a specified amount of the privacy currency; and allowing processing of the query only when, for each one of the first data and second data that must be accessed to service the query, the specified amount of privacy currency is equal to or less than a remaining privacy allowance for the data.

A system and method for scrubbing data to be shared between organizations to test a joint solution, and for preventing the introduction of unscrubbed data. Each organization captures a subset of data, which may be customer data from a line of business. The first organization scrubs its data according to scrubbing rules, and then passes the scrubbed data to its test environment, while the second organization passes its unscrubbed data to its test environment. The scrubbed data is communicated to the second organization and is applied to the unscrubbed data in order to scrub it, and then communicate it to the first organization. Both organizations use the scrubbed data in their respective test environments to test the joint solution or joint testing. Scrubbing the data may involve scrubbing only specific data fields containing sensitive information.

The application discloses a data cleaning method and device, equipment and a storage medium. The method comprises the following steps: determining the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleaning strategy; performing hierarchical cleaning on the logical blocks based on the target cleaning strategy of each logical block until the storage medium to be cleaned is completely cleaned; wherein the target cleaning strategy is one of a first strategy for full cleaning, a second strategy for partial cleaning and a third strategy without data cleaning. In this way, by performing hierarchical cleaning on each logical block of the partition of the storage medium to be cleaned based on the first strategy, the second strategy and the third strategy, a targeted cleaning strategy is provided for different functional logical blocks of the storage medium to be cleaned, so that the data cleaning efficiency is improved while the data security is taken into account, and the user experience is improved.

The application belongs to the technical field of power monitoring, and discloses a power monitoring video fast splicing method based on key frame extraction, which comprises the following steps: video preprocessing, acquiring multiple power operation site monitoring videos, carrying out data cleaning and data enhancement, and laying a foundation for subsequent splicing; in the aspect of feature extraction network, the modeling capability of the algorithm on space and time sequence information is considered, a time shift module TSM is embedded into a two-dimensional convolution network ResNet101, a two-dimensional convolution network TSM-ResNet101 with space and time sequence information modeling capability is obtained, the network is used as the feature extraction network of the algorithm, non-overlapping areas are directly spliced, and panoramic key frames are generated; panoramic video generation: the panoramic key frames are reorganized according to the time sequence of the original video, a linear interpolation method is used to supplement the interframe transition effect, and a complete and smooth 360° panoramic monitoring video is generated. The application effectively resists electromagnetic interference in the power site, improves the quality of key frames, and provides protection for accurate registration.

This invention relates to the field of computer technology and discloses a data cleaning method, apparatus, electronic device, and storage medium. The method includes: acquiring data characteristic information, data type, and current business mode of each data item in a storage system; determining the access mode of any data item based on its data characteristic information and data type; determining a target data cleaning strategy for the storage system based on the current business mode and the access modes of each data item; and cleaning the storage system according to the target data cleaning strategy. The method provided above improves both the efficiency and reliability of data cleaning results by predictively analyzing the access modes of data items and combining this with the current business mode of the storage system to determine the target data cleaning strategy.

The invention provides a tag storage method and device, electronic equipment and a storage medium, and the tag storage method comprises the steps: adding target tag data into an original tag table, and deleting historical tag data corresponding to the target tag data from the original tag table; performing label data aggregation on a target label corresponding to the target label data in the original label table to obtain target aggregated data; and adding the target aggregation data into a label aggregation table. According to the method, efficient management of the tag data is realized by utilizing the characteristics of the column database, intelligent updating and historical data cleaning of the original tag table are realized through a specific engine mechanism, aggregation processing and storage are performed on the tag data, and while the data consistency and traceability are ensured, the data processing efficiency is improved. The storage and utilization efficiency, the data updating speed and the complex query response speed are greatly improved, and the method is suitable for label management in a high-concurrency scene.

The invention belongs to the field of user portrait analysis, and particularly relates to a user portrait intelligent optimization method and an analysis management system based on cloud computing and AI, which can more quickly obtain the preference of a user by constructing a user portrait through a cloud platform, perform data cleaning on redundant information in the user portrait, reduce the computing pressure of the cloud platform, and improve the user experience. Meanwhile, user data in the user portrait is encrypted to ensure the safety of user information; a pushing strategy is updated for the user portrait, and meanwhile the user purchase degree and the user triggering sensitive behavior frequency in the user portrait are stored; compared with the prior art, the method has the advantages that the pushing result is more accurate, the types of pushed commodities are various, the judgment on sensitive behaviors of the user is more accurate, and disputes between the user and merchants are reduced.

The embodiment of the invention provides a middleware abnormal item tracking method and device based on cloud service and a storage medium. The method comprises the following steps: acquiring original data; performing data cleaning on the original data to obtain an original workload report; comparing the original workload report with a historical abnormal item report in the original data, and identifying at least one governed item, at least one ungoverned item and at least one newly added governed item; and according to the at least one governed item, the at least one ungoverned item and the at least one newly added governing item, generating and displaying a summary report. According to the method, the frequency and timeliness of abnormal item tracking of the middleware of the cloud service can be improved, and the accuracy and effectiveness of efficiency management work of the cloud service are improved, so that reliable operation of a business system based on the cloud service is ensured.

The present application relates to the technical field of secure processing of electronic spreadsheets, and particularly relates to a method, system, device and medium for secure processing of electronic spreadsheets based on RPA. The method comprises the following steps: when an RPA client starts to execute an electronic spreadsheet automation process, detecting a document processing client configuration; the RPA client executes actions in the order of the electronic spreadsheet automation process, performs offline secure interaction with an electronic spreadsheet automation component according to access credentials through a locally deployed add-in, and controls an original electronic spreadsheet in a local document processing memory; when the RPA client finishes executing the electronic spreadsheet automation process, performing data cleaning. Through the pre-positioned detection and initialization configuration, the risk of cache leakage caused by the client's own mechanism is cut off from the source; while ensuring the efficiency of the automation process, the memory-level secure isolation of the original document and the localized permission control are realized; through thorough data cleaning, the risk of illegal dumping of residual data in volatile memory is eliminated.

A system and method for performing standards-compliant data sanitization of Small Computer System Interface (SCSI), Advanced Technology Attachment (ATA), and Non-Volatile Memory Express (NVMe) hard disk drives (HDDs), solid state drives (SSDs), Non-Volatile Memory Express (NVMe) drives, and other portable storage media, e.g. flash storage, USB thumb drives at an optimized speed. The sanitization and overwrite processes herein disclosed are executed by completely web-based applications that use the local hard drive (host) as a server intermediate the web browser and the program files.

The invention relates to the technical field of data processing, in particular to a logistics track data synchronization method, device and equipment and a storage medium, and the method realizes real-time cleaning and updating of abnormal data through an incremental data processing mechanism, namely, real-time analysis and operation log generation based on a message queue, so that the real-time cleaning and updating of the abnormal data are realized. A data processing flow is advanced to a message queue stage, so that disconnection between a data cleaning link and a display link is avoided, and the external network data delay time is effectively shortened; a differential updating strategy is adopted, incremental data only updates local network point data, and full data is subjected to replacement operation after verification is passed, so that global data refreshing in a traditional full coverage mode can be avoided, data updating granularity is refined to a network point level, and the phenomenon of alternation of new and old data in a high-concurrency scene is effectively avoided; and invalid data cleaning is realized through a dual guarantee mechanism, so that the problem of dirty data residue caused by lack of a compensation mechanism in a traditional scheme is solved, and the validity of cached data is ensured.

The application discloses a kind of data cleaning method, system, equipment and medium applied to multi-tenant SaaS.The method comprises the following steps: obtaining the connection information of the database to be cleaned, and the connection information is tested and the authority is checked remotely;When the remote connection test and the authority check of the connection information are passed, it is determined that the database to be cleaned is successfully connected, the all table list of the database to be cleaned is presented through visual interface, and the corresponding data volume and Chinese explanation of each table list are displayed;Receive the table list to be cleaned and cleaning mode selected by the user through the visual interface;According to the selected cleaning mode, the selected table list to be cleaned is executed data cleaning operation;Record and archive the execution result and cleaning log of the data cleaning operation, greatly improve the efficiency of data cleaning, break the barrier of heterogeneous database, improve system versatility, and ensure the security of cleaning operation.

Digital retirement of an IHS may include encrypting, by the IHS, data on the IHS, using an encryption key stored in a trusted platform module (TPM) of the IHS, and incapacitating the TPM. This encryption key may be a (unique (256-bit encryption)) data sanitization key stored protected inside the TPM, thereby, the encryption key is protected inside the TPM by being immutable and unreadable by external devices. Digital retirement of the IHS may also include verifying all data on the IHS is encrypted successfully, prior to incapacitating the TPM. Incapacitation of the TPM may be carried out by instructing a power supply of the IHS to overvolt the TPM. Digital retirement of the IHS may be carried out in response to receipt of a digital retirement command that has been authorized and / or authenticated through two users and may be further manually approved by a third authorized and / or authenticated user.