338 results about "Data scrubbing" patented technology

A multi-source multi-tenant reference data utility and methods for forming and maintaining the same, delivering high quality reference data in response to requests from clients, implemented using a shared infrastructure, and also providing added value services using the client's reference data. Included are data cleansing and quality assurance of the received data with full tracking of the sourcing of each value, storage of resulting entity values in a repository which allows retrievals and enforces source based entitlements, and delivery of retrieved data in the form of on demand datasets supporting a wide range of client application needs. An advantageous implementation has additional services for reporting on data quality and usage, a selection of value adding data driven computations and business document storage. By using a shared infrastructure and amortizing the costs of data quality assurance across a plurality of clients, while ensuring that clients only receive values from data sources to which they are licensed, better quality data at lower cost is delivered.

Neuro-response data including Electroencephalography (EEG), Functional Magnetic Resonance Imaging (fMRI) is filtered, analyzed, and combined to evaluate the effectiveness of stimulus materials such as marketing and entertainment materials. A data collection mechanism including multiple modalities such as, Electroencephalography (EEG), Functional Magnetic Resonance Imaging (fMRI), Electrooculography (EOG), Galvanic Skin Response (GSR), etc., collects response data from subjects exposed to marketing and entertainment stimuli. A data cleanser mechanism filters the response data and removes cross-modality interference.

Central nervous system, autonomic nervous system, and effector data is measured and analyzed to determine the effectiveness of marketing and entertainment stimuli. A data collection mechanism including multiple modalities such as Electroencephalography (EEG), Electrooculography (EOG), Galvanic Skin Response (GSR), etc., collects response data from subjects exposed to marketing and entertainment stimuli. A data cleanser mechanism filters the response data. The response data is enhanced using intra-modality response synthesis and/or a cross-modality response synthesis.

Central nervous system, autonomic nervous system, and effector data is measured and analyzed to determine the effectiveness of marketing and entertainment stimuli. A data collection mechanism including multiple modalities such as Magnetoencephalography (MEG), Electrooculography (EOG), Galvanic Skin Response (GSR), etc., collects response data from subjects exposed to marketing and entertainment stimuli. A data cleanser mechanism filters the response data. The response data is enhanced using intra-modality response synthesis and/or a cross-modality response synthesis.

Provision for scalable cleansing and value enhancement of data in the context of a multi-source multi-tenant data repository. The source data comes from multiple sources and on multiple topics. Evolutionarily tracked source data tags are used to hold tracking information reflecting the nature and sources of each change to the data, as it is affected during the various stages of data processing. The stages of processing include validation, normalization, single-source cleansing and cross-source processes. Various rules are applied during these stages, and evolutionarily tracked source data tags are used to record sources and agents of all changes to the data. As information is processed, transformed, and added to the repository, corresponding evolutionarily tracked source data tags are stored in association with the various information elements. The information contained in these tags can be used to enforce data entitlements in a multi-tenant data repository environment.

An organization's data records are often noisy: because of transcription errors, incomplete information, and lack of standard formats for textual data. A fundamental task during data cleansing and integration is matching strings—perhaps across multiple relations—that refer to the same entity (e.g., organization name or address). Furthermore, it is desirable to perform this matching within an RDBMS, which is where the data is likely to reside. In this paper, We adapt the widely used and established cosine similarity metric from the information retrieval field to the relational database context in order to identify potential string matches across relations. We then use this similarity metric to characterize this key aspect of data cleansing and integration as a join between relations on textual attributes, where the similarity of matches exceeds a specified threshold. Computing an exact answer to the text join can be expensive. For query processing efficiency, we propose an approximate, sampling-based approach to the join problem that can be easily and efficiently executed in a standard, unmodified RDBMS. Therefore the present invention includes a system for string matching across multiple relations in a relational database management system comprising generating a set of strings from a set of characters, decomposing each string into a subset of tokens, establishing at least two relations within the strings, establishing a similarity threshold for the relations, sampling the at least two relations, correlating the relations for the similarity threshold and returning all of the tokens which meet the criteria of the similarity threshold.

A data cleaning process includes the steps of: validating data loaded from at least two source systems; appending the validated data to a normalized data cleaning repository; selecting the priority of the source systems; creating a clean database; loading the consistent, normalized, and cleansed data from the clean database into a format required by data systems and software tools using the data; creating reports; and updating the clean database by a user without updating the source systems. The data cleaning process standardizes the process of collecting and analyzing data from disparate sources for optimization models enabling consistent analysis. The data cleaning process further provides complete auditablility to the inputs and outputs of data systems and software tools that use a dynamic data set. The data cleaning process is suitable for, but not limited to, applications in aircraft industry, both military and commercial, for example for supply chain management.

The invention discloses a household cleaning robot capable of establishing a map by self and a cleaning method. The household cleaning robot comprises a casing, a control center arranged in the casing, a power supply module, a storage unit, a camera, a lighting alarm device, an ultrasonic wave sensor, an infrared sensor, a photosensitive sensor, a wireless module and an RFID reader-writer, wherein the ultrasonic wave sensor, the infrared sensor and the photosensitive sensor are respectively connected with the control center, and the wireless module and the RFID reader-writer are connected with the control center. The household cleaning robot further comprises a plurality of RFID labels bonded under the periphery of a wall, wherein the RFID labels are used for recording position coordinate information and are active RFID labels. The cleaning robot can establish the indoor two-dimensional map to achieve the indoor location function and acquire the coordinate points in a room. Walking areas and non-walking areas are marked on the two-dimensional map, so that the cleaning robot can quickly determine a cleaning path and accurately reach a target to be cleaned. To hygienic dead angles which cannot be cleaned, the cleaning robot is manually operated and controlled to automatically store operation and control data. When the hygienic dead angles are cleaned again, the stored operation and control data can be called to clean the target.

A system and method is disclosed for detecting and/or mitigating an overload condition from one or more first computers, such as a distributed denial of service (DDoS) attack, viral attack, or the like, targeting one or more of a plurality of second computers located on a network. While one or more DDoS attacks are mitigated, a meter, detection apparatus, software, or method, detects the condition being mitigated in a data cleaning center, and provides an alert or notification regarding the mitigated attack. Another preferred embodiment relates, in general terms, to a system and method for detecting and/or mitigating an overload or attempted overload condition targeting a domain name server. A network connection is provided for receiving one or more DNS requests from one or more client computers located on a network. A preferred embodiment includes a processor for providing a response to the one or more DNS requests to the one or more client computers if more than a threshold number of duplicate DNS requests are received. Another preferred embodiment relates, in general terms, to a system and method for detecting and/or mitigating an attempted overload condition targeting a networked computer system that uses a redirection module to divert data until it is deemed to be clean.

An automatic data sanitizer module sanitizes formatted data from an external source system according to stored sanitization rules for release to an external destination system so that the destination system receives only that data for which it is authorized. The module generally includes an Input Comms module, a Message Processor, an Output Guard a Downgrader and an Output Comms module. The Input Comms module supports the communications protocol dictated by the external source system. The Processor sanitizes the message according to the rules written for the specific external system under consideration. The Guard verifies the modifications performed by the Processor. The Downgrader moves the resulting file to the Output Comms working directory and the Output Comms makes the resulting message available to the destination system. The system supports a variety of different formats and greatly facilitates the timely dissemination of information within a multi-level secure environment while protecting security interests.

A RAID subsystem with a plurality of hard disk drives has each disk controller autonomously executing data scrubbing on its disks instead of requiring the RAID controller to execute data scrubbing. The data scrubbing may be interrupted upon an I/O request from the RAID controller, or it may continue if a guaranteed scrub rate is not being achieved. Data is stored in large bands of concentric data tracks, and a band may be preferentially selected for scrubbing based on frequency of reading the band, or recency of writing to the band.

A data scrubbing apparatus corrects disturb data errors occurring in an array of memory cells such as SMT MRAM cells. The data scrubbing apparatus receives an error indication that an error has occurred during a read operation of a grouping of memory cells within the array of memory cells. The data scrubbing apparatus may generate an address describing the location of the memory cells to be scrubbed. The data scrubbing apparatus then commands the array of memory cells to write back the corrected data. Based on a scrub threshold value, the data scrubbing apparatus writes the corrected data back after a specific number of errors. The data scrubbing apparatus may further suspend writing back during a writing of data. The data scrubbing apparatus provides a busy indicator externally during a write back of corrected data.

The invention discloses a method for removing repeated objects on the basis of meta data, relating to meta data removal field, solving the problem that the working quantity of the repeated data removal is large. The method firstly carries out standardized processing to the meta data to be recorded. During the comparison, by reducing the comparison range, the working quantity is reduced and the working efficiency is improved. In the date set record, records which have the same press fields with the meta data to be recorded are selected; in the selected record, isbn, book name, author, press, publishing time and price character are selected as the comparison range. A similarity comparison function with weighing values is used to calculate the similarity value between the meta data to be recorded and the attribute value of the corresponding fields in the data set; the weighing value is multiplied by the similarity value of each field and then added by the gained complex similarity value; the complex similarity value is compared with the prearranged threshold value; if the complex similarity value is not less than the threshold value, the current record in the data set and the meta data to be recorded are repeated data.

A method, system, and apparatus are provided for processing tables embedded within documents wherein a first table header is detected by using semantic groupings of table header terms to identify a minimum number of table header terms in a scanned line of an text document; a potential data zone is extracted by applying white space correlation analysis to a portion of the text document that is adjacent to the first table header; one or more data zone columns from the potential data zone are grouped and aligned with a corresponding header column in the first table header to form a candidate table; data cleansing is performed on the candidate table; and then one or more columns of the candidate table are evaluated using natural language processing to apply a specified table analysis.

A cleansing system for improving operation of a plant. A server is coupled to the cleansing system for communicating with the plant via a communication network. A computer system has a web-based platform for receiving and sending plant data related to the operation of the plant over the network. A display device interactively displays the plant data. A data cleansing unit is configured for performing an enhanced data cleansing process for allowing an early detection and diagnosis of the operation of the plant based on at least one environmental factor. The data cleansing unit calculates and evaluates an offset amount representing a difference between a measurement and a simulation for detecting an error of measurement during the operation of the plant based on the plant data.

The invention discloses a deep learning-based electric power suspected complaint work order recognition method, and relates to complaint recognition method for electric power customer services. At present, electric power suspected complaint work orders are artificially recognized, so that the efficiency is low, the timeliness is bad and human resources are wasted. The method comprises the following steps of: 1) deep learning model configuration; 2) complaint feature label extraction; 3) complaint sample formatting; 4) model learning and training; 5) suspected complaint recognition; and 6) suspected complaint classification. According to the method, a series of work such as data cleaning and sorting, complaint tendency word extraction, data modeling and optimization, sample iterative learning and training, machine learning and prediction and the like can be carried out on complaint lists through a deep learning technology, so that deep learning-based intelligent recognition and classification of suspected complaint work orders are realized, the intelligent work experience is enhanced and the service quality control work efficiency is improved.

Methods for removing carbonyl sulfide (COS) from a synthesis gas stream are disclosed. The method entails raising the water level of a wet scrubber so that hydrolysis of the COS may occur in the scrubber itself instead of a COS reduction chamber and no additional catalysts other than those naturally occurring in the production of the synthesis gas need be introduced into the scrubber. The water is raised, in one example, to a level such that water within an inner tube of the wet scrubber flows over an upper end of the inner tube. Raising the water level in the scrubber ensures intimate and vigorous interaction between the water, the COS, and the naturally occurring catalyst, and promotes the hydrolysis of the COS. In one example, the naturally occurring catalyst (e.g., alumina oxide) is present in the coal ash produced when coal is the fuel gasified that creates the synthesis gas stream.

The invention relates to a method and a system for discovering a user residence position based on mobile phone signaling. The method comprises the following steps: the obtained mobile phone signalingdata of the user is taken as the original trajectory data, and the original trajectory data is sequentially processed by data cleaning, data slicing, preliminary clustering, data verification and corecluster calculation, and finally the user staying position is obtained. Adopting a series of algorithms such as track slice aggregation, hierarchical clustering algorithm, kernel density clustering algorithm and machine learning algorithm, etc., considering mobile phone track point density, signaling time interval, track point moving direction and moving distance, etc., we can judge user 's staying position, staying time, entering and leaving staying point time. The algorithm can effectively eliminate the interference of base station drift on user location judgment, improve the identificationaccuracy of user dwell point position and dwell time, and objectively restore the user dwell position.

A data scrubbing apparatus corrects disturb errors occurring in a memory cell array, such as SMT MRAM cells. The data scrubbing apparatus activates scrubbing of the data and associated error correction bits based on a number of errors corrected, at a power up of the memory cell array, or a programmed time interval. The data scrubbing apparatus may generate an address describing the location of the memory cells to be scrubbed. The data scrubbing apparatus then commands the array of memory cells to write back the corrected data, the associated error correction bits, and reference bits. The data scrubbing apparatus provides a busy indicator externally during a write back of corrected data.

The embodiment of the invention discloses an application program control method, an application program control apparatus, a storage medium and a mobile terminal, the method comprises the following steps: within a set period, at intervals of set time, acquiring current behavior characteristics; acquiring a preset behavior prediction model generated based on a machine learning method, wherein the preset behavior prediction model is obtained through training of a behavior characteristics sample of a plurality of known control operations and used for predicting the control operations based on thebehavior characteristics; inputting the current behavior characteristics to the preset behavior prediction model to obtain a prediction result output by the preset behavior prediction model; according to the prediction result, automatically executing the control operations confirming to user behavior habits, wherein the control operations comprise data clearing and data updating operations of anapplication program. The technical scheme provided by the invention can automatically execute the application program control operations confirming to the user behavior habits, thus, intellectualization of application program control is promoted.