1783 results about "Data ingestion" patented technology

An intelligent document recognition-based document management system includes modules for image capture, image enhancement, image identification, optical character recognition, data extraction and quality assurance. The system captures data from electronic documents as diverse as facsimile images, scanned images and images from document management systems. It processes these images and presents the data in, for example, a standard XML format. The document management system processes both structured document images (ones which have a standard format) and unstructured document images (ones which do not have a standard format). The system can extract images directly from a facsimile machine, a scanner or a document management system for processing.

Social media content items are mapped to relevant time-based media events. These mappings may be used as the basis for multiple applications, such as ranking of search results for time-based media, automatic recommendations for time-based media, prediction of audience interest for media purchasing/planning, and estimating social interest in the time-based media. Social interest in time-based media (e.g., video and audio streams and recordings) segments is estimated through a process of data ingestion and integration. The estimation process determines social interest in specific events represented as segments in time-based media, such as particular plays in a sporting event, scenes in a television show, or advertisements in an advertising block. The resulting estimates of social interest also can be graphically displayed.

Methods, systems and processor-readable media for extracting and classifying location information utilizing social media messages and/or data thereof. The social media messages can be sampled from a social media database and the messages filtered based on a heuristic rule. A geolocation entity from the unstructured social media messages can be extracted utilizing a geolocation entity extracting module. The messages with the geoentities can be uploaded onto a crowd sourcing platform to manually annotate the messages with a label. A text classification model can be built and learned from the label utilizing a machine learning algorithm and the messages can be classified by a location classifier in order to extract the user location. The user location can then be transformed into a geocode so that a spatial search can be enabled and the distance between the locations can be easily calculated.

Provided are systems and methods for Web-based intelligent paperless document management where users can collect, store, and share all document from various locations. Also provided are systems and methods requiring minimal data reentry because of data extraction capabilities. Systems and methods described herein can deliver documents via internet to multiple persons without using barcodes or separator sheets for faxing or sending documents.

Business methods and systems that are provided use knowledge-based expert systems for mining (extracting) highly structured clinical information from various structured and unstructured sources of healthcare provider data. In one business method for on-line healthcare management and decision support, a service provider maintaining a collection of structured clinical data, the structured clinical data comprising information automatically mined from various structured and unstructured sources of healthcare provider data from one or more different healthcare providers. The service provider provides a customer on-line access to structured clinical data in the collection, or providing an on-line service to the customer using structured clinical data in the collection, based on a service agreement between the customer and the service provider.

In embodiments of the present invention improved capabilities are described for the steps of identifying, through a monitoring module of a security software component, a data extraction behavior of a software application attempting to extract data from an endpoint computing facility; and in response to a finding that the data extraction behavior is related to extracting sensitive information and that the behavior is a suspicious behavior, causing the endpoint to perform a remedial action. The security software component may be a computer security software program, a sensitive information compliance software program, and the like.

A secure cashless gaming system comprises a plurality of gaming devices which may or may not be connected to a central host network. Each gaming device includes an intelligent data device reader which is uniquely associated with a security module interposed between the intelligent data device reader and the gaming device processor. A portable data device bearing credits is used to allow players to play the various gaming devices. When a portable data device is presented to the gaming device, it is authenticated before a gaming session is allowed to begin. The intelligent data device reader in each gaming device monitors gaming transactions and stores the results for later readout in a secure format by a portable data extraction unit, or else for transfer to a central host network. Gaming transaction data may be aggregated by the portable data extraction unit from a number of different gaming devices, and may be transferred to a central accounting and processing system for tracking the number of remaining gaming credits for each portable data unit and/or player. Individual player habits can be monitored and tracked using the aggregated data. The intelligent data device reader may be programmed to automatically transfer gaming credits from a portable data device the gaming device, and continually refresh the credits each time they drop below a certain minimum level, thus alleviating the need for the player to manually enter an amount of gaming credits to transfer to the gaming device.

A method and system for facilitating information searching for a user of an electronic device is provided. Facilitating searches involves obtaining information about the user interests, identifying potential data of interest to the user, extracting data related to said data of interest to the user, and collecting the extracted related data for presentation to the user on the device.

A method of automatically narrowing data search space and improving accuracy of data extraction using known constraints in a layout of extracted data elements for classified documented is provided. The method includes: analyzing each document to classify it within a document category, each category having a corresponding set of expected layouts; analyzing each electronic document to automatically extract images and text features; automatically constructing a data structure including a layout of the extracted features and layout relationships amongst the extracted features, wherein each of the extracted features in the layout maintains a reference to neighboring features and wherein closely related features are merged to form a combined feature; automatically narrowing data search space by detecting and removing parts of the layout that are not associated with any data elements using the data structure; and automatically detecting data using the extracted feature layout and the layout relationships amongst the extracted features.

A method for extracting an attribute occurrence from template generated semi-structured document comprising multi-attribute data records comprises identifying a first set of attribute occurrences in the template generated semi-structured document using an ontology. The method further comprises determining a boundary of each multi-attribute data record in the template generated semi-structured document, learning a pattern for an attribute corresponding to an identified attribute occurrence of the first set in the template generated semi-structured document, and applying the pattern within the boundary of each multi-attribute data record in the template generated semi-structured document to extract a second set of attribute occurrences.

The invention discloses a prediction method and visualization method of traffic jam. The methods comprise the following steps: GPS data sent back by a taxicab is utilized for being associated to roads in an electronic map through the map matching method; the road section speed calculated in terms of the matched data is utilized for judging the traffic state of road sections; the evolution rule of the traffic jam, including formation and dissipation of the traffic jam, is extracted with the utilization of the historical data; the traffic jam prediction is performed with the utilization of the sliding time window scheme after a real-time traffic information database is associated; the congestion intensity and the influence range of congested road sections can be presented by a thermodynamic chart after the congestion intensity of the congested road sections is calculated based on the prediction result. The prediction method provided by the invention can realize highly accurate traffic jam prediction, and the visualization method of the thermodynamic chart enables the traffic jam to be visualized and understandable, so that the location and the influence range of the traffic jam can be discerned conveniently.

The present system can perform machine learning of prototypical descriptions of data elements for extraction from machine-readable documents. Document templates are created from sets of training documents that can be used to extract data from form documents, such as: fill-in forms used for taxes; flex-form documents having many variants, such as bills of lading or insurance notifications; and some context-form documents having a description or graphic indicator in proximity to a data element. In response to training documents, the system performs an inductive reasoning process to generalize a document template so that the location of data elements can be predicted for the training examples. The automatically generated document template can then be used to extract data elements from a wide variety of form documents.

An extraction-rule generation and training system uses information obtained from multiple markup language documents (e.g. web pages) of similar structure to generate an extraction rule for extracting datapoints from markup language documents. By using information extracted from multiple documents of similar structure, including information regarding correlations between such documents, the method produces data extraction rules that provide improved datapoint extraction reliability. Where the structures of two or more documents are not sufficiently similar, the system maintains separate extraction rules for the same datapoint, and applies these separate extraction rules in combination to particular markup language documents to extract the datapoint.

Remote data collection systems and methods retrieve data including financial, sales, marketing, operational and the like data from a plurality of databases and database types remotely over a network in an automated, platform-agnostic manner. A remote data collection system includes a network interface, a connection to a data source, a processor communicatively coupled to the network interface and the connection, and memory storing instructions for remote data collection that, when executed, cause the processor to: receive a request to extract data from the data source; extract the data in a non-intrusive manner from the data source using a two phase process comprising a reconciliation phase and a collection phase; and transmit one of an entire set and a subset of the extracted data based on the request.

The present method and apparatus provides for automated testing of data integration and business intelligence projects using Extract, Load and Validate (ELV) architecture. The method and computer program product provides a testing framework that automates the querying, extraction and loading of test data into a test result database from plurality of data sources and application interfaces using source specific adaptors. The test data available for extraction using the adaptors include metadata such as the database query generated by the OLAP Tools that are critical to validate the changes in business intelligence systems. A validation module helps define validation rules for verifying the test data loaded into the test result database. The validation module further provides a framework for comparing the test data with previously archived test data as well as benchmark test data.

A portable data reading device, such as a barcode scanner or RFID reader, includes a Web server and a first server-side application to modify one or more settings of the portable data reading device. The settings may include, for example, symbology settings, device settings, and network settings. The Web server may receive formatted data from a client browser representing a requested modification of at least one setting of the portable data reading device. Upon receiving the formatted data, the Web server may automatically invoke the first server-side application to modify the at least one setting responsive to the formatted data.

For long-term data preservation, a storage virtualization system contains a metadata extraction module, an indexing module, a search module, and a virtualization module. The system utilizes two types of virtual volumes: unmarked volumes and marked volumes. The metadata extraction module extracts metadata that describes the data stored in logical volumes located in external storage. The indexing module scans the data and creates an index, and the index and metadata are stored in a local storage. After metadata is extracted for all data in a volume, and all data in the volume are indexed, the virtual volume corresponding to that volume is marked and the volume is ready to be made inactive. The search module allows a user to search for desired data using the metadata and the index stored in the local storage instead having to access the external storage systems where the data is actually stored.

The invention discloses a traffic travel characteristic data extraction method based on big data, and belongs to the field of traffic big data analysis application. Mobile phone data act as the core of the big data, and GPS data, coil data and video data act as the auxiliary of the big data. Traffic travel characteristic data include OD data and job-housing ratio data. The main steps are listed as follows: 1) the mobile phone data, the GPS data, the coil data and the video data are acquired and the mobile phone data are preprocessed; 2) traffic cell allocation is performed through combination of the geographic attributes of administrative regions and base stations; 3) traffic travel chain identification is performed through combination of the multi-source data with the mobile phone data acting as the core, and passing points and stop points are identified; 4) inter-cell OD result output is performed through combination of the mobile phone data and demographic data; and 5) all the traffic travel chains within a week are analyzed, residential places and working places are judged through combination of turn-on and turn-off data and conversation data, and job-housing analysis is performed. Wide range of urban traffic travel characteristic data acquisition can be completed in a short period of time by the traffic travel characteristic data extraction method based on the big data.

A vehicle environment monitoring system is provided that is based on a three-dimensional vector model. The three-dimensional vector model of the vehicle's environment is generated on the basis of the image data captured by at least one three-dimensional camera. Out of the image data, particular data are extracted for generating the three-dimensional vector model in order to reduce the data volume. For data extraction, a data extraction algorithm is applied that is determined in accordance with at least one parameter that relates to the situation of the vehicle. Therefore, targeted data extraction is performed for generating a three-dimensional model that is particularly adapted for an application that is desired in the current vehicle situation. The applications of the vector model include driver assistance, external monitoring and vehicle control, as well as recording in an event data recorder. In one implementation, a sequence of three-dimensional vector models, representing a three-dimensional space-and-time model, is generated.

A log file processing system sorts records from large log files and groups them by session without making a complete copy of the log files by capturing a subset of the log files in a sliding memory window and identifying all records in the window that form a complete user session. Records belonging to a complete session are output for analyzing, and the remaining records are output as raw log data for additional processing. Using a ring buffer to implement the sliding memory window, data structures are used to group records by session, to identify completed sessions, and to index into the ring buffer to retrieve records for completed sessions that are to be directly analyzed. Any records remaining in the ring buffer at the end of slide window processing may be output as raw log file data and are processed as incomplete or malformed session records. An embodiment of the log file processing system provides a significant improvement on the speed of data extraction from log files into analyzable session data.

Selectively managing data conveyance between computing devices may be facilitated by a data distribution device receiving a message including a subscription request for a data acquisition device, determining an attribute of the subscription request, selecting, based on a current state of the data distribution device and the attribute of the subscription request, between a data pull service and a data push service for providing the responsive data, and sending a message regarding the selected service.

The present invention provides a data hiding method of hiding media data in message data and a data extraction method of extracting the hidden data wherein message data is dispersively hidden in media data such as an image or sound to prevent a third person from modifying the message data easily. More specifically, the present invention relates to a data hiding method in which media data is expressed as a media array while message data is expressed as a message array so that the array elements of the message array can be dispersively hidden in the message array based on a state values specifying a particular array element of the media array, comprising the steps of: (a) determining the j-th (j>/=0) state value Sj; (b) determining (j+1)-th state value Sj+1 based on the j-th state value, the array element of the media array indicated by the j-th state value, and the array element of the message array; and (c) hiding data with respect to the array element of the media array indicated by the (j+1)-th state value Sj+1.

A management system for managing a plurality of computers and a plurality of pieces of software in a computer center includes: a selection data storage part for storing data regarding allocation destination selection, the data including at least either one of operation data and temperature data, wherein the operation data represent states of execution of the pieces of software by the computers, respectively, and temperature data represent temperature distribution in the computers; an instruction generation part for extracting an overheated computer that is assumed to emit more heat as compared with the other computers, and a less-heated computer that is assumed to emit less heat as compared with the other computers, by using the data regarding allocation destination selection, and generating an instruction for relocating at least a part of a piece of software allocated to the overheated computer to the less-heated computer; and an instruction part for outputting the instruction. This makes it possible to save energy for cooling the computers.

Systems and methods for providing interactive printing that includes displaying and selecting print or fax jobs on a front panel of a printer device without the print job being despooled to the printer device. In an interactive printing environment, a print job is initiated without specifying a particular printing device. The print job is stored at a client computing device or at a printer server, and the job data (non-print data) corresponding to the print job is stored at the printer server. Job data is obtained at a printing device from the printer server and selectively displayed on the front panel of the printing device. Upon selection of the job data at the printing device, the corresponding print job is despooled to the printing device from the printer server or client computing device. The print job is then rendered at the printer device.

The invention relates to a criminal suspect mining association method and a system based on social network analysis and is characterized in that the method comprises the following steps: pre-processing the user input data, extracting the key information from the input data; establishing the social network graph model; setting the parameter; executing the community discovery algorithm; outputting the discovery result and listing the criminal suspects; compared with the prior art, the method and the system have the beneficial effects as follows: the potential associated criminal suspect can be found from the social circle of the given criminal suspect, the method is good in performance and fast in system operation, by considering the requirement of the real scenes, good expansibility can be achieved for analyzing in the range appointed by the user.

The invention provides an intelligent home optimization system based on data mining, which comprises a user authentication module, a journal memory module, a data extraction module, a data mining algoritic module, a sensor monitoring module, a synchronous renewal module, a database module, a scheduler module, and a communication module. An intelligent home control system is internally added with the function of then data mining algoritic and the communication module which can communicate with other homes to cause that after a home system passes through authentication, the home system counts and analyzes a series of actions of family members after the family members go home, analyzes habits and customs of people by the data mining algoritic so as to obtain some regularity, and then transmits the regularity, namely, associative rules to the scheduler module of the intelligent home control system; the scheduler module transmits a control command to each home which is connected with a central control system; therefore, the system can control home without needing operation of the family members in a humanity intelligent way.

The invention discloses an electromyographic signal gesture recognition method based on a hidden markov model. The method comprises the following steps of: executing smoothing filtering for electromyographic signals; extracting a multi-feature feature set for each window data through a sliding window, and executing normalization and feature dimension reduction of minimum redundancy maximum correlation criterion for feature vectors; designing three classes of hidden markov model classifiers, and optimizing parameters of the hidden markov model classifiers; obtaining classifier models through training with hidden markov classifier model parameters and training data; inputting test data into the models trained well, and according to likelihood output by each class of hidden markov model, determining that the class corresponding to the maximum likelihood is the recognized class. According to the method provided by the invention, three classes of common hidden markov model classifiers are recognized based on a new feature set. By application of a classification method based on the hidden markov model, different gestures of the same testee can be recognized accurately, and gestures of different testees can be relatively recognized accurately.

The invention discloses a combination forecast modeling method of wind farm power by using gray correlation analysis, belonging to the technical field of wind power generation modeling. In particular, the invention is related to a weighted combination forecast method of wind power based on a least square support vector machine and an error back propagation neural network. The forecast method comprises that forecasted values of wind speed and wind direction are acquired in advance from meteorological departments while real-time output power is acquired from a wind farm data acquiring system; that the forecasted values of wind speed and wind direction and the real-time output power are inputted into a data processing module for data analyzing extraction and data normalization, and then normalized data is loaded to a database server; processed data in the database server is extracted by a combination forecast algorithm server to carry out model training and power forecast, and the wind farm sends running data to the data processing module in real time to realize rolling forecasting. The method of the invention achieves the goal of combination forecast of wind farm output in a short time. The method not only maximally utilizes advantages of two algorithms but also increases forecast efficiency by saving computing resources and shortening computing time.

The invention provides a method for implementing repeated data deletion, which comprises the following steps of: calculating similarity of data by a simhash (similarity hash) algorithm; reckoning the similarity into an index library through a similarity positioning storage position; writing the data into a data warehouse; if the data with the same similarity enters the data warehouse, extracting the corresponding data in the data warehouse, and then performing binary comparison; and if the data are the same, recording indexes, otherwise, recording different data parts. The invention also provides a system for implementing the repeated data deletion, which comprises a similarity marking library (BitMap), a data offset marking library, the data warehouse (LBAMap) and a storage library (Resp) for recording initial data. The method and the system ensure data consistency by data comparison based on the simhash algorithm theory, efficiently finish the repeated data deletion, and ensure the consistency of the data.

Provided are a method and system for extracting a target object from a background image, the method including: generating a scalar image of differences between the object image and the background, using a lightness and a color difference between the background and current video frame; initializing a mask to have a value equal to a value for a corresponding pixel of a mask of a previous video frame, where a value of the scalar image of differences for the pixel is less than a threshold, and to have a predetermined value otherwise; clustering the scalar image of differences and the depth data; filling the mask for each pixel position the current video frame, using a centroid of a cluster of the scalar image of differences and the depth data; and updating the background image on the basis of the filled mask and the scalar image of differences.