8951 results about "Data pre-processing" patented technology

The invention discloses a text named entity recognition method based on Bi-LSTM, CNN and CRF. The method includes the following steps: (1) using a convolutional nerve network to encode and convert information on text word character level to a character vector; (2) combining the character vector and word vector into a combination which, as an input, is transmitted to a bidirectional LSTM neural network to build a model for contextual information of every word; and (3) in the output end of the LSTM neural network, utilizing continuous conditional random fields to carry out label decoding to a whole sentence, and mark the entities in the sentence. The invention is an end-to-end model without the need of data pre-processing in the un-marked corpus with the exception of the pre-trained word vector, therefore the invention can be widely applied for statement marking of different languages and fields.

A methodology for process modeling and control and the software system implementation of this methodology, which includes a rigorous, nonlinear process simulation model, the generation of appropriate linear models derived from the rigorous model, and an adaptive, linear model predictive controller (MPC) that utilizes the derived linear models. A state space, multivariable, model predictive controller (MPC) is the preferred choice for the MPC since the nonlinear simulation model is analytically translated into a set of linear state equations and thus simplifies the translation of the linearized simulation equations to the modeling format required by the controller. Various other MPC modeling forms such as transfer functions, impulse response coefficients, and step response coefficients may also be used. The methodology is very general in that any model predictive controller using one of the above modeling forms can be used as the controller. The methodology also includes various modules that improve reliability and performance. For example, there is a data pretreatment module used to pre-process the plant measurements for gross error detection. A data reconciliation and parameter estimation module is then used to correct for instrumentation errors and to adjust model parameters based on current operating conditions. The full-order state space model can be reduced by the order reduction module to obtain fewer states for the controller model. Automated MPC tuning is also provided to improve control performance.

The invention discloses a system used for constructing a user figure. The system comprises a data acquisition module used for acquiring the Internet surfing log data of a user, a data pre-processing module used for pre-processing the Internet surfing log data of the user, a data analysis module used for carrying out characteristic extraction for the Internet surfing log data after pre-processing to acquire attribute characteristics of the user, a data application module used for matching in a multi-dimensional database on the basis of label classification trained by an established multi-dimensional characteristic database according to the attribute characteristics of the user to acquire multi-dimensional attribute labels of the user, and a data business module used for constructing the user figure according to the multi-dimensional attribute labels. The system is advantaged in that the holographic multi-dimensional user figure can be constructed, and rapid precise advertisement delivery of service providers/enterprises/companies and consumption behavior recommendation of use groups can be satisfied.

The invention provides a three-dimensional digital earth-space data organizing and rendering method based on a quad-tree index, which belongs to the technical fields of cartography, geography information systems and virtual reality. The method comprises the following steps of: unifying common spatial data formats under multi-scale and multi-projection conversion, multispectral, multi-temporal and high-resolution remote sensing satellite images and aerial images as well as digital thematic maps with different scales into same coordinate system, carrying out operation on the attribute of each element and the parameter regulation of quad-tree tiles, outputting in the form of the quad-tree tiles, carrying out quad-tree cutting on three-dimensional landscape map data, leading spatial data into a relevant database, and carrying out unified management. By using the method, common vector data, raster data, altitude data and three-dimensional map data are organically integrated and issued into a three-dimensional digital earth prototype, thereby shortening the time of data preprocessing, improving the execution efficiency and providing a new integration method for three-dimensional digital earth fundamental geographic data dissemination.

The method comprises: using a sensor to collect the characteristic signals; the collected signals about each kind of running state are sent to the data processing module; after processed, the signals are sent to the computer system; said computer system comprises a device database, a symptom event database, a diagnosis knowledge database; a device database and signal analysis module, a symptom acquiring module, a diagnostic reasoning module, and a diagnosis explaining module; the software of the data processing module makes data exchange; the symptom event database exchanges data with the diagnostic module, data processing module; the diagnostic knowledge exchanges the data with the failure processing module, knowledge acquiring module and data processing module.

An electronic nose system for detecting quality of agricultural products mainly comprises a gas enrichment module mainly for collecting the low-concentration smell, a gas chamber and gas circuit module and a sensor array mainly for converting the smell signal into the electric signal, a sensor conditioning circuit and a data preprocessing module mainly for filtering, analog-to-digital converting and feature extracting of the output signal of the sensor array, an embedded system with data storage for identifying and judging the signal and a display and result output module, wherein the gas enrichment module consists of an adsorption tube filled with adsorbent, an electric heating wire and a temperature control device. The electronic nose system has the characteristics of small size, convenient carrying, high sensitivity of the system and wide application range of the system.

The invention relates to a method for relationship classification with an LSTM and CNN joint model based on location. The method includes the steps of (1) preprocessing data; (2) training word vectors; (3) extracting location vectors; acquiring the location vector feature and high-dimensional location feature vector of each word in a training set, cascading the word vector of each word with the high-dimensional location feature vector thereof to obtain a joint feature; (4) building a model for a specific task; encoding contextual information and semantic information of entities by use of bidirectional LSTM; outputting the vector of the location corresponding to the marked entities, inputting the output to CNN, outputting two entity nouns and their contextual information and relational wordinformation, and inputting the entity nouns and their contextual information and relational word information into a classifier for classification; (5) training the model by use of a loss function. The method does not need to manually extract any features, the joint model does not need to use additional natural language processing tools to preprocess the data, the algorithm is simple and clear, and the best effect at present is achieved.

The invention relates to the technical field of equipment fault monitoring, and particularly discloses an equipment fault warning and state monitoring method. The equipment fault warning and state monitoring method is characterized by including two processes of model building and model operation; the process of model building includes the steps of acquiring training data, performing data preprocessing operation on the training data, then adopting a nonparametric learning algorithm to select a memory matrix, training a residual generator and acquiring a residual threshold of each parameter; the process of model operation includes the steps of acquiring real-time data, performing data preprocessing operation on the real-time data, then calculating all parameter residuals of the real-time data, analyzing the residuals to judge whether the equipment state is normal or not, and further positioning fault causes. The method has the advantages of data-driven method based universality, robustness and high adaptive capability, the shortcoming that warning results are difficult to analyze and explain is avoided, and additionally accuracy and efficiency of fault warning are both improved due to introduction of the nonparametric learning algorithm.

The invention discloses an image significance detection method which uses confrontation training to generate a convolution neural network model, which belongs to the field of computer vision and image processing. The method comprises the steps of data preprocessing, network structure, suitable parameter selecting, and training with a random gradient descending method and an impulse unit. According to data preprocessing, a large amount of collected data and labels are preprocessed. According to network structure, a network structure and a specific kernel function are designed. Suitable parameters including learning rate, a momentum factor and the number of images inserted into the network each time are selected. The random gradient descending method and the impulse unit are used for training to reduce the possibility of network over-fitting. According to the invention, a significance map can be accurately acquired.

A computer implemented method, system, and computer usable program code for staged tracing, where an initial high-level trace is performed to detect potential problems or issues at a sub-system level, followed by a dynamic tracing state, with a more detailed level of tracing for an identified problematic sub-system. During such dynamic tracing, the CPU consumption or processing time is monitored and if such consumption remains below a given threshold, additional trace points may be added. If such CPU consumption exceeds the given threshold, existing trace-points are selectively backed-out or removed. The dynamic adding and removing of trace-points allows for the CPU to perform in a desired window of execution performance such that the overall system performance is not adversely affected when tracing is enabled.

The invention belongs to the radiation source signal identification technology field and particularly relates to a radar radiation source signal identification method based on the one-dimensional convolutional neural network. The method comprises steps that (1), a radar radiation source signal data set is generated; (2), data pre-processing is carried out; (3), the convolutional neural network is constructed; (4), parameters are set, and the convolutional neural network is trained; (5), prediction classification is carried out; (6), precision calculation is carried out; and (7), the result is outputted. The method is advantaged in that 1, a network structure of the convolutional neural network is utilized to carry out characteristic extraction of signals, and a manual characteristic design process in a traditional algorithm can be avoided; 2, signals with a signal to noise ratio lower than -10dB can be accurately identified; and 3, the method can be simply realized.

The invention provides a high-precision map generation system and a high-precision map generation method based on a high-definition ortho-photo map. The system comprises an image shooting module, a horizontal laser radar, a GPS (global positioning system) processing module, an inertial navigation module, a rotary encoder, an image and data preprocessing module and a geographical information processing module, wherein the vehicle-mounted image shooting module is used for acquiring a road image; the laser radar is used for scanning a barrier and acquiring geographical information data; the precision of GPS information is optimized; the ortho-photo map of the road image is acquired and then is rotated and sheared to generate corresponding geographical information file operation; a full ortho-photo map sequence and a corresponding geographical information file are combined and spliced to generate a base map of a global map; various types of geographical information data are marked on the base map of the map. According to the method and the system, a high-precision navigation map can be generated, and the data precision can reach centimeter level; the method and the system are extremely high in practical value on an advanced auxiliary driving system and an unmanned vehicle.

An optical scanner system for contiguous three-dimensional topographic or volumetric imaging of a surface from an aircraft or spacecraft is disclosed. A servo controller synchronizes the rotation rates of a pair of wedge scanners with high precision to the multi-kilohertz laser fire rate producing an infinite variety of well-controlled scan patterns. This causes the beam pattern to be laid down in precisely the same way on each scan cycle, eliminating the need to record the orientations of the wedges accurately on every laser fire, thereby reducing ancillary data storage or transmission requirements by two to three orders of magnitude and greatly simplifying data preprocessing and analysis. The described system also uses a holographic element to split the laser beam into an array that is then scanned in an arbitrary pattern. This provides more uniform signal strength to the various imaging detector channels and reduces the level of optical crosstalk between channels, resulting in a higher fidelity three-dimensional image.

The invention discloses a text summarization generation system and method based on coding-decoding deep neural networks. The system comprises: an Internet text obtaining module which is used for obtaining text information on the Internet; a data preprocessing module which is used for preprocessing the text information; a summarization model training module which is used for extracting quantified text information from the preprocessed text information and performing training according to a coding-decoding deep neural network model to obtain a summarization training model; a summarization generation module which is used for using the preprocessed text information as input and outputting summarization information with preset length according to the coding-decoding deep neural network model. The text summarization generation system and method based on coding-decoding deep neural networks have following advantages that text information is compressed into a summarization text with a coherent description by means of computer automatic analysis and abstraction or generation of the central content expressed by the text, which facilitate users to understand the text content and then to quickly read and select information of interest; the text is compressed into a summarization to reduce the browse burden of users.

The invention belongs to the field of network resource management, relates to the cooperative filtration technique of network resources and discloses a network resource personalized recommended method based on an ultrafast neural network. The network resource personalized recommended method based on ultrafast neural network comprises the following steps: firstly, data preprocessing, reading information from the journal files of a system user and generating a global user interested matrix, exchanging the global user interested matrix into a single user interested matrix of a current user, and then transforming and reducing dimensionality to mark out a training set A1 and a predicting set A2, secondly, model training, building an interest predicting model with single hidden layer neural network SLFN s structure for a target user, adopting ultrafast learning machine technique to carry out training on the training set A1 and getting various connection power values and hidden layer threshold values of the neural network model with single hidden layer, thirdly, prediction recommending, utilizing the obtained predicting model to calculate the scoring values of every resource in the predicting set A2 given by the target user and then recommending the first several resources with highest predicting scores to the target user.

The invention discloses a floating car state real-time traffic information processing method based on GPS data, which includes the following steps: 1. GPS point data pre-processing and map pre-processing measures; 2. conduct point matching and select alternative road section sets according to projection distance and azimuth angles; 3. determine correct matching road sections and find out travel routes according to an improved optical route selection method considering topological relation between front and rear points (including a special regional node processing method for urban complicate road network); 4. calculate the route average travel speed, and conduct statistics in a speed formula to generate a road network speed thematic map. The floating car state real-time traffic information processing method based on GPS data of the invention is well applicable the modern complicated urban road network full of overpasses and staggered main and side roads, which can not only meet the speed requirement for large-data-quantity GPS data real-time calculation, but also obtain quite high matching precision.

The invention relates to a vehicle flow predicting method based on an integrated LSTM neural network. On the basis of historical data obtained by vehicle flow detection, an integrated LSTM neural network vehicle flow prediction model is established to carry out vehicle flow prediction, so that the generalization error of the prediction model is reduced and the accuracy is improved. The method comprises the following steps that: data preprocessing is carried out; according to a preprocessed vehicle flow time sequence value, a vehicle flow matrix data set is constructed and the vehicle flow of an (n+1)th period of time is predicted by using first n periods of time, wherein each period of time is delta t expressing the time length and the unit is min; a plurality of different LSTM neural network models are constructed by using different initial weights; on the basis of a bagging integrated learning method, a training set and a verification set are constructed; a plurality of LSTM neural networks are trained to obtain an optimized module; a weighting coefficient of the single LSTM model is calculated by using the verification set; and inverse transformation and reverse normalization are carried out on a predicted vehicle flow value to obtain a predicted vehicle flow and integrated weighting is carried out to obtain a vehicle flow value predicted finally by the model.

The invention relates to an electric power telecommunication network device fault prediction method based on improved LSTM. The invention provides a data preprocessing and time sequence input construction method for the first time. Compared with the simple recurrent neural network, the LSTM is more likely to learn long-term dependence and can solve the prediction problems related to sequences. Because of the strong association between device alerts, the independence of variables can be ensured through PCA. The target replication strategy is also used for improving the LSTM, which can bring local error information in every step. Compared with a simple target output only in the last step, the strategy can improve the accuracy of the model and reduce the risk of over fitting. In combination with dropout, the invention proposes a prediction model of LSTM, which can achieve better prediction precision by deep learning. At the same time, the LSTM is usd for modeling electric power telecommunication network alarm data for the first time and identifying the timing sequence mode therein.

The invention discloses a short text classification method based on a topic word vector and a convolutional neural network, which comprises the following steps: 1) a data acquisition stage: acquiringshort text data according to requirements, and labeling the short text data as a training set; 2) a data preprocessing stage: performing word segmentation, stop word removal, useless text filtering and the like on the text; 3) representing short text features, namely respectively representing a theme level and a word vector level; 4) carrying out subject term vector joint training; 5) optimizing and iterating parameters of the convolutional neural network classification model; and 6) performing category prediction on the new sample. According to the invention, short text data characteristics are combined; in the feature representation stage, a topic vector and a word vector are combined for representation; semantic feature expansion is carried out on the data characteristics of the short text, text semantic information is further mined by utilizing the local sensitive information extraction capability of the convolutional neural network in the classification model training stage, and indexes such as short text classification task category prediction accuracy can be improved.

The invention discloses an intelligent questioning-answering system construction method and system based on deep learning and a knowledge atlas. A crawler is utilized to obtain an interrogation medical dataset of the internet, and data preprocessing is conducted to obtain a labeled dataset; a word-splitting dictionary based on the medical field is constructed through the further utilization of a hospital electronic medical record, and is merged with a medical dictionary to serve as a word-splitting dictionary of the system; the knowledge atlas associated with diseases and symptoms is constructed, and disease entity aligning and symptom entity aligning are conducted; according to disease entity aligning, the labeled dataset is obtained; a language model based on deep learning is constructed; a query optimization algorithm which is combined with contextual information of a user and is based on the knowledge atlas is constructed; a training dataset merged by the language model and the knowledge atlas is constructed for model merging training, and a pre-diagnosis merging model based on the language model and the knowledge atlas is obtained. By means of the intelligent questioning-answering system construction method and system based on deep learning and the knowledge atlas, active interrogation interaction through the further utilization of self-reported information of the user anddisease pre-diagnosis according to the self-reported information and interrogation information of the user are achieved.

A solution for reducing interference in noninvasive spectroscopic measurements of tissue and blood analytes is provided. By applying a basis set representing various tissue components to a collected sample measurement, measurement interferences resulting from the heterogeneity of tissue, sampling site differences, patient-to-patient variation, physiological variation, and instrumental differences are reduced. Consequently, the transformed sample measurements are more suitable for developing calibrations that are robust with respect to sample-to-sample variation, variation through time, and instrument related differences. In the calibration phase, data associated with a particular tissue sample site is corrected using a selected subset of data within the same data set. This method reduces the complexity of the data and reduces the intra-subject, inter-subject, and inter-instrument variations by removing interference specific to the respective data subset. In the measurement phase, the basis set correction is applied using a minimal number of initial samples collected from the sample site(s) where future samples will be collected.

The invention provides a remote sensing image semantic segmentation method based on context information and an attention mechanism. For a to-be-labeled high-resolution remote sensing image, data preprocessing and data amplification are firstly carried out, and then the amplified data is used for training a remote sensing image semantic segmentation model. In the training process, preliminary features are extracted from the image through a backbone network, then multi-scale features are extracted and fused through a multi-scale context information module, then features of different levels are combined with an attention fusion module, and finally bilinear interpolation up-sampling is directly utilized to obtain a final segmentation result. For an unlabeled image, firstly the unlabeled imageis segmented into small image blocks, the small image blocks are input into a semantic segmentation network to obtain a corresponding result, and then the small image blocks are spliced to obtain a final large-size segmentation result graph.