Data feature preprocessing method and implementation system and application thereof

A data feature and preprocessing technology, applied in the field of neural networks, can solve the problems of high consumption of manpower and material resources, few processing methods for high-dimensional data features, and lack of quantitative evaluation methods for judgments.

Inactive Publication Date: 2021-01-08
SHANDONG UNIV
View PDF3 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Different preprocessing methods should be selected for different forms of data. For higher-dimensional data features, dimensionality reduction and redundant data removal are required. Different methods will affect the effect of data application to varying degrees, so it can handle structured fields and non- There are few high-dimensional data feature processing methods for structured text information
[0003] At present, when judging whether a prisoner meets the norms of "commutation, parole and suspension of execution", judges need to review a large number of legal documents to make a judgment, which consumes a lot of manpower and material resources, and at the same time produces a certain degree of subjectivity. Quantitative Evaluation Method for Judgment

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data feature preprocessing method and implementation system and application thereof
  • Data feature preprocessing method and implementation system and application thereof
  • Data feature preprocessing method and implementation system and application thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0079] A method of data feature preprocessing, such as Figure 4 shown, including the following steps:

[0080] (1) Data structure

[0081] Raw data can be divided into two categories by data type, including quantifiable fields and text fields;

[0082] Data structuring, constructing feature vectors: structured data refers to data with strict data format and length specifications.

[0083] For quantifiable fields, label encoding is performed on discrete category fields, and normalization is performed on continuous numeric fields;

[0084] For the text field, extract the rules, use information extraction and knowledge representation technology to extract keywords, and express the corresponding rules, and establish a structured knowledge base; for example, input the legal provisions of the Supreme People's Court on handling commutation cases, and output Information points in a fixed format, including "commutation rules", "commutation time", "commutation interval", etc.

[00...

Embodiment 2

[0104] The implementation system of a method for data feature preprocessing described in Embodiment 1, such as figure 1 As shown, it includes sequentially connected data structuring units, feature vector extraction and construction units, and the feature vector extraction and construction units include sequentially connected similarity calculation modules and weight sorting modules;

[0105] The data structuring unit is used to realize the data structuring process of step (1); the similarity calculation module is used to realize the similarity calculation process of step 1); the weight sorting module is used to realize the weight sorting process of step 2).

Embodiment 3

[0107] The application of a method of data feature preprocessing described in Example 1 in judging whether the prisoner meets the temporary conditions for reducing leave, such as figure 2 shown, including the following steps:

[0108] A. Process the prison data through the above data feature preprocessing method to obtain the feature vector

[0109] The prison data includes quantifiable fields and text fields. The quantifiable fields are the multi-dimensional information of the persons to be evaluated, including population data dimensions, social relationship dimensions, physiological dimensions, psychological dimensions, criminal information dimensions, and reformation education dimensions; Relevant content of temporary laws and regulations; demographic data dimensions include the gender, age, education, occupation, special skills, and whether they are three-no persons; social relationship dimensions include the prisoner’s family structure, family economic level, family Edu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a data preprocessing method as well as an implementation system and application thereof. The method comprises the following steps: (1) data structuralization: treating original data which comprises quantizable fields and text fields, specifically, for the quantizable fields, conducting label coding on discrete category fields in the quantizable fields, conducting normalization on continuous numeric fields, performing rule extraction on the text fields, extracting keywords by utilizing an information extraction and knowledge representation technology, representing corresponding rules, and establishing a structured knowledge base; and (2) feature vector extraction and construction: for the quantizable fields processed in the step (1), judging the similarity, deletinginvalid features with small sample similarity distinction, and selecting the most effective feature as a feature vector. Aiming at a textual data file, the invention provides a keyword and rule extraction and quantification method based on a knowledge extraction and representation technology, and a structured knowledge base is established for quantitative evaluation.

Description

technical field [0001] The invention relates to a data feature preprocessing method and its realization system and application, belonging to the technical field of neural network. Background technique [0002] The development of the information society has produced massive amounts of data. People need to process data of various dimensions and forms all the time for production and life, and to obtain various audio and video files, sensor data and other information. Common methods for data preprocessing include data cleaning, data integration, and data transformation. Different preprocessing methods should be selected for different forms of data. For higher-dimensional data features, dimensionality reduction and redundant data removal are required. Different methods will affect the effect of data application to varying degrees, so it can handle structured fields and non- There are few high-dimensional data feature processing methods for structured text information. [0003] ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2458G06N3/04G06N3/08G06Q10/06G06Q50/26
CPCG06F16/2465G06N3/084G06Q10/06395G06Q50/26G06N3/045
Inventor 李玉军邓媛洁魏莹
Owner SHANDONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap