Target characteristic data classification processing method and device

Abstract

The invention discloses a classification processing method and device for target characteristic data. The method comprises the steps that a target characteristic data set is acquired; the target characteristic data set comprises target characteristic data; preprocessing the target characteristic data set to obtain a preprocessed data set; and carrying out classification identification processing on the preprocessed data set to obtain category information. According to the method, effective classification processing and service of complex multi-source heterogeneous target characteristic data are realized, correct extraction of target characteristics is realized, and the production efficiency and competitiveness of equipment are effectively improved.

Description

Technical Field

[0001] The present invention relates to the fields of data processing and computers, and in particular to a method and device for classifying and processing target characteristic data. Background Art

[0002] At present, a large amount of target characteristic data is generated in industrial production, logistics and transportation, equipment testing and other links. These data generally have multi-source heterogeneous characteristics such as complex formats, inconsistent standards, and a lot of data. Among the target characteristic data, many heterogeneous data come from the same type of entity. How to effectively classify and identify heterogeneous data, identify data belonging to the same type of entity, and reduce the complexity of subsequent data processing, there is currently no relevant technology involved.

[0003] How to effectively classify and extract these complex, multi-source and heterogeneous target feature data to obtain the correct target features is a problem that needs to be solved urgently. Summary of the invention

[0004] The present invention aims at effectively classifying and extracting complex multi-source heterogeneous target characteristic data so as to obtain correct target characteristics. The present invention discloses a classification processing method and device for target characteristic data.

[0005] In a first aspect of an embodiment of the present application, a method for classifying and processing target characteristic data is disclosed, comprising:

[0006] S1, obtaining a target characteristic data set; the target characteristic data set includes target characteristic data;

[0007] S2, preprocessing the target characteristic data set to obtain a preprocessed data set;

[0008] S3, performing classification and recognition processing on the pre-processed data set to obtain category information.

[0009] The preprocessing of the target characteristic data set to obtain a preprocessed data set includes:

[0010] S21, performing data cleaning processing on the target characteristic data set to obtain a first data set;

[0011] S22, normalizing the first data set to obtain a second data set;

[0012] S23, performing boundary check processing on the second data set to obtain a preprocessed data set.

[0013] The performing boundary check processing on the second data set to obtain a preprocessed data set includes:

[0014] S231, for each data attribute of the second data set, preset a corresponding value range; the value range includes an upper limit value of the value range and a lower limit value of the value range;

[0015] S232, for each data of the second data set, judging whether the data value is within the value range according to the value range of its data attribute; if the data value is within the value range, not processing the data; if the data value is not within the value range, setting the data value to the upper limit value or the lower limit value of the value range that is closest to the data value;

[0016] S233, performing fusion processing on all data of the second data set after S232 is completed to obtain a preprocessed data set.

[0017] The performing boundary check processing on the second data set to obtain a preprocessed data set includes:

[0018] S2301, for each type of data attribute of the second data set, take the data collection information of the data as a known independent variable, take the data value of the data as a known dependent variable, use the known independent variable and the known dependent variable to construct a curve to be approximated, use the function approximation method to perform curve fitting on the curve to be approximated, and obtain a characteristic polynomial f(Ix) of the type of data attribute; use the characteristic polynomial f(Ix) to calculate and process the known independent variables to obtain an approximate dependent variable; determine whether the absolute value of the difference between the approximate dependent variable and the corresponding known dependent variable is greater than a set first regression discrimination threshold; if it is greater than the first regression discrimination threshold, delete the data from the second data set; if it is less than or equal to the first regression discrimination threshold, do not process the data;

[0019] S2302: Perform fusion processing on all data executed in S2301 of the second data set to obtain a data set after boundary check.

[0020] The classification and identification processing of the preprocessed data set to obtain category information includes:

[0021] S31, obtaining preset classification information; the classification information is represented as a classification matrix; the row vector of the classification matrix is ​​a vector constructed by typical values ​​of all data attributes of each target characteristic type; the row number of the row vector of the classification matrix is ​​a classification number; the classification number represents the category information of the target characteristic type corresponding to the row vector;

[0022] S32, using each target characteristic data of the preprocessed data set, respectively construct a corresponding vector to be identified;

[0023] S33, performing classification calculation processing on each vector to be identified and the classification matrix to obtain category information of the vector to be identified;

[0024] The classification calculation process includes:

[0025] Performing cross-correlation operations on the vector to be identified and each row vector of the classification matrix to obtain corresponding cross-correlation sequences;

[0026] Perform feature transformation processing on each cross-correlation sequence to obtain the corresponding feature sequence;

[0027] Perform distribution calculation on the feature sequence to obtain a distribution sequence;

[0028] Perform logarithmic weighted accumulation processing on each distribution sequence to obtain the corresponding classification feature value;

[0029] The row number of the row vector of the classification matrix corresponding to the distribution sequence with the smallest classification characteristic value is determined as the category information of the vector to be identified.

[0030] The expression of the distribution calculation process is:

[0031]

[0032] Among them, r k,i represents the distribution of the i-th value of the k-th feature sequence, E k (i) represents the i-th value of the k-th feature sequence, and N0 is the total number of elements contained in the feature sequence;

[0033] The expression of the logarithmic weighted accumulation process is:

[0034]

[0035] Among them, H k is the classification feature value of the kth distribution sequence, ω i is the i-th weighted value.

[0036] According to a second aspect of an embodiment of the present invention, a device for classifying and processing target characteristic data is disclosed, the device comprising:

[0037] A memory storing executable program code;

[0038] a processor coupled to the memory;

[0039] The processor calls the executable program code stored in the memory to execute the classification processing method of the target characteristic data.

[0040] According to a third aspect of an embodiment of the present invention, a computer storable medium is disclosed, wherein the computer storable medium stores computer instructions, and when the computer instructions are called by a computer, they are used to execute the classification processing method of the target characteristic data.

[0041] According to a fourth aspect of an embodiment of the present invention, an information data processing terminal is disclosed. The information data processing terminal is used to implement the classification processing method of the target characteristic data.

[0042] The beneficial effects of the present invention are:

[0043] The present invention realizes the effective aggregation, processing and service of complex multi-source heterogeneous target characteristic data, realizes the correct extraction of target characteristics, and effectively improves the production efficiency and competitiveness of equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart for implementing the method of the present invention. DETAILED DESCRIPTION

[0045] In order to better understand the content of the present invention, an embodiment is given here.

[0046] Figure 1 It is a flow chart for implementing the method of the present invention.

[0047] In a first aspect of an embodiment of the present application, a method for classifying and processing target characteristic data is disclosed, comprising:

[0048] S1, obtaining a target characteristic data set; the target characteristic data set includes target characteristic data;

[0049] S2, preprocessing the target characteristic data set to obtain a preprocessed data set;

[0050] S3, classify and identify the preprocessed data set to obtain category information;

[0051] The preprocessing of the target characteristic data set to obtain a preprocessed data set includes:

[0052] S21, performing data cleaning processing on the target characteristic data set to obtain a first data set;

[0053] S22, normalizing the first data set to obtain a second data set;

[0054] S23, performing boundary check processing on the second data set to obtain a preprocessed data set;

[0055] The performing boundary check processing on the second data set to obtain a preprocessed data set includes:

[0056] S231, for each data attribute of the second data set, preset a corresponding value range; the value range includes an upper limit value of the value range and a lower limit value of the value range;

[0057] S232, for each data of the second data set, judging whether the data value is within the value range according to the value range of its data attribute; if the data value is within the value range, not processing the data; if the data value is not within the value range, setting the data value to the upper limit value or the lower limit value of the value range that is closest to the data value;

[0058] S233, constructing a preprocessed data set by using all the data of the second data set after S232 is completed;

[0059] The performing boundary check processing on the second data set to obtain a preprocessed data set includes:

[0060] S2301, for each type of data attribute of the second data set, take the data collection information of the data as a known independent variable, take the data value of the data as a known dependent variable, use the known independent variable and the known dependent variable to construct a curve to be approximated, use the function approximation method to perform curve fitting on the curve to be approximated, and obtain a characteristic polynomial f(Ix) of the type of data attribute; use the characteristic polynomial f(Ix) to calculate and process the known independent variables to obtain an approximate dependent variable; determine whether the absolute value of the difference between the approximate dependent variable and the corresponding known dependent variable is greater than a set first regression discrimination threshold; if it is greater than the first regression discrimination threshold, delete the data from the second data set; if it is less than or equal to the first regression discrimination threshold, do not process the data;

[0061] S2302, constructing a data set after boundary checking by using all data of executing S2301 of the second data set;

[0062] The classification and identification processing of the preprocessed data set to obtain category information includes:

[0063] S31, obtaining preset classification information; the classification information is represented as a classification matrix; the row vector of the classification matrix is ​​a vector constructed by typical values ​​of all data attributes of each target characteristic type; the row number of the row vector of the classification matrix is ​​a classification number; the classification number represents the category information of the target characteristic type corresponding to the row vector;

[0064] S32, using each target characteristic data of the preprocessed data set, respectively construct a corresponding vector to be identified;

[0065] S33, performing classification calculation processing on each vector to be identified and the classification matrix to obtain category information of the vector to be identified;

[0066] The classification calculation process includes:

[0067] Performing cross-correlation operations on the vector to be identified and each row vector of the classification matrix to obtain corresponding cross-correlation sequences;

[0068] Perform feature transformation processing on each cross-correlation sequence to obtain the corresponding feature sequence;

[0069] Perform distribution calculation on the feature sequence to obtain a distribution sequence;

[0070] Perform logarithmic weighted accumulation processing on each distribution sequence to obtain the corresponding classification feature value;

[0071] The row number of the row vector of the classification matrix corresponding to the distribution sequence with the smallest classification characteristic value is determined as the category information of the vector to be identified.

[0072] The expression of the distribution calculation process is:

[0073]

[0074] Among them, r k,i represents the distribution of the i-th value of the k-th feature sequence, E k (i) represents the i-th value of the k-th feature sequence, and N0 is the total number of elements contained in the feature sequence;

[0075] The expression of the logarithmic weighted accumulation process is:

[0076]

[0077] Among them, H k is the classification feature value of the kth distribution sequence, ω i is the i-th weighted value.

[0078] The weighted value may be obtained by presetting or by weight calculation;

[0079] The feature transformation may be a Demon spectrum transformation process.

[0080] The row number of the row vector of the classification matrix corresponding to the distribution sequence with the smallest classification eigenvalue is determined by determining the corresponding feature sequence through the distribution sequence with the smallest classification eigenvalue, determining the cross-correlation sequence based on the feature sequence, and determining the row number of the row vector based on the cross-correlation sequence.

[0081] The method of using each target characteristic data of the preprocessed data set to construct a corresponding vector to be identified is to use each data in each target characteristic data as a vector element to construct a corresponding vector to be identified.

[0082] The linear fitting process uses the characteristic vector element serial number value Ix as a known independent variable and the characteristic vector element value as a known dependent variable, and constructs the curve to be approximated using the known independent variable and the known dependent variable, and uses the function approximation method to perform curve fitting on the curve to be approximated to obtain the characteristic polynomial f(Ix).

[0083] The curve fitting of the curve to be approximated by the function approximation method may be performed by using the best consistent linear approximation method. The characteristic polynomial f(Ix) is expressed as:

[0084] f(Ix)=α P1 (Ix) P1 +α P1-1 (Ix) P1-1 +…+α2(Ix) 2 +α1(Ix)+α0,

[0085] Wherein, P1 is the order of the characteristic polynomial f(Ix), α0, α1, α2, …, α P1 are the coefficients of the characteristic polynomial f(Ix);

[0086] The data cleaning process includes filling missing values, smoothing noise data, and smoothing or deleting outliers; the smoothed noise data is obtained by first determining the noise data, and then smoothing the noise data according to the data before and after the noise data; the noise data is a value whose value is less than the detection sensitivity of the sensor of the observed data, or greater than the measurement upper limit of the sensor of the observed data. The outlier point can be determined by the Kalman filter method. The filling value of the missing value can be determined by averaging the measured values ​​within a certain sampling interval before and after the missing value.

[0087] The weight calculation includes:

[0088] S301, extracting a simulation data subset, an actual data subset and evaluation data volume requirement information of all data attributes of target characteristic data from the preprocessed data set;

[0089] S302, for each type of data attribute simulation data subset, the maximum value, minimum value and median value of the number of entity information of all categories are obtained by statistics, and the maximum value, minimum value and median value are normalized and calculated to obtain the normalized value of the category value; the normalized calculation process, the calculation expression is:

[0090] ρi =P i / (P max -P min ),

[0091] Among them, P max and P min are the maximum and minimum values ​​of the number of entity information of all categories in the simulation data subset of the i-th entity, P i is the median value of the number of entity information of all categories in the simulation data subset of the i-th entity, ρ i is the normalized value of the category value of the i-th entity;

[0092] S303, all entity types are divided into N1 categories, each entity type category has a corresponding entity category coefficient, and the entity category coefficient corresponding to the i-th entity type category is denoted as a i , where i = 1, 2, ..., N1, a i ∈[0,1]; according to the entity category to which each type of data attribute belongs, determine the entity category coefficient of the class entity;

[0093] S304, according to the values ​​of the evaluation data volume demand information, the values ​​are divided into N2 levels from high to low, and each level corresponds to a corresponding demand coefficient; according to the period described by the values ​​of the evaluation data volume demand information of each type of data attribute, the corresponding demand coefficient is determined;

[0094] S305, for each data attribute of the target characteristic data, weighted summation processing is performed on the category numerical normalization values ​​of all entity types, entity category coefficients, and demand coefficients to obtain a weighted value w of the data attribute; the weighted summation processing includes:

[0095]

[0096] Among them, b i is the demand coefficient of the i-th type of entity, [w 1,1 ,w 1,2 ,w 1,3 ] is the first weight sequence;

[0097] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for classifying and processing target characteristic data, characterized in that: include: S1, obtain the target characteristic data set; The target characteristic data set includes target characteristic data; S2, preprocessing the target characteristic data set to obtain a preprocessed data set; S3, performing classification and recognition processing on the pre-processed data set to obtain category information.

2. The method for classifying and processing target characteristic data according to claim 1, characterized in that: The preprocessing of the target characteristic data set to obtain a preprocessed data set includes: S21, performing data cleaning processing on the target characteristic data set to obtain a first data set; S22, normalizing the first data set to obtain a second data set; S23, performing boundary check processing on the second data set to obtain a preprocessed data set.

3. The method for classifying and processing target characteristic data according to claim 2, characterized in that: The performing boundary check processing on the second data set to obtain a preprocessed data set includes: S231, for each data attribute of the second data set, preset a corresponding value range; the value range includes an upper limit value of the value range and a lower limit value of the value range; S232, for each data of the second data set, judging whether the data value is within the value range according to the value range of its data attribute; if the data value is within the value range, not processing the data; if the data value is not within the value range, setting the data value to the upper limit value or the lower limit value of the value range that is closest to the data value; S233, performing fusion processing on all data of the second data set after S232 is completed to obtain a preprocessed data set.

4. The method for classifying and processing target characteristic data according to claim 3, characterized in that: The performing boundary check processing on the second data set to obtain a preprocessed data set includes: S2301, for each type of data attribute of the second data set, take the data collection information of the data as a known independent variable, take the data value of the data as a known dependent variable, use the known independent variable and the known dependent variable to construct a curve to be approximated, use the function approximation method to perform curve fitting on the curve to be approximated, and obtain a characteristic polynomial f(Ix) of the type of data attribute; use the characteristic polynomial f(Ix) to calculate and process the known independent variables to obtain an approximate dependent variable; determine whether the absolute value of the difference between the approximate dependent variable and the corresponding known dependent variable is greater than a set first regression discrimination threshold; if it is greater than the first regression discrimination threshold, delete the data from the second data set; if it is less than or equal to the first regression discrimination threshold, do not process the data; S2302: Perform fusion processing on all data executed in S2301 of the second data set to obtain a data set after boundary check.

5. The method for classifying and processing target characteristic data according to claim 3, characterized in that: The classification and identification processing of the preprocessed data set to obtain category information includes: S31, obtaining preset classification information; the classification information is represented as a classification matrix; the row vector of the classification matrix is ​​a vector constructed by typical values ​​of all data attributes of each target characteristic type; the row number of the row vector of the classification matrix is ​​a classification number; the classification number represents the category information of the target characteristic type corresponding to the row vector; S32, using each target characteristic data of the preprocessed data set, respectively construct a corresponding vector to be identified; S33, performing classification calculation processing on each vector to be identified and the classification matrix to obtain category information of the vector to be identified.

6. The method for classifying and processing target characteristic data according to claim 5, characterized in that: The classification calculation process includes: Performing cross-correlation operations on the vector to be identified and each row vector of the classification matrix to obtain corresponding cross-correlation sequences; Perform feature transformation processing on each cross-correlation sequence to obtain the corresponding feature sequence; Perform distribution calculation on the feature sequence to obtain a distribution sequence; Perform logarithmic weighted accumulation processing on each distribution sequence to obtain the corresponding classification feature value; The row number of the row vector of the classification matrix corresponding to the distribution sequence with the smallest classification characteristic value is determined as the category information of the vector to be identified.

7. The method for classifying and processing target characteristic data according to claim 6, characterized in that: The expression of the distribution calculation process is: Among them, r k,i represents the distribution of the i-th value of the k-th feature sequence, E k (i) represents the i-th value of the k-th feature sequence, and N0 is the total number of elements contained in the feature sequence; The expression of the logarithmic weighted accumulation process is: Among them, H k is the classification feature value of the kth distribution sequence, ω i is the i-th weighted value.

8. A classification processing device for target characteristic data, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the target characteristic data classification processing method according to any one of claims 1 to 7.

9. A computer storable medium, characterized in that: The computer storable medium stores computer instructions, and when the computer instructions are called by a computer, they are used to execute the classification processing method for target characteristic data according to any one of claims 1 to 7.

10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the classification processing method of target characteristic data according to any one of claims 1 to 7.

Images