Structured data classification method, apparatus and terminal device

Abstract

The application is suitable for the technical field of computer application, and provides a structured data classification method, which comprises the following steps: separating the to-be-classified structured data containing multiple fields to determine discrete field data included in the to-be-classified structured data, grouping sentences of a field name and a field value corresponding to each field to determine text data corresponding to the to-be-classified structured data, then classifying the discrete field data by using a preset discrete data classification model to determine a first classification result corresponding to the to-be-classified structured data, further classifying the text data by using a preset text data classification model to determine a second classification result corresponding to the to-be-classified structured data, and finally determining a target classification result corresponding to the to-be-classified structured data according to the first classification result and the second classification result. Thus, by combining the discrete data classification method and the text data classification method, the accuracy of the structured data classification is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer application, and particularly relates to a structured data classification method and device and a terminal device. BACKGROUND

[0002] In recent years, with the rapid development of the Internet, network information and data information show geometric growth. In the face of a variety of network data, how to effectively use has become one of the research hotspots of current information work. In order to quickly and accurately find the information needed by users from the massive data, data automatic analysis has developed rapidly.

[0003] In related technologies, when classifying data, for complex data containing both discrete data and text data, since the data structure and data type contained are not unified, when classifying such data by using a single data classification algorithm, an accurate prediction result cannot be usually obtained, thereby resulting in low data classification accuracy. SUMMARY

[0004] The embodiments of the present application provide a structured data classification method and device and a terminal device, which can solve the problem that for complex data containing both discrete data and text data, since the data structure and data type contained are not unified, when classifying such data by using a single data classification algorithm, an accurate prediction result cannot be usually obtained, thereby resulting in low data classification accuracy.

[0005] In a first aspect, the embodiments of the present application provide a structured data classification method, including: obtaining structured data to be classified, wherein the structured data to be classified includes a plurality of fields, each field includes a field name and a field value; separating the structured data to be classified according to the data type of the field value corresponding to each field, to determine discrete field data included in the structured data to be classified; performing sentence processing on the field name and the field value corresponding to each field, to determine text data corresponding to the structured data to be classified; performing classification processing on the discrete field data by using a preset discrete data classification model, to determine a first classification result corresponding to the structured data to be classified; performing classification processing on the text data by using a preset text data classification model, to determine a second classification result corresponding to the structured data to be classified; and determining a target classification result corresponding to the structured data to be classified according to the first classification result and the second classification result.

[0006] In a second aspect, an embodiment of the present application provides a structured data classification apparatus, comprising: a first obtaining module configured to obtain structured data to be classified, wherein the structured data to be classified comprises a plurality of fields, and each field comprises a field name and a field value; a first determining module configured to separate the structured data to be classified according to a data type of the field value corresponding to each field, so as to determine discrete field data included in the structured data to be classified; a second determining module configured to group sentences of the field name and the field value corresponding to each field, so as to determine text data corresponding to the structured data to be classified; a third determining module configured to perform classification processing on the discrete field data by using a preset discrete data classification model, so as to determine a first classification result corresponding to the structured data to be classified; a fourth determining module configured to perform classification processing on the text data by using a preset text data classification model, so as to determine a second classification result corresponding to the structured data to be classified; and a fifth determining module configured to determine a target classification result corresponding to the structured data to be classified according to the first classification result and the second classification result.

[0007] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the structured data classification method as described above when executing the computer program.

[0008] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is characterized in that the computer program is executed by a processor to implement the structured data classification method as described above.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a terminal device, causes the terminal device to execute the structured data classification method of any one of the first aspect.

[0010] Compared with the prior art, the embodiment of the present application has the beneficial effects that: when classifying structured data, the discrete data and the text data in the structured data are separated, and the discrete data classification algorithm and the text classification algorithm are respectively used to classify the discrete data and the text data corresponding to the structured data, and the final classification result of the structured data is determined by combining the classification results of the two algorithms, thereby reducing the influence of mixed data types on data classification and improving the accuracy of structured data classification. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0012] Figure 1 is a flowchart of a structured data classification method provided by an embodiment of the present application;

[0013] Figure 2 is a flowchart of a structured data classification method provided by another embodiment of the present application;

[0014] Figure 3 is a flowchart of a structured data classification method provided by yet another embodiment of the present application;

[0015] Figure 4 is a flowchart of a structured data classification method provided by yet another embodiment of the present application;

[0016] Figure 5 is a structural diagram of a structured data classification apparatus provided by an embodiment of the present application;

[0017] Figure 6 is a structural diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0018] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0019] It should be understood that the term "comprising" as used in the specification and in the claims indicates the presence of the recited features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0020] It should also be understood that the term "and / or" as used in the specification and in the claims indicates any combination of one or more of the associated listed items and all possible combinations of those items.

[0021] As used in the specification and appended claims herein, the term “if’ can be interpreted as meaning “when” or “upon” or “in response to a determination” or “in response to a detection” depending on the context. Similarly, the phrase “if it is determined” or “if [the described condition or event] is detected” can be interpreted as meaning “upon a determination” or “in response to a determination” or “upon a detection of [the described condition or event]” or “in response to a detection of [the described condition or event]” depending on the context.

[0022] In addition, in the description of the specification and the appended claims herein, the terms “first”, “second”, “third”, etc. are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0023] In the specification of the present application, the reference “one embodiment” or “some embodiments” and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements “in one embodiment”, “in some embodiments”, “in other some embodiments”, “in further some embodiments” and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean “one or more but not all embodiments”, unless otherwise specifically emphasized. The terms “include”, “contain”, “have” and their variants mean “include but not limited to”, unless otherwise specifically emphasized.

[0024] The structured data classification method, device, terminal equipment, storage medium and computer program provided by the present application are described in detail below with reference to the accompanying drawings.

[0025] Figure 1 A flowchart of a structured data classification method provided by an embodiment of the present application is shown.

[0026] Step 101, obtaining structured data to be classified, wherein the structured data to be classified includes a plurality of fields, and each field includes a field name and a field value.

[0027] It should be noted that the structured data classification method of the present application embodiment can be executed by the structured data classification device of the present application embodiment. The structured data classification device of the present application embodiment can be configured in any terminal equipment to execute the structured data classification method of the present application embodiment.

[0028] The structured data to be classified can refer to structured data containing multiple data types. For example, the structured data of the present application embodiment can be table data.

[0029] The field name can refer to the corresponding name in the field of the structured data.

[0030] wherein, the field value, can be a specific value corresponding to each field in the structured data.

[0031] For example, when classifying demographic data, any one piece of data in the demographic data table can be taken as the structured data to be classified. A piece of structured data to be classified can be represented as "name: Zhang San; age: 35; ID number: 123456789; hometown: X province X city; occupation: doctor", wherein the field names of the fields are name, age, ID number, hometown, and occupation, respectively. For another example, the structured data to be classified in the embodiments of the present application is the description data of a collection, and the structured data to be classified in the embodiments of the present application can be any one piece of data in Table 1, wherein the field names of the fields included in each piece of data are collection name, platform for issuing, copyright party, copyright party level, collection type, style, whether series is issued, price, display method, play type, and description level, respectively. When the data in the first row of Table 1 is taken as the structured data to be classified, the field values of the fields are (Yuan) blue and white peony pattern vase, meta universe platform, ceramic museum, level one, ceramic, national style, yes, 50, flat, pricing and issuing, high, respectively.

[0032] Table 1

[0033]

[0034]

[0035] In the embodiments of the present application, the input structured data can be acquired in real time as the structured data to be classified, and the structured data to be classified is classified to determine the classification result corresponding to the structured data to be classified. For example, when a table as shown in Table 1 is acquired, the classification method of the structured data in the embodiments of the present application can determine each piece of data in the table as the structured data to be classified in turn, so as to classify each piece of structured data.

[0036] In step 102, the structured data to be classified is separated according to the data type of the field value corresponding to each field, so as to determine the discrete field data included in the structured data to be classified.

[0037] The discrete field data can be the fixed value range data of the field value included in the structured data to be classified. For example, the fields with the field names of copyright party level, collection type, style, whether series is issued, price, display method, play type, and description level in Table 1 are discrete fields.

[0038] In the embodiment of the present application, in order to avoid that the structured data to be classified contains both discrete data and text data, thereby causing the classification result to be inaccurate, the structured data to be classified can be classified, so as to respectively use different classification algorithms to classify and process the discrete data and the text data, so as to improve the accuracy of the structured data classification. Therefore, whether the value range of the field value corresponding to each field in the structured data to be classified is fixed can be determined according to the data type of the field value corresponding to each field in the structured data to be classified, and the fields whose field value ranges are fixed are determined as discrete field data included in the structured data to be classified.

[0039] For example, for each row of structured data in Table 1, since the value of the collection name field is random and does not have a fixed value range, it can be determined that the collection name field does not belong to discrete field data; since all collections can be divided into a fixed number of collection types, the field value of the collection type field can have a fixed value range, so the collection type can be determined as discrete field data. By analogy, whether each field in the structured data to be classified is discrete field data can be determined.

[0040] In step 103, the field name and field value corresponding to each field are processed to form a sentence, so as to determine the text data corresponding to the structured data to be classified.

[0041] The text data can be text data that describes the structured data to be classified as a whole.

[0042] In the embodiment of the present application, the field name and field value of each field in the structured data to be classified can be combined to form a sentence, so as to generate the text data corresponding to the structured data to be classified, so that the generated text data can describe all the information contained in the structured data to be classified.

[0043] For example, if the structured data to be classified is the first row of data in Table 1, after the field name and field value corresponding to each field of the structured data to be classified are processed to form a sentence, the generated text data can be "The collection name is (Yuan) blue and white peony pattern vase, the issuing platform is the metaverse platform, the copyright party is the ceramic museum, the copyright party level is first-class, the collection type is ceramic, the style is national style, whether it is a series is yes, the price is 50, the display mode is plane, the play type is pricing, and the description degree is high."

[0044] In step 104, the discrete field data is classified by using a preset discrete data classification model, so as to determine the first classification result corresponding to the structured data to be classified.

[0045] The preset discrete data classification model can be a data classification model that is pre-trained and can perform classification processing on discrete data.

[0046] It should be noted that the preset discrete data classification model can be any type of discrete data classification model. In actual use, a suitable type of discrete data classification model can be selected according to actual needs and specific application scenarios to generate a preset discrete data classification model suitable for actual classification scenarios. The embodiments of the present application do not limit this. For example, the preset discrete data classification model can be any one of a support vector machine model, a language model neural network model, a multi-classification logistic ordered regression model, a K-nearest neighbor model, a decision tree model, and a random forest prediction model.

[0047] The first classification result can be a classification result obtained by using the preset discrete data classification model to perform classification processing on the discrete field data in the structured data to be classified.

[0048] In the embodiments of the present application, the discrete field data in the structured data to be classified can be extracted first, and then the preset discrete data classification model is used to perform classification processing on the discrete field data to obtain a first classification result corresponding to the structured data to be classified.

[0049] It should be noted that in the embodiments of the present application, a large amount of training data and labeled data can be obtained according to a specific classification target to train the discrete data classification model, so that the preset discrete data classification model generated by training can generate a desired result of classifying the structured data to be classified according to the structured data to be classified. For example, when the classification target is to predict the sales of a collection, i.e., the classification target is to divide the collection data into multiple categories (such as low sales, medium sales, high sales, etc.) according to sales, the structured data as shown in Table 1 can be obtained as training data, and the actual sales corresponding to each piece of structured data can be obtained as labeled data. Then, the discrete data in each piece of training data is input into an initial discrete data classification model in sequence to generate a prediction result for each piece of training data. Then, the network parameters of the initial discrete data classification model are updated according to the difference between the prediction result of each piece of training data and the labeled data, and the updated discrete data classification model is used for further training until an updated discrete data classification model with a loss value less than a certain threshold is generated. After that, the updated discrete data classification model is determined as the preset discrete data classification model.

[0050] Step 105, using the preset text data classification model to perform classification processing on the text data to determine a second classification result corresponding to the structured data to be classified.

[0051] The text type data classification model can be a pre-trained data classification model that can classify text type data.

[0052] It should be noted that the preset discrete data classification model can be any type of natural language processing text classification model. In actual use, a suitable type of text data classification model can be selected for training to generate a preset discrete data classification model suitable for an actual classification scenario, and the embodiments of the present application do not limit this. For example, the preset discrete data classification model can be any one of a text classification algorithm based on deep learning, a related model for generating word vectors (Word to Vector, Word2Vec), a FastText fast text classification, a bidirectional encoding representation from Transformers (Bidirectional Encoder Representation from Transformers, BERT) based on a Transformer algorithm, and the like.

[0053] The second classification result can be a classification result obtained by using the preset text type data classification model to classify the text type data corresponding to the structured data to be classified.

[0054] In the embodiments of the present application, the text type data corresponding to the structured data to be classified can be input into the preset text type data classification model to classify the text type data, so as to obtain a second classification result corresponding to the structured data to be classified.

[0055] It should be noted that in the embodiments of the present application, a large amount of training data and labeled data need to be obtained according to the specific classification target to train the text type data classification model, so that the preset text type data classification model generated by training can generate the expected result of classifying the structured data to be classified according to the structured data to be classified. For example, when the classification target is to predict the sales of the collection, that is, the classification target is to divide the collection data into multiple categories (such as low sales, medium sales, high sales, etc.) according to sales, the structured data as shown in Table 1 can be obtained as training data, and the actual sales corresponding to each piece of structured data can be obtained as labeled data, and then each field in each piece of training data is grouped and input into the initial text type data classification model in turn to generate a prediction result for each piece of training data. Then, according to the difference between the prediction result corresponding to each piece of training data and the labeled data, the network parameters of the initial text type data classification model are updated, and the updated text type data classification model is used for further training until an updated text type data classification model with a loss value less than a certain threshold is generated. After that, the updated text type data classification model is determined as the preset text type data classification model.

[0056] Step 106, determining the target classification result corresponding to the structured data to be classified according to the first classification result and the second classification result.

[0057] Among them, the target classification result can be the final classification result of the structured data to be classified obtained by combining the first classification result and the second classification result of the structured data to be classified.

[0058] In the embodiments of the present application, after obtaining the first classification result and the second classification result corresponding to the structured data to be classified, the target classification result corresponding to the structured data to be classified can be obtained by combining the first classification result and the second classification result, so as to ensure the accuracy of the classification result of the structured data fused with discrete data and text type data.

[0059] Further, since the preset discrete data classification model and the preset discrete classification model can also output the confidence of the prediction result when predicting the category of the structured data to be classified, the target classification result corresponding to the structured data to be classified can also be determined according to the confidence output by the classification model. That is, in one possible implementation manner of the embodiments of the present application, the first classification result can include a first category and a first confidence corresponding to the first category, and the second classification result includes a second category and a second confidence corresponding to the second category. Correspondingly, the step 106 can include:

[0060] When the first confidence is greater than or equal to the second confidence, the first category is determined as the target classification result;

[0061] determining the second category as the target classification result when the first confidence is less than the second confidence.

[0062] The first category can refer to a category to which the structured data to be classified belongs, which is generated by performing classification processing on the structured data to be classified by using a preset discrete classification model.

[0063] The first confidence can refer to a confidence that the structured model to be classified belongs to the first category, which is output by the preset discrete classification model.

[0064] The second category can refer to a category to which the structured data to be classified belongs, which is generated by performing classification processing on the structured data to be classified by using a preset text classification model.

[0065] The second confidence can refer to a confidence that the structured model to be classified belongs to the second category, which is output by the preset text classification model.

[0066] In the embodiments of the present application, when the first category is the same as the second category, the target classification result corresponding to the structured data to be classified can be determined as the first category or the second category; when the first category is different from the second category, the category with a higher confidence can be determined as the target classification result corresponding to the structured data to be classified according to the first confidence corresponding to the first category and the second confidence corresponding to the second category. That is, when the first confidence corresponding to the first category is greater than or equal to the second confidence corresponding to the second category, the first category can be determined as the target classification result; when the first confidence corresponding to the first category is less than the second confidence corresponding to the second category, the second category can be determined as the target classification result.

[0067] For example, assuming that the structured data classification method of the embodiments of the present application is used to predict the sales of an inventory, if the data in the first row in Table 1 is determined as the structured data to be classified, and the first classification result output by the preset discrete data classification model is "high sales; confidence: 0.9", and the second classification result output by the preset text data classification model is "high sales; confidence: 0.8", at this time the first category is the same as the second category, the target classification result can be directly determined as high; assuming that the first classification result output by the preset discrete data classification model is "high sales; confidence: 0.9", and the second classification result output by the preset text data classification model is "medium sales; confidence: 0.8", at this time the first category is different from the second category, the category with a higher confidence can be determined as the target classification result, that is, the target classification result can be determined as high.

[0068] The method for classifying structured data provided in the application separates the structured data to be classified according to the data types of the field values corresponding to each field in the structured data to be classified, determines the discrete field data included in the structured data to be classified, and processes the field names and field values corresponding to each field to form a sentence to determine the text data corresponding to the structured data to be classified. Then, the discrete field data is classified by using a preset discrete data classification model to determine the first classification result corresponding to the structured data to be classified, and the text data is classified by using a preset text data classification model to determine the second classification result corresponding to the structured data to be classified. Finally, the target classification result corresponding to the structured data to be classified is determined according to the first classification result and the second classification result. In this way, when the structured data is classified, the discrete data in the structured data is first separated, the text data corresponding to the structured data is generated by forming a sentence, then the discrete data classification model and the text data classification model are used to classify the discrete data and the text data respectively, and the final classification result of the structured data is determined by combining the classification results of the two algorithms, thereby reducing the influence of mixed data types on data classification and improving the accuracy of structured data classification.

[0069] In a possible implementation form of the application, when the structured data includes a numerical field with a numerical field value, the numerical field can be discretized, and then each field is combined to form a sentence and subjected to discrete data classification processing, so as to further improve the accuracy of structured data classification.

[0070] The method for classifying structured data provided in the application will be further described below. Figure 2 The method for classifying structured data provided in the application will be further described below.

[0071] Figure 2 A flowchart of another method for classifying structured data provided in the application is shown.

[0072] In step 201, structured data to be classified is obtained, wherein the structured data to be classified includes a plurality of fields, and each field includes a field name and a field value.

[0073] In step 202, the structured data to be classified is separated according to the data types of the field values corresponding to each field to determine the discrete field data included in the structured data to be classified.

[0074] The specific implementation process and principles of steps 201-202 described above can refer to the detailed description of the above embodiments, which will not be described here again.

[0075] In step 203, the discrete field with a numerical value is determined as a numerical field according to the field value corresponding to each discrete field.

[0076] The numerical field can refer to a field with a numerical value in the discrete field.

[0077] In the embodiment of the present application, after the discrete field included in the structured data to be classified is determined, the field with a numerical value is determined as a numerical field according to whether the field value of each discrete field is numerical data.

[0078] For example, the price field can be determined as a numerical field according to the data shown in Table 1.

[0079] In step 204, the text field value corresponding to each numerical field is determined according to the preset correspondence between the text field value and the value range and the value range of the field value corresponding to each numerical field.

[0080] The text field value can refer to the field value converted into a literal form according to the value range of the numerical field value.

[0081] In the embodiment of the present application, for each numerical field, the correspondence between the text field value corresponding to the numerical field and the value range can be preset. Then, when the structured data to be classified contains the numerical field, the preset correspondence between the text field value corresponding to the numerical field and the value range can be obtained, and then the text field value corresponding to the value range can be determined as the text field value corresponding to the numerical field according to the value range of the numerical field.

[0082] For example, if a numerical field value is 10, the value range of the field value is 1 to 100, and the preset text field value of the field is low, medium, and high, corresponding to the value range of 1 to 25, 25 to 75, and 75 to 100, respectively, the text field value corresponding to the numerical field can be determined as low.

[0083] In step 205, the text field value corresponding to each numerical field is determined as the field value corresponding to each numerical field, respectively, to texturize the field value corresponding to each numerical field.

[0084] In the embodiment of the present application, after the text field value corresponding to each numerical field is determined, the field value of the numerical field in the discrete field can be converted from numerical data to the corresponding text field value, to texturize each numerical field.

[0085] Step 206, for each text type field, the field name and field value corresponding to the field, each discrete type field, the field name and field value corresponding to the field and each numerical type field, the field name and the text field value after the field are processed to generate the text type data corresponding to the structured data to be classified.

[0086] In the embodiment of the application, after each numerical type field in the structured data to be classified is texted, the texted fields can be processed to generate the text data corresponding to the structured data to be classified. That is, the field name and field value of the text type field corresponding to the structured data to be classified, the field name and the text field value after the text of each numerical type field in the discrete type field, and the field name and field value of the non-numerical type field in the discrete type field are combined into a sentence to generate the text type data corresponding to the structured data to be classified.

[0087] For example, as shown in Table 1, the numerical type field is price, and when the first row of data in Table 1 is used as the structured data to be classified, the field value of the price field of the structured data to be classified can be texted as "low". Thus, after the texted fields are processed, the text type data corresponding to the structured data to be classified is "Collection name: blue and white peony pattern vase, issuing platform: metaverse platform, copyright party: ceramic museum, copyright party level: first level, collection type: ceramic, style: national style, whether series: yes, price: low, display mode: plane, play type: pricing, description: high".

[0088] Step 207, using a preset discrete type data classification model to classify the discrete type field data to determine the first classification result corresponding to the structured data to be classified.

[0089] In the embodiment of the application, if the numerical type field in the discrete type data is discretized in the foregoing steps, the discretized discrete type data can be input into the preset discrete type data classification model for classification processing to determine the first classification result.

[0090] Step 208, using a preset text type data classification model to classify the text type data to determine the second classification result corresponding to the structured data to be classified.

[0091] Step 209, determining the target classification result corresponding to the structured data to be classified according to the first classification result and the second classification result.

[0092] The specific implementation process and principles of steps 207-209 described above can refer to the detailed description of the above embodiments, which will not be repeated here.

[0093] The method for classifying structured data provided in the application separates the structured data to be classified according to the data types of the field values corresponding to each field in the structured data to be classified, to determine discrete field data included in the structured data to be classified, and performs text processing on the field values of the numerical fields in the discrete fields, and then performs sentence processing on the discrete numerical fields and the field names and field values corresponding to each field, to determine text data corresponding to the structured data to be classified, and then classifies the discrete field data by using a preset discrete data classification model, and classifies the text data by using a preset text data classification model, and finally determines the target classification result corresponding to the structured data to be classified according to the classification results of the two classification models. In this way, the numerical fields in the structured data to be classified are discretized to reduce the influence of data fluctuations of the numerical fields on the classification result, and the accuracy of the classification of the structured data is further improved.

[0094] In a possible implementation form of the application, the redundant fields in the structured data to be classified that affect the classification accuracy can be removed, and then the remaining fields are classified by using the classification model, to further improve the accuracy of the classification of the structured data.

[0095] The method for classifying structured data provided in the application will be further described below. Figure 3 The method for classifying structured data provided in the application will be further described below.

[0096] Figure 3 A flowchart of another method for classifying structured data provided in the application is shown.

[0097] In step 301, structured data to be classified is obtained, wherein the structured data to be classified includes a plurality of fields, and each field includes a field name and a field value.

[0098] In step 302, the structured data to be classified is separated according to the data types of the field values corresponding to each field, to determine discrete field data included in the structured data to be classified.

[0099] The specific implementation process and principles of steps 301-302 described above can refer to the detailed description of the above embodiments, which will not be described herein again.

[0100] In step 303, it is determined whether the discrete field data contains a preset redundant field according to the preset redundant field.

[0101] The preset redundant field can be a field included in the structured data to be classified that causes the prediction accuracy of the preset discrete classification model to decrease.

[0102] In the embodiments of the present application, it is determined that the discrete field data contained in the structured data to be classified, whether the discrete field data contains a preset redundant field, so as to remove the redundant field contained in the discrete field data.

[0103] Further, the redundant field affecting the accuracy of the preset discrete data classification model can be determined by training the preset discrete data classification model through a large amount of training data. That is, in a possible implementation manner of the embodiments of the present application, before the step 303, the method can further include:

[0104] obtaining a first test data set, wherein the first test data set includes a plurality of first structured data and a first real category corresponding to each first structured data, and the second structured data includes N discrete fields to be screened, wherein N is a positive integer;

[0105] removing the i th discrete field to be screened included in each first structured data to generate an i th reference test data set, wherein i is a positive integer greater than or equal to 1 and less than or equal to N;

[0106] using the preset discrete data classification model to perform classification processing on the plurality of discrete fields to be screened corresponding to each first structured data in the i th reference test data set, to determine an i th first predicted category corresponding to each first structured data;

[0107] determining a first classification accuracy of the preset discrete data classification model using the i th reference training data set for classification according to the difference between the i th first predicted category corresponding to each first structured data and the first real category corresponding to each first structured data;

[0108] selecting the preset redundant field from the N discrete fields to be screened according to the N first classification accuracies.

[0109] The first test data set can be a data set for testing the preset discrete data classification model to determine the redundant field contained in the first structured data, that is, a data set for testing the classification result of the first structured data.

[0110] The first real category can be the real classification result of the first structured data.

[0111] For example, when the classification target of the embodiments of the present application is to predict the sales classification of the collection, the first test data set can be the data as shown in Table 2, wherein each row of data in Table 2 can be a piece of first structured data, and columns 2-9 in Table 2 are discrete fields to be screened included in the first structured data, i.e., copyright level, collection type, style, whether series is issued, price, display method, play type, and description detail level, and column 10 can be the first true category corresponding to the first structured data. It should be noted that 9 pieces of first structured data are exemplarily shown in Table 2 for illustration, and in actual use, the first test data set can include a large number (such as 1000, 10000, etc.) of first structured data.

[0112] Table 2

[0113]

[0114] The reference test data set can be a test data set generated by removing any one of the discrete fields to be screened in each piece of first structured data in the first test data set.

[0115] For example, after removing the first discrete field to be screened “copyright level” in each piece of first structured data in Table 2, a first parameter test data set can be generated, as shown in Table 3; and similarly, after removing the second discrete field to be screened in each piece of first structured data, a second parameter test data set can be generated, and so on.

[0116] Table 3

[0117]

[0118] The first predicted category can be a prediction result obtained by using a preset discrete data classification model to classify the first structured data from which the discrete field to be screened is removed.

[0119] The first classification accuracy can be determined according to the difference between the first predicted category and the first true category corresponding to each piece of first structured data.

[0120] In the embodiment of the present application, after the i-th discrete field to be screened is removed from the first test data set in the above manner and the i-th reference test data set is generated, the remaining discrete field data contained in each first structured data included in the i-th reference test data set can be input into the preset discrete data classification model to determine the i-th first predicted category corresponding to each first structured data, and then according to the difference between the i-th first predicted category of each first structured data and the corresponding first real category, the first classification accuracy of the preset discrete data classification model using the i-th reference training data set for classification is determined. In the same manner as described above, N first classification accuracies of the preset discrete data classification model using N reference training data sets for classification can be determined.

[0121] As a possible implementation, after the N first classification accuracies are determined, one or more reference test data sets with the highest first classification accuracy can be determined, and the discrete field to be screened not contained in the one or more reference test data sets is determined as the preset redundant field.

[0122] As a possible implementation, a classification accuracy threshold can also be preset, and the reference test data set with the first classification accuracy greater than the classification accuracy threshold is determined, and the discrete field to be screened not contained in the reference test data set with the first classification accuracy greater than the classification accuracy threshold is determined as the preset redundant field.

[0123] For example, after the "copyright party level" and "play type" fields in Table 2 are removed, the two generated reference test data sets have the highest first classification accuracy, and then the "copyright party level" and "play type" fields can be determined as the preset redundant field.

[0124] Step 304, when the discrete field data contains the preset redundant field, the preset redundant field is removed from the discrete data.

[0125] In the embodiment of the present application, when it is determined that the discrete field data contains the preset redundant field, the preset redundant field can be removed to avoid the influence of the preset redundant field on the classification accuracy.

[0126] Step 305, when the discrete field data contains the preset redundant field, the preset redundant field is removed from the field to generate a screened field.

[0127] Step 306, the field name and field value corresponding to each screened field are processed to determine the text type data corresponding to the structured data to be classified.

[0128] In the embodiment of the present application, after removing the preset redundant fields contained in the discrete field data, the filtered fields generated after removing the redundant fields can also be used for grouping when performing grouping processing, so as to generate the text type data corresponding to the structured data to be classified.

[0129] For example, as shown in Table 1, if the preset redundant fields are the "copyright party level" and "play type" fields, when the data in the first row of Table 1 is taken as the structured data to be classified, the text type data generated after grouping can be "The name of the collection is (Yuan) blue and white peony pattern vase, the issuing platform is the meta universe platform, the copyright party is the ceramic museum, the collection type is ceramic, the style is national style, whether it is a series is yes, the price is low, the display mode is plane, and the description degree is high".

[0130] In step 307, a preset discrete data classification model is used to perform classification processing on the discrete field data, so as to determine the first classification result corresponding to the structured data to be classified.

[0131] In step 308, a preset text type data classification model is used to perform classification processing on the text type data, so as to determine the second classification result corresponding to the structured data to be classified.

[0132] In step 309, the target classification result corresponding to the structured data to be classified is determined according to the first classification result and the second classification result.

[0133] The specific implementation process and principles of steps 307-309 described above can refer to the detailed description of the above embodiment, which will not be described here.

[0134] The structured data classification method provided in the present application separates the structured data to be classified according to the data type of the field value corresponding to each field in the structured data to be classified, determines the discrete field data included in the structured data to be classified, removes the redundant fields in the discrete field data, and then performs grouping processing on the field name and field value corresponding to each field after removing the redundant fields, so as to determine the text type data corresponding to the structured data to be classified. Then, a preset discrete data classification model is used to perform classification processing on the discrete field data after removing the redundant fields, and a preset text type data classification model is used to perform classification processing on the text type data. Finally, the target classification result corresponding to the structured data to be classified is determined according to the classification results of the two classification models. Therefore, by removing the redundant fields in the structured data to be classified which affect the classification accuracy before performing classification on the structured data, and then performing classification processing on the remaining fields by using the classification model, the accuracy of the structured data classification is further improved.

[0135] In a possible implementation form of the present application, the preset discrete classification model can be selected from the plurality of candidate discrete classification models according to an actual application scenario and a classification target, and the selected discrete classification model has the highest accuracy in the current classification scenario, so as to further improve the accuracy of the structured data classification.

[0136] The classification method for structured data provided by the embodiments of the present application will be further described below. Figure 4 The classification method for structured data provided by the embodiments of the present application will be further described below.

[0137] Figure 4 A flowchart of another classification method for structured data provided by the embodiments of the present application is shown.

[0138] In step 401, a second test data set is obtained, wherein the second test data set includes a plurality of second structured data and a second true class corresponding to each second structured data, and the second structured data includes a plurality of discrete fields.

[0139] The second test data set can refer to a data set related to the classification target of the structured data.

[0140] The second true class can refer to the true classification result of the second structured data.

[0141] For example, when the classification target of the embodiments of the present application is to predict the classification of the sales of the collection, the second test data set can be the data shown in Table 2, wherein each row of data in Table 2 can be a second structured data, and columns 2-9 in Table 2 are a plurality of discrete fields included in the first structured data, i.e., copyright party level, collection type, style, whether series is issued, price, display method, play type, and description detail level, and column 10 can be the second true class corresponding to the second structured data.

[0142] In the embodiments of the present application, the plurality of discrete data classification models can be tested by the second test data set including a large amount of second structured data, so as to select the discrete data classification model with the highest classification accuracy as the preset discrete data classification model.

[0143] In step 402, M candidate discrete data classification models are obtained, wherein M is a positive integer greater than 1.

[0144] The candidate discrete data classification model can refer to a data classification model selected in advance and suitable for classifying discrete data. For example, the candidate discrete data classification model can include a support vector machine model, an LM neural network model, a multi-classification logistic ordered regression model, a K-nearest neighbor model, a decision tree model, a random forest prediction model, and the like, which are not limited in the embodiments of the present application.

[0145] In step 403, each second structured data is classified by using the jth candidate discrete data classification model to determine the jth second predicted category corresponding to each second structured data, where j is a positive integer greater than or equal to 1 and less than or equal to M.

[0146] The second predicted category can refer to a predicted result obtained by classifying the discretized field in each second structured data by using the candidate discrete data classification model.

[0147] The jth predicted category can refer to a predicted result generated by classifying the second structured data by using the jth candidate discrete data classification model.

[0148] In the embodiments of the present application, each second structured data in the second test data set can be classified by using each candidate discrete data classification model in sequence to determine the second classification accuracy corresponding to each candidate discrete data classification model. For the jth candidate discrete data classification model, each second structured data in the second test data set can be input into the jth candidate discrete data classification model in sequence to generate the jth second predicted category corresponding to each second structured data.

[0149] In step 404, the jth second classification accuracy corresponding to the jth candidate discrete data classification model is determined according to the jth second predicted category corresponding to each second structured data and the corresponding second real category.

[0150] The jth second classification accuracy corresponding to the jth candidate discrete data classification model can be determined according to the difference between the jth second predicted category corresponding to each second structured data and the corresponding second real category.

[0151] In the embodiments of the present application, the jth second classification accuracy corresponding to the jth candidate discrete data classification model can be determined according to the difference between the jth second predicted category corresponding to each second structured data and the corresponding second real category. In the same way, the second classification accuracy corresponding to each candidate discrete data classification model can be determined.

[0152] Further, the classification accuracy of the model can be measured by the precision and recall of the model, so in one possible implementation manner of the embodiments of the present application, the above step 404 can include:

[0153] The jth precision and recall corresponding to the jth candidate discrete data classification model are determined according to the jth second predicted category corresponding to each second structured data and the corresponding second real category.

[0154] According to the precision and recall corresponding to the jth candidate discrete classification model, a second classification accuracy corresponding to the jth candidate discrete classification model is determined.

[0155] In the embodiments of the present application, the jth second predicted category corresponding to each second structured data is set to false negative (FN) if the actual category corresponding to each second structured data is correct; the jth second predicted category corresponding to each second structured data is set to true negative (TN) if the actual category corresponding to each second structured data is false; the jth second predicted category corresponding to each second structured data is set to true positive (TP) if the actual category corresponding to each second structured data is correct; and the jth second predicted category corresponding to each second structured data is set to false positive (FP) if the actual category corresponding to each second structured data is false, so that the precision corresponding to the jth candidate discrete classification model can be obtained as:

[0156]

[0157] Meanwhile, the recall corresponding to the jth candidate discrete classification model can also be obtained as:

[0158]

[0159] In the embodiments of the present application, because the second classification accuracy corresponding to the jth candidate discrete classification model is the difference between the second predicted category and the second actual category, it can be expressed as the harmonic mean of the precision and the recall, and the maximum is 1 and the minimum is 0, that is:

[0160]

[0161] Step 405, the candidate discrete data classification model with the maximum second classification accuracy is determined as the preset discrete data classification model.

[0162] In the embodiments of the present application, the greater the corresponding second classification accuracy, the higher the accuracy of the candidate discrete data classification model, so the candidate discrete data classification model with the maximum second classification accuracy can be determined as the preset discrete data classification model.

[0163] Step 406, the structured data to be classified is obtained, wherein the structured data to be classified includes a plurality of fields, and each field includes a field name and a field value.

[0164] Step 407, the structured data to be classified is separated according to the data type of the field value corresponding to each field, so as to determine the discrete field data included in the structured data to be classified.

[0165] Step 408, the field name and the field value corresponding to each field are processed to determine the text type data corresponding to the structured data to be classified.

[0166] Step 409, the discrete type data classification model is used to classify the discrete type field data to determine the first classification result corresponding to the structured data to be classified.

[0167] Step 410, the text type data classification model is used to classify the text type data to determine the second classification result corresponding to the structured data to be classified.

[0168] Step 411, the first classification result and the second classification result are used to determine the target classification result corresponding to the structured data to be classified.

[0169] The specific implementation process and principle of steps 406-411 described above can refer to the detailed description of the above embodiments, which will not be repeated here.

[0170] The structured data classification method provided in the present application obtains a plurality of candidate discrete type data classification models, and respectively classifies each second structured data in the second test data set by using each candidate discrete type data classification model. Then, according to the classification accuracy of each candidate discrete type data classification model, the candidate discrete type data classification model with the highest classification accuracy is determined as the preset discrete type data classification model. After that, the preset discrete type data classification model determined can be used to classify the discrete type field data in the structured data to be classified to determine the first classification result corresponding to the structured data to be classified. In combination with the second classification result of the structured data to be classified by using the preset text type data classification model, the target classification result of the structured data to be classified is determined. Thus, by selecting the model with the highest classification accuracy in the current classification scenario from the plurality of candidate discrete type classification models as the preset discrete type data classification model, the accuracy of the structured data classification is further improved.

[0171] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0172] The structured data classification method according to the above embodiments, Figure 5 The structural block diagram of the structured data classification device provided by the embodiments of the present application is shown. For ease of illustration, only the part related to the embodiments of the present application is shown.

[0173] Referring to Figure 5 The device 50 comprises:

[0174] The first obtaining module 51 is configured to obtain structured data to be classified, wherein the structured data to be classified comprises a plurality of fields, and each field comprises a field name and a field value.

[0175] The first determining module 52 is configured to separate the structured data to be classified according to the data type of the field value corresponding to each field, to determine discrete field data included in the structured data to be classified.

[0176] The second determining module 53 is configured to perform sentence processing on the field name and the field value corresponding to each field, to determine text data corresponding to the structured data to be classified.

[0177] The third determining module 54 is configured to perform classification processing on the discrete field data by using a preset discrete data classification model, to determine a first classification result corresponding to the structured data to be classified.

[0178] The fourth determining module 55 is configured to perform classification processing on the text data by using a preset text data classification model, to determine a second classification result corresponding to the structured data to be classified.

[0179] The fifth determining module 56 is configured to determine a target classification result corresponding to the structured data to be classified according to the first classification result and the second classification result.

[0180] In actual use, the structured data classification apparatus provided by the embodiments of the present application can be configured in any terminal device to perform the above-mentioned structured data classification method.

[0181] The classification device of structured data provided in the application separates the structured data to be classified according to the data type of the field value corresponding to each field in the structured data to be classified, to determine the discrete field data included in the structured data to be classified, and performs sentence processing on the field name and field value corresponding to each field, to determine the text data corresponding to the structured data to be classified. Then, the discrete field data is classified by using a preset discrete data classification model, to determine the first classification result corresponding to the structured data to be classified. Then, the text data is classified by using a preset text data classification model, to determine the second classification result corresponding to the structured data to be classified. Finally, the target classification result corresponding to the structured data to be classified is determined according to the first classification result and the second classification result. Thus, when the structured data is classified, the discrete data in the structured data is first separated, and the text data corresponding to the structured data is generated by sentence processing. Then, the discrete data classification model and the text data classification model are used to classify the discrete data and the text data, respectively, and the final classification result of the structured data is determined by combining the classification results of the two algorithms. Thus, the influence of mixed data types on data classification is reduced, and the accuracy of structured data classification is improved.

[0182] In a possible implementation manner of the embodiment of the application, the first classification result includes a first category and a first confidence degree corresponding to the first category, and the second classification result includes a second category and a second confidence degree corresponding to the second category. Correspondingly, the fifth determination module 56 includes:

[0183] A first determination unit, configured to determine the first category as the target classification result when the first confidence degree is greater than or equal to the second confidence degree.

[0184] A second determination unit, configured to determine the second category as the target classification result when the first confidence degree is less than the second confidence degree.

[0185] Further, in another possible implementation manner of the embodiment of the application, the fields include at least one discrete field and at least one text field, and the discrete data includes at least one discrete field. Correspondingly, the second determination module 53 includes:

[0186] A third determination unit, configured to determine a discrete field with a numerical value type value as a numerical value type field according to the field value corresponding to each discrete field.

[0187] A fourth determination unit, configured to determine a text field value corresponding to each numerical value type field according to a preset corresponding relationship between the text field value and the value range and the value range in which the field value corresponding to each numerical value type field is located.

[0188] The fifth determining unit is configured to determine the text type field value corresponding to each numerical type field as the field value corresponding to each numerical type field respectively, so as to text the field value corresponding to each numerical type field;

[0189] The first generating unit is configured to perform sentence processing on the field name and field value corresponding to each text type field, the field name and field value corresponding to each discrete type field, and the field name and text type field value corresponding to each numerical type field, so as to generate the text type data corresponding to the structured data to be classified.

[0190] Further, in another possible implementation of the embodiment of the application, the discrete type field data includes at least one discrete type field; and correspondingly, the apparatus 50 further includes:

[0191] The first judging module is configured to judge whether the discrete type field data includes the preset redundant field according to the preset redundant field;

[0192] The first removing module is configured to remove the preset redundant field from the discrete type data when the discrete type field data includes the preset redundant field.

[0193] Further, in another possible implementation of the embodiment of the application, the apparatus 50 further includes:

[0194] The second acquiring module is configured to acquire a first test data set, wherein the first test data set includes a plurality of first structured data and a first true class corresponding to each first structured data, and the second structured data includes N discrete type fields to be screened, wherein N is a positive integer;

[0195] The second removing module is configured to remove the i-th discrete type field to be screened included in each first structured data, so as to generate an i-th reference test data set, wherein i is a positive integer greater than or equal to 1 and less than or equal to N;

[0196] The sixth determining module is configured to perform classification processing on the plurality of discrete type fields to be screened corresponding to each first structured data in the i-th reference test data set by using the preset discrete data classification model, so as to determine an i-th first predicted class corresponding to each first structured data;

[0197] The seventh determining module is configured to determine a first classification accuracy of the preset discrete data classification model using the i-th reference training data set for classification according to the difference between the i-th first predicted class corresponding to each first structured data and the corresponding first true class;

[0198] The first selecting module is configured to select the preset redundant field from the N discrete type fields to be screened according to the N first classification accuracies.

[0199] Further, in a possible implementation of the embodiment of the present application, the second determining module 53 comprises:

[0200] The first removing unit is configured to remove the preset redundant field from the field to generate a filtered field when the preset redundant field is contained in the discrete field data.

[0201] The sixth determining unit is configured to perform sentence grouping processing on the field name and the field value corresponding to each filtered field to determine the text type data corresponding to the structured data to be classified.

[0202] Further, in a possible implementation of the embodiment of the present application, the device 50 further comprises:

[0203] The third obtaining module is configured to obtain a second test data set, wherein the second test data set comprises a plurality of second structured data and a second true class corresponding to each second structured data, and the second structured data comprises a plurality of discretized fields.

[0204] The fourth obtaining module is configured to obtain M candidate discrete data classification models, wherein M is a positive integer greater than 1.

[0205] The eighth determining module is configured to perform classification processing on the plurality of discretized fields included in each second structured data by using the jth candidate discrete data classification model to determine a jth second predicted class corresponding to each second structured data, wherein j is a positive integer greater than or equal to 1 and less than or equal to M.

[0206] The ninth determining module is configured to determine a second classification accuracy corresponding to the jth candidate discrete data classification model according to the jth second predicted class corresponding to each second structured data and the second true class corresponding thereto.

[0207] The tenth determining module is configured to determine the candidate discrete data classification model with the maximum second classification accuracy as the preset discrete data classification model.

[0208] Further, in a possible implementation of the embodiment of the present application, the ninth determining module comprises:

[0209] The seventh determining unit is configured to determine the precision and the recall corresponding to the jth candidate discrete classification model according to the jth second predicted class corresponding to each second structured data and the second true class corresponding thereto.

[0210] The eighth determining unit is configured to determine the second classification accuracy corresponding to the jth candidate discrete classification model according to the precision and the recall corresponding to the jth candidate discrete classification model.

[0211] It should be noted that the information interaction, execution process and the like between the above apparatuses / units are based on the same concept as the method embodiments of the present application, and the specific functions and the brought technical effects can be referred to the method embodiments part, which will not be repeated here.

[0212] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the above-described functions. The functional units and modules in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0213] In order to realize the above-mentioned embodiments, the present application further provides a terminal device.

[0214] Figure 6 The structural schematic diagram of the terminal device of one embodiment of the present application.

[0215] As shown in Figure 6 The terminal device 200 includes:

[0216] The memory 210 and the at least one processor 220, the bus 230 connecting different components including the memory 210 and the processor 220, the memory 210 storing a computer program, when the processor 220 executes the program, realizing the structured data classification method of the present application.

[0217] The bus 230 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to industry standard architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video electronics standards association (VESA) local bus and peripheral component interconnect (PCI) bus.

[0218] Terminal device 200 typically includes a variety of computer system readable media. These media can be any available media that is accessible by terminal device 200 and includes both volatile and non- volatile media, removable and non-removable media.

[0219] Memory 210 also can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 240 and / or cache memory 250. Terminal device 200 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 260 can be provided for reading from and writing to a non-removable, non-volatile magnetic media (e.g., a "hard drive"). Figure 4 Although not shown, a magnetic disk drive can also be utilized to read from and write to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive can be utilized to read from and / or write to a removable, non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, etc.). Figure 6 In such instances, each drive can be connected to the bus 230 by one or more data media interfaces. The memory 210 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the application.

[0220] Program / utility 280, having a set (at least one) of program modules 270, can be stored in, for example, memory 210 by way of example, such as an operating system, one or more application programs, other program modules, and program data, each or some combination thereof, can include implementation of a networking environment. Program modules 270 generally carry out the functions and / or methodologies of embodiments of the application as described herein.

[0221] Terminal device 200 can also be in communication with one or more external devices 290 such as a keyboard, a pointing device, a display 291, etc.; one or more devices that enable a user to interact with terminal device 200; and / or any devices (e.g., network card, modem, etc.) that enable terminal device 200 to communicate with one or more other computing devices. Such communication can be facilitated by an input / output (I / O) interface 292. Still yet, terminal device 200 can be in communication with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or the Internet) through a network adapter 293. As depicted, network adapter 293 communicates with the other components of terminal device 200 through bus 230. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with terminal device 200. For example, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc., can be used in conjunction with terminal device 200.

[0222] Processor 220 performs various function applications and data processing by running programs stored in memory 210.

[0223] It should be noted that the implementation process and technical principles of the terminal device of the embodiment are described above in the classification method of the structured data of the embodiment of the application, and will not be described here.

[0224] The embodiment of the application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps in each method embodiment.

[0225] The embodiment of the application provides a computer program product, when the computer program product is run on a terminal device, the terminal device is enabled to realize the steps in each method embodiment.

[0226] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods through a computer program to instruct relevant hardware to complete, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can at least include any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium can not be an electrical carrier signal and a telecommunication signal.

[0227] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0228] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0229] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / terminal equipment and method can be implemented in other ways. For example, the above-described apparatus / terminal equipment embodiments are merely schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0230] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0231] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of classifying structured data, characterized by, The method comprises the following steps: acquiring structured data to be classified, wherein the structured data to be classified comprises a plurality of fields, each of the fields comprises a field name and a field value; separating the structured data to be classified according to the data type of the field value corresponding to each of the fields, to determine discrete field data included in the structured data to be classified; performing sentence processing on the field name and the field value corresponding to each of the fields, to determine text data corresponding to the structured data to be classified, wherein the sentence processing on the field name and the field value corresponding to each of the fields refers to combining the field name and the field value of each of the fields in the structured data to be classified into a sentence, to generate the text data corresponding to the structured data to be classified, which is used to describe all information contained in the structured data to be classified; performing classification processing on the discrete field data by using a preset discrete data classification model, to determine a first classification result corresponding to the structured data to be classified; performing classification processing on the text data by using a preset text data classification model, to determine a second classification result corresponding to the structured data to be classified; determining a target classification result corresponding to the structured data to be classified according to the first classification result and the second classification result.

2. The method of claim 1, wherein, The first classification result comprises a first category and a first confidence degree corresponding to the first category, the second classification result comprises a second category and a second confidence degree corresponding to the second category, and the determination of the target classification result corresponding to the structured data to be classified according to the first classification result and the second classification result comprises: when the first confidence degree is greater than or equal to the second confidence degree, determining the first category as the target classification result; and when the first confidence degree is less than the second confidence degree, determining the second category as the target classification result.

3. The method according to claim 1, wherein, The fields comprise at least one discrete field and at least one text field, the discrete data comprises the at least one discrete field, and the sentence processing on the field name and the field value corresponding to each of the fields, to determine the text data corresponding to the structured data to be classified, comprises: determining the discrete field with a numerical value type field value as a numerical value type field according to the field value corresponding to each of the discrete fields; determining a text field value corresponding to each of the numerical value type fields according to a preset corresponding relationship between a text field value and a value range, and a value range in which the field value corresponding to each of the numerical value type fields is located; determining the text field value corresponding to each of the numerical value type fields as the field value corresponding to each of the numerical value type fields, respectively, to texturize the field value corresponding to each of the numerical value type fields. The field name and field value corresponding to each of the text type field, the field name and field value corresponding to each of the discrete type field, and the field name and text field value corresponding to each of the numerical type field are processed to generate the text type data corresponding to the structured data to be classified.

4. The method of claim 1, wherein, The discrete type field data includes at least one discrete type field, and before the classification processing of the discrete type field data by using the preset discrete type data classification model is performed to determine the first category corresponding to the structured data to be classified, the method further includes: According to the preset redundant field, it is judged whether the preset redundant field is included in the discrete type field data; When the preset redundant field is included in the discrete type field data, the preset redundant field is removed from the discrete type data.

5. The method according to claim 4, wherein, Before the step of judging whether the preset redundant field is included in the discrete type field data according to the preset redundant field, the method further includes: Obtaining a first test data set, wherein the first test data set includes a plurality of first structured data and a first true category corresponding to each of the first structured data, and each of the first structured data includes N discrete type fields to be screened, wherein N is a positive integer; Removing the i-th discrete type field to be screened included in each of the first structured data to generate an i-th reference test data set, wherein i is a positive integer greater than or equal to 1 and less than or equal to N; Using the preset discrete type data classification model, the plurality of discrete type fields to be screened corresponding to each of the first structured data in the i-th reference test data set are classified to determine an i-th first predicted category corresponding to each of the first structured data; According to the difference between the i-th first predicted category corresponding to each of the first structured data and the first true category corresponding to each of the first structured data, a first classification accuracy of the preset discrete data classification model using the i-th reference training data set for classification is determined; According to N first classification accuracies, the preset redundant field is selected from N discrete type fields to be screened.

6. The method of claim 4, wherein, The method further includes: When the preset redundant field is included in the discrete type field data, the preset redundant field is removed from the field to generate a screened field; The field name and field value corresponding to each of the screened field are processed to determine the text type data corresponding to the structured data to be classified.

7. The method of any one of claims 1-6, wherein, The method further includes: Obtaining a second test data set, wherein the second test data set includes a plurality of second structured data and a second true category corresponding to each of the second structured data, and the second structured data includes a plurality of discrete type fields; Obtaining M candidate discrete type data classification models, wherein M is a positive integer greater than 1; Classify each of the second structured data by using the jth candidate discrete data classification model, to determine a jth second predicted category corresponding to each of the second structured data, wherein j is a positive integer greater than or equal to 1 and less than or equal to M; Determine a second classification accuracy corresponding to the jth candidate discrete data classification model according to the jth second predicted category corresponding to each of the second structured data and the second real category corresponding to each of the second structured data; Determine the candidate discrete data classification model with the maximum second classification accuracy as the preset discrete data classification model.

8. The method of claim 7, wherein, The method further includes: Determine a precision and a recall corresponding to the jth candidate discrete data classification model according to the jth second predicted category corresponding to each of the second structured data and the second real category corresponding to each of the second structured data; Determine the second classification accuracy corresponding to the jth candidate discrete data classification model according to the precision and the recall corresponding to the jth candidate discrete data classification model.

9. A classification apparatus for structured data, characterized by The method further includes: A first obtaining module, configured to obtain structured data to be classified, wherein the structured data to be classified includes a plurality of fields, and each of the fields includes a field name and a field value; A first determining module, configured to separate the structured data to be classified according to a data type of the field value corresponding to each of the fields, to determine discrete field data included in the structured data to be classified; A second determining module, configured to group the field name and the field value corresponding to each of the fields, to determine text data corresponding to the structured data to be classified, wherein the grouping of the field name and the field value corresponding to each of the fields refers to combining the field name and the field value of each of the fields in the structured data to be classified into a sentence, to generate the text data corresponding to the structured data to be classified, and the text data corresponding to the structured data to be classified is used to describe all information contained in the structured data to be classified; A third determining module, configured to classify the discrete field data by using a preset discrete data classification model, to determine a first classification result corresponding to the structured data to be classified; A fourth determining module, configured to classify the text data by using a preset text data classification model, to determine a second classification result corresponding to the structured data to be classified; A fifth determining module, configured to determine a target classification result corresponding to the structured data to be classified according to the first classification result and the second classification result.

10. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 8.

Images