Data classification model training method, data processing method and storage medium

Abstract

The present application discloses a data classification model training method, a data processing method, an apparatus, an electronic device and a storage medium, wherein the data classification model training method comprises: inputting a first feature tensor into a data classification model and outputting a second feature tensor; each row of first vectors in the first feature tensor corresponds to a first attribute name representing a data object; each row of elements in the second feature tensor corresponds to the distance between any two rows of first vectors; based on the output second feature tensor, clustering each first vector to obtain at least one cluster; labeling the first attribute names corresponding to a set number of first vectors under each cluster to obtain a first sample set; determining a loss value based on the first sample set, and updating the weight parameters of the data classification model according to the determined loss value until all determined first sample sets meet the set end training conditions.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data classification model training method, a data processing method, a device, an electronic device and a storage medium. Background Art

[0002] The large scale and loose organizational structure of Internet content pose challenges to users in obtaining information and knowledge. Knowledge Graph, with its powerful semantic processing capabilities, has laid the foundation for intelligent information applications in the Internet era. Knowledge Graph aims to describe various entities and relationships that exist in the real world. Common description forms include: "Entity 1-Relationship-Entity 2" or "Entity-Attribute Name-Attribute Value".

[0003] In the related art, when determining a standardized attribute name from multiple attribute names, all candidate attribute names are recalled through literal matching, and then the standard attribute names are manually screened. Since attribute names that literally match may correspond to different attributes, attribute names that literally do not match may also correspond to the same attributes, that is to say, the correlation of the candidate attribute names recalled for manual screening is not high, resulting in low efficiency and poor effect in screening standardized attribute names. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a data classification model training method, a data processing method, an apparatus, an electronic device and a storage medium to at least solve the problem of low efficiency and poor effect in attribute name screening in related technologies.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a data classification model training method, the method comprising:

[0007] The first feature tensor is input into the data classification model, and the second feature tensor is output; each row of the first vector in the first feature tensor corresponds to a first attribute name representing the data object; each row of the elements in the second feature tensor corresponds to the distance between any two rows of the first vectors;

[0008] Based on the output second feature tensor, clustering each first vector to obtain at least one cluster;

[0009] Labeling the first attribute names corresponding to a set number of first vectors under each cluster to obtain a first sample set;

[0010] A loss value is determined based on the first sample set, and a weight parameter of the data classification model is updated according to the determined loss value until all the determined first sample sets meet the set training end condition.

[0011] Among them, in the above scheme, the first attribute names corresponding to the set number of first vectors under each cluster are labeled to obtain the first sample set, including:

[0012] Combining first attribute names corresponding to any two first vectors of a set number of first vectors under each cluster to obtain at least one first data group;

[0013] Labeling each first data group in the at least one first data group according to whether two first attribute names included in the first data group correspond to the same type of attribute, to obtain a labeling result corresponding to each first data group;

[0014] The first sample set is determined based on the labeling result.

[0015] In the above solution, determining the first sample set based on the labeling result includes:

[0016] determining at least one second data group based on the annotation result;

[0017] The first sample set is obtained according to all determined second data groups; wherein the second data group is composed of two first data groups that satisfy set composition conditions; the set composition conditions indicate that the two first data groups correspond to different labeling results, and the two first data groups have one and only one identical first attribute name.

[0018] In the above solution, the step of inputting the first feature tensor into the data classification model includes:

[0019] Based on at least one type of first information, determining a corresponding first feature tensor;

[0020] The determined first feature tensor is input into the data classification model.

[0021] In the above solution, determining the corresponding first feature tensor based on at least one type of first information includes:

[0022] Construct a bipartite graph based on the at least one type of first information, and determine a corresponding first feature tensor;

[0023] and / or,

[0024] The at least one type of first information is segmented, and a corresponding first feature tensor is determined based on a word vector corresponding to the segmentation result.

[0025] In the above solution, the corresponding first feature tensor is determined at least twice; and the step of inputting the determined first feature tensor into the data classification model includes:

[0026] Performing a nonlinear transformation on the determined at least two first eigentensors to obtain a third eigentensor;

[0027] The obtained third feature tensor is input into the data classification model.

[0028] In the above scheme, the set training end condition includes:

[0029] The labeling result indicates that the first attribute names corresponding to a set number of first vectors under the same cluster correspond to the same type of attributes; the labeling result indicates the result obtained by labeling the first attribute names corresponding to the set number of first vectors under each cluster.

[0030] The present application also provides a data processing method, which includes:

[0031] The fourth feature tensor is input into the data classification model, and a fifth feature tensor is output; each row of the second vector in the fourth feature tensor corresponds to a second attribute name representing the data object; each row of elements in the fifth feature tensor corresponds to the distance between any two rows of the second vector;

[0032] Based on the output fifth feature tensor, clustering each second vector to obtain at least one cluster;

[0033] Based on the cluster center corresponding to each cluster obtained by clustering, determine the third attribute name corresponding to each cluster; the third attribute name represents the attribute name of the corresponding cluster; wherein,

[0034] The data classification model is trained using any of the data classification model training methods described above.

[0035] Wherein, in the above solution, inputting the fourth feature tensor into the data classification model includes:

[0036] Based on at least one type of second information, determining a corresponding fourth feature tensor;

[0037] The determined fourth feature tensor is input into the data classification model.

[0038] In the above solution, determining the corresponding fourth feature tensor based on at least one type of second information includes:

[0039] Construct a bipartite graph based on the at least one type of second information, and determine a corresponding fourth eigentensor; and / or,

[0040] The at least one type of second information is segmented, and a corresponding fourth feature tensor is determined based on a word vector corresponding to the segmentation result.

[0041] In the above solution, the cluster center corresponding to each cluster obtained by clustering is used to determine the third attribute name corresponding to each cluster, including:

[0042] Based on the cluster center corresponding to each cluster obtained by clustering, the second attribute name corresponding to the second vector with the shortest distance under the corresponding cluster is determined as the third attribute name of the corresponding cluster.

[0043] The present application also provides a data classification model training device, including:

[0044] A first classification unit is used to input a first feature tensor into a data classification model and output a second feature tensor; each row of first vectors in the first feature tensor corresponds to a first attribute name representing a data object; each row of elements in the second feature tensor corresponds to a distance between any two rows of first vectors;

[0045] A first clustering unit, used for clustering each first vector based on the output second feature tensor to obtain at least one cluster;

[0046] A labeling unit, used for labeling first attribute names corresponding to a set number of first vectors under each cluster to obtain a first sample set;

[0047] A training unit is used to determine a loss value based on the first sample set, and update a weight parameter of the data classification model according to the determined loss value until all the determined first sample sets meet a set end training condition.

[0048] The present application also provides a data processing device, including:

[0049] A second classification unit is used to input the fourth feature tensor into a data classification model and output a fifth feature tensor; each row of second vectors in the fourth feature tensor corresponds to a second attribute name representing a data object; each row of elements in the fifth feature tensor corresponds to a distance between any two rows of second vectors;

[0050] A second clustering unit, used for clustering each second vector based on the output fifth feature tensor to obtain at least one cluster;

[0051] A processing unit is used to determine a third attribute name corresponding to each cluster based on the cluster center corresponding to each cluster obtained by clustering; the third attribute name represents the attribute name of the corresponding cluster; wherein,

[0052] The data classification model is trained using any of the data classification model training methods described above.

[0053] The embodiment of the present application further provides a first electronic device, comprising: a first processor and a first memory for storing a computer program that can be run on the processor,

[0054] Wherein, when the first processor is used to run the computer program, it executes the steps of any of the above-mentioned data classification model training methods.

[0055] The embodiment of the present application further provides a second electronic device, comprising: a second processor and a second memory for storing a computer program that can be run on the processor,

[0056] Wherein, the second processor is used to execute the steps of any one of the data processing methods when running the computer program.

[0057] An embodiment of the present application also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned data classification model training methods or the steps of any of the above-mentioned data processing methods are implemented.

[0058] The embodiment of the present application provides a data classification model training method, a data processing method, a device, an electronic device and a storage medium. In the data classification model training method, each row of the first vector in the first feature tensor corresponds to a first attribute name representing a data object, the first feature tensor is input into the data classification model, and the second feature tensor is output. Based on the distance between any two rows of first vectors corresponding to each row of elements in the output second feature tensor, each first vector is clustered to obtain at least one cluster, and the first attribute name corresponding to the first vector under each cluster is labeled to obtain a first sample set. The weight parameters of the data classification model are updated based on the first sample set until all the determined first sample sets meet the set end training conditions. In this way, each first vector is clustered based on the distance between the first vectors, the corresponding first attribute name is labeled based on the clustering result, and the labeled data classification model is used as a sample set to train the data classification model, so that the trained data classification model can accurately judge the correlation between the first attribute names. On this basis, the trained data classification model is used for data processing, which can efficiently filter attribute names from a large amount of data and improve the data processing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of a data classification model training method implementation flow provided in an embodiment of the present application;

[0060] Figure 2 A schematic diagram of a key-value pair information extraction scenario provided in an embodiment of the present application;

[0061] Figure 3 A schematic diagram of a text information extraction scenario provided in an embodiment of the present application;

[0062] Figure 4 A schematic diagram of graph relationship learning of a bipartite graph provided in an embodiment of the present application;

[0063] Figure 5 A schematic diagram of word vector learning provided in an embodiment of the present application;

[0064] Figure 6 A schematic diagram of a data processing method implementation flow provided in an embodiment of the present application;

[0065] Figure 7 A schematic diagram of a data processing framework provided for an application embodiment of the present application;

[0066] Figure 8 A schematic diagram of a data processing flow provided for an application embodiment of the present application;

[0067] Fig. 9 A schematic diagram of the structure of a data classification model training device provided in an embodiment of the present application;

[0068] Fig.10 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0069] Fig.11 A schematic diagram of the structure of a first electronic device provided in an embodiment of the present application;

[0070] Fig.12 A schematic diagram of the structure of a second electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The large scale and loose organizational structure of Internet content pose challenges to users in obtaining information and knowledge. Knowledge Graph, with its powerful semantic processing capabilities, has laid the foundation for intelligent information applications in the Internet era. Knowledge Graph aims to describe various entities and relationships that exist in the real world. Common description forms include: "Entity 1-Relationship-Entity 2" or "Entity-Attribute Name-Attribute Value".

[0072] Attribute name standardization is an important step in structuring knowledge graph information, which includes obtaining the attribute names of products, determining the standard attribute names corresponding to these attribute names, and using the determined standard attribute names to generate a knowledge graph, thereby enabling information comparison between different products.

[0073] In the related art, when determining a standardized attribute name from multiple attribute names, all candidate attribute names are recalled through literal matching, and then the standard attribute names are manually screened. Since the attribute names that literally match may correspond to different attributes, the attribute names that literally do not match may also correspond to the same attributes. For example, in electrical products, the attribute names "frame grade" and "optional shell" have the same meaning, but the literal matching degree is low; for another example, the attribute names "optional shell" and "optional rated voltage" both contain "optional", but the corresponding attributes are completely different. In other words, the candidate attribute names recalled for manual screening have a low correlation, and it is easy to miss candidate attribute names with a high correlation. It is necessary to rely on manual screening of standard attribute names from the candidate attribute names, resulting in low efficiency and poor effect in screening attribute names.

[0074] Based on this, the embodiments of the present application provide a data classification model training method, a data processing method, a device, an electronic device and a storage medium. In the data classification model training method, each row of the first vector in the first feature tensor corresponds to a first attribute name representing a data object, the first feature tensor is input into the data classification model, and the second feature tensor is output. Based on the distance between any two rows of first vectors corresponding to each row of elements in the output second feature tensor, each first vector is clustered to obtain at least one cluster, and the first attribute name corresponding to the first vector under each cluster is labeled to obtain a first sample set. The weight parameters of the data classification model are updated based on the first sample set until all the determined first sample sets meet the set end training conditions. In this way, each first vector is clustered based on the distance between the first vectors, the corresponding first attribute name is labeled based on the clustering result, and the labeled data classification model is used as a sample set to train the data classification model, so that the trained data classification model can accurately judge the correlation between the first attribute names. On this basis, the trained data classification model is used for data processing, which can efficiently filter attribute names from a large amount of data and improve the data processing effect. At the same time, compared with the solution in the related art that recalls candidate attribute names for manual screening based on literal matching, the embodiment of the present application can exclude attribute names with low correlation during labeling, thereby reducing the amount of data to be processed and improving data processing efficiency.

[0075] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0076] The present application embodiment provides a data classification model training method, such as Figure 1 As shown, the method includes:

[0077] Step 101: Input the first feature tensor into the data classification model and output the second feature tensor.

[0078] Among them, each row of the first vector in the first feature tensor corresponds to a first attribute name representing the data object; each row of elements in the second feature tensor corresponds to the distance between any two rows of the first vectors.

[0079] The first feature tensor is input into the data classification model, and the data classification model outputs the second feature tensor, and each row element in the second feature tensor corresponds to the distance between the two rows of first vectors in the first feature tensor. Here, the data object can be defined by a set of attributes, including but not limited to external entities, things, incidents or events, roles, organizational units, places or structures, and an attribute name of the data object represents the name of an attribute in a set of attributes of the data object. In practical applications, the data object can be various products, such as bookcases, wardrobes, etc. The attribute names of wardrobes include size, volume, etc. The distance can be the Euclidean distance between the two rows of first vectors, or it can be obtained by calculating the distance through the metric matrix M.

[0080] For example, the first feature tensor of the input data classification model is Including three first vectors, first vector A (0 1 0 1), first vector B (1 0 0 0), first vector C (0 1 0 0), the output second feature tensor is Each row represents the Euclidean distance between the first vector A and the first vector B, the first vector A and the first vector C, and the first vector B and the first vector C, respectively.

[0081] In one embodiment, inputting the first feature tensor into the data classification model includes:

[0082] Based on at least one type of first information, determining a corresponding first feature tensor;

[0083] The determined first feature tensor is input into the data classification model.

[0084] Here, corresponding types of data can be obtained based on each type of information. For feature extraction of information, at least one type of first information can be input into a feature extraction layer, and the feature extraction layer performs feature extraction on each type of first information in the at least one type of first information input to obtain a corresponding first feature tensor. The types of first information include but are not limited to key-value pair information and / or text information. The form of the first information includes structured data and unstructured data, and the source is but not limited to that obtained through crawling technology and stored in a set database.

[0085] In practical applications, such as Figure 2The schematic diagram of the product specification manual image shown is extracted by optical character recognition (OCR) to obtain attribute-related key-value pair information, which includes attribute name and optional attribute value. For example, the attribute name "rated current" and the corresponding optional attribute value are extracted, including "2.5A", "6.3A", "12.5A", "16A", etc.

[0086] like Figure 3 The specification table schematic diagram shown in the figure is used to extract the data in the schematic diagram to obtain text information related to the attribute. For example, text information such as "leakage protection module", "×× relay", "characteristics", and "isolating switch" is extracted.

[0087] In one embodiment, determining the corresponding first feature tensor based on at least one type of first information includes:

[0088] Construct a bipartite graph based on the at least one type of first information, and determine a corresponding first feature tensor; and / or,

[0089] The at least one type of first information is segmented, and a corresponding first feature tensor is determined based on a word vector corresponding to the segmentation result.

[0090] Here, the types of first information include but are not limited to key-value pair information and / or text information. The method of obtaining the first feature tensor by performing feature extraction based on at least one type of first information can be a feature representation based on a graph relationship learning model such as a LIINE model. A bipartite graph is constructed based on the attribute name and optional attribute value of the key-value pair information, and the optional attribute value represents the value range of the attribute value. The attribute name and attribute value are considered to be two different entities, and the value relationship can be regarded as an edge between the two entities. The feature representation of the attribute name entity reflects a physical property of the attribute name. For example, the values ​​of "frame", "rated current", and "frame current code" are all current values, so these are attribute names related to the attribute of current, and the correlation between attribute names is higher. For another example, "clothing color" and "shoe color" are both attribute names related to color, and the correlation between attribute names is also higher. Therefore, this value relationship can be represented by constructing a bipartite graph about the attribute, such as Figure 4 As shown, a graph mining or graph learning algorithm is used based on the bipartite graph to extract the feature representation of the node.

[0091] In practical applications, there is a correlation between the literal meaning of the attribute name and the attribute represented by the attribute name. For example, "frame" and "frame current code" are both attribute names describing attributes related to the frame, so the correlation between them is higher than the correlation between "frame" and "rated current". Therefore, the feature extraction layer can be further trained in combination with literal matching, so that the obtained first feature tensor can more accurately represent the first information.

[0092] The method of obtaining the first feature tensor based on feature extraction of at least one type of first information may be a feature representation based on word embedding. Semantic analysis is performed by extracting word vectors from text data related to the attribute name. Specifically, extracting word vectors based on text data of attribute names includes: segmenting the attribute name to obtain at least one word segmentation corresponding to the attribute name, and processing the semantic vector of at least one word segmentation to obtain the first vector corresponding to the attribute name. Here, the processing method includes but is not limited to: concatenating the semantic vector of at least one word segmentation and weighted addition of the semantic vector of at least one word segmentation. The first vector of the attribute name can be used to calculate the contextual semantic similarity of the attribute name. Here, the word vector model may be a Skip-gram model. Such as Figure 5 As shown in Figure 1, the word vectors output by the model reflect the semantic similarity of the context between words. In this way, the attribute names can be represented by the word vectors output by the model, thereby judging the correlation between different attribute names.

[0093] In practical applications, the semantic similarity between the contexts of "frame" and "shell" is higher, the semantic similarity between the contexts of "frame" and "current" is lower, and the semantic similarity between the contexts of "frame" and "rated" is even lower.

[0094] The first feature tensor obtained by performing feature extraction based on at least one type of first information may be a combination of feature representation based on graph relationship learning and feature representation based on word embedding.

[0095] In this way, the first information is processed by at least one feature representation method, and the universality of the data classification model is improved by inputting the feature tensors corresponding to data from more sources and training the data classification model. At the same time, based on the feature representation of the value relationship corresponding to the attribute name and the semantic combination, the characteristics of the corresponding attribute name can be more accurately characterized, so that the data classification effect of the model is better. In addition, by decoupling the processes of the two models of representation learning and metric learning, the feature extraction process of representation learning and the metric learning process are made independent, and the trained data classification model can be applied to more application scenarios and applied to more electronic devices.

[0096] In one embodiment, the corresponding first feature tensor is determined at least twice; and the step of inputting the determined first feature tensor into the data classification model comprises:

[0097] Performing a nonlinear transformation on the determined at least two first eigentensors to obtain a third eigentensor;

[0098] The obtained third feature tensor is input into the data classification model.

[0099] Here, the corresponding first feature tensor is determined at least twice based on at least one type of first information to obtain at least two first feature tensors, and a nonlinear transformation is performed on the determined at least two first feature tensors, and the at least two first feature tensors are mapped to a new feature space to obtain a third feature tensor. The third feature tensor can be obtained by mapping at least two first feature tensors to a new feature space through formula (1):

[0100]

[0101] in,

[0102] f i,j Represents the element in the i-th row and j-th column of the third feature tensor;

[0103] Represents the element in the i-th row and j-th column of the first feature tensor;

[0104] Represents the element in the i-th row and j-th column of another first feature tensor;

[0105] W g Characterization The transposed matrix of

[0106] W s Characterization The transposed matrix of .

[0107] Here, the method of mapping at least two first feature tensors to the new feature space can be obtained by transforming the hidden layer of the deep neural network. g and W s Optimization can be performed based on the metric learning results, so that the distance between attribute names corresponding to the same attribute is as small as possible, and the distance between attribute names corresponding to different attributes is as large as possible.

[0108] Step 102: Clustering each first vector based on the output second feature tensor to obtain at least one cluster.

[0109] Based on the distance between two rows of first vectors represented by each row element in the second feature tensor, each first vector is clustered to obtain at least one cluster. Here, a clustering algorithm such as k-means can be used to cluster based on the similarity of each first vector. In the clustering result, the similarity between the first vectors in the same cluster can be higher than the similarity threshold.

[0110] Step 103: labeling the first attribute names corresponding to a set number of first vectors in each cluster to obtain a first sample set.

[0111] A set number of first vectors are randomly selected under each cluster, and the corresponding first attribute names are determined so that the ratio of the set number to the number of attribute names under each cluster satisfies the set ratio. The selected set number of first attribute names are annotated based on the set rules, and the first sample set is determined based on the annotation results. Here, when annotating the selected set number of first attribute names, grouping may be performed or not, and the grouping method used includes but is not limited to: classifying the set number of first attribute names as a group, and evenly distributing the first attribute names to each group with a set ratio of groups. The annotation of the first attribute names may be performed by manual labeling.

[0112] For example, if the ratio is set to 5%, and there are 200 first attribute names under Cluster A, then the number is set to 10, and 10 first attribute names under Cluster A are randomly selected for annotation.

[0113] In one embodiment, labeling a set number of first attribute names under each cluster to obtain a first sample set includes:

[0114] Combining any two first attribute names from a set number of first attribute names under each cluster to obtain at least one first data group;

[0115] Labeling each first data group in the at least one first data group according to whether two first attribute names included in the first data group correspond to the same type of attribute, to obtain a labeling result corresponding to each first data group;

[0116] The first sample set is determined based on the labeling result.

[0117] Here, any two first attribute names among the set number of first attribute names under each cluster are combined in pairs to obtain at least one first data group (x, y) including the two first attribute names. The way of labeling the set number of first attribute names under each cluster can be to manually judge and label whether the two first attribute names in the first data group correspond to the same type of attribute, obtain the labeling result corresponding to each first data group, and determine the first sample set from the first attribute names based on the labeling result of whether the two first attribute names in the characterized first data group belong to the same type of attribute.

[0118] In practical applications, manual labeling is performed for binary classification tasks. If the two first attribute names correspond to the same type of attributes, they are labeled as 0, otherwise they are labeled as 1.

[0119] In this way, the manual screening task of a large number of attribute names is transformed into a manually labeled binary classification task, which simplifies the difficulty of user processing attribute names, reduces the error rate of screening attribute names and improves efficiency. At the same time, since the distance of the first attribute name under each cluster is close but does not correspond to the same type of attribute, training the model based on the first attribute name under each cluster can accelerate the model learning process and improve the training efficiency of the model.

[0120] In one embodiment, determining the first sample set based on the labeling result includes:

[0121] determining at least one second data group based on the annotation result;

[0122] The first sample set is obtained according to all determined second data groups; wherein the second data group is composed of two first data groups that satisfy set composition conditions; the set composition conditions indicate that the two first data groups correspond to different labeling results, and the two first data groups have one and only one identical first attribute name.

[0123] Based on the labeling result of the first data group, two first data groups that meet the set composition condition are used as data constituting the second data group. Here, the set composition condition indicates that the two first data groups correspond to different labeling results, and the two first data groups have and only have one same first attribute name.

[0124] In practical applications, the three first attribute names For example, based on the annotation results, any of the following two second data groups can be generated: triple data group Or binary data set positive and negative sample pairs and A corresponding first sample set is obtained using any one of the two second data sets.

[0125] Step 104: Determine a loss value based on the first sample set, and update the weight parameters of the data classification model according to the determined loss value until all the determined first sample sets meet the set end training conditions.

[0126] Here, based on the type of the data group of the first sample set being a binary data group or a ternary data group, the loss function of the corresponding data classification model is set to be a Contrastive Loss binary group loss and a Triplet Loss triple group loss.

[0127] In one embodiment, the set training end condition includes:

[0128] The labeling result indicates that the first attribute names corresponding to a set number of first vectors under the same cluster correspond to the same type of attributes; the labeling result indicates the result obtained by labeling the first attribute names corresponding to the set number of first vectors under each cluster.

[0129] Here, a set number of first attribute names under each cluster are labeled. When the obtained labeling results indicate that the set number of first attribute names under each cluster correspond to the same type of attributes, it can be considered that the data classification model obtained based on training can accurately cluster the attribute names according to the corresponding attribute types, so that the attribute names in each cluster obtained by clustering correspond to the same type of attributes.

[0130] In an embodiment of the present application, each first vector is clustered based on the distance between the first vectors, the first attribute name corresponding to the clustering result is labeled, and the labeled data is used as a sample set to train the data classification model, so that the trained data classification model can accurately judge the correlation between the first attribute names, and on this basis, the trained data classification model is used for data processing, which can efficiently screen attribute names from a large amount of data and improve the data processing effect. At the same time, compared with the solution for manual screening based on literal matching to recall candidate attribute names in the related art, the embodiment of the present application can exclude attribute names with low correlation when labeling, thereby reducing the amount of data to be processed and improving data processing efficiency. In addition, by decoupling the processes of the two models of representation learning and metric learning, the feature extraction process and the metric learning process are made independent, and the trained data classification model can be applied to more application scenarios and more electronic devices.

[0131] Figure 6 A schematic diagram showing a flow chart of a data processing method provided in an embodiment of the present application is shown. Figure 6 , the data processing methods include:

[0132] Step 601: Input the fourth feature tensor into the data classification model and output the fifth feature tensor.

[0133] Among them, each row of the second vector in the fourth feature tensor corresponds to a second attribute name representing the data object; each row of elements in the fifth feature tensor corresponds to the distance between any two rows of second vectors; and the data classification model is trained using any of the data classification model training methods described above.

[0134] The fourth feature tensor is input into a data classification model trained by any of the data classification model training methods described above, and the data classification model outputs a fifth feature tensor, and each row of elements in the fifth feature tensor represents the distance between the corresponding two rows of second vectors in the fourth feature tensor. Here, the data object can be defined by a set of attributes, including but not limited to external entities, things, incidents or events, roles, organizational units, places or structures, and an attribute name of the data object represents the name of an attribute in a set of attributes of the data object. In practical applications, the data object can be various products, such as bookcases, wardrobes, etc., and the attribute names of wardrobes include size, volume, etc.

[0135] In one embodiment, inputting the fourth feature tensor into the data classification model includes:

[0136] Based on at least one type of second information, determining a corresponding fourth feature tensor;

[0137] The determined fourth feature tensor is input into the data classification model.

[0138] Here, at least one type of second information can be input into the feature extraction layer, and the feature extraction layer extracts features from each type of second information in the at least one type of second information input to obtain a corresponding fourth feature tensor. The types of second information include, but are not limited to, key-value pair information and / or text information, and the forms of second information include structured data and unstructured data, and the sources include, but are not limited to, those obtained through crawling technology and stored in a set database.

[0139] In one embodiment, determining the corresponding fourth feature tensor based on at least one type of second information includes:

[0140] Construct a bipartite graph based on the at least one type of second information, and determine a corresponding fourth eigentensor; and / or,

[0141] The at least one type of second information is segmented, and a corresponding fourth feature tensor is determined based on a word vector corresponding to the segmentation result.

[0142] Here, the types of the second information include but are not limited to key-value pair information and / or text information. The fourth feature tensor obtained by feature extraction based on at least one type of second information can be a feature representation based on a graph relationship learning model such as a LIINE model. A bipartite graph is constructed based on the attribute name and optional attribute value of the key-value pair information, and the optional attribute value represents the value range of the attribute value. The attribute name and attribute value are considered to be two different entities, and the value relationship can be regarded as an edge between the two entities. The feature representation of the attribute name entity reflects a physical property of the attribute name.

[0143] In practical applications, there is a correlation between the literal meaning of the attribute name and the attribute represented by the attribute name. For example, "frame" and "frame current code" are both attribute names describing attributes related to the frame, so the correlation between them is higher than the correlation between "frame" and "rated current". Therefore, the feature extraction layer can be further trained in combination with literal matching, so that the fourth feature tensor obtained can more accurately represent the second information.

[0144] The fourth feature tensor obtained by feature extraction based on at least one type of second information may be a feature representation based on word embedding. Semantic analysis is performed by extracting word vectors from text data related to the attribute name. Specifically, extracting word vectors based on text data of attribute names includes: segmenting the attribute name to obtain at least one word segmentation corresponding to the attribute name, and processing the semantic vector of at least one word segmentation to obtain a second vector corresponding to the attribute name. Here, the processing method includes but is not limited to: concatenating the semantic vector of at least one word segmentation and weighted addition of the semantic vector of at least one word segmentation. The second vector of the attribute name can be used to calculate the contextual semantic similarity of the attribute name. The word vector model may be a Skip-gram model. Such as Figure 5 As shown in Figure 1, the word vectors output by the model reflect the semantic similarity of the context between words. In this way, the correlation between attribute names can be judged by the word vectors output by the model.

[0145] The fourth feature tensor obtained by performing feature extraction based on at least one type of second information can be a combination of feature representation based on graph relationship learning and feature representation based on word embedding.

[0146] In this way, the second information is processed by at least one feature representation method, and the universality of the data classification model is improved by inputting the feature tensors corresponding to data from more sources and training the data classification model. At the same time, based on the feature representation of the value relationship corresponding to the attribute name and the semantic combination, the characteristics of the corresponding attribute name can be more accurately characterized, so that the data classification effect of the model is better. In addition, by decoupling the processes of the two models of representation learning and metric learning, the feature extraction process and the metric learning process are made independent, and the trained data classification model can be applied to more application scenarios and applied to more electronic devices.

[0147] Step 602: Based on the output fifth feature tensor, cluster each second vector to obtain at least one cluster.

[0148] Based on the distance between two rows of second vectors represented by each row element in the fifth feature tensor, each second vector is clustered to obtain at least one cluster. Here, a clustering algorithm such as k-means can be used to cluster based on the similarity of each second vector. In the clustering result, the similarity between the second vectors in the same cluster can be higher than the similarity threshold.

[0149] Step 603: Based on the cluster center corresponding to each cluster obtained by clustering, determine the third attribute name corresponding to each cluster.

[0150] The third attribute name represents the attribute name of the corresponding cluster.

[0151] For each cluster, according to the distance from the cluster center vector obtained by clustering, the third attribute name is determined from the second vector under the corresponding cluster, and the third attribute name is determined as the standard attribute name of the corresponding cluster.

[0152] In one embodiment, determining the third attribute name corresponding to each cluster based on the cluster center corresponding to each cluster obtained by clustering includes:

[0153] Based on the cluster center corresponding to each cluster obtained by clustering, the second attribute name corresponding to the second vector with the shortest distance under the corresponding cluster is determined as the third attribute name of the corresponding cluster.

[0154] The cluster center vector can characterize the common characteristics of the second attribute name corresponding to the second vector in this cluster. Here, the second attribute name closest to the cluster center vector is determined as the third attribute name as the standard attribute name of the corresponding cluster. The determined third attribute name can characterize the corresponding second attribute name.

[0155] In this way, through the fifth feature tensor representing the distance output by the data classification model, a large number of second attribute names can be classified, and the standard attribute name corresponding to each type of second attribute name can be determined. Compared with the related art that recalls all candidate attribute names through literal matching and then manually screens the standard attribute names, the embodiment of the present application overcomes the problems of low efficiency and poor effect of standardized attribute names, and can efficiently screen attribute names from a large amount of data to improve data processing effects. At the same time, compared with the solution for manual screening based on literal matching to recall candidate attribute names in the related art, the embodiment of the present application can exclude attribute names with low correlation when annotating, thereby reducing the amount of data to be processed and improving data processing efficiency. In addition, by decoupling the processes of the two models of representation learning and metric learning, the feature extraction process and the metric learning process are made independent, and the trained data classification model can be applied to more application scenarios and more electronic devices.

[0156] In the related art, the standardization of attribute names relies on manual judgment and screening, which requires manual judgment in a large amount of data, which is inefficient and has a high error rate. The following is an application example of the embodiment of the present application:

[0157] Combination Figure 7The schematic diagram of a data processing framework is shown. Based on representation learning, information is extracted for feature extraction, and attribute names are described from different perspectives. Based on metric learning, the extracted features are fused, and the distance between attribute names is calculated. Then, through active learning, representation learning and metric learning are decoupled. On the one hand, existing features in representation learning can be continuously optimized and new features can be determined. On the other hand, difficult samples can be screened, and supervised learning of the metric learning model can be performed in combination with manual annotation. In the whole process, representation learning, active learning, and metric learning are carried out simultaneously, which can accelerate the model learning process and improve the training efficiency of the model. In addition, the data processing framework based on active learning can reduce the labor cost of data processing tasks.

[0158] like Figure 8 A schematic diagram of a data processing flow chart shown includes:

[0159] Step 801: Data information preparation.

[0160] Captures information about the data associated with the attribute, including key-value pairs and text information.

[0161] Step 802: Perform representation learning based on data.

[0162] Features are extracted, and the distance metric model is continuously updated based on metric learning under the active learning framework to optimize the distance matrix of attribute names in high-dimensional space.

[0163] Here, feature representation of nodes can be extracted based on graph mining or graph learning algorithms based on graph relationship learning, or feature representation can be performed based on word vector models such as the Skip-gram model.

[0164] Clustering is performed based on the distance matrix of the attribute name. The distance matrix stores the distance calculation results between the vectors representing the attribute names. Clustering is performed based on the vectors representing the distance calculation results to obtain at least one attribute name cluster. Based on the clustering results, the set query strategy framework is adopted to realize the selection of difficult samples in the sample space. Here, the set query strategy framework can be a set strategy to select difficult samples according to the clustering results. Difficult samples refer to samples that are very close but do not correspond to the same type of attributes. In this way, active learning selects difficult samples based on the set rules, making model learning efficient.

[0165] Randomly sample a certain proportion of attribute names from each cluster, and combine the attribute names of the same cluster in pairs. Manually label the data group of difficult samples for the binary classification task, and label them by the identification bit. 0 means that the data group belongs to the same attribute, and 1 means that the data group belongs to different attributes. Generate a difficult sample training set based on the labeled data group, and you can generate any of the following two second data groups: triple data group Or binary data set positive and negative sample pairs and The data classification model is trained using either of the two second data groups as training data, and based on the data group type of the first sample set being a binary data group or a ternary data group, the loss function of the data classification model is set to be Contrastive Loss binary loss and Triplet Loss triplet loss. Through training, the distance results output by the data classification model are made as small as possible for the attribute names corresponding to the same attribute, and as large as possible for the attribute names corresponding to different attributes. Based on the training of the data classification model, the distance matrix corresponding to the attribute name is updated, and the above steps are repeated.

[0166] Step 803: perform distance clustering based on attribute names, and select standard attribute names of clusters based on clustering results.

[0167] Here, based on the distance matrix output by the optimized data classification model, a better clustering result is obtained, and the attribute name corresponding to the vector closest to the cluster center vector in each cluster is determined as the standard attribute name of the cluster. In this way, the standard attribute name corresponding to each cluster can be obtained, and the obtained standard attribute name is used as the attribute name dictionary. Based on the obtained attribute name dictionary, the attribute name can be normalized.

[0168] In the application embodiment of the present application, the process of attribute name standardization is modeled as sample distance optimization in a high-dimensional vector space. At the same time, active learning is used to decouple the two processes of representation learning and metric learning, so that the feature iteration of representation learning and the process of metric learning are independent. In addition, in active learning, attribute names are standardized, and the dependence on manual labor (man-made standard tasks, collected, sorted, compared and screened by manual labor) is transformed into a weak dependence on manual labor (a binary classification task with fixed standards). In this way, by performing simple and standardized labeling tasks on part of the sample set, the efficiency of the task is improved and the difficulty of the task is reduced.

[0169] To implement the method of the embodiment of the present application, the embodiment of the present application also provides a data classification model training device, which is arranged on a first electronic device, such as Fig. 9 As shown, the device comprises:

[0170] The first classification unit 901 is used to input the first feature tensor into the data classification model and output the second feature tensor; each row of the first vector in the first feature tensor corresponds to a first attribute name representing the data object; each row of elements in the second feature tensor corresponds to the distance between any two rows of the first vectors;

[0171] A first clustering unit 902 is used to cluster the first vectors based on the output second feature tensor to obtain at least one cluster;

[0172] A labeling unit 903 is used to label the first attribute names corresponding to a set number of first vectors under each cluster to obtain a first sample set;

[0173] The training unit 904 is used to determine a loss value based on the first sample set, and update the weight parameters of the data classification model according to the determined loss value until all the determined first sample sets meet the set end training conditions.

[0174] In one embodiment, the labeling unit 903 is used to:

[0175] Combining first attribute names corresponding to any two first vectors of a set number of first vectors under each cluster to obtain at least one first data group;

[0176] Labeling each first data group in the at least one first data group according to whether two first attribute names included in the first data group correspond to the same type of attribute, to obtain a labeling result corresponding to each first data group;

[0177] The first sample set is determined based on the labeling result.

[0178] In one embodiment, the labeling unit 903 is used to:

[0179] determining at least one second data group based on the annotation result;

[0180] The first sample set is obtained according to all determined second data groups; wherein the second data group is composed of two first data groups that satisfy set composition conditions; the set composition conditions indicate that the two first data groups correspond to different labeling results, and the two first data groups have one and only one identical first attribute name.

[0181] In one embodiment, the first classification unit 901 is used to:

[0182] Based on at least one type of first information, determining a corresponding first feature tensor;

[0183] The determined first feature tensor is input into the data classification model.

[0184] In one embodiment, the first classification unit 901 is further used for:

[0185] Construct a bipartite graph based on the at least one type of first information, and determine a corresponding first feature tensor; and / or,

[0186] The at least one type of first information is segmented, and a corresponding first feature tensor is determined based on a word vector corresponding to the segmentation result.

[0187] In one embodiment, the first classification unit 901 is further used for:

[0188] The step of inputting the determined first feature tensor into a data classification model comprises:

[0189] Performing a nonlinear transformation on the determined at least two first eigentensors to obtain a third eigentensor;

[0190] The obtained third feature tensor is input into the data classification model.

[0191] In one embodiment, the set training end condition includes:

[0192] The labeling result indicates that the first attribute names corresponding to a set number of first vectors under the same cluster correspond to the same type of attributes; the labeling result indicates the result obtained by labeling the first attribute names corresponding to the set number of first vectors under each cluster.

[0193] In actual application, the first classification unit 901, the first clustering unit 902, the labeling unit 903, and the training unit 904 can be implemented by a processor in a data classification model training device, such as a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU) or a programmable gate array (FPGA).

[0194] It should be noted that: when the data classification model training device provided in the above embodiment performs data classification model training, only the division of the above program modules is used as an example. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the data classification model training device provided in the above embodiment and the data classification model training method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0195] To implement the method of the embodiment of the present application, the embodiment of the present application further provides a data processing device, which is arranged on a second electronic device, such as Fig.10 As shown, the device comprises:

[0196] The second classification unit 1001 is used to input the fourth feature tensor into the data classification model and output a fifth feature tensor; each row of the second vector in the fourth feature tensor corresponds to a second attribute name representing the data object; each row of elements in the fifth feature tensor corresponds to the distance between any two rows of the second vector;

[0197] A second clustering unit 1002 is used to cluster the second vectors based on the output fifth feature tensor to obtain at least one cluster;

[0198] The processing unit 1003 is used to determine the third attribute name corresponding to each cluster based on the cluster center corresponding to each cluster obtained by clustering; the third attribute name represents the attribute name of the corresponding cluster; wherein,

[0199] The data classification model is trained using any of the data classification model training methods described above.

[0200] In one embodiment, the second classification unit 1001 is used to:

[0201] Based on at least one type of second information, determining a corresponding fourth feature tensor;

[0202] The determined fourth feature tensor is input into the data classification model.

[0203] In one embodiment, the second classification unit 1001 is further used for:

[0204] Construct a bipartite graph based on the at least one type of second information, and determine a corresponding fourth eigentensor; and / or,

[0205] The at least one type of second information is segmented, and a corresponding fourth feature tensor is determined based on a word vector corresponding to the segmentation result.

[0206] In one embodiment, the processing unit 1003 is used to:

[0207] Based on the cluster center corresponding to each cluster obtained by clustering, the second attribute name corresponding to the second vector with the shortest distance under the corresponding cluster is determined as the third attribute name of the corresponding cluster.

[0208] In actual application, the second classification unit 1001, the second clustering unit 1002, and the processing unit 1003 can be implemented by a processor in a data processing device, such as a CPU, a DSP, an MCU, or an FPGA.

[0209] It should be noted that: when the data processing device provided in the above embodiment performs data processing, only the division of the above program modules is used as an example. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the data processing device provided in the above embodiment and the data processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0210] Based on the hardware implementation of the above program modules, and in order to implement the data classification model training method of the embodiment of the present application, the embodiment of the present application also provides a first electronic device, such as Fig.11 As shown, the first electronic device 1100 includes:

[0211] The first communication interface 1101 can exchange information with other devices such as network devices;

[0212] The first processor 1102 is connected to the first communication interface 1101 to implement information exchange with other devices, and is used to execute the method provided by one or more technical solutions of the first electronic device side when running a computer program. The computer program is stored in the first memory 1103.

[0213] Of course, in actual application, the various components in the first electronic device 1100 are coupled together through the first bus system 1104. It can be understood that the first bus system 1104 is used to realize the connection and communication between these components. In addition to the first data bus, the first bus system 1104 also includes a first power bus, a first control bus and a first status signal bus. However, for the sake of clarity, Fig.11 Various buses are labeled as a first bus system 1104 .

[0214] The first memory 1103 in the embodiment of the present application is used to store various types of data to support the operation of the first electronic device 1100. Examples of such data include: any computer program used to operate on the first electronic device 1100.

[0215] It can be understood that the first memory 1103 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory, a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The first memory 1103 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memories.

[0216] The method disclosed in the above embodiment of the present application can be applied to the first processor 1102, or implemented by the first processor 1102. The first processor 1102 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the first processor 1102 or an instruction in software form. The above-mentioned first processor 1102 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The first processor 1102 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the first memory 1103. The first processor 1102 reads the program in the first memory 1103 and completes the steps of the above method in combination with its hardware.

[0217] Optionally, when the first processor 1102 executes the program, it implements the corresponding processes implemented by the first electronic device in each method of the embodiments of the present application, which will not be described again for the sake of brevity.

[0218] Based on the hardware implementation of the above program modules, and in order to implement the data processing method on the second electronic device side of the embodiment of the present application, the embodiment of the present application also provides a second electronic device, such as Fig.12 As shown, the second electronic device 1200 includes:

[0219] The second communication interface 1201 can exchange information with other devices such as network devices;

[0220] The second processor 1202 is connected to the second communication interface 1201 to implement information exchange with other devices and is used to execute the method provided by one or more technical solutions on the second electronic device side when running a computer program. The computer program is stored in the second memory 1203.

[0221] Of course, in actual application, the various components in the second electronic device 1200 are coupled together through the second bus system 1204. It can be understood that the second bus system 1204 is used to realize the connection and communication between these components. In addition to the second data bus, the second bus system 1204 also includes a second power bus, a second control bus, and a second status signal bus. However, for the sake of clarity, Fig.12 Various buses are labeled as a second bus system 1204.

[0222] The second memory 1203 in the embodiment of the present application is used to store various types of data to support the operation of the second electronic device 1200. Examples of such data include: any computer program used to operate on the second electronic device 1200.

[0223] It is understood that the second memory 1203 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be ROM, PROM, EPROM, EEPROM, FRAM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM; the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a RAM, which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available, such as SRAM, SSRAM, DRAM, SDRAM, DDRSDRAM, ESDRAM, SLDRAM, DRRAM. The second memory 1203 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memory.

[0224] The method disclosed in the above embodiment of the present application can be applied to the second processor 1202, or implemented by the second processor 1202. The second processor 1202 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of the hardware in the second processor 1202 or an instruction in the form of software. The above-mentioned second processor 1202 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The second processor 1202 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the second memory 1203, and the second processor 1202 reads the program in the second memory 1203, and completes the steps of the above method in combination with its hardware.

[0225] Optionally, when the second processor 1202 executes the program, it implements the corresponding process implemented by the second electronic device in each method of the embodiments of the present application, which will not be repeated here for the sake of brevity.

[0226] In an exemplary embodiment, the embodiment of the present application further provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, including a first memory 1103 and a second memory 1203 storing a computer program, and the computer program can be executed by a first processor 1102 and a second processor 1202 of an electronic device, respectively, to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM.

[0227] In the several embodiments provided in the present application, it should be understood that the disclosed devices, electronic devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0228] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0229] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0230] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0231] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0232] It should be noted that the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict. Unless otherwise specified and limited, the term "connection" should be understood in a broad sense, for example, it can be an electrical connection, or it can be the internal connection of two components, it can be a direct connection, or it can be an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0233] In addition, in the examples of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the objects distinguished by "first\second\third" can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0234] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0235] The term "and / or" herein is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality. For example, at least one of A, B, and C can represent any one or more elements selected from the set consisting of A, B, and C.

[0236] The various specific technical features in the various embodiments described in the specific implementation methods can be combined in various ways without contradiction. For example, different implementation methods can be formed by combining different specific technical features. In order to avoid unnecessary repetition, the various possible combinations of the specific technical features in this application will not be described separately.

Claims

1. A data classification model training method, characterized in that: The method comprises: The first feature tensor is input into the data classification model, and the second feature tensor is output; each row of the first vector in the first feature tensor corresponds to a first attribute name representing the data object; each row of the elements in the second feature tensor corresponds to the distance between any two rows of the first vectors; Based on the output second feature tensor, clustering each first vector to obtain at least one cluster; Labeling the first attribute names corresponding to a set number of first vectors under each cluster to obtain a first sample set; Determine a loss value based on the first sample set, and update a weight parameter of the data classification model according to the determined loss value until all the determined first sample sets meet a set end training condition; The set training end conditions include: The labeling result indicates that the first attribute names corresponding to a set number of first vectors under the same cluster correspond to the same type of attributes; the labeling result indicates the result obtained by labeling the first attribute names corresponding to the set number of first vectors under each cluster.

2. The method according to claim 1, characterized in that: The first attribute names corresponding to the set number of first vectors under each cluster are labeled to obtain a first sample set, including: Combining first attribute names corresponding to any two first vectors of a set number of first vectors under each cluster to obtain at least one first data group; Labeling each first data group in the at least one first data group according to whether two first attribute names included in the first data group correspond to the same type of attribute, to obtain a labeling result corresponding to each first data group; The first sample set is determined based on the labeling result.

3. The method according to claim 2, characterized in that The determining the first sample set based on the labeling result includes: determining at least one second data group based on the annotation result; The first sample set is obtained according to all determined second data groups; wherein the second data group is composed of two first data groups that satisfy set composition conditions; the set composition conditions indicate that the two first data groups correspond to different labeling results, and the two first data groups have one and only one identical first attribute name.

4. The method according to claim 1, characterized in that: The step of inputting the first feature tensor into the data classification model comprises: Based on at least one type of first information, determining a corresponding first feature tensor; The determined first feature tensor is input into the data classification model.

5. The method according to claim 4, characterized in that The determining, based on at least one type of first information, a corresponding first feature tensor includes: Construct a bipartite graph based on the at least one type of first information, and determine a corresponding first feature tensor; and / or, The at least one type of first information is segmented, and a corresponding first feature tensor is determined based on a word vector corresponding to the segmentation result.

6. The method according to claim 4, characterized in that determining the corresponding first eigentensor at least twice; The step of inputting the determined first feature tensor into the data classification model comprises: Performing a nonlinear transformation on the determined at least two first eigentensors to obtain a third eigentensor; The obtained third feature tensor is input into the data classification model.

7. A data processing method, characterized in that: The method comprises: The fourth feature tensor is input into the data classification model, and a fifth feature tensor is output; each row of the second vector in the fourth feature tensor corresponds to a second attribute name representing the data object; each row of elements in the fifth feature tensor corresponds to the distance between any two rows of the second vector; Based on the output fifth feature tensor, clustering each second vector to obtain at least one cluster; Based on the cluster center corresponding to each cluster obtained by clustering, determine the third attribute name corresponding to each cluster; the third attribute name represents the attribute name of the corresponding cluster; wherein, The data classification model is trained using the data classification model training method according to any one of claims 1 to 6.

8. The method according to claim 7, characterized in that The step of inputting the fourth feature tensor into the data classification model comprises: Based on at least one type of second information, determining a corresponding fourth feature tensor; The determined fourth feature tensor is input into the data classification model.

9. The method according to claim 8, characterized in that The determining, based on at least one type of second information, a corresponding fourth feature tensor includes: Construct a bipartite graph based on the at least one type of second information, and determine a corresponding fourth eigentensor; and / or, The at least one type of second information is segmented, and a corresponding fourth feature tensor is determined based on a word vector corresponding to the segmentation result.

10. The method according to any one of claims 7 to 9, characterized in that: Determining the third attribute name corresponding to each cluster based on the cluster center corresponding to each cluster obtained by clustering includes: Based on the cluster center corresponding to each cluster obtained by clustering, the second attribute name corresponding to the second vector with the shortest distance under the corresponding cluster is determined as the third attribute name of the corresponding cluster.

11. A data classification model training device, characterized in that: include: A first classification unit, used for inputting the first feature tensor into a data classification model and outputting a second feature tensor; Each row of the first vector in the first feature tensor corresponds to a first attribute name representing a data object; each row of the elements in the second feature tensor corresponds to the distance between any two rows of the first vectors; A first clustering unit, used for clustering each first vector based on the output second feature tensor to obtain at least one cluster; A labeling unit, used for labeling first attribute names corresponding to a set number of first vectors under each cluster to obtain a first sample set; A training unit, configured to determine a loss value based on the first sample set, and update a weight parameter of the data classification model according to the determined loss value until all the determined first sample sets meet a set training end condition; The set end training condition includes: the first attribute names corresponding to the set number of first vectors under the same cluster represented by the labeling result correspond to the same type of attributes; The labeling result represents a result obtained by labeling the first attribute names corresponding to a set number of first vectors under each cluster.

12. A data processing device, characterized in that: include: a second classification unit, configured to input the fourth feature tensor into a data classification model and output a fifth feature tensor; Each row of the second vector in the fourth feature tensor corresponds to a second attribute name representing the data object; each row of elements in the fifth feature tensor corresponds to the distance between any two rows of the second vector; A second clustering unit, used for clustering each second vector based on the output fifth feature tensor to obtain at least one cluster; A processing unit is used to determine a third attribute name corresponding to each cluster based on the cluster center corresponding to each cluster obtained by clustering; the third attribute name represents the attribute name of the corresponding cluster; wherein, The data classification model is trained using the data classification model training method according to any one of claims 1 to 6.

13. A first electronic device, characterized in that: include: a first processor and a first memory for storing a computer program executable on the processor, Wherein, when the first processor is used to run the computer program, the steps of the method described in any one of claims 1 to 6 are executed.

14. A second electronic device, characterized in that: include: a second processor and a second memory for storing a computer program executable on the processor, Wherein, when the second processor is used to run the computer program, the steps of the method described in any one of claims 7 to 10 are executed.

15. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 6, or implements the steps of the method according to any one of claims 7 to 10.

Images