Model training method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a model training method and device, electronic equipment and storage medium, computer technology, artificial intelligence technology field, specifically related to natural language processing, image processing, deep learning and big data and other artificial intelligence technology fields. The specific implementation scheme is: obtaining sample data; classifying the sample data according to a sample classification strategy to generate a label value corresponding to the sample data; extracting features of the sample data to generate sample data features; obtaining a data classification model to be trained; and training the data classification model according to the sample data features and the label value to generate a trained data classification model. Thus, the effect of data classification can be improved by establishing and applying a data classification model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer technology, in particular to the technical field of artificial intelligence such as natural language processing, image processing, deep learning and big data, and more particularly to a model training method and device, an electronic device and a storage medium. BACKGROUND

[0002] Data classification and grading is the basis of data use management and security protection. Formulating a classification and grading system for data, especially important data, and managing it according to the rules is an important work to achieve the goal of data security.

[0003] The data collected by the traditional data system has a single source and a small amount, and the classification and grading is simple and controllable. However, the data collected by the big data system has a large amount and a variety of types, including structured, semi-structured and unstructured data. At present, in the big data system, the focus is on the source and type of data, and less attention is paid to the grading of data. SUMMARY

[0004] The present disclosure provides a model training method, device, apparatus and storage medium.

[0005] According to an aspect of the present disclosure, a model training method is provided, comprising:

[0006] obtaining sample data;

[0007] grading the sample data according to a sample grading strategy to generate a label value corresponding to the sample data;

[0008] extracting features of the sample data to generate sample data features;

[0009] obtaining a data grading model to be trained; and

[0010] training the data grading model according to the sample data features and the label value to generate the data grading model after training.

[0011] According to another aspect of the present disclosure, a model training device is provided, comprising:

[0012] a first obtaining module configured to obtain sample data;

[0013] a first generating module configured to grade the sample data according to a sample grading strategy to generate a label value corresponding to the sample data;

[0014] a second generating module configured to extract features of the sample data to generate sample data features;

[0015] The second acquisition module is configured to acquire a data classification model to be trained.

[0016] The training module is configured to train the data classification model according to the sample data features and the label values, to generate the data classification model after training.

[0017] According to another aspect of the present disclosure, an electronic device is provided, comprising:

[0018] at least one processor; and

[0019] a memory connected to the at least one processor in communication; wherein

[0020] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the model training method.

[0021] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the model training method.

[0022] According to another aspect of the present disclosure, a computer program product comprising a computer program is provided, which, when executed by a processor, implements the steps of the model training method.

[0023] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0025] Figure 1 is a flowchart of a model training method according to one embodiment of the present disclosure;

[0026] Figure 2 is a flowchart of a model training method according to another embodiment of the present disclosure;

[0027] Figure 3 is a flowchart of a model training method according to another embodiment of the present disclosure;

[0028] Figure 4 is a flowchart of a model training method according to another embodiment of the present disclosure;

[0029] Figure 5 is a flowchart of a model training method according to one specific embodiment of the present disclosure;

[0030] Figure 6 FIG. 1 is a structural schematic diagram of a model training apparatus according to an embodiment of the present disclosure; and

[0031] Figure 7 FIG. 4 is a block diagram of an electronic device for implementing a model training method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding. These should be considered in their context only, and should not be considered limiting the scope of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0033] A model training method, apparatus, electronic device, and storage medium according to an embodiment of the present disclosure are described below with reference to the accompanying drawings.

[0034] Artificial intelligence is a discipline that studies using a computer to simulate some thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.) of a person, and has both a hardware technical field and a software technical field. Artificial intelligence hardware technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, etc.; artificial intelligence software technology includes computer vision technology, speech recognition technology, natural language processing technology, and deep learning, big data processing technology, knowledge graph technology, etc.

[0035] Natural language processing is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics.

[0036] Image processing is a technology that uses a computer to analyze images to achieve desired results. It is also known as image processing. Image processing generally refers to digital image processing. A digital image is a large two-dimensional array obtained by shooting with an industrial camera, a video camera, a scanner, etc. The elements of the array are called pixels, and their values are called gray values. Image processing techniques generally include image compression, enhancement and restoration, matching, description and recognition.

[0037] Deep learning is a new research direction in the field of machine learning. Deep learning is to learn the internal rules and representation levels of sample data, and the information obtained in the learning process is very helpful for the interpretation of data such as text, images and sound. The ultimate goal is to enable machines to have analysis and learning ability like people, and to be able to recognize text, images and sound data. Deep learning is a complex machine learning algorithm, and the effect achieved in speech and image recognition far exceeds that of previous related technologies.

[0038] Big data, an IT (Internet Technology) industry term, refers to a collection of data that cannot be captured, managed and processed within a certain time range by conventional software tools, and is a massive, high-growth and diversified information asset that requires new processing modes to have stronger decision-making, insight discovery and process optimization capabilities. In Victor Mayer-Schonberger and Kenneth Cukier's "Big Data Era", big data refers to the analysis and processing of all data without using shortcuts such as random analysis (sample survey). The 5V characteristics of big data (proposed by IBM): Volume (large amount), Velocity (high speed), Variety (variety), Value (low value density), Veracity (truthfulness).

[0039] The model training method provided by the embodiments of the present disclosure can be executed by an electronic device, which can be a PC (Personal Computer), a tablet computer, a palm computer, a server, etc., and is not limited here.

[0040] In the embodiments of the present disclosure, a processing component, a storage component and a driving component can be arranged in the electronic device. Optionally, the driving component and the processing component can be arranged integrally, and the storage component can store an operating system, an application program or other program modules. The processing component implements the model training method provided by the embodiments of the present disclosure by executing the application program stored in the storage component.

[0041] Figure 1 A flowchart of a model training method provided by the embodiments of the present disclosure.

[0042] The model training method of the embodiments of the present disclosure can also be executed by the model training apparatus provided by the embodiments of the present disclosure. The apparatus can be configured in an electronic device to achieve the following: obtaining sample data, grading the sample data according to a sample grading strategy to generate label values corresponding to the sample data, extracting features of the sample data to generate sample data features, obtaining a data grading model to be trained, and training the data grading model according to the sample data features and the label values to generate the data grading model after training. Thus, the effect of data grading can be improved by establishing and applying the data grading model.

[0043] As a possible case, the model training method of the embodiments of the present disclosure can also be executed on a server side. The server can be a cloud server, and the model training method can be executed on the cloud.

[0044] As shown in Figure 1 , the model training method can include the following steps:

[0045] In step 101, sample data is obtained. The sample data can be multiple, and can include unstructured data such as text, pictures and videos, and structured data such as databases and data tables.

[0046] In the embodiments of the present disclosure, there can be multiple ways to obtain sample data. The obtained sample data can be stored in the storage space of the electronic device for subsequent retrieval and use. The storage space is not limited to a storage space based on an entity, such as a hard disk. The storage space can also be a storage space of a network hard disk connected to the electronic device (cloud storage space).

[0047] In step 102, the sample data is graded according to a sample grading strategy to generate label values corresponding to the sample data. The sample grading strategy can be calibrated according to actual conditions and requirements.

[0048] In the embodiments of the present disclosure, relevant personnel can manually annotate the sample data according to the grading rules to obtain the label values corresponding to the sample data. The grading rules can be calibrated according to actual conditions.

[0049] Specifically, after the electronic device obtains the sample data, the relevant personnel can manually annotate the sample data according to the grading rules by operating the electronic device to obtain the label values corresponding to the sample data, that is, to annotate the sample data with the corresponding grade, and to take the grade as the label value of the sample data.

[0050] For example, if the level of the sample data is divided into four levels according to the grading rule: level 1, level 2, level 3 and level 4, the label value can be 1, 2, 3 and 4, and the sample data can be labeled with the corresponding label value according to the grading rule; if the level of the sample data is divided into two levels: important (sensitive) and unimportant (non-sensitive) according to the grading rule, represented by numerical values 1 and 0 respectively, the label value can be 1 and 0, and the sample data can be labeled with the corresponding label value according to the grading rule.

[0051] As a possible case, the sample data can be processed according to the experience grading model trained based on the inventory rule (for example, the grading rule) to generate the label value corresponding to the sample data. It should be noted that the experience grading model described in this embodiment can be trained in advance and pre-stored in the storage space of the electronic device for easy retrieval and use.

[0052] The training and generation of the experience grading model can be performed by a related training server. The training server can be a cloud server or a host of a computer. The training server and the electronic device that can execute the model training method provided by the embodiments of the present disclosure establish a communication connection. The communication connection can be at least one of a wireless network connection and a wired network connection. The training server can send the trained experience grading model to the electronic device, so that the electronic device can call it when needed, thereby greatly reducing the computing pressure of the electronic device.

[0053] Specifically, the electronic device can retrieve the sample data and the experience grading model described above from the storage space of the electronic device, and input the sample data into the experience grading model, so as to process the sample data through the experience grading model to obtain the label value corresponding to the sample data output by the experience grading model.

[0054] Therefore, by manually labeling the sample data and / or processing the sample data based on the experience grading model trained based on the inventory rule (for example, the grading rule), the label value corresponding to the sample data can provide more extensive support for the training of the model, so that the subsequent training of the data grading model is more accurate.

[0055] Step 103, feature extraction is performed on the sample data to generate sample data features.

[0056] The sample data features can include sample data content features, sample data statistical features and sample metadata features.

[0057] In the embodiments of the present disclosure, the sample data can be feature extracted based on a pre-set feature extraction strategy to generate sample data features. The pre-set feature extraction strategy can be calibrated according to actual conditions and requirements.

[0058] Specifically, after obtaining the sample data, the electronic device can perform feature extraction on the sample data based on a preset feature extraction strategy to generate sample data features.

[0059] As one possible case, the sample data can be subjected to feature extraction according to a feature extraction model to generate sample data features. It should be noted that the feature extraction model described in this embodiment can be pre-trained and pre-stored in the storage space of the electronic device for easy retrieval and use.

[0060] Specifically, after obtaining the sample data, the electronic device can retrieve (obtain) the feature extraction model from its own storage space, and input the sample data into the feature extraction model, so as to perform feature extraction on the sample data by the feature extraction model to obtain sample data features output by the feature extraction model.

[0061] As another possible case, the electronic device can also use a feature extraction tool (for example, a plug-in) to perform feature extraction on the sample data to obtain sample data features.

[0062] Step 104: Obtain a data classification model to be trained.

[0063] It should be noted that the data classification model to be trained described in this embodiment can be pre-stored in the storage space of the electronic device for easy retrieval and use. The data classification model to be trained can be a tree model, a neural network, a deep learning model, etc., which is not limited herein.

[0064] Step 105: Train the data classification model according to the sample data features and the label values to generate a trained data classification model.

[0065] Specifically, after obtaining the sample data features and the label values, the electronic device can retrieve (obtain) the data classification model to be trained from its own storage space, and can train the data classification model according to the sample data features and the label values to generate a trained data classification model.

[0066] Further, after obtaining the trained data classification model, the electronic device can perform classification detection on the data to be classified according to the data classification model to generate the level of the data to be classified. Thus, through the trained data classification model, fast and automatic data classification inference can be realized, and the closest deep understanding of data security to data classification can be achieved, and automatic classification recognition based on data features can be realized.

[0067] In the embodiments of the present disclosure, first, sample data is acquired, and the sample data is classified according to a sample classification strategy to generate a label value corresponding to the sample data, then feature extraction is performed on the sample data to generate sample data features, and a data classification model to be trained is acquired, and finally the data classification model is trained according to the sample data features and the label value to generate the data classification model after training. Thus, the effect of data classification can be improved by establishing and applying the data classification model.

[0068] To clearly illustrate the above embodiment, in an embodiment of the present disclosure, as shown in Figure 2 The sample data features can include sample data content features and sample data statistical features, and the feature extraction on the sample data to generate the sample data features can include:

[0069] In step 201, content feature extraction is performed on the sample data to generate sample data content features.

[0070] The sample data content features can be features of the sample data itself, for example, one or more of natural language features, graphic features, picture and video features, and text recognition derived features in pictures.

[0071] In the embodiments of the present disclosure, if the sample data is unstructured data such as text, pictures and videos, content feature extraction can be performed on the sample data to generate corresponding content features. It should be noted that different feature extraction techniques can be used to perform content feature extraction on sample data of different data types, which is not limited here.

[0072] Specifically, if the sample data is text, a text feature extraction technique (for example, a TF-IDF (Term Frequency-Inverse Document Frequency) algorithm or the like) can be used to perform text feature extraction on the sample data, including extracting text features such as topic words, key words and sensitive words to generate content features of the sample data, and in particular, sensitive words that can indicate the data sensitivity level need to be extracted to facilitate classification of the sample data; if the sample data is a picture or a video, a text recognition technique (for example, an OCR (Optical Character Recognition) text recognition technique or the like) can be used to perform text recognition on the picture or the video to be recognized to extract text content in the picture or the video, and then a text feature extraction technique is used to perform feature extraction on the text content to generate content features of the sample data.

[0073] As a possible case, if the sample data is a picture, picture understanding technology can be used to extract features of the picture, including sensitive information and picture content vector features, to generate content features of the sample data. In particular, sensitive information that can indicate the sensitivity level in the picture needs to be extracted to facilitate classification of the sample data.

[0074] As another possible case, if the sample data is a video, video understanding technology can be used to extract features of the video, including sensitive information and video content vector features, to generate content features of the sample data. In particular, sensitive information that can indicate the sensitivity level in the video needs to be extracted to facilitate classification of the sample data.

[0075] It should be noted that the text feature extraction technology, text recognition technology, picture understanding technology and video understanding technology described in this embodiment can be calibrated according to actual conditions and needs, and are not limited here.

[0076] Step 202, obtaining related information of the sample data. The related information can include coverage information of the sample data and distribution information of the sample data.

[0077] Step 203, performing statistical feature extraction on the related information and the sample data to generate sample data statistical features.

[0078] In the embodiments of the present disclosure, the statistical feature extraction strategy can be preset to perform statistical feature extraction on the related information and the sample data to generate sample data statistical features. The preset statistical feature extraction strategy can be calibrated according to actual conditions and needs.

[0079] Specifically, after the electronic device obtains the related information and the sample data, the statistical feature extraction strategy can be preset to perform statistical feature extraction on the related information and the sample data to generate sample data statistical features. The sample data statistical features can include coverage of the sample data, distribution of the sample data, and data length of the sample data.

[0080] Thus, the content features of the sample data and the statistical features of the sample data can be obtained to support the training of the model.

[0081] Further, in an embodiment of the present disclosure, the sample data features described above can also include sample metadata features. The sample data features are extracted to generate sample data features, and the sample metadata features are extracted in response to determining that the type of the sample data is a target type. The target type can include a database type and a data table type.

[0082] In the embodiments of the present disclosure, if the type of the sample data is a target type (i.e., the data type of the sample data is a database type or a data table type), the sample data can be subjected to metadata feature extraction based on a preset metadata feature extraction strategy to generate sample metadata features. The preset metadata feature extraction strategy can be calibrated according to actual conditions and requirements.

[0083] Specifically, after the electronic device obtains the sample data, it can first determine whether the sample data is of the target type. If yes, the sample data can be subjected to metadata feature extraction based on the preset metadata feature extraction strategy to generate sample metadata features. If no, the sample data is not subjected to metadata feature extraction.

[0084] In this way, sample metadata features can be obtained to support the training of the model.

[0085] In the embodiments of the present disclosure, the sample data features obtained through deep analysis (learning) of the sample data are used to train the data classification model, which can further improve the accuracy and application range of the data classification model and solve the problem in the related art that the inventory classification rules cannot accurately classify and identify data.

[0086] In one embodiment of the present disclosure, as shown in Figure 3 training the data classification model according to the sample data features and the label values to generate the trained data classification model can include:

[0087] Step 301: Preprocessing the sample data features to generate target data features.

[0088] In the embodiments of the present disclosure, the sample data features can be processed (preprocessed) based on a preset preprocessing strategy to generate target data features. The preset preprocessing strategy can be calibrated according to actual conditions and requirements.

[0089] Specifically, after the electronic device obtains the sample data features, the sample data features can be processed (preprocessed) based on the preprocessing strategy, for example, converted (e.g., normalized), combined (character concatenation, numerical addition, subtraction, multiplication, and division, etc.), and the like, to form new conversion and combination features, i.e., target data features.

[0090] For example, if the sample data features are numerical values, the numerical values can be subjected to certain addition, subtraction, multiplication, and division operations, and then the results of the operations can be normalized to obtain normalized results, which are taken as the target data features.

[0091] Step 302: Training the data classification model according to the target data features and the label values to generate the trained data classification model.

[0092] Specifically, after obtaining the target data feature and the label value, the electronic device can train the data classification model according to the target data feature and the label value to generate the data classification model after training.

[0093] In this way, the sample data feature can be preprocessed, and the data classification model can be trained by the preprocessed sample data feature to generate a better data classification model, thereby realizing automatic classification of data and improving the classification effect of data.

[0094] To make the above embodiment clearer, in an embodiment of the present disclosure, as shown in Figure 4 training the data classification model according to the target data feature and the label value can include:

[0095] Step 401: input the target data feature into the data classification model to generate a predicted label value.

[0096] Step 402: generate a loss value according to the predicted label value and the label value corresponding to the sample data.

[0097] Step 403: train the data classification model according to the loss value.

[0098] Specifically, after obtaining the target data feature and the label value corresponding to the sample data, the electronic device can call the data classification model from the storage space of the electronic device, input the target data feature into the data classification model, process the target data feature by the data classification model to generate a predicted label value, generate a loss value according to the predicted label value and the label value corresponding to the sample data, and train the data classification model according to the loss value until the training is completed to generate a better data classification model.

[0099] In this way, the data classification model to be trained can be trained to obtain a better data classification model, thereby improving the effect of data classification.

[0100] To make the above embodiment clearer, in an embodiment of the present disclosure, as shown in Figure 5 is a flowchart of data classification. Referring to Figure 5First, sample data is acquired, and the sample data is classified to generate a label value corresponding to the sample data, then a sample data feature is extracted, and the data classification model to be trained is trained according to the sample data feature and the label value, to generate a high-quality data classification model, and finally, the data is predicted by the trained data classification model, that is, the data is classified, and a predicted classification result is generated. Thus, the data classification model is established and applied, which can realize fast and automatic data classification reasoning, and achieve the closest understanding of data security to data classification, realize automatic classification recognition based on data characteristics, solve the problem that the classification rules in the related art cannot accurately classify and recognize data, and enable the above-established data classification model to be applied in more scenarios.

[0101] Figure 6 A structural schematic diagram of a model training apparatus provided by an embodiment of the present disclosure is shown.

[0102] The model training apparatus of the embodiment of the present disclosure can be configured in an electronic device to acquire sample data, classify the sample data according to a sample classification strategy to generate a label value corresponding to the sample data, and extract features of the sample data to generate sample data features, then acquire a data classification model to be trained, and train the data classification model according to the sample data features and the label value to generate a trained data classification model, so that the effect of data classification can be improved by establishing and applying the data classification model.

[0103] As shown in Figure 6 The model training apparatus 600 can include a first acquisition module 610, a first generation module 620, a second generation module 630, a second acquisition module 640, and a training module 650.

[0104] The first acquisition module 610 is configured to acquire sample data. The sample data can be multiple, and can include unstructured data such as text, pictures, and videos, and can also include structured data such as databases and data tables.

[0105] In the embodiment of the present disclosure, there can be multiple ways to acquire sample data, wherein the acquired sample data can be stored in the storage space of the electronic device for subsequent retrieval and use. The storage space is not limited to an entity-based storage space, for example, a hard disk, and can also be a storage space of a network hard disk connected to the electronic device (cloud storage space).

[0106] The first generation module 620 is configured to classify the sample data according to a sample classification strategy to generate a label value corresponding to the sample data. The sample classification strategy can be calibrated according to actual conditions and requirements.

[0107] In the embodiments of the present disclosure, the relevant personnel can manually label the sample data according to the grading rules to obtain the label value corresponding to the sample data, wherein the grading rules can be calibrated according to actual conditions.

[0108] Specifically, after the first acquisition module 610 acquires the sample data, the relevant personnel can manually label the sample data according to the grading rules by operating the first generation module 620 to obtain the label value corresponding to the sample data, that is, label the sample data with the corresponding level, and take the level as the label value of the sample data. For example, if the level of the sample data is divided into four levels according to the grading rules: level 1, level 2, level 3 and level 4, the label value can be 1, 2, 3 and 4, and the sample data can be labeled with the corresponding label value according to the grading rules. If the level of the sample data is divided into two levels according to the grading rules: important (sensitive) and unimportant (non-sensitive), represented by numerical values 1 and 0 respectively, the label value can be 1 and 0, and the sample data can be labeled with the corresponding label value according to the grading rules.

[0109] As a possible case, the first generation module 620 can process the sample data according to the experience grading model trained based on the inventory rule (grading rule) to generate the label value corresponding to the sample data. It should be noted that the experience grading model described in this embodiment can be trained in advance and pre-stored in the storage space of the electronic device for easy retrieval and application. The training and generation of the experience grading model can be performed by a related training server. The training server can be a cloud server or a host of a computer. The training server and the electronic device that can execute the model training method provided by the embodiments of the present disclosure are in communication connection, which can be at least one of wireless network connection and wired network connection. The training server can send the trained experience grading model to the electronic device, so that the electronic device can call it when needed, thereby greatly reducing the computing pressure of the electronic device.

[0110] Specifically, the first generation module 620 can call the above-mentioned experience grading model from the storage space of the electronic device, and input the sample data into the experience grading model, so as to process the sample data through the experience grading model to obtain the label value corresponding to the sample data output by the experience grading model.

[0111] The second generation module 630 is configured to extract features of the sample data to generate sample data features.

[0112] The sample data features can include sample data content features, sample data statistical features and sample metadata features.

[0113] In the embodiments of the present disclosure, the second generation module 630 can perform feature extraction on the sample data based on a preset feature extraction strategy to generate sample data features. The preset feature extraction strategy can be calibrated according to actual conditions and requirements.

[0114] Specifically, after the first acquisition module 610 acquires the sample data, the second generation module 630 can perform feature extraction on the sample data features based on the preset feature extraction strategy to generate sample data features of the sample data.

[0115] As a possible case, the second generation module 630 can perform feature extraction on the sample data according to a feature extraction model to generate sample data features. It should be noted that the feature extraction model described in this embodiment can be pre-trained and pre-stored in the storage space of the electronic device for easy retrieval and use.

[0116] Specifically, after the first acquisition module 610 acquires the sample data, the second generation module 630 can retrieve (acquire) the feature extraction model from the storage space of the electronic device, and input the sample data into the feature extraction model, so as to perform feature extraction on the sample data by the feature extraction model to obtain sample data features output by the feature extraction model.

[0117] As another possible case, the second generation module 630 can also use a feature extraction tool (for example, a plug-in) to perform feature extraction on the sample data to obtain sample data features.

[0118] The second acquisition module 640 is configured to acquire a data classification model to be trained.

[0119] It should be noted that the data classification model to be trained described in this embodiment can be pre-stored in the storage space of the electronic device for easy retrieval and use. The data classification model to be trained can be a tree model, a neural network, a deep learning model, etc., which is not limited here.

[0120] The training module 650 is configured to train the data classification model according to the sample data features and the label values to generate a trained data classification model.

[0121] Specifically, after the first generation module 620 obtains the label values and the first acquisition module 610 acquires the sample data features obtained by the second generation module 630, the second acquisition module 640 can retrieve (acquire) the data classification model to be trained from the storage space of the electronic device, and then the training module 650 can train the data classification model according to the sample data features and the label values to generate a trained data classification model.

[0122] In the embodiment of the present disclosure, the first obtaining module obtains sample data, the first generating module classifies the sample data according to a sample classification strategy to generate a label value corresponding to the sample data, the second generating module extracts features of the sample data to generate sample data features, the second obtaining module obtains a data classification model to be trained, and the training module trains the data classification model according to the sample data features and the label value to generate the data classification model after training, so that the data classification model can be established and applied to improve the effect of data classification.

[0123] In one embodiment of the present disclosure, the sample data features include sample data content features and sample data statistical features, the second generating module 630 is specifically configured to extract content features of the sample data to generate sample data content features, obtain related information of the sample data, and extract statistical features of the related information and the sample data to generate sample data statistical features.

[0124] In one embodiment of the present disclosure, the sample data features further include sample metadata features, and the second generating module 630 is further configured to, in response to determining that the type of the sample data is a target type, extract metadata features of the sample data to generate sample metadata features, wherein the target type includes a database type and a data table type.

[0125] In one embodiment of the present disclosure, the training module 650 can include a preprocessing unit 651 and a training unit 652.

[0126] The preprocessing unit 651 is configured to preprocess the sample data features to generate target data features.

[0127] The training unit 652 is configured to train the data classification model according to the target data features and the label value to generate the data classification model after training.

[0128] In one embodiment of the present disclosure, the training unit 652 is specifically configured to input the target data features into the data classification model to generate a predicted label value, generate a loss value according to the predicted label value and the label value corresponding to the sample data, and train the data classification model according to the loss value.

[0129] It should be noted that the foregoing explanation and description of the model training method embodiment are also applicable to the model training device of the embodiment, and will not be repeated here.

[0130] The device of the embodiment of the present disclosure acquires sample data through the first acquisition module, classifies the sample data according to a sample classification strategy through the first generation module to generate a label value corresponding to the sample data, extracts features of the sample data through the second generation module to generate sample data features, acquires a data classification model to be trained through the second acquisition module, and finally trains the data classification model according to the sample data features and the label value through the training module to generate the data classification model after training. Thus, the data classification effect can be improved by establishing and applying the data classification model.

[0131] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0132] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0133] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the present disclosure described and / or claimed in this document.

[0134] As shown in Figure 7 The device 700 includes a computing unit 701 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded into a random access memory (RAM) 703 from a storage unit 708. Various programs and data required for the operation of the device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0135] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through computer networks, such as the Internet, and / or various telecommunication networks.

[0136] The computing unit 701 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above, such as the model training method. For example, in some embodiments, the model training method can be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded onto the RAM 703 and executed by the computing unit 701, one or more steps of the model training method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the model training method by any other appropriate means, such as by means of firmware.

[0137] Various implementations of the systems and techniques described above herein can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0138] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.

[0139] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0140] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0141] The systems and techniques described herein can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0142] The computer system can include clients and servers. This relationship can be. The servers are generally remote from the users and can be accessed via the Internet using a communication network. The relationship can be facilitated by a computer program running on the computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system. It solves the defects of large management difficulty and weak business scalability in traditional physical host and VPS (Virtual Private Server, or VPS for short) services. The server can also be a server of a distributed system or a server combined with a blockchain.

[0143] It should be understood that the various forms of flow shown above can be reordered, additional or deleted steps. For example, the steps described in the present disclosure can be executed in parallel, sequentially or in different order, as long as the desired results of the technical solutions of the present disclosure can be achieved, which are not limited herein.

[0144] The above detailed description does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A model training method, comprising: Obtain sample data, which includes unstructured data and structured data. The unstructured data includes at least one of the following: text, images, and videos. The structured data includes at least one of the following: databases and data tables. The sample data is classified according to the sample classification strategy to generate label values ​​corresponding to the sample data; Content features are extracted from the sample data to generate the sample data content features; Obtain relevant information about the sample data, including coverage information and distribution information of the sample data; Statistical features are extracted from the relevant information and the sample data to generate statistical features of the sample data; In response to determining that the type of the sample data is a target type, metadata features are extracted from the sample data to generate sample metadata features, wherein the target type includes database type and data table type; Obtain the hierarchical model of the data to be trained; and The data classification model is trained based on the sample data content features, sample data statistical features, sample metadata features, and the label values ​​to generate the trained data classification model.

2. The method according to claim 1, wherein, The step of training the data classification model based on the sample data features and the label values ​​to generate the trained data classification model includes: The sample data features are preprocessed to generate target data features; The data classification model is trained based on the target data features and the label values ​​to generate the trained data classification model.

3. The method according to claim 2, wherein, The step of training the data classification model based on the target data features and the label values ​​includes: The target data features are input into the data classification model to generate predicted label values; A loss value is generated based on the predicted label value and the label value corresponding to the sample data; The data classification model is trained based on the loss value.

4. A model training device, comprising: The first acquisition module is used to acquire sample data, which includes unstructured data and structured data. The unstructured data includes at least one of the following: text, images, and videos. The structured data includes at least one of the following: databases and data tables. The first generation module is used to classify the sample data according to the sample classification strategy in order to generate the label value corresponding to the sample data. The second generation module is used to extract content features from the sample data to generate the content features of the sample data. Obtain relevant information about the sample data, including coverage information and distribution information of the sample data; extract statistical features from the relevant information and the sample data to generate statistical features of the sample data; In response to determining that the type of the sample data is a target type, metadata features are extracted from the sample data to generate sample metadata features, wherein the target type includes database type and data table type; The second acquisition module is used to acquire the hierarchical model of the data to be trained. The training module is used to train the data classification model based on the sample data content features, sample data statistical features, sample data metadata features and the label values, so as to generate the trained data classification model.

5. The apparatus according to claim 4, wherein, The training module includes: A preprocessing unit is used to preprocess the sample data features to generate target data features; The training unit is used to train the data classification model based on the target data features and the label values ​​to generate the trained data classification model.

6. The apparatus according to claim 5, wherein, The training unit is specifically used for: The target data features are input into the data classification model to generate predicted label values; A loss value is generated based on the predicted label value and the label value corresponding to the sample data; The data classification model is trained based on the loss value.

7. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the model training method according to any one of claims 1-4.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the model training method according to any one of claims 1-4.

9. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the model training method according to any one of claims 1-4.

Images