Data classification and grading system, method and device for ultra-short text

Abstract

The present application discloses a data classification and grading system, method and device for ultra-short texts, including: determining corresponding data to be processed based on a received classification and grading request; constructing a classification and grading standard vector set based on the classification and grading request and using pre-loaded word vectors and multiple classification and grading standards; constructing a field vector matrix using word vectors and field names of the data to be processed, wherein the field names are parsed into original words, core words, modifiers, and segmented words; matching the field vector matrix with each classification and grading standard vector matrix respectively, and judging whether there is a first candidate item with a matching degree greater than a first preset threshold; in the case of determining that there is a first candidate item, marking the data to be processed with a corresponding classification and grading label; in the case of determining that there are multiple first candidate items, constructing a context matrix for matching; in the case of determining that there is no first candidate item, marking the data to be processed with a non-classification and grading label.

Description

Technical Field

[0001] The present application relates to the field of software management technology, and in particular to a data classification and grading system, method and device for ultra-short texts. Background Art

[0002] At present, the methods commonly used for data classification and grading are hard matching of text, regular matching, word vector similarity matching, classification model training and inference prediction. The current data classification and grading work mainly focuses on the classification and grading of table names and field names in the database (among which, there are also classification and grading by scanning data content, which is generally done by scanning data content features to correct metadata, such as Chinese names in tables and Chinese names in fields, so this type of classification and grading is ultimately still classified as classification and grading based on table names and field names). Since the number of table names and field names in the above-mentioned database is very large, if classification and grading is performed manually, the workload is huge. Furthermore, since the table names and field names of the database only include a few words, they belong to ultra-short texts, and the ultra-short texts themselves are very lacking in semantics, the accuracy of classification and grading for ultra-short texts is generally not high.

[0003] For example, if a hard text matching method is used to classify and grade ultra-short texts, the accuracy rate is very low.

[0004] Second, if regular matching is used to classify and grade ultra-short text data, the accuracy rate is slightly improved, but it is difficult to exhaust all regular rules, so the applicability is not ideal, and the difficulty and workload of maintaining regular expressions are increased.

[0005] 3. Although word vector similarity matching is currently a widely used data classification and grading method, it is only a similarity match between words by converting them into vectors, which cannot solve the ambiguity of words and the one-to-many problem. For example, the field name "name" may appear in multiple categories. Typical examples are personal information and legal representative information. These two pieces of information belong to different categories, one for private business and the other for public business. When the keyword "name" is configured in both the private and public categories, and the field name of a table is "name", the field name and the keywords corresponding to the two categories are 100% matched. However, the field names correspond to two different categories, and it is not clear which category the field name "name" belongs to.

[0006] Fourth, if classification model training and inference prediction are used to classify and grade ultra-short texts, ultra-short texts need to be pre-labeled, and classification and grading can only be performed on the data range covered by the labeled samples. For example, if the samples only cover private business and not public business, the trained model can only be used for private business. If the classification is increased or changed, the ultra-short text needs to be re-labeled, which has low applicability and high maintenance costs.

[0007] With respect to the technical problems existing in the above-mentioned prior art of data classification and grading for ultra-short texts, such as low accuracy, low applicability and high cost, no effective solution has been proposed so far. Summary of the invention

[0008] The embodiments of the present disclosure provide a data classification and grading system, method and device for ultra-short texts, so as to at least solve the technical problems that the data classification and grading methods for ultra-short texts in the prior art generally have low accuracy, low applicability and high cost.

[0009] According to one aspect of an embodiment of the present disclosure, a data classification and grading system for ultra-short texts is provided, comprising: an intelligent engine, a word vector library respectively connected to the intelligent engine in communication, a standard management module, a task management module and a metadata management module, wherein the intelligent engine is configured to perform the following operations: receiving a classification and grading request sent by the task management module, and determining corresponding to-be-processed data from the metadata management module based on the classification and grading request, wherein the to-be-processed data includes a field name; loading a corresponding classification and grading standard table from the standard management module based on the classification and grading request, wherein the classification and grading standard table includes a plurality of classification and grading standards; constructing a classification and grading standard vector set using word vectors pre-loaded from the word vector library and a plurality of classification and grading standards, wherein the classification and grading standard vector set includes a plurality of classification and grading standard vector matrices; constructing a field vector matrix using word vectors pre-loaded from the word vector library and the field name of the to-be-processed data; matching the field vector matrix with each classification and grading standard vector matrix respectively, and determining whether there is a first candidate item whose matching degree is greater than a first preset threshold; and in the case of determining that there is a first candidate item, determining a classification directory and a security level corresponding to the first candidate item, and marking the to-be-processed data with a corresponding classification and grading label based on the classification directory and the security level.

[0010] According to another aspect of the embodiment of the present disclosure, a data classification and grading method for ultra-short texts is also provided, including: receiving a classification and grading request sent by a task management module, and based on the classification and grading request, determining corresponding data to be processed from a metadata management module, wherein the data to be processed includes a field name; based on the classification and grading request, loading a corresponding classification and grading standard table from a standard management module, wherein the classification and grading standard table includes a plurality of classification and grading standards; constructing a classification and grading standard vector set using word vectors pre-loaded from a word vector library and a plurality of classification and grading standards, wherein the classification and grading standard vector set includes a plurality of classification and grading standard vector matrices; constructing a field vector matrix using word vectors pre-loaded from a word vector library and the field name of the data to be processed; matching the field vector matrix with each classification and grading standard vector matrix respectively, and determining whether there is a first candidate item whose matching degree is greater than a first preset threshold; and in the case of determining that there is a first candidate item, determining a classification directory and a security level corresponding to the first candidate item, and based on the classification directory and the security level, marking the data to be processed with a corresponding classification and grading label.

[0011] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is running, a processor executes any one of the methods described above.

[0012] According to another aspect of the embodiment of the present disclosure, there is also provided a data classification and grading device for ultra-short texts, comprising: a to-be-processed data determination module, for receiving a classification and grading request sent by a task management module, and determining corresponding to-be-processed data from a metadata management module based on the classification and grading request, wherein the to-be-processed data includes a field name; a loading module, for loading a corresponding classification and grading standard table from a standard management module based on the classification and grading request, wherein the classification and grading standard table includes a plurality of classification and grading standards; a set construction module, for constructing a classification and grading standard vector set using word vectors pre-loaded from a word vector library and a plurality of classification and grading standards, wherein the classification and grading standard vector set includes a plurality of classification and grading standard vector matrices; a field vector matrix construction module, for constructing a field vector matrix using word vectors pre-loaded from a word vector library and the field name of the to-be-processed data; a first matching module, for matching the field vector matrix with each classification and grading standard vector matrix respectively, and determining whether there is a first candidate item whose matching degree is greater than a first preset threshold; and a first label determination module, for determining a classification directory and a security level corresponding to the first candidate item when it is determined that there is a first candidate item, and marking the to-be-processed data with a corresponding classification and grading label based on the classification directory and the security level.

[0013] According to another aspect of the embodiment of the present disclosure, there is also provided a data classification and grading device for ultra-short texts, comprising: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: receiving a classification and grading request sent by a task management module, and based on the classification and grading request, determining corresponding data to be processed from a metadata management module, wherein the data to be processed includes a field name; based on the classification and grading request, loading a corresponding classification and grading standard table from a standard management module, wherein the classification and grading standard table includes a plurality of classification and grading standards; constructing a classification and grading standard vector set using word vectors pre-loaded from a word vector library and a plurality of classification and grading standards, wherein the classification and grading standard vector set includes a plurality of classification and grading standard vector matrices; constructing a field vector matrix using word vectors pre-loaded from a word vector library and the field name of the data to be processed; matching the field vector matrix with each classification and grading standard vector matrix respectively, and determining whether there is a first candidate item whose matching degree is greater than a first preset threshold; and in the case of determining that there is a first candidate item, determining a classification directory and a security level corresponding to the first candidate item, and based on the classification directory and the security level, marking the data to be processed with a corresponding classification and grading label.

[0014] The present application provides a data classification and grading system for ultra-short texts. The system includes an intelligent engine, a word vector library, a standard management module, a task management module, and a metadata management module that are respectively connected to the intelligent engine. The intelligent engine can also perform the following operations: First, the intelligent engine receives a classification and grading request sent by the task management module, and determines the corresponding data to be processed from the metadata management module based on the classification and grading request. Then, the intelligent engine loads the corresponding classification and grading standard from the standard management module based on the classification and grading request. After that, the intelligent engine uses the word vectors pre-loaded from the word vector library and multiple classification and grading standards to construct a classification and grading standard vector set. Further, the intelligent engine uses the word vectors pre-loaded from the word vector library and the field names of the data to be processed to construct a field vector matrix. Then, the intelligent engine matches each classification and grading standard vector matrix with the field vector matrix respectively, and determines whether there is a first candidate item with a matching degree greater than a first preset threshold. Finally, when it is determined that there is a first candidate item, the classification directory and security level corresponding to the first candidate item are determined, and based on the classification directory and security level, the data to be processed is marked with a corresponding classification and grading label.

[0015] Referring to the above content, the intelligent engine can more accurately determine the mapping relationship between the two based on the semantic information of the data to be processed (for example, including the field name and table name of the data to be processed) and the pre-set classification and grading standards, thereby determining the classification directory and security level of the data to be processed, and marking the data to be processed with corresponding classification and grading labels.

[0016] Furthermore, by using the above-mentioned data classification and grading system and data classification and grading method to perform semantic analysis (including context construction) on ultra-short texts, the intelligent engine can avoid the situation that the table name or field name in the data to be processed matches multiple different classification and grading standards during the matching process, making it impossible to classify and grade the data to be processed. This can further improve the matching accuracy. In addition, by using the above-mentioned data classification and grading system and data classification and grading method to disassemble the original words of the ultra-short text and construct the context, the intelligent engine can improve the matching coverage.

[0017] This solves the technical problems of low accuracy, low applicability and high cost that are common in the prior art data classification and grading methods for ultra-short texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0019] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to Embodiment 1 of the present application;

[0020] Figure 2 is a schematic diagram of a data classification and grading system for ultra-short texts according to Example 1 of the present application;

[0021] Figure 3 It is a flowchart of the data classification and grading method for ultra-short texts according to Example 1 of the present application;

[0022] Figure 4 This is an overall flow chart of determining the classification and grading labels of the data to be processed according to the intelligent engine described in Example 1 of the present application;

[0023] Figure 5 is a schematic diagram of a data classification and grading device for ultra-short texts according to Embodiment 2 of the present application;

[0024] Figure 6 It is a schematic diagram of the data classification and grading device for ultra-short texts according to Example 3 of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0027] Example 1

[0028] According to this embodiment, a method embodiment for data classification and grading of ultra-short texts is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] The method embodiment provided in this embodiment can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 The hardware structure block diagram of a computing device for implementing a method for classifying and grading data for ultra-short texts is shown. Figure 1 As shown, the computing device may include one or more processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, the transmission device, and the input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1More or fewer components as shown, or with Figure 1 Different configurations shown.

[0030] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computing device. As involved in the embodiments of the present disclosure, the data processing circuitry acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0031] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data classification and grading method for ultra-short texts in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the data classification and grading method for ultra-short texts of the above-mentioned application program is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0032] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computing device. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0033] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of the computing device.

[0034] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. Figure 1This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing devices described above.

[0035] Figure 2 Schematic diagram of the data classification and grading system for ultra-short text according to this embodiment. Figure 2 As shown, the system 10 includes: a word vector library 110, a standard management module 120, a metadata management module 130, an intelligent engine 140 and a task management module 150. Among them, the intelligent engine 140 is respectively connected to the word vector library 110, the standard management module 120, the task management module 150 and the metadata management module 130. And wherein, the intelligent engine 140 is bidirectionally connected to the task management module 150. In addition, the standard management module 120 is connected to the task management module 150. The metadata management module 130 is connected to the task management module 150.

[0036] The word vector library 110 stores a plurality of word vectors. By training a large amount of corpus, each word is represented in the form of a vector, so that each word corresponds to a unique vector value, thereby generating a plurality of word vectors.

[0037] The standard management module 120 forms a set of classification and grading standard tables according to safety regulations, industry specifications, and actual business of the enterprise (i.e., the set includes multiple different classification and grading standard tables). The classification and grading standard table includes a classification directory, a security word, a security level, and associated words. And the classification and grading standard table includes multiple classification and grading standards. The classification directory and the security word determine a unique classification and grading standard, each classification and grading standard corresponds to a security level, and each classification and grading standard includes multiple associated words.

[0038] Table 1

[0039]

[0040]

[0041] Table 1 shows a classification and grading standard table. Referring to Table 1, each classification and grading standard table includes multiple classification directories, for example, Table 1 includes classification directory 1 and classification directory 2. And each classification directory includes multiple safety words. The classification directory and any one of the multiple safety words determine a unique standard, for example, classification directory 1 and safety word 1 determine a unique standard. And each standard corresponds to a security level, for example, the standard determined by classification directory 1 and safety word 1 corresponds to security level 1. In addition, each standard corresponds to multiple associated words, for example, the standard determined by classification directory 1 and safety word 1 includes associated word 1 and associated word 2.

[0042] In addition, it should be noted that the above is only an illustrative example of the structure of any one classification and grading standard table in the classification and grading standard table set. The structures of other classification and grading standard tables in the classification and grading standard table can refer to the above Table 1, so the applicant will not elaborate on it here.

[0043] Furthermore, the contents of the safety word ID and the associated word ID will be described in detail later, so they will not be repeated here.

[0044] Table 2

[0045]

[0046]

[0047] Table 2 shows the classification and grading standard table corresponding to the classification catalog 1 in Table 1. Referring to Table 2, the classification catalog 1 includes multiple levels of classification. Among them, the classification catalog 1 may include, for example, a primary classification, a secondary classification, a tertiary classification, and a quaternary classification. It should be noted by those skilled in the art that Table 2 is only an example, and the actual classification level of the classification catalog is not limited thereto.

[0048] The metadata management module 130 includes metadata information such as the table name and field name of the database, and also includes corresponding data source connection information. The intelligent engine 140 obtains the data to be processed from the metadata management module 130 .

[0049] The intelligent engine 140 establishes a mapping relationship between the data to be processed (table name or field name) and multiple associated words and multiple security words in the classification and grading standard table through an intelligent algorithm, thereby determining the classification directory and security level of the data to be processed, and labeling the data to be processed with a classification and grading label.

[0050] In addition, since the amount of data assets is too large, it is impossible to process the classification and grading of all data assets at once, so it is necessary to classify and grade the data to be processed in a task manner according to the classification and grading work needs. Therefore, a task management table is set in the task management module 150.

[0051] Table 3

[0052]

[0053]

[0054] Table 3 shows the task management table. Referring to Table 3, the task management table includes a task list and a new task. The new task includes a task name, a task description, setting the type of data to be processed (i.e., table name or field name), selecting the object to be classified and graded (i.e., table name or field name), selecting the classification and grading algorithm, setting algorithm parameters, immediate execution or scheduled execution, manual confirmation and correction of the classification and grading results, and publishing the classification and grading results.

[0055] It should be noted that the intelligent engine 140, word vector library 110, standard management module 120, task management module 150 and metadata management module 130 in the system can all be applied to the hardware structure described above.

[0056] Under the above operating environment, according to the first aspect of this embodiment, a data classification and grading method for ultra-short text is provided. The method comprises: Figure 2 The system 10 shown in FIG. Figure 3 A schematic diagram showing the process of the method is shown in FIG. Figure 3 As shown, the method includes:

[0057] S302: receiving a classification and grading request sent by the task management module, and determining corresponding to-be-processed data from the metadata management module based on the classification and grading request, wherein the to-be-processed data includes field names;

[0058] S304: Based on the classification and grading request, load a corresponding classification and grading standard table from the standard management module, wherein the classification and grading standard table includes a plurality of classification and grading standards;

[0059] S306: constructing a classification and grading standard vector set by using the word vectors preloaded from the word vector library and a plurality of classification and grading standards, wherein the classification and grading standard vector set includes a plurality of classification and grading standard vector matrices;

[0060] S308: construct a field vector matrix using the word vectors pre-loaded from the word vector library and the field names of the data to be processed;

[0061] S310: Matching the field vector matrix with each classification and grading standard vector matrix respectively, and determining whether there is a first candidate item with a matching degree greater than a first preset threshold; and

[0062] S312: When it is determined that there is a first candidate item, determine the classification directory and security level corresponding to the first candidate item, and based on the classification directory and security level, label the data to be processed with a corresponding classification and grading label.

[0063] Specifically, first, the intelligent engine 140 starts the classification and grading service, receives the classification and grading request sent by the task management module 150, and determines the corresponding data to be processed from the metadata management module 130 based on the classification and grading request (S302). The data to be processed includes field names.

[0064] For example, the intelligent engine 140 receives a classification and grading request sent by the task management module 150, wherein the classification and grading request is used to indicate a request to classify and grade task 1. Thus, the intelligent engine 140 determines the data source connection information corresponding to task 1 from the metadata management module 130 according to the received classification and grading request corresponding to task 1, and obtains the to-be-processed data corresponding to task 1 from the corresponding data source.

[0065] Then, the intelligent engine 140 parses the received classification and grading request to determine the task number, the type of data to be processed (i.e., table name or field name), the type of classification and grading operation (i.e., classification, grading, or classification and grading), the classification and grading standard table, the classification and grading algorithm, the matching threshold (including the first matching threshold, the second matching threshold, and the third matching threshold), the method of obtaining the data to be processed, and the method of returning the classification and grading results. It is worth noting that since each data to be processed corresponds to a different task, the type of classification and grading operation required for each data to be processed is different, the classification and grading standard table used is also different, the matching threshold corresponding to each data to be processed is also different, and the method of obtaining the data to be processed and the method of returning the classification and grading results are also different.

[0066] Further, after parsing the classification and grading request and determining the classification and grading standard, the intelligent engine 140 loads the corresponding classification and grading standard table from the standard management module 120 (S304). The classification and grading standard table includes multiple classification and grading standards. Specifically, since the standard management module 120 includes a classification and grading standard table set (i.e., multiple different classification and grading standard tables), based on the classification and grading standard in the parsed classification and grading request, the intelligent engine 140 can determine which classification and grading standard table in the classification and grading standard table set to use.

[0067] Afterwards, the intelligent engine 140 uses the word vectors pre-loaded from the word vector library 110 and the multiple classification and grading standards in the classification and grading standard table to construct a classification and grading standard vector set (S306). Among them, the classification and grading standard vector set includes multiple classification and grading standard vector matrices. Specifically, the classification and grading standard vector set includes multiple classification and grading standard vector matrices. Among them, the classification and grading standard vector matrix is ​​composed of classification directory word vectors, safety word word vectors, and associated word vectors. And because the word vector library includes multiple word vectors, the intelligent engine 140 can pre-load the word vectors corresponding to the classification directory, word vectors, safety words, and associated words in the classification and grading standard table from the word vector library 110.

[0068] In addition, in the classification and grading standard table, the classification directory and the safety word determine a unique rule, there is a one-to-many relationship between the safety word and the associated word, and the associated words corresponding to each safety word are not repeated. However, in the entire classification and grading standard table (each classification and grading standard table includes multiple classification directories), different safety words may correspond to the same associated words (that is, the associated words are not necessarily unique in the classification and grading standard table). For example, the associated word "name" can be the name of the legal representative or the name of an individual customer, and can appear in both public and private business. Therefore, if only associated words are used for matching, there may be a problem of matching errors caused by multiple associated words. Therefore, in this application, unique IDs are also assigned to associated words and safety words. In other words, even if the same associated words appear in the same classification and grading standard table, since the associated words are assigned unique IDs, the IDs corresponding to the same associated words are also different, that is, the IDs of the associated words are globally unique and point to unique safety words.

[0069] Among them, the classification and grading standard vector set S1, for example, includes the classification and grading standard vector matrix D1, the classification and grading standard vector matrix D2, the classification and grading standard vector matrix D3, ..., the classification and grading standard vector matrix DN.

[0070] And among them, the classification and grading standard vector matrix D1 = {associated word 1 word vector, associated word 1ID, safe word 1 word vector, safe word 1ID, classified catalog word vectors at all levels}, the classification and grading standard vector matrix D2 = {associated word 2 word vector, associated word 2ID, safe word 1 word vector, safe word 1ID, classified catalog word vectors at all levels}, the classification and grading standard vector matrix D3 = {associated word 3 word vector, associated word 3ID, safe word 2 word vector, safe word 2ID, classified catalog word vectors at all levels}, ..., the classification and grading standard vector matrix DN = {associated word N word vector, associated NID, safe word M word vector, safe word MID, classified catalog word vectors at all levels}. Among them, M<N.

[0071] Further, the intelligent engine 140 constructs a field vector matrix (S308) using the word vectors pre-loaded from the word vector library 110 and the field names of the data to be processed. Specifically, the field vector matrix is ​​composed of the original word vector, the core word vector, the modifier word vector, and the segmentation word vector. And since the word vector library 110 includes multiple word vectors, the intelligent engine 140 can pre-load the word vectors corresponding to the original word, the core word, the modifier word, and the segmentation word from the word vector library 110. Among them, the original word is used to indicate the word of the field name itself, and the segmentation word is used to indicate the word generated by the segmentation technology for the field name.

[0072] And among them, for example, the core words and modifiers of the field name can also be identified and extracted through lexical and part-of-speech analysis technology. For example, the field name is "customer name", the core word is "name", and the modifier is "customer". For another example, the field name is "validity period of identity card", the core word is "validity period", and the modifier is "identity card". However, when extracting the core words and modifiers of the field name, not all field names can be extracted into the core words and modifiers. For example, the field name is "name", which is an atomic word and cannot extract modifiers.

[0073] Thus, based on the above steps, the field vector matrix can be determined. For example, the field vector matrix L = {original word vector, core word vector, modified word vector, segmentation 1 vector, segmentation 2 vector, ..., segmentation N vector}.

[0074] After determining the classification and grading standard vector set including multiple classification and grading standard vector matrices and the field vector matrix, the intelligent engine 140 matches the field vector matrix with each classification and grading standard vector matrix respectively, and determines whether there is a first candidate item with a matching degree greater than the first preset threshold (S310). Among them, there are three possible matching results. Matching result one, there is only one first candidate item with a matching degree greater than the first preset threshold. In this case, the intelligent engine 140 determines the classification directory and security level corresponding to the first candidate item, and based on the classification directory and security level, marks the corresponding classification and grading label on the data to be processed (S312). Matching result two, there is no first candidate item with a matching degree greater than the first preset threshold. In this case, the intelligent engine 140 determines that there is no classification directory and security level corresponding to the first candidate item, and marks the data to be processed with an unclassified and graded label. Matching result three, there are multiple first candidates greater than the first preset threshold. In this case, the intelligent engine 140 needs to further determine the classification directory and security level corresponding to the data to be processed.

[0075] Specifically, when the intelligent engine 140 determines multiple classification and grading standard vector matrices and field vector matrices, the cosine similarity algorithm is used to match the field vector matrix with each classification and grading standard vector matrix respectively.

[0076] Among them, the intelligent engine 140 performs matching according to the following steps: First, if the original word has a matching associated word, the associated word that matches the original word is given priority, and the remaining words (i.e., core words, modifiers, and participles) are no longer matched. For example, the original word is "legal representative name", associated word 1 is "legal representative name", and the original word "legal representative name" can just correspond to associated word 1, that is, the matching degree between the original word and associated word 1 is 100%, so the remaining words do not need to be matched. In the above case, there is a first candidate item with a matching degree greater than the first preset threshold. And because the associated word corresponds to a unique associated word ID, and the associated word ID points to a unique safe word, the above first candidate item is used to indicate the associated word that matches the original word in the classification and grading standard vector set and the classification and grading standard vector matrix corresponding to the associated word ID. That is, it corresponds to the above matching result one.

[0077] Then, in the absence of an associated word that matches the original word 100%, the core word is matched first. If the core word corresponds to a unique associated word, the security level corresponding to the associated word is taken. In the above case, there is a first candidate item with a matching degree greater than the first preset threshold. The same as the above is that since the associated word corresponds to a unique associated word ID, and the associated word ID points to a unique security word, the above first candidate item is used to indicate the associated word that matches the core word in the classification and grading standard vector set and the classification and grading standard vector matrix corresponding to the associated word ID. That is, it corresponds to the above matching result one.

[0078] In addition to the above situations, there may be a situation where a core word matches multiple associated words. For example, the field name is "validity period of identity card", where the core word is "validity period" and the modifier is "identity card". And referring to Table 1, the classification and grading standard table includes multiple classification directories, where classification directory 1 is personal natural information and classification directory 2 is business processing information. Furthermore, classification directory 1 corresponding to personal natural information includes the associated word "validity period", and classification directory 2 corresponding to business processing also includes the associated word "validity period".

[0079] Therefore, when the core word matches multiple associated words, it is necessary to further use modifiers for matching. For example, the modifier of the field name "validity period of ID card" is "ID card". The classification directories corresponding to the associated words matched by the modifier "ID card" are "personal natural information" and "executive customer information". Therefore, if the classification directories corresponding to the associated words matched with the modifier "ID card" include "personal natural information" and "executive customer information", the classification directories corresponding to the associated words matched with the core word "validity period" include "personal natural information" and "business processing information". Then take the intersection of the two to determine that the field name "validity period of ID card" corresponds to "personal natural information", and then determine the corresponding security level.

[0080] In the above case, there is only one first candidate item with a matching degree greater than the first preset threshold, and the first candidate item at this time is used to indicate the classification and grading standard matrix corresponding to the intersection of the associated words matching the core word and the classification catalog corresponding to the associated word ID, and the associated words matching the modifier word and the classification catalog corresponding to the associated word ID. That is, it corresponds to the above matching result one.

[0081] If there is no first candidate item with a matching degree greater than the first preset threshold, it means that the original word, core word and modifier cannot match the corresponding classification directory and security level, so in the above case, it is necessary to further use the segmentation to match. Further, if the original word, core word, modifier and segmentation cannot match the corresponding classification directory and security level, it means that there is no first candidate item with a matching degree greater than the first preset threshold, so the intelligent engine 140 determines that there is no classification directory and security level corresponding to the first candidate item, and marks the data to be processed with an unclassified label. That is, it corresponds to the above matching result 2.

[0082] However, if the category directory corresponding to "executive customer information" also includes the associated word "validity period", then there are two possibilities, namely "personal natural information" and "executive customer information". That is, there are multiple first candidates at this time. That is, it corresponds to the above matching result three.

[0083] In the case where there are multiple first candidate items, it is necessary to further construct a table vector matrix, and match each first candidate item with the table vector matrix respectively, so as to determine the second candidate item. Further, it is determined whether the matching degree of the second candidate item is greater than the second preset threshold. If the matching degree of the second candidate item is greater than the second preset threshold, the classification directory and security level corresponding to the second candidate item are determined, and the corresponding classification and grading label is marked on the data to be processed.

[0084] When the matching degree of the second candidate item is less than or equal to the second preset threshold, the intelligent engine 140 performs matching based on the target context vector matrix. That is, first, the intelligent engine 140 determines the field context vector matrix. Then the table context vector matrix is ​​determined. Further, based on the field context vector matrix and the table context vector matrix, the target context vector matrix is ​​determined. Finally, each first candidate item is matched with the target context vector matrix to determine the third candidate item, and when the matching degree of the third candidate item is greater than the third preset threshold, the classification directory and security level corresponding to the third candidate item are determined, and the data to be processed are marked with corresponding classification and grading labels. The above content will be described in detail later, so it will not be repeated here.

[0085] When the intelligent engine 140 determines the classification directory and security level corresponding to the data to be processed based on the above method, the classification directory and security level (i.e., classification and grading result) corresponding to the data to be processed are sent to the task management module 150. The task management module 150 stores the received classification directory and security level corresponding to the data to be processed in the task management table. At the same time, the intelligent engine 140 calls back the service interface corresponding to the requester, and notifies the requester of the storage location and access method of the classification and grading result of the data to be processed, so that the requester can process it in time.

[0086] As described in the background technology, the current data classification and grading work mainly focuses on the classification and grading of database tables and field names. Since the number of database tables and field names is very large, the workload is huge if the classification and grading is performed manually. Furthermore, since the database tables and field names only include a few words and belong to ultra-short texts, and the ultra-short texts themselves are very lacking in semantics, the accuracy of the classification and grading of ultra-short texts is generally not high.

[0087] In view of this, the present application provides a data classification and grading system for ultra-short texts. And referring to the above-mentioned content, the intelligent engine can more accurately determine the mapping relationship between the semantic information of the data to be processed (for example, including the field name and table of the data to be processed) and the pre-set classification and grading standards, thereby determining the classification directory and security level of the data to be processed, and marking the data to be processed with corresponding classification and grading labels.

[0088] Furthermore, by using the above data classification system and method to perform semantic analysis (including context construction) on ultra-short texts, the intelligent engine can avoid the situation that the table or field name in the data to be processed matches multiple different classification and grading standards, thereby failing to classify and grade the data to be processed. This can further improve the accuracy of matching. In addition, by using the above data classification system and method to disassemble the original words of the ultra-short text and construct the context, the intelligent engine can improve the matching coverage.

[0089] This solves the technical problems of low accuracy, low applicability and high cost that are common in the existing data classification and grading methods for ultra-short texts.

[0090] Optionally, the field vector matrix is ​​matched with each classification and grading standard vector matrix respectively, and an operation is performed to determine whether there is a first candidate item with a matching degree greater than a first preset threshold, including: when there is no first candidate item, determining that there is no classification directory and security level corresponding to the first candidate item, and marking the processed data with an unclassified and ungraded label; and when it is determined that there are multiple first candidate items, determining a second candidate item, and based on the classification directory and security level corresponding to the second candidate item, marking the processed data with a corresponding classification and grading label.

[0091] Specifically, Figure 4 This is an overall flow chart of determining the classification and grading labels of the data to be processed according to the intelligent engine described in the embodiment of the present application. Figure 4 As shown, referring to the above contents, it can be known that when the matching degree is greater than the first preset threshold, the classification directory and security level corresponding to the first candidate item can be directly determined. When there is no first candidate item with a matching degree greater than the first preset threshold, it is determined that there is no classification directory and security level corresponding to the first candidate item. That is, there is no classification directory and security level corresponding to the field name of the data to be processed in the corresponding classification and grading standard table.

[0092] In the case where there are multiple first candidate items with a matching degree greater than the first preset threshold, it is necessary to further determine the second candidate item, and based on the classification directory and security level corresponding to the second candidate item, the data to be processed is labeled with a corresponding classification and grading label. Among them, the second candidate item is used to indicate the first candidate item with the highest matching degree when each first candidate item is matched with the table vector matrix. The above content will be described in detail later, so it will not be repeated here.

[0093] Optionally, the data to be classified and graded includes a table name, and when it is determined that there are multiple first candidate items, a second candidate item is determined, and based on the classification directory and security level corresponding to the second candidate item, the operation of labeling the data to be processed with a classification and grading label is performed: based on multiple first candidate items, a corresponding candidate set is generated; a table vector matrix is ​​constructed using word vectors pre-loaded from a word vector library and the table name of the data to be processed; each first candidate item is matched with the table vector matrix respectively, and a second candidate item is determined, wherein the second candidate item is used to indicate the first candidate item with the highest matching degree with the table vector matrix; it is determined whether the matching degree of the second candidate item is greater than a second preset threshold; and when the matching degree of the second candidate item is greater than the second preset threshold, the classification directory and security level corresponding to the second candidate item are determined, and based on the classification directory and security level, the data to be processed is labeled with a corresponding classification and grading label.

[0094] Specifically, refer to Figure 4 As shown, when it is determined that there are multiple first candidate items whose matching degree is greater than the first preset threshold, the intelligent engine 140 generates a corresponding candidate set S based on the multiple first candidate items.

[0095] Then, the intelligent engine 140 constructs a table vector matrix using the word vectors pre-loaded from the word vector library 110 and the table name of the data to be processed. Among them, since the candidate set S includes multiple first candidates, it is necessary to further determine which first candidate in the candidate set S is selected. Therefore, the table name can be determined by the context of the field name, that is, the field name context information. The context of the field name includes the table to which it belongs, the corresponding business information, and the field name of the table to which it belongs. This embodiment takes the table information.

[0096] Thus, the intelligent engine 140 extracts the core words, modifiers and segmentation words of the table name in the data to be processed, and constructs the table vector matrix according to the above method of constructing the field name vector matrix. For example, the table vector matrix X = {original word vector, core word vector, modifier word vector, segmentation 1 vector, segmentation 2 vector, ..., segmentation M vector}.

[0097] Further, the intelligent engine 140 matches each first candidate with the table vector matrix respectively, and determines the second candidate. Among them, the second candidate is used to indicate the first candidate with the highest matching degree with the table vector matrix. Specifically, since the above field vector matrix has been matched with the associated word vector in the classification and grading standard vector matrix, and multiple first candidates have been determined. Therefore, the table vector matrix does not need to be matched with the associated word vector in the classification and grading standard vector matrix, and can be directly matched with the safe word in the classification and grading standard vector matrix.

[0098] Thus, the word vector of the safety word, the safety word ID corresponding to the safety word, and the word vector of each level of the directory are extracted from each first candidate item, thereby generating a plurality of fourth candidate items corresponding to each first candidate item. Among them, the fourth candidate item is a vector matrix. And among them, for example, the fourth candidate item A1 = {word vector of safety word 1, safety word 1ID, word vector of directory category at each level}, the fourth candidate item A2 = {word vector of safety word 1, safety word 1ID, word vector of directory category at each level}, ..., the fourth candidate item AM = {word vector of safety word M, safety word MID, word vector of directory category at each level}.

[0099] Furthermore, the intelligent engine 140 matches each fourth candidate item with the table vector matrix according to the cosine similarity algorithm and determines the second candidate item.

[0100] Then, the intelligent engine 140 determines whether the matching degree of the second candidate item is greater than a second preset threshold.

[0101] Finally, when the matching degree of the second candidate item is greater than the second preset threshold, the classification directory and security level corresponding to the second candidate item are determined, and the corresponding classification and grading label is marked for the data to be processed based on the classification directory and security level. For example, it has been determined that the matching degree of the second candidate item and the table vector matrix is ​​greater than the second preset threshold, and the second candidate item is the first candidate item corresponding to the fourth candidate item A2, so the classification directory and security level corresponding to the second candidate item can be determined, and then the classification and grading label is marked for the data to be processed.

[0102] Optionally, when the matching degree of the second candidate item is less than or equal to the second preset threshold, the intelligent engine is further configured to perform the following operations: determine the field context vector matrix based on the contextual relationship of the field name, and the field vector matrix and the table vector matrix; determine the table context vector matrix based on the contextual relationship of the table, and the field vector matrix and the system vector matrix, wherein the system vector matrix corresponds to the system to which the table belongs; determine the target context vector matrix based on the field context vector matrix and the table context vector matrix; match each first candidate item with the target context vector matrix respectively, and determine the third candidate item, wherein the third candidate item is used to indicate the first candidate item with the highest matching degree with the target context vector matrix; determine whether the matching degree of the third candidate item is greater than the third preset threshold; and when the matching degree of the third candidate item is greater than the third preset threshold, determine the classification directory and security level corresponding to the third candidate item, and based on the classification directory and security level, label the data to be processed with the corresponding classification and grading label. Further optionally, when the matching degree of the third candidate item is less than or equal to the third preset threshold, the intelligent engine is further configured to perform the following operations: determine that there is no classification directory and security level corresponding to the third candidate item, and label the data to be processed with no classification and grading label.

[0103] Specifically, refer to Figure 4 As shown in the figure, for a certain field, the context of the field name refers to other field names in the same table and the name of the table to which it belongs. For a certain table, the context of the table includes the corresponding field name and the name of the system to which it belongs.

[0104] Therefore, the intelligent engine 140 can determine the field context vector matrix H based on the context relationship corresponding to the field name. The specific calculation formula is as follows:

[0105]

[0106] Wherein, j = 1 to N. α is the weight value corresponding to the table vector matrix, and β is the weight value corresponding to the field vector matrix.

[0107] And the intelligent engine 140 can also determine the table context vector matrix K based on the context relationship corresponding to the table. The specific calculation formula is as follows:

[0108]

[0109] Where Y represents the system vector matrix of the system to which the table belongs.

[0110] Thus, the intelligent engine 140 may determine the target context vector matrix based on the field context vector matrix H and the table context vector matrix K. The intelligent engine 140 may, for example, merge the field context vector matrix H with the table context vector matrix K to generate the target context vector matrix.

[0111] Furthermore, the intelligent engine 140 matches the target context vector matrix with the associated word vectors in each first candidate item according to the cosine similarity algorithm, and determines a third candidate item, wherein the third candidate item is used to indicate the first candidate item with the highest matching degree with the target context vector matrix.

[0112] Then, the intelligent engine 140 determines whether the matching degree corresponding to the third candidate item is greater than the third preset threshold. If the matching degree corresponding to the third candidate item is greater than the third preset threshold, the classification directory and security level corresponding to the third candidate item are determined, and the corresponding classification and grading label is marked for the data to be processed based on the classification directory and security level; if the matching degree corresponding to the third candidate item is less than or equal to the third preset threshold, it means that there is no classification directory and security level corresponding to the data to be processed, and the data to be processed is marked with a label of no optimal classification directory and security level.

[0113] Optionally, the intelligent engine is also configured to perform the following operations: when determining the classification directory and security level corresponding to the data to be processed, call back the service interface corresponding to the requester, and notify the requester of the storage location and access method of the classification and grading results of the data to be processed.

[0114] Specifically, due to the large amount of data to be classified and graded, asynchronous execution is generally adopted. After the execution is completed, the requester service interface is called back and the classification and grading execution is completed, the location information of the classification and grading results and the access method are notified, so as to facilitate the requester to process in time.

[0115] Therefore, according to the first aspect of this embodiment, the technical effect of improving the accuracy and applicability of data classification and grading for ultra-short texts and reducing the cost is achieved.

[0116] In addition, reference Figure 1 As shown, according to the second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0117] Therefore, according to this embodiment, the technical effect of improving the accuracy and applicability of data classification and grading for ultra-short texts and reducing the cost is achieved.

[0118] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0120] Example 2

[0121] Figure 5FIG. 5 shows a data classification and grading device 500 for short texts according to this embodiment, which corresponds to the method according to embodiment 1. Figure 5 As shown, the device 500 includes: a to-be-processed data determination module 510, which is used to receive a classification and grading request sent by a task management module, and based on the classification and grading request, determine the corresponding to-be-processed data from the metadata management module, wherein the to-be-processed data includes a field name; a loading module 520, which is used to load the corresponding classification and grading standard table from the standard management module based on the classification and grading request, wherein the classification and grading standard table includes multiple classification and grading standards; a set construction module 530, which is used to construct a classification and grading standard vector set using the word vectors pre-loaded from the word vector library and the multiple classification and grading standards, wherein the classification and grading standard vector set includes Multiple classification and grading standard vector matrices; a field vector matrix construction module 540, used to construct a field vector matrix using word vectors pre-loaded from a word vector library and the field name of the data to be processed; a first matching module 550, matching the field vector matrix with each classification and grading standard vector matrix respectively, and determining whether there is a first candidate item with a matching degree greater than a first preset threshold; and a first label determination module 560, used to determine the classification directory and security level corresponding to the first candidate item when it is determined that there is a first candidate item, and based on the classification directory and the security level, label the data to be processed with a corresponding classification and grading label.

[0122] Optionally, the first matching module 550 includes: a second label determination module, which is used to determine that there is no classification directory and security level corresponding to the first candidate item when there is no first candidate item, and mark the data to be processed with an unclassified and graded label; and a third label determination module, which is used to determine the second candidate item when it is determined that there are multiple first candidate items, and mark the data to be processed with a corresponding classification and graded label based on the classification directory and security level corresponding to the second candidate item.

[0123] Optionally, the data to be classified and graded includes a table name, and the third label determination module includes: a candidate set generation module, which is used to generate a corresponding candidate set based on multiple first candidate items; a table vector matrix construction module, which is used to construct a table vector matrix using word vectors pre-loaded from a word vector library and the table name of the data to be processed; a second candidate item determination module, which is used to match each first candidate item with the table vector matrix respectively, and determine the second candidate item, wherein the second candidate item is used to indicate the first candidate item with the highest matching degree with the table vector matrix; a second matching module, which is used to determine whether the matching degree of the second candidate item is greater than a second preset threshold; and a third label determination submodule, which is used to determine the classification directory and security level corresponding to the second candidate item when the matching degree of the second candidate item is greater than the second preset threshold, and based on the classification directory and the security level, label the data to be processed with a corresponding classification and grading label.

[0124] Optionally, the third matching module includes: a field context vector matrix determination module, which is used to determine the field context vector matrix based on the context relationship of the field name, and the field vector matrix and the table vector matrix; a table context vector matrix determination module, which is used to determine the table context vector matrix based on the context relationship of the table, and the field vector matrix and the system vector matrix, wherein the system vector matrix corresponds to the system to which the table name belongs; a target context vector matrix determination module, which is used to determine the target context vector matrix based on the field context vector matrix and the table context vector matrix; a third candidate item determination module, which is used to match each first candidate item with the target context vector matrix respectively, and determine the third candidate item, wherein the third candidate item is used to indicate the first candidate item with the highest matching degree with the target context vector matrix; a third matching submodule, which is used to determine whether the matching degree of the third candidate item is greater than a third preset threshold; and a fourth label determination module, which is used to determine the classification directory and security level corresponding to the third candidate item when the matching degree of the third candidate item is greater than the third preset threshold, and based on the classification directory and the security level, label the data to be processed with a corresponding classification and grading label.

[0125] Optionally, the fourth matching module includes: a fifth label determination module, used to determine that there is no classification directory and security level corresponding to the third candidate item, and to mark the data to be processed with an unclassified label.

[0126] Optionally, the device 500 also includes: a callback module, which is used to call back the service interface corresponding to the requester when determining the classification directory and security level corresponding to the data to be processed, and notify the requester of the storage location and access method of the classification and grading results of the data to be processed.

[0127] Therefore, according to this embodiment, the technical effect of improving the accuracy and applicability of data classification and grading for ultra-short texts and reducing the cost is achieved.

[0128] Example 3

[0129] Figure 6 The data classification and grading device 600 for ultra-short text according to this embodiment is shown, and the device 600 corresponds to the method according to embodiment 1. Figure 6 As shown, the device 600 includes: a processor 610; and a memory 620, which is connected to the processor 610 and is used to provide the processor 610 with instructions for processing the following processing steps: receiving a classification and grading request sent by the task management module, and based on the classification and grading request, determining the corresponding data to be processed from the metadata management module, wherein the data to be processed includes a field name; based on the classification and grading request, loading the corresponding classification and grading standard table from the standard management module, wherein the classification and grading standard table includes multiple classification and grading standards; using the word vectors pre-loaded from the word vector library and the multiple classification and grading standards, constructing a classification and grading standard vector set, wherein the classification and grading standard vector set includes multiple classification and grading standard vector matrices; using the word vectors pre-loaded from the word vector library and the field name of the data to be processed, constructing a field vector matrix; matching the field vector matrix with each classification and grading standard vector matrix respectively, and judging whether there is a first candidate item with a matching degree greater than a first preset threshold; and in the case of determining that there is a first candidate item, determining the classification directory and security level corresponding to the first candidate item, and based on the classification directory and the security level, marking the corresponding classification and grading label on the data to be processed.

[0130] Optionally, the field vector matrix is ​​matched with each classification and grading standard vector matrix respectively, and an operation is performed to determine whether there is a first candidate item with a matching degree greater than a first preset threshold, including: when there is no first candidate item, determining that there is no classification directory and security level corresponding to the first candidate item, and marking the processed data with an unclassified and ungraded label; and when it is determined that there are multiple first candidate items, determining a second candidate item, and based on the classification directory and security level corresponding to the second candidate item, marking the processed data with a corresponding classification and grading label.

[0131] Optionally, the data to be classified and graded includes a table name, and when it is determined that there are multiple first candidate items, a second candidate item is determined, and based on the classification directory and security level corresponding to the second candidate item, the operation of labeling the data to be processed with a classification and grading label is performed: based on multiple first candidate items, a corresponding candidate set is generated; a table vector matrix is ​​constructed using word vectors pre-loaded from a word vector library and the table name of the data to be processed; each first candidate item is matched with the table vector matrix respectively, and a second candidate item is determined, wherein the second candidate item is used to indicate the first candidate item with the highest matching degree with the table vector matrix; it is determined whether the matching degree of the second candidate item is greater than a second preset threshold; and when the matching degree of the second candidate item is greater than the second preset threshold, the classification directory and security level corresponding to the second candidate item are determined, and based on the classification directory and security level, the data to be processed is labeled with a corresponding classification and grading label.

[0132] Optionally, when the matching degree of the second candidate item is less than or equal to the second preset threshold, the intelligent engine is further configured to perform the following operations: determine the field context vector matrix based on the contextual relationship of the field name, as well as the field vector matrix and the table vector matrix; determine the table context vector matrix based on the contextual relationship of the table, as well as the field vector matrix and the system vector matrix, wherein the system vector matrix corresponds to the system to which the table name belongs; determine the target context vector matrix based on the field context vector matrix and the table context vector matrix; match each first candidate item with the target context vector matrix respectively, and determine a third candidate item, wherein the third candidate item is used to indicate the first candidate item with the highest matching degree with the target context vector matrix; determine whether the matching degree of the third candidate item is greater than the third preset threshold; and when the matching degree of the third candidate item is greater than the third preset threshold, determine the classification directory and security level corresponding to the third candidate item, and based on the classification directory and the security level, label the data to be processed with a corresponding classification and grading label.

[0133] Optionally, when the matching degree of the third candidate item is less than or equal to the third preset threshold, the intelligent engine is further configured to perform the following operations: determine that there is no classification directory and security level corresponding to the third candidate item, and label the data to be processed as unclassified.

[0134] Optionally, the intelligent engine is also configured to perform the following operations: when determining the classification directory and security level corresponding to the data to be processed, call back the service interface corresponding to the requester, and notify the requester of the storage location and access method of the classification and grading results of the data to be processed.

[0135] Therefore, according to this embodiment, the technical effect of improving the accuracy and applicability of data classification and grading for ultra-short texts and reducing the cost is achieved.

[0136] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0137] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0141] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0142] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data classification and grading system for ultra-short texts, characterized in that: include: An intelligent engine, a word vector library, a standard management module, a task management module and a metadata management module respectively connected to the intelligent engine, wherein The intelligence engine is configured to perform the following operations: receiving a classification and grading request sent by the task management module, and determining corresponding data to be processed from the metadata management module based on the classification and grading request, wherein the data to be processed includes a field name; Based on the classification and grading request, loading a corresponding classification and grading standard table from the standard management module, wherein the classification and grading standard table includes a plurality of classification and grading standards; Using the word vectors preloaded from the word vector library and the plurality of classification and grading standards, construct a classification and grading standard vector set, wherein the classification and grading standard vector set includes a plurality of classification and grading standard vector matrices; Constructing a field vector matrix using the word vectors preloaded from the word vector library and the field names of the data to be processed; Matching the field vector matrix with each of the classification and grading standard vector matrices respectively, and determining whether there is a first candidate item with a matching degree greater than a first preset threshold; as well as When it is determined that there is a first candidate item, the classification directory and security level corresponding to the first candidate item are determined, and based on the classification directory and the security level, the data to be processed is labeled with a corresponding classification and grading label.

2. The system according to claim 1, characterized in that The operation of matching the field vector matrix with each of the classification and grading standard vector matrices respectively and determining whether there is a first candidate item with a matching degree greater than a first preset threshold includes: In the case that the first candidate item does not exist, determining that the classification directory and the security level corresponding to the first candidate item do not exist, and marking the data to be processed with an unclassified and graded label; and When it is determined that there are multiple first candidate items, a second candidate item is determined, and based on the classification directory and security level corresponding to the second candidate item, the data to be processed is labeled with a corresponding classification and grading label.

3. The system according to claim 2, characterized in that The data to be classified and graded includes a table name, and when it is determined that there are multiple first candidate items, a second candidate item is determined, and based on the classification directory and security level corresponding to the second candidate item, a classification and grading label is marked on the data to be processed: Based on the multiple first candidate items, generating a corresponding candidate set; Constructing a table vector matrix using the word vectors preloaded from the word vector library and the table name of the data to be processed; Matching each first candidate item with the table vector matrix respectively, and determining the second candidate item, wherein the second candidate item is used to indicate the first candidate item with the highest matching degree with the table vector matrix; Determining whether the matching degree of the second candidate item is greater than a second preset threshold; as well as When the matching degree of the second candidate item is greater than the second preset threshold, the classification directory and security level corresponding to the second candidate item are determined, and based on the classification directory and the security level, the data to be processed is marked with a corresponding classification and grading label.

4. The system according to claim 3, characterized in that When the matching degree of the second candidate item is less than or equal to the second preset threshold, the intelligent engine is further configured to perform the following operations: Determine a field context vector matrix based on the context relationship of the field name, the field vector matrix and the table vector matrix; Determine a table context vector matrix based on the context relationship of the table, the field vector matrix and the system vector matrix, wherein the system vector matrix corresponds to the system to which the table name belongs; Determining a target context vector matrix based on the field context vector matrix and the table context vector matrix; Matching each of the first candidate items with the target context vector matrix respectively, and determining a third candidate item, wherein the third candidate item is used to indicate the first candidate item with the highest matching degree with the target context vector matrix; Determining whether the matching degree of the third candidate item is greater than a third preset threshold; as well as When the matching degree of the third candidate item is greater than the third preset threshold, the classification directory and security level corresponding to the third candidate item are determined, and based on the classification directory and the security level, the data to be processed is marked with a corresponding classification and grading label.

5. The system according to claim 4, characterized in that When the matching degree of the third candidate item is less than or equal to the third preset threshold, the intelligent engine is further configured to perform the following operations: It is determined that there is no classification directory and security level corresponding to the third candidate item, and the data to be processed is marked with an unclassified label.

6. The system according to claim 1, characterized in that The intelligence engine is also configured to perform the following operations: When the classification directory and security level corresponding to the data to be processed are determined, the service interface corresponding to the requester is called back, and the storage location and access method of the classification and grading results of the data to be processed are notified to the requester.

7. A data classification and grading method for ultra-short texts, characterized in that: include: Receiving a classification and grading request sent by a task management module, and determining corresponding data to be processed from a metadata management module based on the classification and grading request, wherein the data to be processed includes a field name; Based on the classification and grading request, loading a corresponding classification and grading standard table from a standard management module, wherein the classification and grading standard table includes a plurality of classification and grading standards; Using the word vectors preloaded from the word vector library and the plurality of classification and grading standards, a classification and grading standard vector set is constructed, wherein the classification and grading standard vector set includes a plurality of classification and grading standard vector matrices; Constructing a field vector matrix using the word vectors preloaded from the word vector library and the field names of the data to be processed; Matching the field vector matrix with each of the classification and grading standard vector matrices respectively, and determining whether there is a first candidate item with a matching degree greater than a first preset threshold; as well as When it is determined that there is a first candidate item, the classification directory and security level corresponding to the first candidate item are determined, and based on the classification directory and the security level, the data to be processed is labeled with a corresponding classification and grading label.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 6.

9. A data classification and grading device for ultra-short texts, characterized in that: include: A to-be-processed data determination module, configured to receive a classification and grading request sent by a task management module, and determine corresponding to-be-processed data from a metadata management module based on the classification and grading request, wherein the to-be-processed data includes a field name; A loading module, configured to load a corresponding classification and grading standard table from a standard management module based on the classification and grading request, wherein the classification and grading standard table includes a plurality of classification and grading standards; A set construction module, used to construct a classification and grading standard vector set by using the word vectors preloaded from the word vector library and the plurality of classification and grading standards, wherein the classification and grading standard vector set includes a plurality of classification and grading standard vector matrices; A field vector matrix construction module, used to construct a field vector matrix using the word vectors pre-loaded from the word vector library and the field names of the data to be processed; A first matching module matches the field vector matrix with each of the classification and grading standard vector matrices, and determines whether there is a first candidate item with a matching degree greater than a first preset threshold; as well as The first label determination module is used to determine the classification directory and security level corresponding to the first candidate item when it is determined that there is a first candidate item, and to label the data to be processed with a corresponding classification and grading label based on the classification directory and the security level.

10. A data classification and grading device for ultra-short texts, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Receiving a classification and grading request sent by a task management module, and determining corresponding data to be processed from a metadata management module based on the classification and grading request, wherein the data to be processed includes a field name; Based on the classification and grading request, loading a corresponding classification and grading standard table from a standard management module, wherein the classification and grading standard table includes a plurality of classification and grading standards; Using the word vectors preloaded from the word vector library and the plurality of classification and grading standards, a classification and grading standard vector set is constructed, wherein the classification and grading standard vector set includes a plurality of classification and grading standard vector matrices; Constructing a field vector matrix using the word vectors preloaded from the word vector library and the field names of the data to be processed; Matching the field vector matrix with each of the classification and grading standard vector matrices respectively, and determining whether there is a first candidate item with a matching degree greater than a first preset threshold; as well as When it is determined that there is a first candidate item, the classification directory and security level corresponding to the first candidate item are determined, and based on the classification directory and the security level, the data to be processed is labeled with a corresponding classification and grading label.

Images