Data classification method, occupation category identification method, and data classification system

Abstract

Embodiments of the present specification provide a data classification method, a professional category identification method and a data classification system, wherein the data classification method comprises: performing keyword analysis on to-be-processed data to obtain at least one keyword in the to-be-processed data; searching for candidate category information and multi-level weights corresponding to each keyword through a pre-constructed keyword library, wherein the candidate category information comprises at least one candidate category, the keyword library comprises a plurality of sample keywords, sample category information and sample multi-level weights corresponding to each sample keyword, and the sample multi-level weights are weights of the sample keywords under different classification granularities; and filtering at least one candidate category corresponding to each keyword according to each keyword and the multi-level weights corresponding to each keyword to obtain a target category of the to-be-processed data. Since the weights are multi-level weights obtained according to different classification granularities, each keyword can be accurately represented, and thus the target category can be accurately obtained.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a data classification method. One or more embodiments of this specification also relate to a method for identifying occupational categories, a data classification system, a data classification device, an occupational category identification device, a computing device, a computer-readable storage medium, and a computer program. Background Technology

[0002] With the development of computer technology, data classification has gradually become a hot topic in the field of Natural Language Processing (NLP). Taking occupational category recognition as an example, constructing a scientific and accurate occupational classification system can objectively reflect the development and structural changes in the economic, social, and technological fields, provide a basis for the standardization of economic information statistics and population censuses, serve as the foundation for the scientific, standardized, and modern management of the workforce, and provide conditions for vocational education and training and employment services.

[0003] Currently, data is usually classified manually based on experience. However, with the explosive growth and frequent changes in data, manual classification has become extremely labor-intensive and inefficient. Furthermore, manual classification inevitably introduces a large number of uncertainties, resulting in poor accuracy of the classification results. Therefore, there is an urgent need for an efficient and accurate data classification scheme. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a data classification method. One or more embodiments of this specification also relate to a method for identifying occupational categories, a data classification system, a data classification device, an occupational category identification device, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a data classification method is provided, comprising:

[0006] Perform keyword analysis on the data to be processed to obtain at least one keyword in the data;

[0007] By using a pre-built keyword library, we can find the candidate category information and multi-level weights corresponding to each keyword. The candidate category information includes at least one candidate category. The keyword library includes multiple sample keywords, the sample category information corresponding to each sample keyword, and the multi-level weights of the samples. The multi-level weights of the samples are the weights of the sample keywords at different classification granularities.

[0008] Based on each keyword and its corresponding multi-level weights, at least one candidate category corresponding to each keyword is filtered to obtain the target category of the data to be processed.

[0009] According to a second aspect of the embodiments of this specification, a method for identifying occupational categories is provided, comprising:

[0010] Perform keyword analysis on the job description data to obtain at least one keyword from the job description data;

[0011] By using a pre-built keyword library, we can find the candidate category information and multi-level weights corresponding to each keyword. The candidate category information includes at least one candidate category. The keyword library includes multiple sample keywords, the sample category information corresponding to each sample keyword, and the multi-level weights of the samples. The multi-level weights of the samples are the weights of the sample keywords at different classification granularities.

[0012] Based on each keyword and its corresponding multi-level weights, at least one candidate category corresponding to each keyword is filtered to obtain the occupational category corresponding to the occupational description data.

[0013] According to a third aspect of the embodiments of this specification, a data classification system is provided, including a client and a server;

[0014] The client is used to send data classification requests to the server, whereby the data classification requests carry the data to be processed.

[0015] The server performs keyword analysis on the data to be processed to obtain at least one keyword. Using a pre-built keyword library, it searches for candidate category information and multi-level weights corresponding to each keyword. The candidate category information includes at least one candidate category, and the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and multi-level weights for each sample keyword. The multi-level weights represent the weights of the sample keywords at different classification granularities. Based on each keyword and its corresponding multi-level weights, it filters at least one candidate category for each keyword to obtain the target category of the data to be processed. Finally, it sends the target category of the data to be processed to the client.

[0016] The client is also used to receive the target category of the data to be processed.

[0017] According to a fourth aspect of the embodiments of this specification, a data classification apparatus is provided, comprising:

[0018] The first analysis module is configured to perform keyword analysis on the data to be processed to obtain at least one keyword in the data to be processed.

[0019] The first search module is configured to search for candidate category information and multi-level weights corresponding to each keyword through a pre-built keyword library. The candidate category information includes at least one candidate category, and the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights. The sample multi-level weights are the weights of the sample keywords at different classification granularities.

[0020] The first filtering module is configured to filter at least one candidate category corresponding to each keyword based on each keyword and the multi-level weights corresponding to each keyword, so as to obtain the target category of the data to be processed.

[0021] According to a fifth aspect of the embodiments of this specification, an occupational category identification device is provided, comprising:

[0022] The second analysis module is configured to perform keyword analysis on the occupational description data to obtain at least one keyword in the occupational description data.

[0023] The second search module is configured to search for candidate category information and multi-level weights corresponding to each keyword through a pre-built keyword library. The candidate category information includes at least one candidate category, and the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights. The sample multi-level weights are the weights of the sample keywords at different classification granularities.

[0024] The second filtering module is configured to filter at least one candidate category corresponding to each keyword based on each keyword and the multi-level weights corresponding to each keyword, so as to obtain the occupation category corresponding to the occupation description data.

[0025] According to a sixth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0026] Memory and processor;

[0027] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method provided in the first or second aspect above.

[0028] According to a seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the methods provided in the first or second aspect described above.

[0029] According to an eighth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the method provided in the first or second aspect described above.

[0030] This specification provides a data classification method in one embodiment, which involves keyword analysis of the data to be processed to obtain at least one keyword in the data; using a pre-built keyword library, searching for candidate category information and multi-level weights corresponding to each keyword, wherein the candidate category information includes at least one candidate category, the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights, where the sample multi-level weights are the weights of the sample keywords at different classification granularities; and filtering at least one candidate category corresponding to each keyword based on each keyword and its corresponding multi-level weights to obtain the target category of the data to be processed. By using a pre-built keyword library to classify the data, manual analysis costs are saved, and the efficiency of data classification is improved. Furthermore, since the weights corresponding to each keyword in the data to be processed are multi-level weights obtained according to different classification granularities, they can accurately represent each keyword. Therefore, by filtering at least one candidate category corresponding to each keyword based on each keyword and its corresponding multi-level weights, the target category of the data to be processed can be accurately obtained. Attached Figure Description

[0031] Figure 1 This is an architecture diagram of a data classification system provided in one embodiment of this specification;

[0032] Figure 2 This is a flowchart illustrating a data classification method provided in one embodiment of this specification;

[0033] Figure 3 This is a schematic diagram of a sample set in a data classification method provided in one embodiment of this specification;

[0034] Figure 4 This is a flowchart illustrating the processing of a data classification model in a data classification method provided in one embodiment of this specification.

[0035] Figure 5 This is a flowchart illustrating an occupational category identification method provided in one embodiment of this specification;

[0036] Figure 6 This is a flowchart illustrating the processing procedure of a data classification method provided in one embodiment of this specification;

[0037] Figure 7 This is a flowchart illustrating a method for identifying occupational categories according to one embodiment of this specification.

[0038] Figure 8 This is a schematic diagram of a data classification interface provided in one embodiment of this specification;

[0039] Figure 9This is a schematic diagram of the structure of a data classification device provided in one embodiment of this specification;

[0040] Figure 10 This is a schematic diagram of the structure of an occupational category identification device provided in one embodiment of this specification;

[0041] Figure 11 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0042] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0043] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0044] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0045] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0046] Stop words: Stop words are words that are automatically filtered out before or after processing natural language data in information retrieval to save storage space and improve search efficiency. These stop words are manually entered and not automatically generated, forming a stop word list.

[0047] BERT Model: BERT (Bidirectional Encoder Representation from Transformers) is a pre-trained language representation model. It emphasizes that instead of using traditional unidirectional language models or shallow concatenation of two unidirectional language models for pre-training, it employs a novel Masked Language Model (MLM) to generate deep bidirectional language representations.

[0048] ES (Elasticsearch) is an open-source, distributed, highly scalable, and real-time search and data analysis engine that makes it easy to search, analyze, and explore large amounts of data.

[0049] Inverse Document Frequency (IDF) is a measure of the general importance of a word. Its magnitude is inversely proportional to the frequency of a word.

[0050] Term frequency (TF) refers to the number of times a given word appears in a document.

[0051] With the development of computer technology, data classification has gradually become a hot topic in the field of natural language processing. Taking occupational category recognition as an example, constructing a scientific and accurate occupational classification system can objectively reflect the development and structural changes in the fields of economy, society, and science and technology, provide a basis for the standardization of economic information statistics and population census, serve as the foundation for the scientific, standardized, and modern management of the labor force, and provide conditions for vocational education and training and employment services.

[0052] Currently, occupational types can be determined in labor force surveys in the following ways: First, census takers manually enter occupational types. Second, relying on expert experience, occupational types are manually selected by analyzing keywords in occupational descriptions. Third, supervised training of a classifier using manually labeled data is used to classify occupations.

[0053] However, manual classification is labor-intensive and extremely inefficient. Furthermore, manual classification inevitably introduces a large number of uncertainties, resulting in poor accuracy of the classification results. Relying solely on supervised classifier schemes also requires a large number of experts to annotate the data. Due to the significant differences in occupational descriptions, and with limited expert annotation resources, it is difficult to build an accurate classification system in the short term.

[0054] To address the aforementioned issues, this specification provides a data classification method. The method involves performing keyword analysis on the data to be processed to obtain at least one keyword. Using a pre-built keyword library, candidate category information and multi-level weights corresponding to each keyword are retrieved. The candidate category information includes at least one candidate category, and the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and multi-level weights. The multi-level weights represent the weights of the sample keywords at different classification granularities. Based on each keyword and its corresponding multi-level weights, at least one candidate category corresponding to each keyword is filtered to obtain the target category of the data to be processed. By utilizing a pre-built keyword library for data classification, manual analysis costs are saved, and the efficiency of data classification is improved. Furthermore, since the weights corresponding to each keyword in the data to be processed are multi-level weights obtained according to different classification granularities, they can accurately represent each keyword. Therefore, when filtering at least one candidate category corresponding to each keyword based on its multi-level weights, the target category of the data to be processed is accurately obtained.

[0055] It should be noted that the embodiments in this specification may involve the use of user data. In practical applications, user-specific personal data may be used in the scheme described herein within the scope permitted by applicable laws and regulations, provided that it complies with the applicable laws and regulations of the country (e.g., with the user's explicit consent, with the user being properly notified, etc.).

[0056] This specification provides a data classification method, and also relates to an occupational category identification method, a data classification system, a data classification device, an occupational category identification device, a computing device, a computer-readable storage medium, and a computer program, which will be described in detail in the following embodiments.

[0057] See Figure 1 , Figure 1 This specification illustrates an architecture diagram of a data classification system provided in one embodiment of the specification. The data classification system includes a client 100 and a server 200.

[0058] Client 100 is used to send a data classification request to server 200, wherein the data classification request carries the data to be processed;

[0059] Server 200 is used to perform keyword analysis on the data to be processed, obtaining at least one keyword in the data; using a pre-built keyword library, it searches for candidate category information and multi-level weights corresponding to each keyword, where the candidate category information includes at least one candidate category, the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights, where the sample multi-level weights are the weights of sample keywords at different classification granularities; based on each keyword and its corresponding multi-level weights, it filters at least one candidate category corresponding to each keyword to obtain the target category of the data to be processed; and sends the target category of the data to be processed to client 100.

[0060] Client 100 is also used to receive the target category of the data to be processed.

[0061] The solution implemented in this specification saves on manual analysis costs and improves the efficiency of data classification by using a pre-built keyword library to classify data. Furthermore, since the weights corresponding to each keyword in the data to be processed are multi-level weights obtained according to different classification granularities, each keyword can be accurately represented. Thus, when filtering at least one candidate category corresponding to each keyword based on each keyword and its corresponding multi-level weights, the target category of the data to be processed can be accurately obtained.

[0062] In practical applications, a data classification system can have multiple clients 100. These clients 100 can establish communication connections through a server 200. In the data classification scenario, the server 200 provides data classification services among the multiple clients 100. Each client 100 can act as a sender or receiver, communicating through the server 200. Users can interact with the server 200 through clients 100 to receive data sent by other clients 100, or send data to other clients 100. In the data classification scenario, a user can publish a data stream to the server 200 through a client 100. The server 200 generates classification results based on this data stream and pushes the classification results to other communicating clients. The connection between clients 100 and server 200 is established through a network. The network provides the medium for the communication link between the clients and the server. The network can include various connection types, such as wired, wireless communication links, or fiber optic cables. The data transmitted by clients 100 may need to undergo encoding, transcoding, compression, or other processing before being published to the server 200.

[0063] It should be noted that client 100 can be a browser, an app (application), a web application such as an H5 (HyperText Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application, etc. Client 100 can be developed based on the software development kit (SDK) of the corresponding service provided by the server, such as based on a real-time communication (RTC) SDK. Client 100 can be deployed on electronic devices and depends on the device to run or on certain apps on the device. Electronic devices may have displays and support information browsing, such as personal mobile terminals such as mobile phones, tablets, and personal computers. Various other types of applications can also be configured on electronic devices, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0064] Server 200 may include servers providing various services, such as servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It should be noted that server 200 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0065] It is worth noting that the data classification method provided in the embodiments of this specification is generally executed by the server. However, in other embodiments of this specification, the client may also have similar functionality to the server, thereby executing the data classification method provided in the embodiments of this specification. In other embodiments, the data classification method provided in the embodiments of this specification may also be executed jointly by the client and the server.

[0066] See Figure 2 , Figure 2 This specification illustrates a flowchart of a data classification method according to an embodiment, which specifically includes the following steps:

[0067] Step 202: Perform keyword analysis on the data to be processed to obtain at least one keyword in the data.

[0068] In one or more embodiments of this specification, the data to be processed carried by the data classification request can be classified in response to the data classification request. When classifying the data to be processed, a word segmentation tool (such as Jieba) can be directly used to segment the data, and candidate category information and multi-level weights corresponding to each segmented word in the data to be processed can be searched in a pre-built keyword library. To reduce the amount of data processing and improve the efficiency of data classification, keyword analysis can also be performed on the data to be processed, stop words can be deleted, at least one keyword can be obtained from the data to be processed, and further data classification can be performed based on at least one keyword.

[0069] Specifically, the data to be processed is the object of processing in the data classification process. The data to be processed can be data from different fields, such as e-commerce, population statistics, education, etc. The data to be processed can be in various forms, such as text, images, audio, etc., selected according to the actual situation; this specification does not impose any limitations on this. For example, the data to be processed might be "I usually clean the house," where at least one keyword is "cleaning" or "sanitation."

[0070] It should be noted that if the data to be processed is audio, an audio conversion tool can be used to convert the audio to obtain the corresponding text. If the data to be processed is an image, optical character recognition (OCR) can be performed on the image to obtain the corresponding text. Alternatively, keyword analysis can be performed directly on the audio or image; the choice depends on the specific circumstances, and this specification does not impose any limitations on this approach.

[0071] In practical applications, there are multiple ways to perform keyword analysis on the data to be processed to obtain at least one keyword from the data. The specific method chosen depends on the actual situation, and the embodiments in this specification do not impose any limitations on this. One possible implementation of this specification is to match the data to be processed with a pre-set keyword list, and use the words that appear in both the data and the keyword list as keywords in the data to be processed. Another possible implementation of this specification is to input the data to be processed into a keyword recognition model (such as the BERT model), and obtain at least one keyword from the data through the keyword recognition model's processing.

[0072] Step 204: Using a pre-built keyword library, find the candidate category information and multi-level weights corresponding to each keyword. The candidate category information includes at least one candidate category. The keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights. The sample multi-level weights are the weights of the sample keywords at different classification granularities.

[0073] In one or more embodiments of this specification, after performing keyword analysis on the data to be processed to obtain at least one keyword in the data to be processed, the candidate category information and multi-level weights corresponding to each keyword can be found through a pre-built keyword library.

[0074] Specifically, a keyword database can also be called a keyword set. Candidate categories refer to the categories to which each keyword in the data to be processed belongs; a keyword can correspond to one candidate category or multiple candidate categories. Multi-level weights refer to the relative importance of indicators to keywords at different classification granularities.

[0075] It should be noted that there are multiple ways to find candidate category information and multi-level weights for each keyword using a pre-built keyword library. The specific method should be selected according to the actual situation, and this specification does not limit this approach. In one optional embodiment of this specification, the entire keyword library can be traversed to find candidate category information and multi-level weights for each keyword. In another possible implementation of this specification, the attributes of each keyword can be determined first, and candidate category information and multi-level weights can be found in the keyword library based on the attributes of each keyword. For example, if the attribute of "apple" is "food", then the candidate category information and multi-level weights corresponding to "apple" can be found among the multiple sample keywords under "food" in the keyword library.

[0076] In practical applications, the construction of the above keyword database can include the following steps:

[0077] Obtain a sample set, which includes multiple sample categories, each sample category includes multiple sample subcategories, the classification granularity of the sample subcategories is smaller than that of the sample categories, and each sample subcategories carry sample description text;

[0078] Extract at least one sample keyword corresponding to each sample subcategory from the sample description text carried by each sample subcategory;

[0079] Based on multiple sample categories, multiple sample subcategories included in each sample category, and at least one sample keyword corresponding to each sample subcategory, determine the multi-level weight of at least one sample keyword.

[0080] A keyword library is constructed based on multiple sample subcategories, at least one sample keyword corresponding to each sample subcategory, and the multi-level weight of at least one sample keyword.

[0081] In the embodiments of this specification, there are multiple ways to obtain the sample set, and the specific method should be selected according to the actual situation. This specification does not impose any limitations on this method. In one possible implementation, multi-level descriptive documents (such as occupational classification dictionaries) can be read from other data acquisition devices or databases, and these multi-level descriptive documents can be used as the sample set. In another possible implementation, a large number of multi-level descriptive documents can be manually input to form the sample set. After obtaining the sample set, at least one sample keyword corresponding to each sample sub-category can be extracted from the sample description text carried by each sample sub-category. The implementation method of this step is the same as the above-described method of "performing keyword analysis on the data to be processed to obtain at least one keyword in the data to be processed," and will not be described again in this specification.

[0082] In one optional embodiment of this specification, the multi-level weighting of samples includes a first sample weight and a second sample weight; the determination of the multi-level weighting of at least one sample keyword based on multiple sample categories, multiple sample subcategories included in each sample category, and at least one sample keyword corresponding to each sample subcategory may include the following steps:

[0083] For any given sample keyword, obtain the number of sample subcategories and the number of sample categories corresponding to that sample keyword;

[0084] The first sample weight of the keyword is determined based on the number of sample subcategories and the total number of sample subcategories in the sample set;

[0085] The second sample weight of the keyword is determined based on the number of sample categories and the total number of sample categories in the sample set.

[0086] In the embodiments of this specification, when determining the multi-level weight of at least one sample keyword, the weight of each sample keyword at different classification granularities can be calculated using the inverse text frequency index. The IDF of the current sample keyword is Log(D / i), where the number of categories of the current sample keyword is i, and the total number of categories is D.

[0087] It is worth noting that since this is a hierarchical classification, a sample keyword may belong to multiple categories at the same time. Therefore, the sample keyword corresponds to a list of categories (category information). However, these categories all belong to the same level. Therefore, the embodiments of this specification propose the concept of hierarchical IDF. For example, IDF values ​​are calculated for categories at the 3-bit coding level (2-01) and the 5-bit coding level (such as 2-01-01) respectively. For each sample keyword, it is defined as IDF3 and IDF5.

[0088] By applying the scheme of the embodiments of this specification, since the weights corresponding to each keyword in the data to be processed are multi-level weights obtained according to different classification granularities, each keyword can be accurately represented. Thus, when filtering multiple candidate categories according to category keywords and the multi-level weights of each category keyword, the target category of the data to be processed can be accurately obtained.

[0089] See Figure 3 , Figure 3This diagram illustrates a sample set in a data classification method provided in one embodiment of this specification. The sample set includes a first-level (major category) sample category "Professional and Technical Personnel" and its corresponding sample description text "Personnel engaged in scientific research and professional and technical work". This major category includes the following subcategories: 2-01 Scientific Researchers; 2-02 Engineering and Technical Personnel; 2-03 Agricultural Technicians; 2-04 Aircraft and Ship Technicians; 2-05 Health Professionals; 2-06 Economic and Financial Professionals; 2-07 Legal and Social Professionals; 2-08 Teaching Staff; 2-09 Literature, Art, and Sports Professionals; 2-10 News, Publishing, and Cultural Professionals; ...; 2-99 Other Professionals. It also includes a second-level "subcategory" sample category "2-01 Scientific Researchers" and its corresponding sample description text "Professionals engaged in social science and natural science research". This category includes the following subcategories: 2-01-01 Philosophy Researchers; 2-01-02 Economics Researchers; 2-01-03 Law Researchers; 2-01-04 Education Researchers; 2-01-05 History Researchers; 2-01-06 Natural and Earth Science Researchers; 2-01-07 Agricultural Science Researchers; 2-01-08 Medical Researchers; 2-01-09 Management Researchers; 2-01-10 Literature and Art Researchers; 2-01-99 Other Scientific Researchers; It also includes the third-level "subcategory" sample category "2-01-01 Philosophy Researchers" and its corresponding sample description text "Professionals engaged in the study of the general laws of nature, society, and thought." This subcategory includes the following occupations: 2-01-01-00 Philosophy Researchers; the sample description text for "2-01-01-00 Philosophy Researchers" is "Professionals engaged in the study of the general laws of nature, society, and thought." Their main work tasks include: ...

[0090] For example, for the category 2-01-01, the sample description text for this category is concatenated or punctuation is removed to organize the sample description text into a category-description pair format: "2-01-01 Philosophy Researcher: A professional engaged in the study of the general laws of nature, society, and thought. Philosophy Researcher. A professional engaged in the study of the general laws of nature, society, and thought. Main tasks: Research on ontology, epistemology, axiology, logic, ethics, aesthetics, psychology, atheism, philosophy of science and technology, and historical and philosophical views. Research on the history of Chinese philosophy and the history of Western philosophy. Comparative study of Eastern and Western philosophy." This is achieved using keyword extraction and summarization algorithms (TextRank) or the Term Frequency-Inverse Document Frequency (TF-IDF) index. Algorithms such as Frequency were used to extract keywords from the sample descriptive text, ultimately organizing them into a category-keyword list pair format: "2-01-01 Philosophy Researchers: Ethics, China, Historical View, Axiology, Research, Psychology, Professional, Philosophy, Worldview, Epistemology, Society, Ontology, Law, Thinking, General, Aesthetics, Science and Technology, History of Philosophy, Logic". The correspondence between the category-keyword list pairs was further reversed to obtain the keyword-category mapping relationship: "Ethics: 2-01-01; China: 1-01-00, 1-03-00, 1-04-01, 2-01-01; Historical View: 2-01-01; Axiology: 2-01-01; Research: 1-04-01, 2-01-01, 2-01-02, 2-01-03; ...". Finally, the multi-level weights of each word were calculated, and a keyword library was constructed. See Table 1 below, which shows an example keyword library.

[0091] Table 1 Keyword Database

[0092] Sample keywords Sample category information IDF3 IDF5 Sample word features K1 c1_1, c1_2, c1_3…… IDF3_1 IDF5_1 V1 K2 C2_1, c2_2, c2_3…… IDF3_2 IDF5_2 V2 K3 C3_1, c3_2, c3_3…… IDF3_3 IDF5_3 V3 …… …… …… …… ……

[0093] It should be noted that some keywords may belong to many categories, and these keywords do not have a clear category indication function. Therefore, we can count the number of categories corresponding to each keyword. If the number of categories for any keyword is greater than the category threshold, then filter the keyword and calculate the multi-level weight of each keyword to reduce the amount of data calculation.

[0094] The solution implemented in this specification is based on the multi-level weights of samples obtained by dividing samples according to different classification granularities. This can accurately represent the keywords of each sample, thereby accurately obtaining the target category of the data to be processed when filtering multiple candidate categories based on category keywords and the multi-level weights of each category keyword.

[0095] In one optional embodiment of this specification, the keyword library may not include certain keywords from the data to be processed; these keywords are called specified keywords. During data classification, these specified keywords can be ignored. Furthermore, to ensure the accuracy of data classification, sample keywords with high similarity to the specified keywords in the keyword library can be used to replace the specified keywords. That is, after finding the candidate category information and multi-level weights corresponding to each keyword through a pre-constructed keyword library, the following steps can be further included:

[0096] If the specified keyword is not included in the keyword library, the specified keyword is encoded to obtain the specified word features of the specified keyword.

[0097] Based on the specified word features and the sample word features of each sample keyword in the keyword library, select the specified sample keywords from the keyword library;

[0098] Replace specified keywords in the data to be processed with specified sample keywords, and find candidate category information and multi-level weights corresponding to the specified sample keywords through a pre-built keyword library.

[0099] In the embodiments of this specification, there are multiple ways to encode specified keywords to obtain specified word features. The specific method chosen depends on the actual situation, and this specification does not impose any limitations on this approach. One possible implementation of this specification is to use one-hot encoding to encode the specified keywords and obtain specified word features. Another possible implementation of this specification is to use an encoder (such as the encoder in BERT) to generate specified word features for the specified keywords.

[0100] It should be noted that before selecting specified sample keywords from the keyword database, the sample word features of each sample keyword in the keyword database can also be considered. That is, before selecting specified sample keywords from the keyword database based on the specified word features and the sample word features of each sample keyword in the keyword database, the following steps may also be included:

[0101] Obtain multiple sample keywords from the keyword library;

[0102] Encode multiple sample keywords to obtain the sample word features of each sample keyword.

[0103] In this embodiment of the specification, the implementation method of "encoding multiple sample keywords to obtain sample word features of each sample keyword" is the same as the implementation method of "encoding a specified keyword to obtain a specified word feature of the specified keyword" described above, and will not be repeated in this embodiment of the specification. As shown in Table 1 above, after obtaining the sample word features of each sample keyword, the sample word features can be added to the keyword library, and can be directly retrieved when word features are needed.

[0104] By applying the scheme of the embodiments in this specification, multiple sample keywords are encoded to obtain the sample word features of each sample keyword, which facilitates the subsequent screening of specified sample keywords based on specified word features and sample word features, thereby improving the accuracy of data classification.

[0105] Furthermore, after obtaining the specified word features of the designated keyword and the sample word features of each sample keyword, cosine similarity can be calculated, or vector retrieval engines such as Elasticsearch can be directly used to determine the words in each sample keyword that have a high similarity to the designated keyword. It should be noted that when selecting designated sample keywords from the keyword database, the top N sample keywords with the highest similarity to the designated keyword can be selected, and a similarity threshold can be set to include sample keywords with a similarity greater than the threshold as the designated sample keywords. In the subsequent calculation of category indicators, the designated sample keywords can be multiplied by a similarity weight during the calculation process.

[0106] For example, suppose the data to be processed is "consulting company + medical health + emotional counseling", and the keywords in the data are "consulting, medical health, emotional counseling". In the keyword library, "consulting, medical health, emotional counseling" can be found. At this time, the specified keyword in the data to be processed is determined to be "emotion". Then, using a vector retrieval engine, the sample keywords with a similarity to "emotion" exceeding the similarity threshold of 0.9 are found in the keyword library, such as "psychological (similarity 0.95)" and "spiritual (similarity 0.92)". "Psychological" is randomly selected as the specified sample keyword. The specified sample keyword is used to replace the specified keyword in the data to be processed. Through the pre-constructed keyword library, the candidate category information and multi-level weights corresponding to the specified sample keyword are found, thereby finding the candidate category information and multi-level weights corresponding to each keyword in the data to be processed.

[0107] By applying the scheme of the embodiments of this specification, based on the specified word features and the sample word features of each sample keyword in the keyword library, specified sample keywords are selected from the keyword library, and specified keywords in the data to be processed are replaced with specified sample keywords. Thus, in the data classification process, each keyword in the data to be processed can be taken into account, thereby making the data classification results more accurate.

[0108] In one optional embodiment of this specification, since many words are polysemous—for example, "apple" can be mapped to the agricultural category or the information technology category—it is necessary to combine the contextual semantics of the data to be processed in order to accurately classify the data. That is, after searching for candidate category information and multi-level weights corresponding to each keyword through a pre-built keyword library, the following steps may also be included:

[0109] Obtain the sample description text corresponding to at least one candidate category from the candidate category information;

[0110] Encode the sample description text corresponding to at least one candidate category to obtain the sample encoding features of each sample description text;

[0111] Encode the data to be processed to obtain its encoded features;

[0112] Based on the coding features and the sample coding features of each sample description text, candidate categories that do not meet the preset category filtering conditions are deleted from the candidate category information.

[0113] It should be noted that the methods of "encoding the sample description text corresponding to at least one candidate category to obtain the sample encoding features of each sample description text" and "encoding the data to be processed to obtain the encoding features of the data to be processed" are the same as the implementation method of "encoding the specified keyword to obtain the specified word features of the specified keyword" mentioned above, and will not be described again in the embodiments of this specification.

[0114] Furthermore, after obtaining the encoded features representing the contextual semantics of the data to be processed, the encoded features can be matched with the encoded features of the descriptive text of each sample. Candidate categories that do not meet the preset category filtering conditions can be deleted. The preset category filtering conditions can be a similarity greater than 0.8, and the specific selection is made according to the actual situation. This specification does not limit this in any way. For example, assuming the data to be processed is "growing apples", the information technology category has almost no weight here and can be directly deleted.

[0115] By applying the scheme of the embodiments of this specification, based on the encoding features and the sample encoding features of each sample description text, candidate categories that do not meet the preset category screening conditions are deleted from the candidate category information. This can incorporate the semantic information of the data to be processed into the data classification process, reduce the number of candidate categories, and further improve the efficiency and accuracy of data classification.

[0116] Step 206: Based on each keyword and its corresponding multi-level weight, filter at least one candidate category corresponding to each keyword to obtain the target category of the data to be processed.

[0117] In one or more embodiments of this specification, keyword analysis is performed on the data to be processed to obtain at least one keyword in the data to be processed. After searching for candidate category information and multi-level weights corresponding to each keyword through a pre-built keyword library, the candidate category corresponding to each keyword can be further filtered according to each keyword and the multi-level weights corresponding to each keyword to obtain the target category of the data to be processed.

[0118] The solution implemented in this specification saves on manual analysis costs and improves the efficiency of data classification by using a pre-built keyword library to classify data. Furthermore, since the weights corresponding to each keyword in the data to be processed are multi-level weights obtained according to different classification granularities, each keyword can be accurately represented. Thus, when filtering multiple candidate categories based on category keywords and the multi-level weights of each category keyword, the target category of the data to be processed can be accurately obtained.

[0119] In one optional embodiment of this specification, the above-mentioned filtering of at least one candidate category corresponding to each keyword based on each keyword and the multi-level weight corresponding to each keyword to obtain the target category of the data to be processed may include the following steps:

[0120] Based on each keyword and at least one candidate category corresponding to each keyword, determine at least one category keyword corresponding to each candidate category;

[0121] Based on at least one category keyword and at least one category keyword with multi-level weights, at least one candidate category corresponding to each keyword is filtered to obtain the target category of the data to be processed.

[0122] It should be noted that after obtaining the keywords and their corresponding candidate category information in the data to be processed, for any candidate category in the candidate category information, the keywords belonging to that candidate category are searched. These keywords are at least one category keyword corresponding to the candidate category.

[0123] For example, assuming that the candidate categories corresponding to keyword A are c1_1 and c1_2, and the candidate categories corresponding to keyword B are c1_2 and c1_3, then the category keyword corresponding to candidate category c1_1 is keyword A, the category keywords corresponding to candidate category c1_2 are keyword A and keyword B, and the category keyword corresponding to candidate category c1_3 is keyword B.

[0124] The scheme of the embodiments of this specification determines at least one category keyword corresponding to each candidate category based on each keyword and at least one candidate category corresponding to each keyword; and filters at least one candidate category corresponding to each keyword based on at least one category keyword and at least one category keyword multi-level weights to obtain the target category of the data to be processed, thereby improving the accuracy of the target category.

[0125] It should be noted that there are multiple ways to filter at least one candidate category corresponding to each keyword based on at least one category keyword and at least one category keyword's multi-level weight to obtain the target category of the data to be processed. The specific method should be selected according to the actual situation, and this specification does not limit this approach. In one optional embodiment of this specification, if multiple candidate categories are obtained through filtering, the category to which multiple candidate categories belong can be used as the target category of the data to be processed. In another optional embodiment of this specification, the category index value of each candidate category can also be calculated, and the target category can be filtered based on the category index value. That is, the above-mentioned filtering of at least one candidate category corresponding to each keyword based on at least one category keyword and at least one category keyword's multi-level weight to obtain the target category of the data to be processed may include the following steps:

[0126] For any candidate category, the category index value of the candidate category is determined based on at least one category keyword and the multi-level weight of at least one category keyword corresponding to the candidate category.

[0127] Based on the category index value of at least one candidate category corresponding to each keyword, the target category of the data to be processed is selected.

[0128] Specifically, the category index value of a candidate category represents the degree of correlation between the candidate category and the data to be processed. The larger the category index value, the higher the degree of correlation.

[0129] It should be noted that when determining the category index value of a candidate category based on category keywords and their multi-level weights, if there is only one category keyword corresponding to the candidate category, the category index of the candidate category can be obtained by multiplying the multi-level weights of the category keywords; if there are multiple category keywords corresponding to the candidate category, the category index can be obtained by multiplying the multi-level weights of each category keyword and then summing the results.

[0130] In practical applications, when selecting a target category from multiple candidate categories based on their category index values, the category indices can be sorted, and the target category for the data to be processed can be recommended from highest to lowest score. Alternatively, the target category can be randomly selected from candidate categories whose category index values ​​exceed the category index threshold.

[0131] For example, the data to be processed is "XX Catering Management Co., Ltd. + Catering Industry + Waiter / Waitress". The keywords and corresponding candidate category information in the data to be processed are "Catering: 4-02-01; Management: 1-02-01, 1-02-04; Waiter / Waitress: 4-02-01, 4-02-04; Serving: 4-03-02". The category index value of candidate category 4-03-02 = IDF3_Serving * IDF5_Serving, and the category index value of candidate category 4-02-01 = IDF3_Catering * IDF5_Catering + IDF3_Waiter * IDF5_Waiter.

[0132] By applying the scheme of the embodiments of this specification, the target category of the data to be processed is filtered according to the category index value of at least one candidate category corresponding to each keyword, thereby accurately determining the target category of the data to be processed from the candidate categories.

[0133] In one optional embodiment of this specification, after filtering at least one candidate category corresponding to each keyword based on at least one category keyword and at least one category keyword's multi-level weight to obtain the target category of the data to be processed, the following steps may be further included:

[0134] The keyword library is updated based on at least one keyword in the data to be processed and the target category of the data to be processed, resulting in an updated keyword library.

[0135] It should be noted that when updating the keyword library based on at least one keyword in the data to be processed and the target category of the data, data to be processed within the same category can be concatenated, such as "Target Category XXXX: Data to be Processed". The top M keywords can then be extracted as keywords for the current category, and the updated category-keyword list generated in this step can be returned, thus updating the keyword library. Alternatively, keywords can be sent to professionals for confirmation before updating the keyword library.

[0136] By applying the scheme of the embodiments of this specification, the keyword library is updated according to at least one keyword in the data to be processed and the target category of the data to be processed, and an updated keyword library is obtained, which makes the keyword library contain richer content and further improves the accuracy of data classification.

[0137] In one optional embodiment of this specification, to avoid the problems of low universality of a single category description caused by semantic diversity, incomplete keyword database, conceptual conflicts, and unsupervised feature matching, a data classification model can be trained through supervised training. This model is then used to classify low-confidence data. That is, after filtering at least one candidate category corresponding to each keyword based on at least one category keyword and its multi-level weights to obtain the target category of the data to be processed, the following steps may also be included:

[0138] The data classification model is trained using the data to be processed and the target category of the data to be processed, and the trained data classification model is obtained.

[0139] Specifically, data classification models are machine learning models, including but not limited to BERT and Convolutional Neural Networks (CNNs). Machine learning models can be understood as trained programs that can discover patterns and make predictions in new data. These models are represented as mathematical functions that receive requests in the form of input data, make predictions on the input data, and then provide outputs as responses.

[0140] It should be noted that when training the data classification model, a large amount of historical processing data can also be obtained. This historical processing data carries historical categories obtained through the above data classification methods. Professionals can also efficiently and accurately review and label the historical processing data and historical categories, and further utilize the historical processing data and historical categories, as well as the data to be processed and the target category, to conduct supervised training of the data classification model.

[0141] By applying the scheme of the embodiments in this specification, the data classification model is trained using the data to be processed and the target category of the data to be processed, and a trained data classification model is obtained, so that the data classification model can be directly used for data classification in the future, thereby improving the efficiency and accuracy of data classification.

[0142] In practical applications, data classification models include a decoding layer and multiple encoding layers. The above-mentioned method of training the data classification model using the data to be processed and its target category to obtain a trained data classification model can include the following steps:

[0143] For any given coding layer, the output of this layer is determined based on the data to be processed and the coding features output by the previous coding layer. The first coding layer takes the data to be processed as input, and the last coding layer outputs the fused coding features.

[0144] The fused encoded features are input into the decoding layer to obtain the predicted category corresponding to the data to be processed;

[0145] The data classification model is trained based on the predicted category and the target category of the data to be processed, resulting in a fully trained data classification model.

[0146] It should be noted that the target category of the data to be processed is the true label in the data classification process. When training the data classification model based on the predicted category and the target category of the data to be processed, the classification loss value can be calculated based on the predicted category and the target category. The parameters of the data classification model can be adjusted based on the classification loss value, and the data to be processed can be input into the data classification model to obtain the predicted category corresponding to the data to be processed, until the preset stopping condition is reached, and the trained data classification model is obtained.

[0147] In practical applications, there are many functions for calculating classification loss, such as cross-entropy loss, L1 norm loss, maximum loss, mean squared error loss, and logarithmic loss. The specific function to be selected depends on the actual situation, and this specification does not impose any limitations on this. The preset stopping condition includes the classification loss value being less than or equal to a first preset threshold. The first preset threshold is specifically selected based on the actual situation, and this specification does not impose any limitations on this. Specifically, if the classification loss value is greater than the first preset threshold, it indicates a large difference between the predicted category and the target category of the data to be processed, and the data classification model has poor predictive ability for the data to be processed. At this time, the model parameters of the data classification model can be adjusted, and the data to be processed can be input back into the data classification model to obtain the predicted category corresponding to the data to be processed. Training of the data classification model continues until the classification loss value is less than or equal to the first preset threshold, indicating a small difference between the predicted category and the target category of the data to be processed, thus reaching the preset stopping condition and obtaining a fully trained data classification model.

[0148] Furthermore, in addition to comparing the classification loss value with the first preset threshold, the number of iterations can also be used to determine whether the current data classification model has been trained successfully.

[0149] Specifically, if the classification loss value is greater than the first preset threshold, the parameters of the data classification model are adjusted, and the data to be processed is returned to be input into the data classification model to obtain the predicted category corresponding to the data to be processed. The data classification model is then trained again. When the first preset number of iterations is reached, the iteration is stopped, and the trained data classification model is obtained. The first preset number of iterations is selected according to the actual situation, and this embodiment does not limit it in any way.

[0150] See Figure 4 , Figure 4This document illustrates a flowchart of a data classification model in an embodiment of a data classification method provided in this specification. The data to be processed is input into the data classification model, where a hierarchical classification algorithm is used to process the data. For example, a 5-bit encoded feature depends on a 3-bit encoded feature determined in the previous layer, and a 3-bit encoded feature depends on a 1-bit encoded feature determined in the previous layer. Therefore, the FC3 encoding layer depends on the FC2 encoding layer, and the FC2 encoding layer depends on the FC1 encoding layer. The FC2 encoding layer can be obtained by concatenating the original encoding layer and the FC1 encoding layer and then mapping it through a simple neural network layer. Similarly, the FC3 encoding layer can be obtained by concatenating the original encoding layer and the FC2 encoding layer and then mapping it through a simple neural network layer. To avoid error propagation, the original vector of the data to be processed can be directly connected to the FC2 and FC3 encoding layers using direct connection channels.

[0151] By applying the solutions in the embodiments of this specification and utilizing a data classification model that includes a decoding layer and multiple encoding layers, different classification granularities can be considered during the data classification process, thereby further improving the accuracy of data classification.

[0152] See Figure 5 , Figure 5 This specification illustrates a flowchart of an occupational category identification method according to an embodiment, which specifically includes the following steps:

[0153] Step 502: Perform keyword analysis on the job description data to obtain at least one keyword from the job description data.

[0154] Step 504: Using a pre-built keyword library, find the candidate category information and multi-level weights corresponding to each keyword. The candidate category information includes at least one candidate category. The keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights. The sample multi-level weights are the weights of the sample keywords at different classification granularities.

[0155] Step 506: Based on each keyword and its corresponding multi-level weight, filter at least one candidate category corresponding to each keyword to obtain the occupational category corresponding to the occupational description data.

[0156] It should be noted that occupational description data refers to data used to describe occupations, such as occupational description text, occupational description images, occupational description audio, etc. The specific implementation methods of steps 502-506 are the same as described above. Figure 2 The implementation methods of the provided data classification methods are the same, so the embodiments in this specification will not be described in detail again.

[0157] The scheme described in this specification involves keyword analysis of occupational description data to obtain at least one keyword. A pre-built keyword library is used to search for candidate category information and multi-level weights corresponding to each keyword. The candidate category information includes at least one candidate category, and the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights. The sample multi-level weights represent the weights of the sample keywords at different classification granularities. Based on each keyword and its corresponding multi-level weights, at least one candidate category corresponding to each keyword is filtered to obtain the occupational category corresponding to the occupational description data. By using a pre-built keyword library to classify occupational description data, manual analysis costs are saved, and the efficiency of occupational description data classification is improved. Furthermore, since the weights corresponding to each keyword in the occupational description data are multi-level weights obtained according to different classification granularities, they can accurately represent each keyword. Therefore, when filtering at least one candidate category corresponding to each keyword based on its corresponding multi-level weights, the occupational category corresponding to the occupational description data is accurately obtained.

[0158] The following is in conjunction with the appendix Figure 6 Taking the application of the data classification method provided in this specification in a job category identification scenario as an example, the data classification method will be further explained. Figure 6 The present specification illustrates a flowchart of a data classification method according to an embodiment, which includes the following steps:

[0159] Step 602: Perform keyword analysis on the job description data to obtain at least one keyword from the job description data.

[0160] Step 604: Using a pre-built keyword library, find the candidate category information and multi-level weights corresponding to each keyword.

[0161] Step 606: Obtain the sample description text corresponding to at least one candidate category from the candidate category information.

[0162] Step 608: Encode the sample description text corresponding to at least one candidate category to obtain the sample encoding features of each sample description text.

[0163] Step 610: Encode the occupational description data to obtain the encoded features of the occupational description data.

[0164] Step 612: Based on the coding features and the sample coding features of each sample description text, delete the candidate categories in the candidate category information that do not meet the preset category filtering conditions.

[0165] Step 614: Based on each keyword and at least one candidate category corresponding to each keyword, determine at least one category keyword corresponding to each candidate category.

[0166] Step 616: For any candidate category, determine the category index value of the candidate category based on at least one category keyword and the multi-level weight of at least one category keyword corresponding to the candidate category.

[0167] Step 618: Based on the category index value of at least one candidate category corresponding to each keyword, filter the occupational categories corresponding to the occupational description data.

[0168] It should be noted that the specific implementation methods of steps 602-618 are the same as those described above. Figure 2 The implementation methods of the provided data classification methods are the same, so the embodiments in this specification will not be described in detail again.

[0169] The solution implemented in this specification saves on manual analysis costs and improves the efficiency of data classification by using a pre-built keyword library to classify data. Furthermore, since the weights corresponding to each keyword in the occupational description data are multi-level weights obtained by dividing according to different classification granularities, each keyword can be accurately represented. Thus, when filtering at least one candidate category corresponding to each keyword based on each keyword and its corresponding multi-level weights, the occupational category corresponding to the occupational description data can be accurately obtained.

[0170] It is worth noting that the data classification method provided in the embodiments of this specification can be applied to occupational category identification scenarios, such as in labor force population surveys and household migrant worker surveys, to determine the occupational type of the survey subjects. See also Figure 7 , Figure 7This document illustrates a flowchart of a method for identifying occupational categories according to an embodiment of this specification. The method involves inputting occupational description data (e.g., company name + industry + job details) from survey respondents into a keyword extraction model to obtain at least one keyword (e.g., withdrawal, agency, summary) from the occupational description data. Sample description data from a positive occupational keyword library and the keywords from the occupational description data are then input into a vector generation model to obtain sample description features and occupational description features. The sample description data and occupational description features are input into a semantic matching engine to determine the first candidate occupational category corresponding to the occupational description data (e.g., 40500 financial service personnel; 40501 banking service personnel; 40502 securities service personnel; 40503 insurance service personnel) through positive keywords. Simultaneously, positive keywords can be summarized and added to the positive occupational keyword library or a new occupational keyword library after manual review. Furthermore, an additional occupational knowledge base constructed through the keyword extraction model and the occupational description data of the survey respondents can be input into an occupational classification model to determine the occupational category of the occupational description data. Then, the positive keywords and occupational categories from the occupational description data can be combined to obtain second candidate occupational categories (e.g., <Withdrawal, 40501 Bank Service Personnel>, <Agent, 40500 Financial Service Personnel>). Next, negative keyword records in the first and second candidate occupational categories are filtered out. The first and second candidate occupational categories, along with the new occupational keyword library, are input into the project rule engine, integrated with the actual project requirements, and the combined output classification results corresponding to the occupational description data of the survey respondents are "40501 Bank Service Personnel, 40500 Financial Service Personnel".

[0171] It should be noted that the additional occupational knowledge base is a retrieval database existing in the historical standard occupational categories. This database is not necessarily composed of keywords; it could be a description or a pre-coded vector library. The occupational positive keyword database is a database of keywords corresponding to occupations existing in the historical standard occupational categories. The new occupational keyword database is a thesaurus built from keywords that may become new occupations discovered in new datasets and are not present in the historical standard occupational categories. Positive keywords are those that, if they appear in a description, are very likely to belong to the corresponding category. Negative keywords are those that, if they appear in a description, definitely do not belong to a certain category.

[0172] Applying the solutions described in the embodiments of this specification, this specification proposes a multi-angle data classification scheme that combines a keyword library, vector matching, and supervised model classification. This scheme allows for rapid classification using an initial keyword library, followed by more efficient annotation to achieve more accurate supervised classification. After accurate classification, the keyword library can be iteratively optimized to achieve an efficient classification scheme in a short time. Furthermore, this specification proposes a scheme to process the sample set into a keyword library. Hierarchical keyword extraction yields more effective weighted representations of keywords at different levels and categories. The combination of precise and fuzzy vector matching allows for more effective data classification using the constructed keyword library.

[0173] See Figure 8 , Figure 8 This diagram illustrates a data classification interface according to an embodiment of this specification. The data classification interface is divided into a data input interface and a classification result display interface. The data input interface includes a data input box, an "OK" control, and a "Cancel" control. The classification result display interface includes a classification result display box. The user inputs data to be processed through the data input box displayed on the client and clicks the "OK" control. The server receives the data to be processed from the client, performs keyword analysis on the data to obtain at least one keyword; uses a pre-built keyword library to search for candidate category information and multi-level weights corresponding to each keyword; determines at least one category keyword corresponding to each candidate category based on each keyword and its corresponding candidate category information; filters multiple candidate categories corresponding to the data to be processed based on the at least one category keyword and its multi-level weights to obtain the target category of the data to be processed; and sends the target category of the data to be processed to the client so that the client displays the target category of the data to be processed in the classification result display box.

[0174] In practical applications, users can interact with controls in various ways, including clicking, double-clicking, touching, hovering, swiping, long-pressing, voice control, or shaking. The specific method chosen depends on the actual situation, and this specification does not impose any limitations on this method.

[0175] Corresponding to the above method embodiments, this specification also provides embodiments of a data classification device. Figure 9 A schematic diagram of a data classification device according to one embodiment of this specification is shown. Figure 9 As shown, the device includes:

[0176] The first analysis module 902 is configured to perform keyword analysis on the data to be processed to obtain at least one keyword in the data to be processed.

[0177] The first search module 904 is configured to search for candidate category information and multi-level weights corresponding to each keyword through a pre-built keyword library. The candidate category information includes at least one candidate category, and the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights. The sample multi-level weights are the weights of the sample keywords at different classification granularities.

[0178] The first filtering module 906 is configured to filter at least one candidate category corresponding to each keyword based on each keyword and the multi-level weights corresponding to each keyword, so as to obtain the target category of the data to be processed.

[0179] Optionally, the device further includes: a keyword library construction module configured to acquire a sample set, wherein the sample set includes multiple sample categories, each sample category includes multiple sample subcategories, the classification granularity of the sample subcategories is smaller than that of the sample categories, and each sample subcategory carries sample description text; extract at least one sample keyword corresponding to each sample subcategory from the sample description text carried by each sample subcategory; determine the multi-level sample weight of at least one sample keyword based on the multiple sample categories, the multiple sample subcategories included in each sample category, and the at least one sample keyword corresponding to each sample subcategory; and construct a keyword library based on the multiple sample subcategories, the at least one sample keyword corresponding to each sample subcategory, and the multi-level sample weight of the at least one sample keyword.

[0180] Optionally, the multi-level weighting of samples includes a first sample weight and a second sample weight; the keyword library construction module is further configured to, for any sample keyword, obtain the number of sample subcategories corresponding to the sample keyword and the number of sample categories; determine the first sample weight of the sample keyword based on the number of sample subcategories and the total number of sample subcategories in the sample set; and determine the second sample weight of the sample keyword based on the number of sample categories and the total number of sample categories in the sample set.

[0181] Optionally, the device further includes: an encoding module configured to encode the specified keywords in the data to be processed when the specified keywords are not included in the keyword library, thereby obtaining the specified word features of the specified keywords; to filter the specified sample keywords from the keyword library based on the specified word features and the sample word features of each sample keyword in the keyword library; to replace the specified keywords in the data to be processed with the specified sample keywords, and to find the candidate category information and multi-level weights corresponding to the specified sample keywords through a pre-built keyword library.

[0182] Optionally, the device further includes: a deletion module configured to obtain sample description text corresponding to at least one candidate category in the candidate category information; encode the sample description text corresponding to at least one candidate category to obtain sample encoding features of each sample description text; encode the data to be processed to obtain encoding features of the data to be processed; and delete candidate categories in the candidate category information that do not meet the preset category screening conditions based on the encoding features and the sample encoding features of each sample description text.

[0183] Optionally, the first filtering module 906 is further configured to determine at least one category keyword corresponding to each candidate category based on each keyword and at least one candidate category corresponding to each keyword; and to filter at least one candidate category corresponding to each keyword based on at least one category keyword and at least one category keyword's multi-level weight to obtain the target category of the data to be processed.

[0184] Optionally, the first filtering module 906 is further configured to, for any candidate category, determine the category index value of the candidate category based on at least one category keyword and the multi-level weight of at least one category keyword; and filter the target category of the data to be processed based on the category index value of at least one candidate category corresponding to each keyword.

[0185] Optionally, the device further includes a training module configured to train a data classification model using the data to be processed and the target category of the data to be processed, thereby obtaining a trained data classification model.

[0186] Optionally, the data classification model includes a decoding layer and multiple encoding layers; the training module is further configured to determine the output of any encoding layer based on the data to be processed and the encoding features output by the previous encoding layer, wherein the first encoding layer takes the data to be processed as input and the last encoding layer outputs fused encoding features; the fused encoding features are input into the decoding layer to obtain the predicted category corresponding to the data to be processed; the data classification model is trained according to the predicted category and the target category of the data to be processed to obtain the trained data classification model.

[0187] Optionally, the device further includes an update module configured to update the keyword library based on at least one keyword in the data to be processed and the target category of the data to be processed, thereby obtaining an updated keyword library.

[0188] The scheme described in this specification involves keyword analysis of the data to be processed to obtain at least one keyword. A pre-built keyword library is used to search for candidate category information and multi-level weights corresponding to each keyword. The candidate category information includes at least one candidate category, and the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and multi-level weights. The multi-level weights represent the weights of the sample keywords at different classification granularities. Based on each keyword and its corresponding multi-level weights, at least one candidate category corresponding to each keyword is filtered to obtain the target category of the data to be processed. By using a pre-built keyword library to classify the data, manual analysis costs are saved, and the efficiency of data classification is improved. Furthermore, since the weights corresponding to each keyword in the data to be processed are multi-level weights obtained according to different classification granularities, they can accurately represent each keyword. Therefore, when filtering at least one candidate category corresponding to each keyword based on its multi-level weights, the target category of the data to be processed is accurately obtained.

[0189] The above is an illustrative scheme of a data classification device according to this embodiment. It should be noted that the technical solution of this data classification device and the technical solution of the data classification method described above belong to the same concept. For details not described in detail in the technical solution of the data classification device, please refer to the description of the technical solution of the data classification method described above.

[0190] Corresponding to the above method embodiments, this specification also provides embodiments of occupational category identification devices. Figure 10 A schematic diagram of the structure of an occupational category identification device provided in one embodiment of this specification is shown. Figure 10 As shown, the device includes:

[0191] The second analysis module 1002 is configured to perform keyword analysis on the occupational description data to obtain at least one keyword in the occupational description data.

[0192] The second search module 1004 is configured to search for candidate category information and multi-level weights corresponding to each keyword through a pre-built keyword library. The candidate category information includes at least one candidate category, and the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights. The sample multi-level weights are the weights of the sample keywords at different classification granularities.

[0193] The second filtering module 1006 is configured to filter at least one candidate category corresponding to each keyword based on each keyword and the multi-level weights corresponding to each keyword, so as to obtain the occupation category corresponding to the occupation description data.

[0194] The scheme described in this specification involves keyword analysis of occupational description data to obtain at least one keyword. A pre-built keyword library is used to search for candidate category information and multi-level weights corresponding to each keyword. The candidate category information includes at least one candidate category, and the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights. The sample multi-level weights represent the weights of the sample keywords at different classification granularities. Based on each keyword and its corresponding multi-level weights, at least one candidate category corresponding to each keyword is filtered to obtain the occupational category corresponding to the occupational description data. By using a pre-built keyword library to classify occupational description data, manual analysis costs are saved, and the efficiency of occupational description data classification is improved. Furthermore, since the weights corresponding to each keyword in the occupational description data are multi-level weights obtained according to different classification granularities, they can accurately represent each keyword. Therefore, when filtering at least one candidate category corresponding to each keyword based on its corresponding multi-level weights, the occupational category corresponding to the occupational description data is accurately obtained.

[0195] The above is a schematic scheme of an occupational category identification device according to this embodiment. It should be noted that the technical solution of this occupational category identification device and the technical solution of the occupational category identification method described above belong to the same concept. For details not described in detail in the technical solution of the occupational category identification device, please refer to the description of the technical solution of the occupational category identification method described above.

[0196] Figure 11 A structural block diagram of a computing device according to one embodiment of this specification is shown. The components of the computing device 1100 include, but are not limited to, a memory 1110 and a processor 1120. The processor 1120 is connected to the memory 1110 via a bus 1130, and a database 1150 is used to store data.

[0197] The computing device 1100 also includes an access device 1140, which enables the computing device 1100 to communicate via one or more networks 1160. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1140 may include one or more of any type of wired or wireless network interface (e.g., Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Networks (WLAN) interface, a Wi-MAX (World Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0198] In one embodiment of this specification, the aforementioned components of the computing device 1100 and Figure 11 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 11 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0199] The computing device 1100 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1100 can also be a mobile or stationary server.

[0200] The processor 1120 is used to execute computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned data classification method or occupational category identification method.

[0201] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solutions of the data classification method and the occupational category identification method described above. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solutions of the data classification method or the occupational category identification method described above.

[0202] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described data classification method or occupational category identification method.

[0203] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solutions of the data classification method and the occupational category identification method described above. Details not described in detail in the technical solution of the storage medium can be found in the descriptions of the technical solutions of the data classification method or the occupational category identification method described above.

[0204] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described data classification method or occupational category identification method.

[0205] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solutions of the data classification method and the occupational category identification method described above. Details not described in detail in the technical solution of the computer program can be found in the descriptions of the technical solutions of the data classification method or the occupational category identification method described above.

[0206] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0207] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0208] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0209] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0210] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data classification method, comprising: Perform keyword analysis on the data to be processed to obtain at least one keyword in the data; By using a pre-built keyword library, candidate category information and multi-level weights corresponding to each keyword are found. The candidate category information includes at least one candidate category. The keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights. The sample multi-level weights are the weights of the sample keywords at different classification granularities. The sample multi-level weights are obtained based on multiple sample categories, multiple sample subcategories included in each sample category, and at least one sample keyword corresponding to each sample subcategory. The at least one sample keyword corresponding to each sample subcategory is extracted from the sample description text carried by each sample subcategory. Based on each keyword and its corresponding multi-level weight, at least one candidate category corresponding to each keyword is filtered to obtain the target category of the data to be processed.

2. The method according to claim 1, wherein the keyword library is constructed in a manner comprising: Obtain a sample set, wherein the sample set includes multiple sample categories, each sample category includes multiple sample subcategories, the classification granularity of each sample subcategory is smaller than that of the sample category, and each sample subcategory carries sample description text; Extract at least one sample keyword corresponding to each sample subcategory from the sample description text carried by each sample subcategory; Based on the multiple sample categories, the multiple sample subcategories included in each sample category, and at least one sample keyword corresponding to each sample subcategory, determine the multi-level weight of the at least one sample keyword; A keyword library is constructed based on the multiple sample subcategories, at least one sample keyword corresponding to each sample subcategory, and the multi-level weight of the at least one sample keyword.

3. The method according to claim 2, wherein the multi-level sample weights include a first sample weight and a second sample weight; The step of determining the multi-level weight of the at least one sample keyword based on the multiple sample categories, the multiple sample subcategories included in each sample category, and at least one sample keyword corresponding to each sample subcategory includes: For any given sample keyword, obtain the number of sample subcategories and the number of sample categories corresponding to that sample keyword; The first sample weight of the keyword is determined based on the number of sample subcategories and the total number of sample subcategories in the sample set; The second sample weight of the sample keyword is determined based on the number of sample categories and the total number of sample categories in the sample set.

4. The method according to claim 1, further comprising, after searching for candidate category information and multi-level weights corresponding to each keyword through a pre-constructed keyword library: If the specified keyword is not included in the keyword library, the specified keyword is encoded to obtain the specified word features of the specified keyword. Based on the specified word features and the sample word features of each sample keyword in the keyword library, the specified sample keywords are selected from the keyword library; The specified keywords in the data to be processed are replaced with the specified sample keywords, and the candidate category information and multi-level weights corresponding to the specified sample keywords are found through a pre-built keyword library.

5. The method according to claim 1, further comprising, after searching for candidate category information and multi-level weights corresponding to each keyword through a pre-constructed keyword library: Obtain the sample description text corresponding to at least one candidate category from the candidate category information; The sample description text corresponding to the at least one candidate category is encoded to obtain the sample encoding features of each sample description text; The data to be processed is encoded to obtain the encoded features of the data to be processed; Based on the encoding features and the sample encoding features of each sample description text, candidate categories that do not meet the preset category filtering conditions are deleted from the candidate category information.

6. The method according to any one of claims 1-5, wherein filtering at least one candidate category corresponding to each keyword based on each keyword and the multi-level weight corresponding to each keyword to obtain the target category of the data to be processed includes: Based on each keyword and at least one candidate category corresponding to each keyword, at least one category keyword corresponding to each candidate category is determined; Based on the at least one category keyword and the multi-level weight of the at least one category keyword, at least one candidate category corresponding to each keyword is filtered to obtain the target category of the data to be processed.

7. The method according to claim 6, wherein filtering at least one candidate category corresponding to each keyword based on the at least one category keyword and the multi-level weight of the at least one category keyword to obtain the target category of the data to be processed includes: For any candidate category, the category index value of the candidate category is determined based on at least one category keyword corresponding to the candidate category and the multi-level weight of the at least one category keyword; The target category of the data to be processed is selected based on the category index value of at least one candidate category corresponding to each keyword.

8. The method according to claim 1, after filtering at least one candidate category corresponding to each keyword based on each keyword and the multi-level weight corresponding to each keyword to obtain the target category of the data to be processed, further comprising: The data classification model is trained using the data to be processed and the target category of the data to be processed, and the trained data classification model is obtained.

9. The method according to claim 8, wherein the data classification model comprises a decoding layer and multiple encoding layers; The step of training a data classification model using the data to be processed and the target category of the data to obtain a trained data classification model includes: For any given coding layer, the output of this layer is determined based on the data to be processed and the coding features output by the previous coding layer. The first coding layer takes the data to be processed as input, and the last coding layer outputs the fused coding features. The fused encoded features are input into the decoding layer to obtain the predicted category corresponding to the data to be processed; The data classification model is trained based on the predicted category and the target category of the data to be processed, and the trained data classification model is obtained.

10. The method according to claim 1, further comprising, after filtering at least one candidate category corresponding to each keyword based on each keyword and the multi-level weight corresponding to each keyword to obtain the target category of the data to be processed: The keyword library is updated based on at least one keyword in the data to be processed and the target category of the data to be processed, to obtain an updated keyword library.

11. A method for identifying occupational categories, comprising: Perform keyword analysis on the occupational description data to obtain at least one keyword from the occupational description data; By using a pre-built keyword library, candidate category information and multi-level weights corresponding to each keyword are found. The candidate category information includes at least one candidate category. The keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights. The sample multi-level weights are the weights of the sample keywords at different classification granularities. The sample multi-level weights are obtained based on multiple sample categories, multiple sample subcategories included in each sample category, and at least one sample keyword corresponding to each sample subcategory. The at least one sample keyword corresponding to each sample subcategory is extracted from the sample description text carried by each sample subcategory. Based on each keyword and its corresponding multi-level weight, at least one candidate category corresponding to each keyword is filtered to obtain the occupational category corresponding to the occupational description data.

12. A data classification system, comprising a client and a server; The client is used to send a data classification request to the server, wherein... The data classification request carries the data to be processed. The server is used to perform keyword analysis on the data to be processed to obtain at least one keyword in the data; using a pre-built keyword library, it searches for candidate category information and multi-level weights corresponding to each keyword, wherein the candidate category information includes at least one candidate category, the keyword library includes multiple sample keywords, sample category information corresponding to each sample keyword, and sample multi-level weights, the sample multi-level weights being the weights of the sample keywords at different classification granularities, the sample multi-level weights being obtained based on multiple sample categories, multiple sample subcategories included in each sample category, and at least one sample keyword corresponding to each sample subcategory, the at least one sample keyword corresponding to each sample subcategory being extracted from the sample description text carried by each sample subcategory; based on each keyword and the multi-level weights corresponding to each keyword, it filters the at least one candidate category corresponding to each keyword to obtain the target category of the data to be processed; and sends the target category of the data to be processed to the client; The client is also used to receive the target category of the data to be processed.

13. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 10 or claim 11.

14. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of any one of claims 1 to 10 or the method of claim 11.

Images