Enterprise data processing method based on commercial asset carding

Through a method based on commercial asset sorting, combined with the BERT model and the CNN-LSTM network model, the automated classification and grading of enterprise data is realized, and the data grading problem of traditional manual dependence is solved, efficiency and accuracy are improved, and the company's size is adapted to the need.

CN119066497AActive Publication Date: 2024-12-03ZHEJIANG ZHONGDIAN YUANWEI TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411238528.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-05
Publication Date
2024-12-03
Estimated Expiration
2044-09-05

AI Technical Summary

Technical Problem

Traditional enterprise asset data classification relies on manual operations, consumes a lot of time and human resources, and is prone to errors and inconsistencies, and is particularly difficult to effectively process unlabeled data.

Method used

Using a method based on commercial asset sorting, through the issuance of rule knowledge base strategies, terminal PC data collection, document file hierarchical processing, combined with the BERT model and the CNN-LSTM network model, automated data classification and grading are realized, and deep natural language processing is used. Technology performs accurate grading of document data.

Benefits of technology

It realizes automated management of enterprise data, improves work efficiency, optimizes data management processes, reduces manual intervention, ensures the accuracy and scalability of data classification, and adapts to the needs of enterprises of different sizes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066497A_ABST
    Figure CN119066497A_ABST
Patent Text Reader

Abstract

The invention relates to an enterprise data processing method based on commercial asset carding, which is applied to the technical field of data processing, and comprises the steps of rule knowledge base strategy issuing, terminal PC data acquisition, document file grading processing, model and rule updating, asset carding result expansion and the like. By combining data collection, a rule knowledge base and an advanced natural language processing model, automatic management of enterprise digital information is realized, then an intelligent digital asset management system is constructed for enterprises, the working efficiency is greatly improved, the data management process is optimized, and the enterprise data processing method in the application has the advantages that the enterprise data processing efficiency is greatly improved. File classification is performed by using a rule knowledge base model, deep analysis and hierarchical screening of text data are performed by using a network model based on a BERT model and CNN-LSTM, and the network model can identify and understand complex text content, so that the accuracy and rationality of data classification are ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an enterprise data processing method, and in particular to an enterprise data processing method based on business asset sorting and applied in the field of data processing technology. Background Art

[0002] Trade secrets refer to corporate information that is not known to the public, has practical value, and for which the right holder has taken protective measures, including technical information and business information. With the rapid development of the information digital age, enterprises have accumulated a large amount of digital assets, including personnel contract files, financial statements, engineering design drawings, program development codes, telephone consultation recordings, and various video materials. These data are not only an important part of corporate operations, but also a core strategic resource of the enterprise. Therefore, it is crucial to protect the security and integrity of these data.

[0003] In order to deal with data leakage and information security threats, the country has successively issued the "Cybersecurity Law" and the "Data Security Law" to strictly classify and grade data. These regulations require companies to establish a complete data management system and take corresponding protection measures for different types of data.

[0004] In this context, enterprises are faced with the challenge of how to comprehensively scan and sort out their asset data. Effective data scanning and sorting can not only help enterprises understand the status of their own data assets, but also identify potential security risks. Enterprises need to adopt advanced methods and technologies to ensure comprehensive sorting and classification of data, improve data management and protection capabilities, and effectively respond to the increasingly complex network security environment and regulatory requirements.

[0005] Traditional enterprise asset data classification is often based on manual processes and operations, requiring employees to review and classify data, relying on their knowledge and experience, which consumes a lot of time and human resources, and is prone to errors and inconsistencies. Although the current use of traditional machine learning algorithms such as support vector machine algorithms, naive Bayes algorithms, decision tree algorithms, and neural network algorithms to classify text categories continues to deepen, the data labeling process also relies on manual labor and is suitable for the classification of large quantities of labeled text data, but not for the classification of large amounts of unlabeled data. At the same time, traditional machine learning methods are not easy to extract features. Therefore, we propose an enterprise data processing method based on business asset combing. Summary of the invention

[0006] In view of the above-mentioned prior art, the technical problem to be solved by the present invention is that the traditional enterprise asset data classification and grading is often based on manual processes and operations, requiring employees to review and classify the data, relying on the employees' knowledge and experience, which consumes a lot of time and human resources, and is prone to errors and inconsistencies.

[0007] To solve the above problems, the present invention provides an enterprise data processing method based on business asset combing, comprising the following steps: S1. Rule knowledge base strategy issuance: S11, construction of rule knowledge base; S12. Account registration and binding; S13, rule knowledge base is issued; S2. Terminal PC data collection: S21, full disk scan; S211, disk file scanning algorithm: the terminal PC scans the file system using a disk depth-first search algorithm; S212, file rule matching and classification: during the scanning process, character string matching is performed according to the rules in the rule knowledge base, and the matching result includes the file name, path and the rule category that meets the requirements; S22, file classification and uploading; S221, classification result generation: after the scanning is completed, all matching files and their classification information are sorted into classification results in JSON format through the rule knowledge base; S222, uploading classification results: the terminal PC uploads the file classification results to the data management server in segments through the HTTP protocol. During the uploading process, a JSON string is sent to the classification result processing endpoint of the server using a POST request; S223, document upload: filter out files marked as document type, upload these document files together with their classification information, and perform further classification training; S3. Document file classification processing: S31, data input processing: S311, document cleaning: remove unnecessary content in the document through regular expressions, and remove stop words according to the Chinese stop word list; S312, document segmentation: split the document into tokens of word granularity through the Jieba tokenizer; S32, BERT embedding encoding vector: The token after word segmentation passes through the BERT embedding layer to generate word embedding vector, type embedding vector and position vector; S33, BERT weighted coding fusion: The first layer output vector of the low-level Transformer and the last layer output vector of the high-level Transformer are weightedly superimposed through residual connection design; S34, constructing a CNN-LSTM decoding layer: The output of the BERT encoding layer is input into the textCNN network and the Bi-STM network respectively, and the output feature vector of the textCNN network and the output feature vector of the Bi-LSTM network are weighted averaged using a predetermined weight; S35, fully connected layer classification output: The fused vector is input into the fully connected layer, and then the output of the fully connected layer is calculated through softmax to output the relevant probability and perform the final classification task; S36, document classification result generation and storage: the document data is classified by using the network model based on the BERT model and the CNN-LSTM, and the classification results are stored in the data management server; S37, hierarchical model storage: storing the hierarchical model after each scanning task is completed in the data management server; S4. Model and rule updates: S41. Update of rule knowledge base: Based on user feedback and new file types, the rules of the rule knowledge base are regularly updated to ensure that they can adapt to the ever-changing data environment. After the rules are updated, they are sent to each terminal PC through an automated mechanism. S5. Display of asset sorting results: S51, database storage: after each grading and classification task is completed, the classification results of all files and the classification results of document files are stored in the MySQL database; S52. Front-end display: Provide a RESTful API interface to transmit the classification results to the Web front-end through the HTTP protocol.

[0008] The above-mentioned enterprise data processing method based on business asset sorting realizes the automated management of enterprise digital information by combining data collection, rule knowledge base and advanced natural language processing models, and then builds an intelligent digital asset management system for the enterprise, which greatly improves work efficiency and optimizes data management processes.

[0009] As a further improvement of the present application, step S11 includes: S111. Demand research: Cooperate with relevant departments of the enterprise to determine the file types, file suffixes and naming rules that need to be classified. Based on these requirements, define the structure and content of the rule knowledge base to ensure that it can cover all file types that need to be classified; S112. Rule writing: writing specific rules of the rule knowledge base; S113, Knowledge base testing: Testing the rules of the rule knowledge base to ensure the accuracy and coverage of the rules; S114. Knowledge base optimization: Optimize the rules of the rule knowledge base based on the test results, eliminate false matches and improve the classification standards.

[0010] As a further improvement of the present application, step S12 includes: S121. Account registration: When each terminal PC is registered, a unique identifier needs to be generated to create an account on the data management server and ensure the uniqueness of the device identity, which is used to create and maintain the account of the terminal on the data management server; S122, account binding: using HTTP protocol, the unique identification and authentication information of the terminal PC is sent to the data management server through an API endpoint. After the server verifies the information, it binds the terminal PC to its corresponding account and returns confirmation information of successful binding; S123, information storage: the account information, unique identifier, and binding status of the terminal PC are stored in the database through the data management server for subsequent policy distribution and data upload.

[0011] As a further improvement of the present application, step S13 includes: S131, policy delivery: the data management server delivers the rule knowledge base through the HTTP protocol. The rule knowledge base is encoded in the JSON string format and sent to the terminal PC through an HTTP POST request. S132, terminal reception and parsing: After receiving the rule knowledge base, the terminal PC parses the JSON string, extracts the rule content, and stores the rule in the local database, preparing for full disk scanning and classification tasks.

[0012] As a further improvement of the present application, step S4 further includes: S42. Model retraining: Regularly retrain the network models based on the BERT model and CNN-LSTM, and use newly collected data files and classification results for fine-tuning to ensure that the network models always maintain efficient classification capabilities; S43. Performance optimization: Optimize server performance, transmission speed, and data processing flow to ensure that the network model can efficiently handle the increasing amount of data.

[0013] To sum up, the enterprise data processing method in this application realizes the automated management of enterprise digital information by combining data collection, rule knowledge base and advanced natural language processing models, and then builds an intelligent digital asset management system for the enterprise, which greatly improves work efficiency and optimizes the data management process. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a method flow chart of the first implementation mode of this application; Figure 2 This is a structural block diagram of the network model based on the BERT model and CNN-LSTM in the first embodiment of the present application. DETAILED DESCRIPTION

[0015] An implementation of the present application is described in detail below with reference to the accompanying drawings.

[0016] The first implementation method: Figure 1-2 A method for processing enterprise data based on business asset combing is shown, comprising the following steps: S1. Rule knowledge base strategy issuance: S11. Construction of rule knowledge base: S111. Demand research: Cooperate with relevant departments of the enterprise to determine the file types, file suffixes and naming rules that need to be classified. Based on these requirements, define the structure and content of the rule knowledge base to ensure that it can cover all file types that need to be classified; S112. Rule writing: writing specific rules of the rule knowledge base. The rules may include keywords in the file name, file extensions, etc. For example, a file name containing the word "contract" or a file ending with ".docx" will be classified as a "contract document" category; S113, Knowledge base test: Test the rules of the rule knowledge base to ensure the accuracy and coverage of the rules. The test content includes whether the specified files can be accurately matched, whether there are omissions or misjudgments; S114, Knowledge base optimization: optimize the rules of the rule knowledge base based on the test results, eliminate false matches and improve the classification standards to ensure the wide applicability and accuracy of the rule knowledge base; S12. Account registration and binding: S121. Account registration: When each terminal PC is registered, a unique identifier (ID) needs to be generated to create an account on the data management server and ensure the uniqueness of the device identity, which is used to create and maintain the account of the terminal on the data management server; S122, account binding: using HTTP protocol, the unique identification and authentication information of the terminal PC is sent to the data management server through an API endpoint. After the server verifies the information, it binds the terminal PC to its corresponding account and returns confirmation information of successful binding; S123, information storage: the account information, unique identifier, and binding status of the terminal PC are stored in the database through the data management server for subsequent policy distribution and data upload; S13. Rule knowledge base is issued: S131, policy delivery: the data management server delivers the rule knowledge base through the HTTP protocol. The rule knowledge base is encoded in the JSON string format and sent to the terminal PC through an HTTP POST request. S132, terminal reception and parsing: After receiving the rule knowledge base, the terminal PC parses the JSON string, extracts the rule content, and stores the rule in the local database, preparing for full disk scanning and classification tasks; S2. Terminal PC data collection: S21, Full Scan: S211, disk file scanning algorithm: the terminal PC scans the file system using a disk depth-first search algorithm; S212, file rule matching and classification: during the scanning process, character string matching is performed according to the rules in the rule knowledge base, and the matching result includes the file name, path and the rule category that meets the requirements; S22. File classification and upload: S221, classification result generation: After the scanning is completed, all matching files and their classification information are sorted into classification results in JSON format through the rule knowledge base. The classification information includes file name, path, matching rule, classification category, etc.; S222, uploading classification results: the terminal PC uploads the file classification results to the data management server in segments through the HTTP protocol. During the uploading process, a JSON string is sent to the classification result processing endpoint of the server using a POST request; S223, document upload: filter out files marked as document type, upload these document files together with their classification information, and perform further classification training; S3. Document file classification processing: S31, data input processing; S311, Document cleaning: Use regular expressions to remove unnecessary content in the document (such as special characters, HTML tags, extra spaces, etc.), and remove stop words based on the Chinese stop word list; S312, document segmentation: split the document into tokens of word granularity through the Jieba tokenizer; S32, BERT embedding coding vector: After the word segmentation processing, the token passes through the BERT embedding layer to generate the word embedding vector, type embedding vector and position vector. In particular, the position vector in this model adopts the absolute position encoding method based on sine and cosine functions. The reference rules are as follows: Among them, pos represents the number of the word, i represents the dimension of embedding, and i∈[0,768); S33, BERT weighted coding fusion: Fusion principle: The Google team provides two versions of the model, BERT-base and BERT-large. The encoder of the BERT-base model consists of 12 layers of Transformers, and the encoding layer of the BERT-large model consists of 24 layers of Transformers. Different Transformer layers encode feature information in different levels. For example, low-level Transformers capture word-level feature information, while high-level Transformers tend to encode text-level feature information. Fusion process: The first layer output vector of the low-level Transformer and the last layer output vector of the high-level Transformer are weightedly superimposed through residual connection design; S34. Construct CNN-LSTM decoding layer: Construction principle: The textCNN network uses multiple convolution kernels of different sizes to extract key information in sentences (similar to n-grams with multiple window sizes), so as to better capture local features in the text. The Bi-LSTM network uses a bidirectional LSTM layer to process text sequences, which helps to deal with long-distance text dependency problems and obtain contextual word order information of the text. Construction process: The output of the BERT encoding layer is input into the textCNN network and the Bi-STM network respectively, and the output feature vector of the textCNN network and the output feature vector of the Bi-LSTM network are weighted averaged using the predetermined weights; S35, fully connected layer classification output: The fused vector is input into the fully connected layer, and then the output of the fully connected layer is calculated through softmax to output the relevant probability and perform the final classification task; S36. Document classification result generation and storage: Document data is classified through a network model based on the BERT model and CNN-LSTM, and the classification results are stored in the data management server. The data needs to be structured and stored according to the document category and classification results to facilitate subsequent query and management; S37, hierarchical model storage: storing the hierarchical model after each scanning task is completed in the data management server; S4. Model and rule updates: S41. Update of rule knowledge base: Based on user feedback and new file types, the rules of the rule knowledge base are regularly updated to ensure that they can adapt to the ever-changing data environment. After the rules are updated, they are sent to each terminal PC through an automated mechanism. S42. Model retraining: Regularly retrain the network models based on the BERT model and CNN-LSTM, and use newly collected data files and classification results for fine-tuning to ensure that the network models always maintain efficient classification capabilities; S43, Performance Optimization: Optimize server performance, transmission speed and data processing flow to ensure that the network model can efficiently handle the increasing amount of data; S5. Display of asset sorting results: S51. Database storage: After each grading and classification task is completed, the classification results of all files and the classification results of document files are stored in the MySQL database, including the file name, file ID, file classification label, document classification label, etc.

[0017] S52. Front-end display: Provide a RESTful API interface to transmit the classification results to the Web front-end through the HTTP protocol, so that the front-end can display the classification results in the form of tables, charts, etc.

[0018] The advantages of the enterprise data processing method in this application are reflected in many aspects: 1. Automated data processing: It realizes a fully automated process from data collection to data classification, reducing the reliance on manual intervention. By automatically scanning the terminal PC, it can quickly and efficiently collect a large amount of data, and automatically classify and filter according to preset rules; 2. Accurate data classification: Use the rule knowledge base model to classify files and use the network model based on the BERT model and CNN-LSTM to perform in-depth analysis and hierarchical screening of text data. The network model can recognize and understand complex text content to ensure the accuracy and rationality of data classification. The context understanding ability of the network model based on the BERT model and CNN-LSTM enables it to obtain higher classification accuracy when processing document data; 3. Good scalability. This technology is highly scalable and can adapt to the needs of enterprises of different sizes. No matter how the amount of data changes, it can effectively respond and ensure the accuracy of classification and grading. It can flexibly adjust the rule knowledge base according to the specific needs of the enterprise to adapt to different types of files and data.

[0019] In view of current practical needs, the above-mentioned implementation mode adopted in this application is not limited to the scope of protection. Various changes made within the knowledge scope of technical personnel in this field without departing from the concept of this application still fall within the scope of protection of the present invention.

Claims

1. An enterprise data processing method based on business asset combing, characterized in that: The following steps are involved: S1. Rule knowledge base strategy issuance: S11, construction of rule knowledge base; S12. Account registration and binding; S13, rule knowledge base is issued; S2. Terminal PC data collection: S21, full disk scan; S211, disk file scanning algorithm: the terminal PC scans the file system using a disk depth-first search algorithm; S212, file rule matching and classification: during the scanning process, character string matching is performed according to the rules in the rule knowledge base, and the matching result includes the file name, path and the rule category that meets the requirements; S22, file classification and uploading; S221, classification result generation: after the scanning is completed, all matching files and their classification information are sorted into classification results in JSON format through the rule knowledge base; S222, uploading classification results: the terminal PC uploads the file classification results to the data management server in segments through the HTTP protocol. During the uploading process, a JSON string is sent to the classification result processing endpoint of the server using a POST request; S223, document upload: filter out files marked as document type, upload these document files together with their classification information, and perform further classification training; S3. Document file classification processing: S31, data input processing: S311, document cleaning: remove unnecessary content in the document through regular expressions, and remove stop words according to the Chinese stop word list; S312, document segmentation: split the document into tokens of word granularity through the Jieba tokenizer; S32, BERT embedding encoding vector: The token after word segmentation passes through the BERT embedding layer to generate word embedding vector, type embedding vector and position vector; S33, BERT weighted coding fusion: The first layer output vector of the low-level Transformer and the last layer output vector of the high-level Transformer are weightedly superimposed through residual connection design; S34, constructing a CNN-LSTM decoding layer: The output of the BERT encoding layer is input into the textCNN network and the Bi-STM network respectively, and the output feature vector of the textCNN network and the output feature vector of the Bi-LSTM network are weighted averaged using a predetermined weight; S35, fully connected layer classification output: The fused vector is input into the fully connected layer, and then the output of the fully connected layer is calculated through softmax to output the relevant probability and perform the final classification task; S36, document classification result generation and storage: the document data is classified by using the network model based on the BERT model and the CNN-LSTM, and the classification results are stored in the data management server; S37, hierarchical model storage: storing the hierarchical model after each scanning task is completed in the data management server; S4. Model and rule updates: S41. Update of rule knowledge base: Based on user feedback and new file types, the rules of the rule knowledge base are regularly updated to ensure that they can adapt to the ever-changing data environment. After the rules are updated, they are sent to each terminal PC through an automated mechanism. S5. Asset sorting results display: S51, database storage: after each grading and classification task is completed, the classification results of all files and the classification results of document files are stored in the MySQL database; S52. Front-end display: Provide a RESTful API interface to transmit the classification results to the Web front-end through the HTTP protocol.

2. The enterprise data processing method based on business asset combing according to claim 1 is characterized in that: The step S11 comprises: S111. Demand research: Cooperate with relevant departments of the enterprise to determine the file types, file suffixes and naming rules that need to be classified. Based on these requirements, define the structure and content of the rule knowledge base to ensure that it can cover all file types that need to be classified; S112. Rule writing: writing specific rules of the rule knowledge base; S113, Knowledge base testing: Testing the rules of the rule knowledge base to ensure the accuracy and coverage of the rules; S114. Knowledge base optimization: Optimize the rules of the rule knowledge base based on the test results, eliminate false matches and improve the classification standards.

3. The enterprise data processing method based on business asset combing according to claim 1 is characterized in that: The step S12 comprises: S121. Account registration: When each terminal PC is registered, a unique identifier needs to be generated to create an account on the data management server and ensure the uniqueness of the device identity, which is used to create and maintain the account of the terminal on the data management server; S122, account binding: using HTTP protocol, the unique identification and authentication information of the terminal PC is sent to the data management server through an API endpoint. After the server verifies the information, it binds the terminal PC to its corresponding account and returns confirmation information of successful binding; S123, information storage: the account information, unique identifier, and binding status of the terminal PC are stored in the database through the data management server for subsequent policy distribution and data upload.

4. The enterprise data processing method based on business asset combing according to claim 1 is characterized in that: The step S13 comprises: S131, policy delivery: the data management server delivers the rule knowledge base through the HTTP protocol. The rule knowledge base is encoded in the JSON string format and sent to the terminal PC through an HTTP POST request. S132, terminal reception and parsing: After receiving the rule knowledge base, the terminal PC parses the JSON string, extracts the rule content, and stores the rule in the local database, preparing for full disk scanning and classification tasks.

5. The enterprise data processing method based on business asset combing according to claim 1 is characterized in that: The step S4 further comprises: S42. Model retraining: Regularly retrain the network models based on the BERT model and CNN-LSTM, and use newly collected data files and classification results for fine-tuning to ensure that the network models always maintain efficient classification capabilities; S43. Performance optimization: Optimize server performance, transmission speed, and data processing flow to ensure that the network model can efficiently handle the increasing amount of data.

Citation Information

Patent Citations

  • Data grading and classifying method based on financial industry data security

    CN114238727A

  • Power grid text data grading and classifying method and system

    CN117251573A

  • Method and device for realizing large-scale data asset classification and grading processing based on transfer learning, processor and readable storage medium thereof

    CN117688446A

  • Text matching model training method and device, equipment, medium and program product

    CN118277551A

  • Intelligent judgment method and system for file security level marks

    CN118445709A