A data classification method and device, computer equipment and a storage medium

Abstract

This application discloses a data classification method, apparatus, computer equipment, and storage medium, belonging to the fields of big data technology and property insurance financial technology. This application involves parsing the data to be classified to obtain its data type, determining a matching data classification model based on the data type, importing the data to be classified into the matching classification model to obtain initial classification data, obtaining the corresponding business scenario data, importing the business scenario data and the initial classification data into a classification prediction model to obtain a classification prediction result, and combining the initial classification data based on the classification prediction result to obtain the final data classification result. This application also relates to the field of blockchain technology, where the data to be classified is stored in a blockchain network. This application improves data quality and accuracy through data classification, reduces the data usage costs for developers, and improves R&D efficiency.

Description

Technical Field

[0001] This application belongs to the fields of big data technology and property insurance financial technology, specifically involving a data classification method, device, computer equipment and storage medium. Background Technology

[0002] Data processing in the financial and property insurance sector involves extensive data cleaning, transformation, analysis, and classification. However, due to the lack of a unified processing approach, each developer may employ their own unique methods. For example, in insurance claims, different insurance companies have different claims processes and rules, resulting in varying data processing methods for each company. This lack of consistency and standardization can affect data quality and accuracy. Furthermore, with the increase in new business, insurance data processing is becoming increasingly complex. During collaborative project development, developers need to continuously familiarize themselves with various data processing methods to understand and maintain existing code. This process consumes significant time and effort, leading to decreased development efficiency. Summary of the Invention

[0003] The purpose of this application is to propose a data classification method, apparatus, computer equipment, and storage medium to solve the technical problems of insufficient data quality and accuracy caused by the lack of a unified processing method in existing data processing solutions in the financial and property insurance field, and the need for developers to constantly familiarize themselves with various data processing methods, which leads to a decline in R&D efficiency.

[0004] To address the aforementioned technical problems, this application provides a data classification method, employing the following technical solution:

[0005] A data classification method, comprising:

[0006] Receive data classification instructions uploaded by the client, obtain the data to be classified, and parse the data to be classified to obtain the data type of the data to be classified;

[0007] Based on the data type, a data classification model that matches the data to be classified is determined, and a matching classification model is obtained.

[0008] Import the data to be classified into the matching classification model to obtain the initial classification data;

[0009] Obtain the business scenario data corresponding to the data to be classified, import the business scenario data and the initial classification data into the classification prediction model, and obtain the classification prediction result;

[0010] The classification prediction results are sent to the client, and feedback information is received from the client.

[0011] The initial classification data is combined based on feedback information and classification prediction results to obtain the data classification results.

[0012] Furthermore, it receives data classification instructions uploaded by the client, obtains the data to be classified, and parses the data to be classified to obtain the data type of the data to be classified, specifically including:

[0013] Receive data classification instructions uploaded by the client, and obtain the data to be classified based on the data classification instructions;

[0014] Analyze the data to be classified to determine the data source.

[0015] The data type of the data to be classified is determined based on the data source of the data to be classified.

[0016] Furthermore, the data classification model includes decision tree models and support vector machines. When the matching classification model is a decision tree model, the data to be classified is imported into the matching classification model to obtain initial classification data, specifically including:

[0017] Preprocessing is performed on the data to be classified, including data deduplication, missing value handling, and numerical standardization.

[0018] Feature extraction is performed on the preprocessed data to be classified to obtain the first data features;

[0019] Load the pre-trained decision tree model, import the first data features into the decision tree model, and obtain the classification labels output by the decision tree model;

[0020] The data to be classified is classified according to the classification labels to obtain the initial classification data.

[0021] Furthermore, the data classification model includes decision tree models and support vector machines. When the matching classification model is a support vector machine, the data to be classified is imported into the matching classification model to obtain initial classification data, specifically including:

[0022] Preprocessing is performed on the data to be classified, including data deduplication, missing value handling, and numerical standardization.

[0023] Feature extraction is performed on the preprocessed data to be classified to obtain the second data features;

[0024] Load the pre-trained support vector machine and import the second data features into the support vector machine to obtain the decision boundary output by the support vector machine;

[0025] The data to be classified is classified according to the decision boundary to obtain the initial classification data.

[0026] Furthermore, the classification prediction model is a pre-trained Transformer model, which includes an input layer, a self-attention layer, a feedforward neural network layer, and an output layer. It acquires the business scenario data corresponding to the data to be classified, imports the business scenario data and the initial classification data into the classification prediction model, and obtains the classification prediction result, specifically including:

[0027] Feature extraction is performed on business scenario data and initial classification data to obtain scenario data features;

[0028] The scene data features are imported into the classification prediction model through the input layer, and the scene data features are encoded and self-attention weights are assigned through the self-attention layer.

[0029] Vector mapping is performed on the encoded and weighted scene data features through a feedforward neural network layer;

[0030] The vector mapping result is obtained through the output layer, and the vector mapping result is decoded to obtain the classification prediction result, which is then output.

[0031] Furthermore, before acquiring the business scenario data corresponding to the data to be classified, importing the business scenario data and the initial classification data into the classification prediction model, and obtaining the classification prediction result, the following steps are also included:

[0032] Acquire training data, which includes historical scene data and historical classification data;

[0033] Construct training and validation datasets based on the training data;

[0034] The initial prediction model is trained using the training dataset, and the trained initial prediction model is validated using the validation dataset to obtain the classification prediction model.

[0035] Furthermore, the initial prediction model includes an input layer, a self-attention layer, a feedforward neural network layer, and an output layer. The pre-defined initial prediction model is trained using a training dataset, and then validated using a validation dataset to obtain a classification prediction model, specifically including:

[0036] Extract features from the training samples in the validation dataset to obtain the training sample features;

[0037] The features of the training samples are imported into the initial prediction model through the input layer of the initial prediction model;

[0038] The features of the training samples are encoded and self-attention weights are assigned through the self-attention layer of the initial prediction model;

[0039] Vector mapping is performed on the encoded and weighted training sample features through the feedforward neural network layer of the initial prediction model;

[0040] The vector mapping result of the training sample features is obtained through the output layer of the initial prediction model, and the vector mapping result of the training sample features is decoded to obtain the training prediction result.

[0041] The initial prediction model is iterated by comparing the training prediction results with the preset standard results to obtain the trained initial prediction model.

[0042] The initial prediction model is validated using a validation dataset. Once the initial prediction model passes validation, a classification prediction model is obtained.

[0043] To address the aforementioned technical problems, this application also provides a data classification device, which employs the following technical solution:

[0044] A data classification device, comprising:

[0045] The data type determination module is used to receive data classification instructions uploaded by the client, obtain the data to be classified, and parse the data to be classified to obtain the data type of the data to be classified.

[0046] The classification model matching module is used to determine the data classification model that matches the data to be classified based on the data type, and to obtain the matching classification model.

[0047] The initial data classification module is used to import the data to be classified into the matching classification model to obtain the initial classification data;

[0048] The data classification and prediction module is used to obtain the business scenario data corresponding to the data to be classified, import the business scenario data and the initial classification data into the classification and prediction model, and obtain the classification and prediction results.

[0049] The data classification feedback module is used to send the classification prediction results to the client and receive the feedback information returned by the client.

[0050] The final data classification module is used to combine the initial classification data based on feedback information and classification prediction results to obtain the data classification result.

[0051] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0052] A computer device includes a memory and a processor, the memory storing computer-readable instructions, the processor executing the computer-readable instructions to implement the steps of the data classification method as described in any of the preceding claims.

[0053] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0054] A computer-readable storage medium storing computer-readable instructions that, when executed by a processor, implement the steps of the data classification method as described in any one of the preceding descriptions.

[0055] Compared with the prior art, the embodiments of this application have the following main advantages:

[0056] This application discloses a data classification method, apparatus, computer equipment, and storage medium, belonging to the fields of big data technology and property insurance technology. This application receives data classification instructions uploaded by a client, obtains the data to be classified, parses the data to be classified to obtain its data type, determines a data classification model matching the data to be classified based on the data type, obtains a matching classification model, imports the data to be classified into the matching classification model to obtain initial classification data, obtains the business scenario data corresponding to the data to be classified, imports the business scenario data and the initial classification data into a classification prediction model to obtain a classification prediction result, sends the classification prediction result to the client, and receives feedback information returned by the client. Based on the feedback information and the classification prediction result, the initial classification data is combined to obtain the data classification result. This application first performs initial data classification using a data classification model matching the data to be classified, and then performs reclassification prediction using a machine learning classification prediction model based on the business scenario, achieving accurate data classification based on the usage scenario. It is particularly suitable for data classification in the financial and property insurance field. This application improves data quality and accuracy through data classification, reduces the data usage cost for developers, and improves R&D efficiency. Attached Figure Description

[0057] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 An exemplary system architecture diagram is shown, in which this application can be applied;

[0059] Figure 2 A flowchart of one embodiment of the data classification method according to this application is shown;

[0060] Figure 3 A schematic diagram of one embodiment of the data classification apparatus according to this application is shown;

[0061] Figure 4 A schematic diagram of the structure of one embodiment of a computer device according to this application is shown. Detailed Implementation

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0063] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0064] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0065] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0066] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0067] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0068] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103. The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0069] It should be noted that the data classification method provided in this application embodiment is generally executed by the server, and correspondingly, the data classification device is generally set in the server.

[0070] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0071] Continue to refer to Figure 2 The diagram illustrates a flowchart of an embodiment of the data classification method according to this application. Embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0072] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0073] In the financial and property insurance sector, data processing lacks a unified approach, leading each developer to employ their own unique methods. For instance, in insurance claims, different insurance companies have varying claims processes and rules, resulting in inconsistent data processing methods. This lack of uniformity and standardization can impact data quality and accuracy. Furthermore, with the increase in new business, insurance data processing becomes increasingly complex. During collaborative project development, developers need to continuously familiarize themselves with various data processing methods to understand and maintain existing code. This process consumes significant time and effort, leading to decreased development efficiency.

[0074] In the above embodiments, this application discloses a data classification method, apparatus, computer equipment, and storage medium, belonging to the fields of big data technology and property insurance technology. This application first performs initial data classification by matching the data to be classified with a data classification model, and then performs reclassification prediction using a machine learning classification prediction model based on the business scenario, thereby achieving accurate data classification based on the usage scenario. It is particularly suitable for data classification in the field of financial property insurance. This application improves data quality and accuracy through data classification, reduces the data usage cost for developers, and improves R&D efficiency.

[0075] The data classification method includes the following steps:

[0076] S201: Receive the data classification instruction uploaded by the client, obtain the data to be classified, parse the data to be classified, and obtain the data type of the data to be classified.

[0077] In this embodiment, after receiving a data classification instruction uploaded by the client, the server obtains the data to be classified corresponding to the instruction, parses the data to be classified, determines the data source, and determines the data type based on the data source. For example, in insurance sales, the data source may include a sales record database, a customer information database, a policy database, etc. If the data source is a sales record database, the data to be classified is determined to be of sales data type.

[0078] In this embodiment, the data classification method operates on an electronic device (e.g., Figure 1 The server shown can receive instructions or acquire data via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other currently known or future wireless connection methods.

[0079] Furthermore, it receives data classification instructions uploaded by the client, obtains the data to be classified, and parses the data to be classified to obtain the data type of the data to be classified, specifically including:

[0080] Receive data classification instructions uploaded by the client, and obtain the data to be classified based on the data classification instructions;

[0081] Analyze the data to be classified to determine the data source.

[0082] The data type of the data to be classified is determined based on the data source of the data to be classified.

[0083] In this embodiment, the server can parse the received data to be classified to determine its data source and data type. By analyzing the data structure, field information, etc., the data source can be determined, and the data type can be determined through the data source. This step is to determine a suitable data classification model and improve the accuracy of data classification.

[0084] S202, Based on the data type, determine the data classification model that matches the data to be classified, and obtain the matching classification model.

[0085] In this embodiment, the server determines a data classification model that matches the data to be classified from a preset model library based on the data type, and obtains a matching classification model. The models in the model library include at least decision tree models and support vector machines. By matching a suitable data classification model, the accuracy of data classification can be improved.

[0086] S203. Import the data to be classified into the matching classification model to obtain the initial classification data.

[0087] In this embodiment, after the server extracts the data features of the data to be classified, it imports the data features of the data to be classified into the matching classification model to obtain the initial classification data.

[0088] Furthermore, the data classification model includes decision tree models and support vector machines. When the matching classification model is a decision tree model, the data to be classified is imported into the matching classification model to obtain initial classification data, specifically including:

[0089] Preprocessing is performed on the data to be classified, including data deduplication, missing value handling, and numerical standardization.

[0090] Feature extraction is performed on the preprocessed data to be classified to obtain the first data features;

[0091] Load the pre-trained decision tree model, import the first data features into the decision tree model, and obtain the classification labels output by the decision tree model;

[0092] The data to be classified is classified according to the classification labels to obtain the initial classification data.

[0093] In this embodiment, the data to be classified is first preprocessed, including data deduplication, missing value handling, and numerical standardization. Then, features are extracted from the data to be classified to identify attributes that describe the data characteristics. These features are then matched with features in the decision tree model. Starting from the root node, the feature values ​​of the data to be classified are compared with the conditions of the decision tree nodes. Based on the matching results, the corresponding child nodes are selected. This process continues until a leaf node is reached, yielding the final classification result of the decision tree, i.e., the classification label. According to the classification decision path of the decision tree, the data to be classified is assigned to the corresponding category label, completing the classification of the data and obtaining the initial classified data.

[0094] It's important to note that decision tree models are composed of a series of feature conditions and decision rules learned from training data. Therefore, it's crucial to ensure that the features of the data to be classified are consistent with the features of the decision tree model. Decision tree models classify the data using a series of feature conditions and decision rules, assigning it to different categories or making binary classification decisions.

[0095] In one specific embodiment of this application, suppose there is some property insurance sales data, which includes the following fields: age, gender, vehicle type, premium, number of claims, and whether the customer has purchased insurance. This property insurance sales data is classified using a decision tree model to predict whether a customer will purchase insurance. Assume one piece of property insurance sales data is as follows: age 30, gender female, vehicle type SUV. After extracting the attributes that describe the above data characteristics, the comparison starts from the root node according to the features and conditions of the decision tree model. If the data meets the condition based on age, it enters the left child node. Then, based on gender, since it is female, it continues to the left child node. Finally, based on vehicle type, since it is an SUV, it reaches the leaf node. The classification label obtained from the leaf node is "≥X years old; female; SUV vehicle type", where X is a preset age condition. The data can be assigned to the corresponding category based on this classification label, such as "purchase insurance".

[0096] Furthermore, the data classification model includes decision tree models and support vector machines. When the matching classification model is a support vector machine, the data to be classified is imported into the matching classification model to obtain initial classification data, specifically including:

[0097] Preprocessing is performed on the data to be classified, including data deduplication, missing value handling, and numerical standardization.

[0098] Feature extraction is performed on the preprocessed data to be classified to obtain the second data features;

[0099] Load the pre-trained support vector machine and import the second data features into the support vector machine to obtain the decision boundary output by the support vector machine;

[0100] The data to be classified is classified according to the decision boundary to obtain the initial classification data.

[0101] In this embodiment, the data to be classified is first preprocessed, including data deduplication, missing value handling, and numerical standardization. Then, the data is converted into feature vectors so that the support vector machine (SVM) can process it. The converted data is then input into a pre-trained SVM model. The SVM determines the category of the data based on its position in the feature space and its relationship to the decision boundary, thus completing the classification and obtaining the initial classification data.

[0102] It's important to note that converting data into numerical vectors requires ensuring that the vector's dimensions match the feature dimensions used in training the model. The Support Vector Machine (SVM) model learns the decision boundaries and the positions of the support vectors from the training data, and it can then classify the data to be classified based on this information.

[0103] In the specific embodiments described above, when using a support vector machine (SVM) for classification, age can be normalized, gender can be encoded (e.g., 0 represents male, 1 represents female), and vehicle type can be one-hot encoded. This way, each piece of data to be classified can be represented as a feature vector. The transformed data is then input into a pre-trained SVM model. The SVM determines the category of the data based on its position in the feature space and its relationship to the decision boundary. For example, suppose there is a piece of data to be classified where the age is 30, the gender is female, and the vehicle type is SUV. Age can be normalized, gender encoded as 1, and the one-hot encoding of SUV as [0,1,0]. These features are combined into a feature vector, which is then input into the pre-trained SVM model. The SVM determines the category of the data point based on the position of the feature vector in the feature space, for example, classifying it as "purchasing insurance".

[0104] S204: Obtain the business scenario data corresponding to the data to be classified, import the business scenario data and the initial classification data into the classification prediction model, and obtain the classification prediction result.

[0105] The classification prediction model is a pre-trained Transformer model. The Transformer model is a deep learning model based on the self-attention mechanism. The basic components of the Transformer model include multiple layers of self-attention layers and feedforward neural network layers. The self-attention layers are used to calculate attention weights and encode the input sequence, while the feedforward neural network layers are used to further map and transform the encoded sequence.

[0106] In this embodiment, a classification prediction model is pre-trained based on the Transformer model. After the initial classification of the data to be classified is completed, the business scenario data corresponding to the data to be classified is obtained. The business scenario data and the initial classification data are then imported into the classification prediction model to obtain the classification prediction result.

[0107] Furthermore, the classification prediction model includes an input layer, a self-attention layer, a feedforward neural network layer, and an output layer. It acquires the business scenario data corresponding to the data to be classified, imports the business scenario data and the initial classification data into the classification prediction model, and obtains the classification prediction result, specifically including:

[0108] Feature extraction is performed on business scenario data and initial classification data to obtain scenario data features;

[0109] The scene data features are imported into the classification prediction model through the input layer, and the scene data features are encoded and self-attention weights are assigned through the self-attention layer.

[0110] Vector mapping is performed on the encoded and weighted scene data features through a feedforward neural network layer;

[0111] The vector mapping result is obtained through the output layer, and the vector mapping result is decoded to obtain the classification prediction result, which is then output.

[0112] In this embodiment, data features are extracted from business scenario data and initial classification data. Feature extraction methods can include text processing, image processing, statistical feature extraction, etc., with the appropriate feature extraction method selected based on the specific business scenario and data type. Then, a self-attention layer encodes the scenario data features, capturing the correlation and importance between features, and determines the contribution of different features to the classification result through self-attention weight assignment. A feedforward neural network layer further maps and transforms the encoded and weighted scenario data features. The feedforward neural network layer can perform a series of nonlinear transformations to represent and abstract the features at a deeper level, extracting higher-level feature information. The output layer extracts relevant information from the vector mapping results output by the feedforward neural network layer and performs decoding operations, transforming the vector mapping results into specific classification prediction results. This can be processed by subsequent neural network layers (such as fully connected layers) and activation functions, ultimately outputting the classification prediction result.

[0113] Furthermore, before acquiring the business scenario data corresponding to the data to be classified, importing the business scenario data and the initial classification data into the classification prediction model, and obtaining the classification prediction result, the following steps are also included:

[0114] Acquire training data, which includes historical scene data and historical classification data;

[0115] Construct training and validation datasets based on the training data;

[0116] The initial prediction model is trained using the training dataset, and the trained initial prediction model is validated using the validation dataset to obtain the classification prediction model.

[0117] In this embodiment, historical scenario data refers to known business scenario data, while historical classification data refers to known classification results for these scenario data. This data can come from internal company databases, data warehouses, or external data sources. By collecting and organizing this data, a labeled training dataset can be established. The purpose of dividing the dataset into training and validation datasets is to evaluate the model's generalization ability and avoid overfitting during model training. Generally, the training dataset occupies the majority of the total data, while the validation dataset occupies a smaller portion. Finally, the preset initial prediction model is trained using the training dataset, and the trained initial prediction model is validated using the validation dataset to obtain the classification prediction model.

[0118] In the above embodiments, historical data is acquired, and training and validation datasets are constructed based on the historical data. A classification prediction model is pre-trained using the training and validation datasets so that the classification prediction model can be used for data classification prediction in the future.

[0119] Furthermore, the initial prediction model includes an input layer, a self-attention layer, a feedforward neural network layer, and an output layer. The pre-defined initial prediction model is trained using a training dataset, and then validated using a validation dataset to obtain a classification prediction model, specifically including:

[0120] Extract features from the training samples in the validation dataset to obtain the training sample features;

[0121] The features of the training samples are imported into the initial prediction model through the input layer of the initial prediction model;

[0122] The features of the training samples are encoded and self-attention weights are assigned through the self-attention layer of the initial prediction model;

[0123] Vector mapping is performed on the encoded and weighted training sample features through the feedforward neural network layer of the initial prediction model;

[0124] The vector mapping result of the training sample features is obtained through the output layer of the initial prediction model, and the vector mapping result of the training sample features is decoded to obtain the training prediction result.

[0125] The initial prediction model is iterated by comparing the training prediction results with the preset standard results to obtain the trained initial prediction model.

[0126] The initial prediction model is validated using a validation dataset. Once the initial prediction model passes validation, a classification prediction model is obtained.

[0127] In this embodiment, features of training samples in the validation dataset are extracted to obtain training sample features. These features are then imported into the initial prediction model through its input layer. The self-attention layer of the initial prediction model encodes and assigns self-attention weights to the training sample features. The feedforward neural network layer of the initial prediction model performs vector mapping on the encoded and weighted training sample features. Through a series of nonlinear transformations, the features are represented and abstracted at a deeper level to extract higher-level feature information. The vector mapping result of the training sample features is obtained through the output layer of the initial prediction model and decoded to obtain the training prediction result. The prediction error of the initial prediction model is calculated by comparing the training prediction result with a preset standard result. The model is iterated based on the prediction error until it fits perfectly, resulting in a trained initial prediction model. The trained initial prediction model is then validated using the validation dataset. Once the initial prediction model passes validation, a classification prediction model is obtained.

[0128] In the above embodiments, this application uses the Transformer model architecture to quickly train a classification prediction model so that the classification prediction model can be used for data classification prediction in the future.

[0129] S205 sends the classification prediction results to the client and receives the feedback information returned by the client.

[0130] In this embodiment, after obtaining the classification prediction result output by the classification prediction model, the classification prediction result is sent back to the client and displayed on the client, instructing the user to confirm the classification prediction result and obtaining the user's feedback operation data.

[0131] S206, Based on the feedback information and classification prediction results, the initial classification data are combined to obtain the data classification results.

[0132] In this embodiment, the server combines the initial classification data based on the feedback information and the classification prediction results to obtain the data classification result. The feedback information includes "agree with the classification prediction result" and "disagree with the classification prediction result." When the feedback information is "agree with the classification prediction result," the initial classification data is combined based on the data classification scheme in the classification prediction result to obtain the data classification result.

[0133] When the feedback message is "disagree with the classification prediction result", repeat steps S201 to S204 to obtain a new classification prediction result, send the new classification prediction result to the client, and receive the feedback message returned by the client. Repeat the above steps until the feedback message is "agree with the classification prediction result".

[0134] In the above embodiments, this application discloses a data classification method, belonging to the fields of big data technology and property insurance technology. This application receives data classification instructions uploaded by a client, obtains data to be classified, parses the data to be classified to obtain its data type, determines a data classification model matching the data to be classified based on the data type, obtains a matching classification model, imports the data to be classified into the matching classification model to obtain initial classification data, obtains the business scenario data corresponding to the data to be classified, imports the business scenario data and the initial classification data into a classification prediction model to obtain a classification prediction result, sends the classification prediction result to the client, and receives feedback information returned by the client. Based on the feedback information and the classification prediction result, the initial classification data is combined to obtain the data classification result. This application first performs initial data classification using a data classification model matching the data to be classified, and then performs reclassification prediction using a machine learning classification prediction model based on the business scenario, achieving accurate data classification based on the usage scenario. It is particularly suitable for data classification in the financial and property insurance field. This application improves data quality and accuracy through data classification, reduces the data usage cost for developers, and improves R&D efficiency.

[0135] It should be emphasized that, in order to further ensure the privacy and security of the data to be classified, the data to be classified can also be stored in a node of a blockchain.

[0136] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When executed, the computer-readable instructions can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0138] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0139] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a data classification device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0140] like Figure 3 As shown, the data classification device 300 described in this embodiment includes:

[0141] The data type determination module 301 is used to receive the data classification instruction uploaded by the client, obtain the data to be classified, and parse the data to be classified to obtain the data type of the data to be classified.

[0142] The classification model matching module 302 is used to determine the data classification model that matches the data to be classified based on the data type, and to obtain the matching classification model.

[0143] The initial data classification module 303 is used to import the data to be classified into the matching classification model to obtain the initial classification data;

[0144] The data classification and prediction module 304 is used to obtain the business scenario data corresponding to the data to be classified, import the business scenario data and the initial classification data into the classification and prediction model, and obtain the classification and prediction results.

[0145] The data classification feedback module 305 is used to send the classification prediction results to the client and receive the feedback information returned by the client.

[0146] The final data classification module 306 is used to combine the initial classification data based on feedback information and classification prediction results to obtain the data classification result.

[0147] Furthermore, the data type determination module 301 specifically includes:

[0148] The instruction receiving unit is used to receive data classification instructions uploaded by the client and obtain the data to be classified based on the data classification instructions.

[0149] The data source determination unit is used to parse the data to be classified and determine the data source of the data to be classified.

[0150] The data type determination unit is used to determine the data type of the data to be classified based on the data source of the data to be classified.

[0151] Furthermore, the data classification model includes decision tree models and support vector machines. When the matching classification model is a decision tree model, the initial data classification module 303 specifically includes:

[0152] The preprocessing unit is used to preprocess the data to be classified, including data deduplication, missing value handling, and numerical standardization.

[0153] The first feature extraction unit is used to extract features from the preprocessed data to be classified to obtain the first data features;

[0154] The decision tree unit is used to load a pre-trained decision tree model, import the first data features into the decision tree model, and obtain the classification label output by the decision tree model.

[0155] The first initial classification unit is used to classify the data to be classified according to the classification labels to obtain the initial classification data.

[0156] Furthermore, the data classification model includes decision tree models and support vector machines. When the matching classification model is a support vector machine, the initial data classification module 303 also includes:

[0157] The preprocessing unit is used to preprocess the data to be classified, including data deduplication, missing value handling, and numerical standardization.

[0158] The second feature extraction unit is used to extract features from the preprocessed data to be classified to obtain the second data features;

[0159] The Support Vector Machine (SVM) unit is used to load a pre-trained SVM and import the second data features into the SVM to obtain the decision boundary output by the SVM.

[0160] The second initial classification unit is used to classify the data to be classified according to the decision boundary to obtain the initial classification data.

[0161] Furthermore, the classification prediction model is a pre-trained Transformer model, which includes an input layer, a self-attention layer, a feedforward neural network layer, and an output layer. The data classification prediction module 304 specifically includes:

[0162] The scene feature extraction unit is used to extract features from business scene data and initial classification data to obtain scene data features;

[0163] The self-attention unit is used to import scene data features into the classification prediction model through the input layer, and to encode the scene data features and assign self-attention weights through the self-attention layer.

[0164] The vector mapping unit is used to perform vector mapping on the encoded and weighted scene data features through the feedforward neural network layer;

[0165] The decoding output unit is used to obtain the vector mapping result through the output layer, decode the vector mapping result to obtain the classification prediction result, and output the classification prediction result.

[0166] Furthermore, the data classification device 300 also includes:

[0167] The training data acquisition module is used to acquire training data, which includes historical scene data and historical classification data.

[0168] The dataset building module is used to build training and validation datasets based on the training data;

[0169] The model training and validation module is used to train a preset initial prediction model using a training dataset and to validate the trained initial prediction model using a validation dataset to obtain a classification prediction model.

[0170] Furthermore, the initial prediction model includes an input layer, a self-attention layer, a feedforward neural network layer, and an output layer. The model training and validation module specifically includes:

[0171] The training feature extraction unit is used to extract features from the training samples in the validation dataset to obtain the training sample features.

[0172] The training feature import unit is used to import the features of the training samples into the initial prediction model through the input layer of the initial prediction model;

[0173] Self-attention layer units are used to encode training sample features and assign self-attention weights through the self-attention layer of the initial prediction model;

[0174] The vector mapping unit is used to perform vector mapping on the encoded and weighted training sample features through the feedforward neural network layer of the initial prediction model;

[0175] The output decoding unit is used to obtain the vector mapping result of the training sample features through the output layer of the initial prediction model, and decode the vector mapping result of the training sample features to obtain the training prediction result.

[0176] The model iteration unit is used to iterate the initial prediction model by comparing the training prediction results with the preset standard results to obtain the trained initial prediction model.

[0177] The model validation unit is used to validate the trained initial prediction model using a validation dataset. Once the initial prediction model passes validation, a classification prediction model is obtained.

[0178] In the above embodiments, this application discloses a data classification device, belonging to the fields of big data technology and property insurance technology. This application receives data classification instructions uploaded by a client, obtains data to be classified, parses the data to be classified to obtain its data type, determines a data classification model matching the data to be classified based on the data type, obtains a matching classification model, imports the data to be classified into the matching classification model to obtain initial classification data, obtains business scenario data corresponding to the data to be classified, imports the business scenario data and the initial classification data into a classification prediction model to obtain a classification prediction result, sends the classification prediction result to the client, and receives feedback information returned by the client. Based on the feedback information and the classification prediction result, the initial classification data is combined to obtain the data classification result. This application first performs initial data classification using a data classification model matching the data to be classified, and then performs reclassification prediction using a machine learning classification prediction model based on the business scenario, achieving accurate data classification based on the usage scenario. It is particularly suitable for data classification in the financial and property insurance field. This application improves data quality and accuracy through data classification, reduces the data usage cost for developers, and improves R&D efficiency.

[0179] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0180] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0181] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0182] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for data classification methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0183] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, such as executing computer-readable instructions for the data classification method.

[0184] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0185] In the above embodiments, this application discloses a computer device belonging to the fields of big data technology and property insurance technology. This application receives data classification instructions uploaded by a client, obtains data to be classified, parses the data to be classified to obtain its data type, determines a data classification model matching the data to be classified based on the data type, obtains a matching classification model, imports the data to be classified into the matching classification model to obtain initial classification data, obtains business scenario data corresponding to the data to be classified, imports the business scenario data and the initial classification data into a classification prediction model to obtain a classification prediction result, sends the classification prediction result to the client, and receives feedback information returned by the client. Based on the feedback information and the classification prediction result, the initial classification data is combined to obtain the data classification result. This application first performs initial data classification using a data classification model matching the data to be classified, and then performs reclassification prediction using a machine learning classification prediction model based on the business scenario, achieving accurate data classification based on usage scenarios. It is particularly suitable for data classification in the financial and property insurance field. This application improves data quality and accuracy through data classification, reduces the data usage costs for developers, and improves R&D efficiency.

[0186] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the data classification method described above.

[0187] In the above embodiments, this application discloses a computer-readable storage medium, belonging to the fields of big data technology and property insurance technology. This application receives data classification instructions uploaded by a client, obtains data to be classified, parses the data to be classified to obtain its data type, determines a data classification model matching the data to be classified based on the data type, obtains a matching classification model, imports the data to be classified into the matching classification model to obtain initial classification data, obtains business scenario data corresponding to the data to be classified, imports the business scenario data and the initial classification data into a classification prediction model to obtain a classification prediction result, sends the classification prediction result to the client, and receives feedback information returned by the client. Based on the feedback information and the classification prediction result, the initial classification data is combined to obtain the data classification result. This application first performs initial data classification using a data classification model matching the data to be classified, and then performs reclassification prediction using a machine learning classification prediction model based on the business scenario, achieving accurate data classification based on usage scenarios. It is particularly suitable for data classification in the financial and property insurance field. This application improves data quality and accuracy through data classification, reduces the data usage costs for developers, and improves R&D efficiency.

[0188] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0189] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0190] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A data classification method, characterized in that, include: Receive data classification instructions uploaded by the client, obtain the data to be classified, and parse the data to be classified to obtain the data type of the data to be classified; Based on the data type, a data classification model matching the data to be classified is determined, and a matching classification model is obtained; The data to be classified is imported into the matching classification model to obtain initial classification data; Obtain the business scenario data corresponding to the data to be classified, and import the business scenario data and the initial classification data into the classification prediction model to obtain the classification prediction result. The classification prediction result is sent to the client, and feedback information is received from the client. The initial classification data is combined based on the feedback information and the classification prediction results to obtain the data classification results; The data classification model includes a decision tree model and a support vector machine. When the matching classification model is a decision tree model, the step of importing the data to be classified into the matching classification model to obtain initial classification data specifically includes: The data to be classified is preprocessed, wherein the preprocessing includes data deduplication, missing value handling, and numerical standardization; Feature extraction is performed on the preprocessed data to be classified to obtain the first data features; Load a pre-trained decision tree model and import the first data feature into the decision tree model to obtain the classification label output by the decision tree model. The comparison starts from the root node according to the features and conditions of the decision tree model, and the first data feature is divided into leaf nodes. Finally, the classification label is obtained according to the path of the leaf nodes. The data to be classified is classified according to the classification labels to obtain the initial classification data, wherein the data to be classified is assigned to the corresponding category labels according to the classification decision path of the decision tree; Alternatively, feature extraction can be performed on the preprocessed data to be classified to obtain second data features; Load the pre-trained support vector machine and import the second data features into the support vector machine to obtain the decision boundary output by the support vector machine; The data to be classified is classified according to the decision boundary to obtain the initial classified data, wherein the support vector machine classifies the data to be classified according to the relationship between the position of the data to be classified in the feature space and the decision boundary; The system receives data classification instructions uploaded by the client, obtains the data to be classified, and parses the data to be classified to obtain the data type of the data to be classified, specifically including: Receive data classification instructions uploaded by the client, and obtain the data to be classified based on the data classification instructions; The data to be classified is parsed to determine the data source. The data type of the data to be classified is determined based on the data source of the data to be classified.

2. The data classification method as described in claim 1, characterized in that, The classification prediction model is a pre-trained Transformer model, which includes an input layer, a self-attention layer, a feedforward neural network layer, and an output layer. The process of obtaining the business scenario data corresponding to the data to be classified, importing the business scenario data and the initial classification data into the classification prediction model, and obtaining the classification prediction result specifically includes: Feature extraction is performed on the business scenario data and the initial classification data to obtain scenario data features; The scene data features are imported into the classification prediction model through the input layer, and the scene data features are encoded and self-attention weights are assigned through the self-attention layer. The feedforward neural network layer performs vector mapping on the encoded and weighted scene data features; The output layer obtains the vector mapping result, decodes the vector mapping result to obtain the classification prediction result, and outputs the classification prediction result.

3. The data classification method as described in claim 2, characterized in that, Before obtaining the business scenario data corresponding to the data to be classified, importing the business scenario data and the initial classification data into the classification prediction model, and obtaining the classification prediction result, the method further includes: Acquire training data, wherein the training data includes historical scene data and historical classification data; Based on the training data, a training dataset and a validation dataset are constructed; The preset initial prediction model is trained using the training dataset, and the trained initial prediction model is validated using the validation dataset to obtain the classification prediction model.

4. The data classification method as described in claim 3, characterized in that, The initial prediction model includes an input layer, a self-attention layer, a feedforward neural network layer, and an output layer. The process involves training the pre-defined initial prediction model using the training dataset and validating the trained initial prediction model using the validation dataset to obtain the classification prediction model. Specifically, this includes: Extract features from the training samples in the validation dataset to obtain the training sample features; The training sample features are imported into the initial prediction model through the input layer of the initial prediction model; The training sample features are encoded and self-attention weights are assigned through the self-attention layer of the initial prediction model; The features of the encoded and weighted training samples are vector-mapped through the feedforward neural network layer of the initial prediction model. The vector mapping result of the training sample features is obtained through the output layer of the initial prediction model, and the vector mapping result of the training sample features is decoded to obtain the training prediction result. The initial prediction model is iterated by comparing the training prediction results with the preset standard results to obtain the trained initial prediction model. The initial prediction model, after being trained, is validated using the validation dataset. Once the initial prediction model passes the validation, the classification prediction model is obtained.

5. A data classification device, characterized in that, The data classification device implements the steps of the data classification method as described in any one of claims 1 to 4, and the data classification device comprises: The data type determination module is used to receive data classification instructions uploaded by the client, obtain the data to be classified, parse the data to be classified, and obtain the data type of the data to be classified. The classification model matching module is used to determine a data classification model that matches the data to be classified based on the data type, and to obtain a matching classification model. The initial data classification module is used to import the data to be classified into the matching classification model to obtain initial classification data; The data classification and prediction module is used to obtain the business scenario data corresponding to the data to be classified, import the business scenario data and the initial classification data into the classification prediction model, and obtain the classification prediction result. The data classification feedback module is used to send the classification prediction results to the client and receive feedback information returned by the client. The final data classification module is used to combine the initial classification data based on the feedback information and the classification prediction results to obtain the data classification results.

6. A computer device, characterized in that, The system includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data classification method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the data classification method as described in any one of claims 1 to 4.

Images